Lessons from the Mainframe Era

Diane Brockman

Over the years, a lot of smart people have built database systems and identified controls and processes required to build and implement complex business applications. Rather than dismiss the procedures and techniques developed in the mainframe era as intolerably burdensome, it behooves us to see how we can benefit from the lessons of the past.

Background

The basic difference in approach (client/server vs. mainframe development) lies with the difference in usage. Mainframe computers represent a considerable investment for any company, and, because of this expense, multiple areas within a company tend to share a single mainframe. The result is that controls and restrictions are put in place to control access,allocate resources (and expenses), and carefully monitor performance.

Personal computers, on the other hand, which started life as small, inexpensive computers for the consumer market, were quickly adopted by long-suffering individuals and departments as a way to circumvent (or, more charitably, complement) IT and avoid long development cycles.

Unfortunately, it wasn’t long before these PC-based applications ran into the same kinds of issues–security, maintenance, and performance–that affected their mainframe siblings.

Why? Complex business solutions require more edits and controls within the software itself (business specifications). The fact that the application can affect clients of the company means that stricter guidelines for development (technical specifications) must be put in place and the software must meet client expectations (testing). As modifications are made to the application, you need to keep track of the changes (version control). The use of the system by many users means that effort must be expended to control access (security) and ensure that the performance is acceptable (performance and tuning). It starts to look like the mainframe again, doesn’t it? Lessons can be learned from the mainframe era as we take only the best of what was developed during the mainframe life cycle and integrate it with modern techniques. Let’s take each of the items listed above and outline what I mean.

Business specifications

There’s no getting away from it–you can’t build it if you don’t know what "it" is. If you’re developing a complex application, then someone needs to write a detailed specification. Furthermore, you’ll probably be part of a development team, so the business needs must be documented in order to ensure that you and all of the other members of the team are working toward the same goal. Technical developers typically don’t know the business rules or requirements without input from the business area, and that’s where the business, technical, and functional specification documents come in.

Business specifications usually start with a verbal description of what’s required, but this is generally very "high level." I find that writing the requirements is the fastest way to identify the holes that need to be filled. Once the details start to appear, additional questions arise that add even more detail until the complete requirements have been documented. Sample screen layouts and reports at this stage will prevent costly rewrites after the fact. Once the specifications are complete, the business unit needs to sign off on the requirements. On every system that I’ve worked on, business units will typically agree with the "final" specification–right up until I ask them to sign off! Then, there’s at least a one-week delay while it’s reviewed again by the end users, and there’s always at least one change or clarification. Forcing that final check and verifying that the requirements are correct before you start in-depth development is one of the best investments you can make.

Technical specifications

Once you know the business needs and processes, turn your attention to the technical implementation of the system. Data modeling, database design, and program design are all part of this phase. A good system starts with a good database design. The more the DBA knows about the data and how it’s used, the better the database design will be. I make it a policy to use stored procedures for database access. This allows the DBA to do what he or she does best and frees the front-end developers to do what they do best. On a recent project, one of the front-end developers prepared a technical specification for a month-end program that went for 10 pages and involved a great deal of complex code. When the DBA reviewed the specification, he was able to identify where stored procedures could be used to do much of the computational work. The 10-page specification was reduced to one and a half with the front-end program becoming a shell to call a series of stored procedures. A program that was estimated to take over a month to develop was completed in less than half that time.

Your front-end developers must understand the business requirements in addition to the technical issues in order to structure properly the application programs. These must be broken into functional units with methods defined for accessing the data. If you plan your development ahead of time, you can take advantage of common routines for such things as error handling, data formatting, and data access. As part of this design, you need to consider how you’ll test the application. Consider scheduling the construction of routines that insert data into the database ahead of retrieval/update routines to provide data for testing those functions. Pre-load your static "lookup" tables prior to the start of testing. These tables are usually set up independently of the application.

Ensure that all members of your development team are familiar with the work of the other team members. Walkthroughs of technical specifications can save a lot of development time, as any misunderstandings can be cleared up in the early stages. Code reviews are helpful–not just to ensure that the code meets the requirements–but also as a training/learning tool, as more junior members of the team can learn from the senior staff and vice versa. It’s during these reviews that the need for comments becomes apparent to everyone–this is one item that will make maintenance easier in the future.

Testing

PC-based applications that are being used in production environments are often "mission-critical" and affect the company’s fundamental business operations, so the integrity of the data and the processes that affect it are critical. Proper testing is essential, and the more complex the application, the more testing is required. You (or your team) should develop a testing strategy at the same time that the development is starting. As each component is being built, the ability to test should already be in place. As you add each new component, it should build on the previous piece until a comprehensive unit is available. This unit is then tested to ensure that it functions correctly, using both positive and negative test cases.

Testing is a touchy subject for most developers. Often, when the code is handed over to the business acceptance team, there are a lot of errors discovered and egos get damaged. I have two approaches to minimize this. First, I give a copy of the system test plan to the team members as they’re writing their code. Once again, this catches potential errors at an earlier stage in development. On a recent project, I had one team member come to me, pointing out what he thought was an error in the test plan. I was able to clarify that the test plan was correct and resolve his misunderstanding of the requirements before the code was written.

Next, I have developers test each other’s code. I have a box of prizes (nothing major–pens, paper cubes, candy dishes, and so on) and the prize goes to the developer if the code is clean and to the tester if errors are found. It becomes a game, but the real winner is the business unit. As the team actually uses the system, they learn more about how the business flows and can then recommend changes that make the product more user-friendly. When the business unit is approached regarding the changes, they usually agree and appreciate that your development team is working in their best interest. With this type of testing being done by your development team, the Business Acceptance test team has to work harder to find errors. A side benefit to this approach is that your team members are cross-trained, which makes future maintenance easier.

I take the same approach with the business acceptance test plan. You have to remember that the goal is to develop a system that meets the business needs. If sharing any of these test plans with your development team aids in meeting that goal, it can only be of benefit to everyone. If the programmer has misunderstood part of the requirement, and that doesn’t become apparent until business acceptance testing has started, then meeting the deadline could be impacted. Now, you are looking at re-working and re-testing. (People have come up to me and said, "I can’t believe you’re giving the test plan to the developers." My response was, "The business requirements are not a secret.")

Before you turn the code over to the business acceptance team, be sure to volume test the application with real data. Often there are interdependencies between reports, screens, and so forth that might be missed if real data isn’t used for at least part of the testing. It’s also easier for the business unit to validate the results if real data is used. From a volume perspective, code that’s running against 10 rows in a table won’t identify a problem when it later runs against a million. You need to test for your expected production volume.

As I’ve mentioned, once the development team is satisfied, a business test team will then exercise the code. This batch of testing focuses on the business process, and as such, tests the business rules and constraints in more depth than system developers would. It’s at this stage that the actual business users would also be looking at test results to ensure that reports and the like are what was expected.

Version control

The bigger the project, the more urgent the need for version control. You need to keep track of the changes that were made during the testing cycle. There are two reasons for this: First, it allows you to go back to a previous version if the current one fails or causes problems in other areas of the application. Second, it provides an audit trail to let you know what code was changed. Don’t forget–backups are an important part of version control.

Security

Access to both the hardware and software must be controlled. The server needs to be in a secured area and security rules implemented to ensure that only authorized users can access/change the data. The implementation of this type of security usually involves designing some type of authorization form and assigning an individual to the db_accessadmin role for the database.

Programmers should be restricted to using the application’s stored procedures to update the database. This ensures that they’re testing the code with the same access rules that the end users will have. It prevents code from working only for the developer that wrote it.

Performance and tuning

As part of the original technical review, the operational requirements of the system need to be addressed. This includes frequency of backup, memory requirements, licensing, performance needs, physical storage space, and so forth. These items are part of the original proposal for the system, but future needs should be addressed as well.

There are performance issues related to the volume testing phase of the project. These involve ensuring that the system will perform in the production environment just as well as it did in the development environment. Consider such things as other applications running on the server, network capacity, number of concurrent users, and so on. As more users are added to the system and the amount of data increases, changes to the server, network, or memory, for instance, might be required. It’s important that this be considered to ensure that the system continues to meet the users’ needs.

Conclusion

As the saying goes, "It’s a long, hard road to overnight success." Mainframe developers learned a long time ago that these steps and processes need to be followed to ensure a successful delivery, and client/server and other PC-based developers are learning the same lessons today. As systems become more complex and involve more users, the controls will inevitably follow. The problem with the mainframe today is that the controls took on a life of their own and became more complex and time-consuming than many of the applications! The challenge for the PC server community is to learn from that experience and take the best of what it has to offer.