Testing Methodology

To be truly robust, distributed applications require more than simple functional testing before release into production. You should perform at least one—and preferably all—of the following types of testing before releasing your app to customers:

  • Performance testing
  • Load testing
  • Stress testing
  • Endurance testing
Each test captures a different perspective of your distributed application that does not necessarily surface in functional tests.

    Performance testing typically indicates the response time for the entire application from the user's perspective. Proper response time for user actions is critical to maintaining and enhancing your user base. You can also performance test individual components in your application to see how these components interact. Visual Studio Analyzer helps you understand the performance of your application by providing information about individual events in your components.

    Load testing demonstrates how your application performs under concurrent user sessions for typical user scenarios. Setting up common scenarios that execute for a short period of time allows you to see how your application operates under a multiple-user load. Generally, load testing accounts for user "think time," which is the average assumed time for a user action. For example, the think time used when load testing for an order app could be the number of seconds that a data entry clerk takes to enter an order. You can use Analyzer to depict how the performance of your app changes under load testing.

    Stress tests allow you to examine how your application behaves under a maximum user load. To stress test your application, remove the think time for your load scripts and execute the scripts against the server to overload use of the application. If you're getting unhandled exceptions in a stress test, your application may not be robust enough to handle a sudden unexpected increase in user activity.

    Stress tests generally execute for a longer period of time, and can be used to catch difficult-to-diagnose problems like subtle memory leaks in your app. Properly automated, stress tests can be activated from every developer's desk when they leave at night. Remote machines can be monitored by Analyzer, and you can review the Analyzer log in the morning. (Note that if you choose to use Analyzer for logging under high stress, the Analyzer log is bound to fill up quickly. You can then write a simple script that archives the log file at a prescribed point in order to preserve disk space.)

    Performing at least one of these testing methods in addition to core functional testing is an important part of delivering a robust application. You can use Analyzer in conjunction with a variety of tools (such as Bounds Checker and other code analysis tools) to track and measure results.