THE WALLACE E. CARROLL SCHOOL OF MANAGEMENT
MD850: Advanced Topics in IT: e-Service Operations
Spring 2002

MD850 Home

Week 8
Capacity Management

Assigned Readings
Note: The following reading list looks like a lot, but most of the articles are actually in the 1-4 page range, with the remainder in the 8-10 page range, so in total it isn't that much.

Prior to (and after) a website or other e-Service is deployed, managers should analyze the service process to ensure that the service will not fail under the expected demand scenarios. Failure to use such tools can lead to failed web applications, as discussed in the following article:

"Making E-Business Work," Newport Group, Inc., 2001. (Paper Index on Mercury Interactive) or (Download from Mercury Interactive) or (Download from Class Website)

Several capacity management strategies are available for making sure that expected service demands will be able to be met by the service process. The tools available for capacity management of e-Services differ substantially based on a couple of factors. First, the amount one is willing to pay for a tool will determine the type of tool available. Second, a desire for a higher (or lower) level of reliability from an e-Service will determine which tools to use.

For smaller businesses, most capacity management tools revolve around ISP-based tools that report on the historical demands experienced by a web site. We review several different types of tools available for small to large businesses in the following paper on capacity management tools:

Witt, M., and G. R. Heim, "An Overview of Capacity Management Techniques and Tools for E-Services"

As e-Service customer demands grow and as the need for better assurance about the reliability of an e-Service grows, managers tend to want to know that the way in which an e-Service system's capacity is tested is similar to the nature of the service demands it will see in the real world. For example:

"RadView Tests MSNBC's Olympic Mettle"

"How MSNBC Beefed Up Its Web Site for the Olympics"

One possible approach for testing the capacity of a web site (or any other e-Service) before it goes live is to have actual humans within your company test the site out. Essentially, humans could generate demand loads for the site until the site fails in the test environment. This approach has numerous shortcomings, as you can imagine after reading the above articles. For example, using this approach, each human at each computer would create a certain set of demands for digital content delivered through their browser (or other delivery device). As a result, if you would want to ensure that your e-Service could successfully service 3000 people simultaneously, you would simultaneously have 3000 of your employees on their 3000 desktop computers make requests of the service process. Obviously, it would be difficult to control these 3000 individuals to test certain features of your e-Service system. As a result, most companies would not use such a testing process to determine the realizable capacity of their service system.

Instead, large companies will use automated tools for capacity management. Basically, a manager would set up the real website’s service-process configuration, test the site with “real” (actually simulated) demand loads generated by a demand-load simulator, and after adjusting the site’s performance, make the site go into operation. The Mercury Interactive articles linked to below lay out various issues that are important in (i) load testing before going live, and (ii) monitoring performance after going live.

In general, tools for testing large websites can be classified as either "active" or "passive" tools:

"New Frontiers in Web Performance Monitoring," META Group, 2001. (Paper Index on Mercury Interactive) or (Download from Mercury Interactive) or (Download from Class Website)

The major applications for analyzing website capacity fall into either (i) software testing applications, or (ii) testing/monitoring services. The following white papers describe the suite of tools offered by Mercury Interactive (which are similar to those of other companies).

This paper provides an overview of the full offering of software applications and services:

"Managing Web Application Performance Beyond 2000," Newport Group, Inc., 2000. (Paper Index on Mercury Interactive) or (Download from Mercury Interactive) or (Download from Class Website)

The following article describes the typical software web testing applications and what they do:

"The Complete Testing Solution for E-Business" (Paper Index on Mercury Interactive) or (Download from Mercury Interactive) or (Download from Class Website)

The following articles provide information about the types of application performance management services available for web testing, and the performance insights that can be identified:

"ActiveWatch: Challenging Industry Perceptions of Web Performance", Mercury Interactive. (Paper Index on Mercury Interactive) or (Download from Mercury Interactive) or (Download from Class Website)

"Common Web Site Problems Identified by ActiveTest," Mercury Interactive.
(Paper Index on Mercury Interactive) or (Download from Mercury Interactive) or (Download from Class Website)

"Mercury Interactive's Top 10 Performance Management Tips," Mercury Interactive. (Paper Index on Mercury Interactive) or (Download from Mercury Interactive) or (Download from Class Website)

If you are interested in additional examples of such technology by other companies, the links found at the bottom of this page present similar tools.

If you are interested in information about the total market size for web testing tools and the major vendors, the following paper can provide some information:

(Optional) "Annual Load Test Market Summary and Analysis," Newport Group, Inc., 2001. (Paper Index on Mercury Interactive) or (Download from Mercury Interactive)

Another approach – not considered here – is to formally model the e-Service system using OR-based mathematical tools for queueing networks. While academically interesting, this approach might not do you much good in a turbulent e-Service competitive environment, and might require you to employ someone with a PhD in Operations Research or Computer Science who knows queueing. If you are interested in this approach, I suggest you buy a book by Daniel A. Menasce and Virgilio A. F. Almeida, either Capacity Planning for Web Performance (1998) or Scaling for E-Business (2000). I'll explain these approaches in class.

Case Study Assignment
None.

Team Web-Based Assignment
(1) Visit the following sites and analyze the following web site load testing software and services:

Segue Software’s Silk Performer
http://www.segue.com/html/s_solutions/s_performer/s_performer.htm
Mercury Interactive’s ActiveWatch
http://www-svca.mercuryinteractive.com/products/activewatch/
GomezNetworks
http://www.gomeznetworks.com/mktg/method/method1.asp

Submit a 1-2 page report critiquing the capabilities of the various approaches.

Keyword Legend
E-SERV = E-Service: 24 Ways to Keep Your Customers …
CRM = CRM at the Speed of Light: Capturing and Keeping Customers …
FUL = E-Commerce Logistics and Fulfillment: Delivering the Goods

Related Readings: For Further Information

Major Vendors

Compuware
http://www.compuware.com/

Empirix
http://www.empirix.com/
http://www.empirix.com/Empirix/web+test+monitoring/products/e-test+suite.html

Gomez Networks
http://www.gomeznetworks.com/

Loadtesting.com
http://www.loadtesting.com/compare.asp

Mercury Interactive
http://www.mercuryinteractive.com/

Radview
http://www.radview.com/

Radware
http://www.radware.co.il/

Rational Software
http://www.rational.com/

Segue
http://www.segue.com/

Softlight Technologies
http://www.softlight.com/

Industry Sites for Web Application Testing and Capacity Management

“Capacity Planning,” Microsoft TechNet
http://www.microsoft.com/technet/ecommerce/capplan.asp?a=printable

Software Testing and Quality Engineering Magazine
http://www.stqemagazine.com/

StickyMinds.com
http://www.stickyminds.com/

Performance Benchmarks

ECperf
http://java.sun.com/j2ee/ecperf/

ECperf Results
http://ecperf.theserverside.com/ecperf/

Articles on Capacity Management, Load Testing, etc.
“How do the large web sites handle the load of millions of visitors per day?”
http://www.howstuffworks.com/question342.htm

Butler, J., “E-Performance Management Gains Attention at CMG,” Software Magazine, May 8, 2000.
http://www.softwaremag.com/archive/2000feb/CMG.html

Tools for Load Testing and Web Performance Monitoring
GomezNetworks Methodology
http://www.gomeznetworks.com/mktg/method/method1.asp

“Rational Software Introduces Rational SiteLoad; Industry’s First Web-Based Load Testing Product,” Press Release, August 28, 2000.
http://www.rational.com/news/press/pr_view.jsp?ID=7978
Data Sheet: SiteLoad - http://www.rational.com/media/news/presskit/siteload.pdf
http://www.rational.com/products/siteload/readme.jsp

“What is Web Performance Monitoring?”
http://www-svca.mercuryinteractive.com/products/monitoring/
Product: ActiveWatch - http://www-svca.mercuryinteractive.com/products/activewatch/
Product: Topaz - http://www-svca.mercuryinteractive.com/products/topaz/

Rapoza, J., “SilkPerformer Tests Web-Site Fabrics,” PC Week, July 29, 1999.
http://www.zdnet.com/products/stories/reviews/0,4161,410219,00.html
Schindler, E., “Smooth Site Testing,” Sm@rtPartner, September 11, 2000.
http://www.segue.com/html/s_news/recent_articles/SmartPartner 9.00.pdf
Product: SilkPerformer - http://www.segue.com/html/s_solutions/s_performer/s_performer.htm
Product: EConfidenceScale - http://www.segue.com/html/s_solutions/s_hosted/scale_home.htm

Hendrick, K., and S. D. Hendrick, “RadView Software’s WebLoad: Integrated Testing for the Web,” IDC Bulletin #21616, February 2000.
http://www.radview.com/products/papers/IDC/paper.asp
Product: WebLoad - http://www.radview.com/products/webload.htm
Product: WebLoad Resource Manager - http://www.radview.com/products/webloadmgr.htm

Capacity Analysis Tools for Period-by-Period Demand and Demand Load Summary

Web Testing and Monitoring Newsletter – Comparison of Tools
http://members.tripod.co.uk/wtmn/

PerfMan for WebServers
http://www.perfman.com/sellSheets/WebServer/PerfManforWebServers.html

Tutorials on almost any type of capacity you might be interested in managing
http://www.iec.org/online/tutorials/

User Tracking
http://www.arsdigita.com/books/panda/user-tracking

So you want to run your own server
http://www.arsdigita.com/books/panda/server

Related Readings: Traditional (Person-to-Person) Services

Fitzsimmons & Fitzsimmons, Service Management: Operations, Strategy, and Information Technology, Irwin/McGraw-Hill, 2001, Chapter 12 ("Queueing Models and Capacity Planning"), Chapter 13 ("Managing Capacity and Demand")

Haksever, et al., Service Management and Operations, Prentice Hall, 2000, Chapter 11 ("Managing Demand and Supply in Services"), Chapter 15 ("Forecasting Demand for Services")