Testing Intelligence Server Performance

In addition to testing reports and documents for execution and for accuracy between projects, Integrity Manager can determine how long it takes an Intelligence Server to execute a given set of reports or documents. This is called a performance test. You can execute the reports and documents in the integrity test multiple times, to get a better idea of the average time it takes to execute each report.

In a performance test, Integrity Manager records the time it takes to execute each report or document. If the reports and documents are being executed more than once, Integrity Manager records each execution time. You can view the minimum, maximum, and average execution time for each report or document in the Results Summary area. In a comparative integrity test, you can also view the difference in time between the baseline and target reports and documents.

Performance Test Best Practices

The results of a performance test can be affected by many factors. The following best practices can help ensure that you get the most accurate results from a performance test:

• Performance comparison tests should be run as single-project integrity tests. This reduces the load on Integrity Manager and ensures that the recorded times are as accurate as possible.

  To compare performance on two Intelligence Servers, MicroStrategy recommends following the steps below:

  1. Perform a single project test against one project, saving the performance results.
  2. Perform a single project test against the second project, saving the performance results.
  3. Compare the two performance results in a baseline-versus-baseline test.
• Wait until the performance test is complete before attempting to view the results of the test in Integrity Manager. Otherwise the increased load on the Integrity Manager machine may cause the recorded times to be increased for reasons not related to Intelligence Server performance.
• If you are using a baseline-versus-project test or a baseline-versus-baseline test, make sure that the tests have processed the reports and/or documents in the same formats. Execution times are not recorded for each format, only for the aggregate generation of the selected formats. Thus, comparing a baseline with SQL and Graph data against a test of only SQL data is likely to give inaccurate results.
• If the Use Cache setting is selected on the Select Execution Settings page of the Integrity Manager Wizard, make sure that a valid cache exists for testing material. Otherwise the first execution cycle of each report takes longer than the subsequent cycles, because it must generate the cache for the other cycles to use. One way to ensure that a cache exists for each object is to run a single-project integrity test of each object before you run the performance test.

  This setting only affects reports, and does not apply to documents.

• In the Integrity Manager wizard, on the Select Execution Settings page, make sure Concurrent Jobs is set to 1. This causes Intelligence Server to run only one report or document at a time, and provides the most accurate benchmark results for that Intelligence Server.
• The Cycles setting on the Select Processing Options page of the Integrity Manager Wizard indicates how many times each report or document is executed. A high value for this setting can dramatically increase the execution time of your test, particularly if you are running many reports or documents, or several large reports and documents.
• Use 64-bit Integrity Manager when the comparison data is large. The default position of 64-bit Integrity Manager is under C:\Program Files (x86)\MicroStrategy\Integrity Manager called MIntMgr_64.exe. Additionally, use 64-bit Integrity Manager if you have memory issues.