Strategy One

Comparison Tests

A comparison test allows you to select two versions of a dashboard or baseline test, and determine whether their outputs match. This is critical for validating system migrations, User Acceptance Testing, or parallel run scenarios.

The purpose of comparison tests are the following:

  • Migration Validation: Ensure results are consistent after moving objects or data to a new environment (for example, development to production).

  • Object Consistency: Compare two versions of an object after modifications or refactoring.

  • Parallel Testing: Validate that two systems or environments produce identical results.

You must have the Integrity Manager privilege to use the Test Center.

Create a Comparison

  1. Open the Workstation window.
  2. In the Navigation pane, under Administration, click Test Center.

  3. Select Comparison Test, choose the content to include (Data and/or SQL), and click Next.

  4. Choose the Source object. A source can be a Baseline or Result.

    • Baseline sources require a result object of the same Baseline as a target. For example:

      • Source: Financial Dashboard Baseline

      • Target: Financial Dashboard Baseline, July 2, 2025 (Result created from the Baseline)

    • Result sources require either another Result from the same Baseline or the original Baseline as a target. For example, if you want to compare two runs to check if the test output has changed between two dates, you would want to compare two Results. To do this, you can have something like the following:

      • Source: Financial Dashboard Baseline, July 2, 2024 (Result)

      • Target: Financial Dashboard Baseline, July 5, 2024 (another Result from the same Baseline)

    • However, if you just wanted to compare a specific Result to the Baseline (same as the first scenario) you can still compare the Result to the original Baseline itself (only the order is different from the first scenario):

      • Source: Financial Dashboard Baseline, July 2, 2024 (Result)

      • Target: Financial Dashboard Baseline

  5. After you select the source and target, click Next.

  6. The test and its results are located in the Test Center.

Comparison Test Results

  1. To see specifics, Right-click a test result and select View Result.

  2. By default, you will see the Overview page. The overview page provides a summary of pass ratios, matches, and mismatches between the two compared objects (for example, baseline versus result, result versus result).

  3. To check for errors and mismatches as well as explanations, click the Details tab.

  4. To switch to the SQL view, click SQL. This page highlights differences between generated queries.

  5. To switch to the Data view, click Data. This page compares data output side by side, showing precisely where discrepancies exist. If there are missing visualizations, the source and target are marked in the objects panel.