Preserve Statistics Integrity During Parallel Upgrades, Metadata Migration, or Clone

Parallel Upgrade or Metadata Migration with Platform Analytics

Platform Analytics is a telemetry tool that supports telemetry capture from multiple MicroStrategy deployments simultaneously. Platform Analytics has a unique way to identify each metadata connection to the Intelligence Server.

When an Administrator reconfigures the Intelligence Server with a different metadata backup, Platform Analytics immediately recognizes the change and starts recording a new telemetry that is associated with the new metadata.

Depending on your objective, no further action is required. For example, if you created a second environment as a test or sandbox environment. If you want to continue capturing telemetry under the same metadata repository, see below:

If any of the following parameters change, Platform Analytics considers your metadata to be new and an additional row appears in the lu_metadata table:

• metadata_guid A unique metadata GUID is generated and stored when new metadata is created via the configuration wizard.
• host The server that hosts the database sever.
• port The port where the database server runs.
• database The metadata database name.
• uid/sid This field is only used for Oracle databases.

The parameters listed above are processed based on the values that are provided in the metadata DSN. The metadata DSN is located in your odbc.ini file for Linux and the ODBC Data Source Administrator for Windows.

To restore telemetry and optionally ensure all telemetry is captured, perform the following steps:

1. Keep your Platform Analytics services up and running. For example, Zookeeper, Kafka, PAconsumer, REDIS, etc.
2. To optionally ensure that Platform Analytics has completed processing the existing telemetry before making configuration changes, check the number of telemetry message matches for offset topics or if they differ by negligible amount between the Telemetry Server (For example, Kafka) and the Platform Analytics Repository:
   1. MicroStrategy suggests that you check messages from the following core topics or subject areas:
      1. IsReportStats
      2. IsDocumentStats
      3. IsSessionStats
   2. To get the result from the Telemetry Server, go to its installation directory (For example, opt/MicroStrategy/Messaging Services/Kafka/kafka xx.xxx/bin) and execute the following query:
      Copy

      ./kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list <ip:port> --topic <topic> --time -1

   3. To get the result from the Platform Analytics Repository, connect to the repository via the DB Query tool or another query editor and execute the following query:

      Copy

      select * from platform_analytics_wh.lu_topic_partition_offset where id like '%<topic>%'

3. Reconfigure the Intelligence Server with a new metadata.

4. Connect to the Platform Analytics Repository via the DB Query tool or another query editor and verify that the new entry in the lu_metadata table is populated.

   

5. Stop the Platform Analytics Consumer (PAconsumer) process.

6. Using the DB Query tool, swap the metadata_ids in the lu_metadata table by using a temporary modified metadata_id to avoid primary key exceptions:

   Outline

   Copy

   UPDATE lu_metadata SET metadata_id = 'TEMPORARY_MD_ID' WHERE metadata_db_connection = 'OLD_MD_CONNECTION';
   UPDATE lu_metadata SET metadata_id = 'OLD_MD_ID' WHERE metadata_db_connection = 'NEW_MD_CONNECTION';
   UPDATE lu_metadata SET metadata_id = 'NEW_MD_ID' WHERE metadata_db_connection = 'OLD_MD_CONNECTION';

   Example

   Copy

   UPDATE lu_metadata SET metadata_id = '7051653456108457985' WHERE metadata_db_connection = 'host=10.23.35.94;port=5432;database=platform_metadata;';
   UPDATE lu_metadata SET metadata_id = '7051653456108457984' WHERE metadata_db_connection = 'host=127.0.0.1;port=5432;database=platform_metadata;';
   UPDATE lu_metadata SET metadata_id = '7051946829721636864' WHERE metadata_db_connection = 'host=10.23.35.94;port=5432;database=platform_metadata;';

7. Start the Platform Analytics Consumer.

8. Confirm transactions are mapped to the correct metadata_id using the following query. The new entries should have the old metadata_id with the new metdata_db_connection.

   Copy

   SELECT object_name, m.metadata_id, metadata_db_connection, tran_timestamp
   FROM access_transactions a
       JOIN lu_object o ON a.object_id = o.object_id
       JOIN lu_project p ON o.project_id = p.project_id
       JOIN lu_metadata m ON p.metadata_id = m.metadata_id
   WHERE tran_timestamp > NOW() - INTERVAL '5 MINUTE'
   ORDER BY tran_timestamp DESC;

9. Purge the Platform Analytics Warehouse to delete all objects in the metadata with the unused metadata_id using the following query:

   Copy

   doTestBeforePurge: true
   commandsToExecute:
   #  - commandName: DELETE_ALL_DELETED_OBJECTS
   
   #  - commandName: DELETE_ALL_DELETED_PROJECTS
   
     - commandName: DELETE_ALL_OBJECTS_IN_METADATA
       metadataList:
         - METADATA_ID_TO_BE_PURGED
   #      - metadataId_2