Advanced Job Telemetry

Advanced job telemetry aims to improve administration and resource monitoring in Platform Analytics to catch jobs and object information before Intelligence server crashes.

Purpose

Prior to MicroStrategy 2021 Update 6, job statistics were not recorded when a job was created. Thus, when the Intelligence server crashed, there was no way to determine which jobs were active or the objects being manipulated.

Starting in MicroStrategy 2021 Update 6, Platform Analytics enhances the data model to provide server statistics to MicroStrategy administrators during the initial job creation so that the end user can understand what users, actions, or cubes caused the Intelligent Server to crash. A new Kafka topic, Mstr.PlatformAnalytics.IsJobStats, is used to process messages of the creation and completion of jobs.

Workflow

Once a job is created, a message is sent to the Kafka Topic with the CREATETIME field. Upon job completion, another message is sent with the COMPLETETIME field.

Platform Analytics processes messages from the Mstr.PlatformAnalytics.IsJobStats Kafka Topic and batch inserts the start of a job in the fact_inprocess_jobs table with creation_time in ThreeSecondETL. Once the job is complete, an upsert is performed with completion_time. Platform Analytics extracts, transforms, and loads (ETL) on an hourly basis to purge the completed jobs and clean the record.