Platform Analytics Health Check Utility

The Platform Analytics Health Check Utility is an end-to-end health check. This utility troubleshoots issues across multiple components required to produce, consume, and report on telemetry from the platform. In particular, this check is recommended if a report in one of your projects is not providing telemetry to the Platform Analytics warehouse.

The Platform Analytics Health Check Utility performs all three health checks that occur in Start-Up Health Checks and end-to-end telemetry checks to verify that data can be produced by the Intelligence Server and consumed by the Platform Analytics Consumer (Telemetry Store).

If you are using Linux, the Platform Analytics Health Check Utility is located at /opt/MicroStrategy/PlatformAnalytics/bin. If you are using Windows, it is located at C:\Program Files (x86)\MicroStrategy\Platform Analytics\bin.

How to Run the Platform Analytics Health Check Utility

To perform an end-to-end Platform Analytics Health Check, run the platform-analytics-health-check.(sh/bat) utility.

The end-to-end telemetry checks performed by the Platform Analytics Health Check utility include:

Health Check
Redis Health Check
Kafka Health Check
Change Journal Health Check
Statistics Health Check

Health Check

During the health check, there are two checks being executed:

Can the consumer/producer connect to the database provided during installation and stored in the PAConsumerConfig.yaml configuration file? If not, additional network connectivity testing occurs to diagnose the cause of the issue.
Does the database user have the required privileges? For a full list of installation prerequisites see, Platform Analytics Prerequisites.

The Health Check report provides a list of the privileges and the resulting status. If all the checks are successful, the final line will read Warehouse health check result is healthy.

If any line reads Failed, check your PAConsumerConfig.yaml file and ensure the database has the correct privileges.

Suggested Troubleshooting for Health Check Errors

If you receive any of the following errors in the Health Check, here are suggested workarounds:

Missing Privileges Error

If the database user stored in the PAConsumerConfig.yaml configuration file is missing privilege(s), then INFO [privilege type] privilege: Failed. To resolve this error, the administrator must grant the missing privilege(s) to the database user and restart the consumer.

How to Grant Missing Privileges:

Stop the Platform Analytics Consumer and the Usher Metadata Producer.
Connect to the database server that contains the Platform Analytics Repository. Execute the following command, replacing the 'someuser' and 'somehost' with the customer specified information:
Copy
```
GRANT DROP ON platform_analytics_wh.* TO ‘someuser’@‘somehost’;
```
Restart the Platform Analytics Consumer and Usher Metadata Producer.

Failed Connection Error

If the consumer or producer is unable to connect to the database using the configuration specified in the PAConsumerConfig.yaml configuration file, you may see the following error:

Copy

2018-11-21 21:43:28,793 INFO HealthCheck main - Failed to connect to the database. Retrying after waiting for 60 seconds.
2018-11-21 21:45:31,797 INFO HealthCheck main - Failed to connect to the database. Retrying after waiting for 60 seconds.
2018-11-21 21:47:34,800 ERROR HealthCheck main - Failed to connect to the database using url:jdbc:mysql://XX.Y.Z.1:3306/platform_analytics_wh?rewriteBatchedStatements=true&useLegacyDatetimeCode=false&serverTimezone=UTC. Please double check your connection parameters.
Communications link failure

The PAConsumerConfig.yaml file is populated based on the database information provided during installation. To resolve this error, connect to the machine hosting Platform Analytics and confirm all fields under the warehouseDbConnection heading are correct in the PAConsumerConfig.yaml file.

Database User Password is Incorrect Error

The consumer or producer will be unable to connect to the database if the encrypted warehouse password is incorrect. To generate a new encrypted password and update the confirmation, see Update the Database User Password Configured to the Platform Analytics Repository .

Database User Created with SSL Enabled Error

Platform Analytics supports MySQL versions 5.6, 5.7, and 8.0. For MySQL 8.0, SSL connection is enabled by default. Currently, Platform Analytics does not support SSL for the database user connecting to MySQL. When you create the database user for the Platform Analytics Consumer or Usher Metadata Producer, specify the SSL/TLS option using the REQUIRE clause.

How to Disable SSL:

Connect to the Platform Analytics Repository and execute the following command:
Copy
```
show variables like '%ssl%';
```
If the result for 'have_ssl' is 'YES' then SSL is enabled. Create the user with mysql_native_password and REQUIRE NONE options to connect without SSL.
Copy
```
CREATE USER 'test'@'%' IDENTIFIED WITH mysql_native_password BY 'password' REQUIRE NONE;
```

Redis Health Check

The Redis Health Check determines if the consumer or producer can successfully connect to the Redis server. The check provides detailed statistics about Redis collected during startup. If all the checks are successful, the final line will read Redis server health check result is healthy.

If you see an error in your check, ensure Redis is running and that your configuration is correct in the PAConsumerConfig.yaml file.

Suggested Troubleshooting for Redis Health Check Errors

If you receive any of the following errors in the Redis Health Check, here are suggested workarounds:

Redis is Stopped Error

If the consumer or producer is unable to connect to Redis, it may be because it is in a stopped state. To resolve this error, start the MicroStrategy In-Memory Cache, the Platform Analytics Consumer, and the Usher Metadata Producer.

Failed to Connect to Redis Error

If the consumer or producer is unable to connect to Redis, it may be because the configuration is not correct in the PAConsumerConfig.yaml file. To resolve this error, connect to the machine hosting Platform Analytics and confirm all fields under the redisConnection heading are correct in the PAConsumerConfig.yaml file.

It's possible the Redis server failed to write the snapshot to the disk. If this is the case, you can disable the RDP snapshotting process on the Redis server.

Stop the platform-analytics-usher-lookup-producer using the following command:
Copy
```
./platform-analytics-usher-lookup-producer.sh stop
```
Stop the platform-analytics-consumer using the following command:
Copy
```
./platform-analytics-consumer.sh stop
```
Stop the Redis server.
Apply the following changes to the redis.conf file:
1. Comment out the three lines under the ###SNAPSHOTTING### heading.
  Copy
```
#save 900 1
#save 300 10
#save 60 10000
```
2. In the same section, enter the line:
  Copy
```
save ""
```
Start the Redis server.
Start the platform-analytics-consumer using the following command:
Copy
```
./platform-analytics-consumer.sh start
```
Start the platform-analytics-usher-lookup-producer using the following command:
Copy
```
./platform-analytics-usher-lookup-producer.sh start
```

Password Authentication Enabled for Redis Error

If the consumer or producer is unable to connect to Redis, it may be because password authentication is enabled. By default, Redis is not configured with password authentication, but it can set after installation.

If Redis has been enabled with password authentication and the password is missing in the PAConsumerConfig.yaml configuration file, the consumer or producer will be unable to connect to Redis. To resolve this error, follow the steps for Enable Password Authentication on the MicroStrategy Telemetry Cache.

Kafka Health Check

The Kafka Health Check ensures the Telemetry Manager (Apache Zookeeper) and the Telemetry Server (Kafka Server) are started and connected. If all the checks are successful, the final line will read Kafka cluster health check result is healthy.

Since the Telemetry Server is dependent on the Telemetry Manager, the Telemetry Manager must be started first.

If you see an error in your check, ensure ZooKeeper and Kafka are started.

Suggested Troubleshooting for Kafka Health Check Errors

How to check if ZooKeeper servers are running on all nodes:

On Linux, run the following command to run the PID.

Copy

ps ax | grep java | grep -i QuorumPeerMain | grep -v grep | awk '{print $1}'

On Windows, open Window's Services and check if the \"Apache ZooKeeper\" service is running.

How to check if Kafka servers are running on all nodes:

On Linux, run the following command to run the PID.

Copy

ps ax | grep -i 'server.prop'| grep java | grep -v grep | awk '{print $1}'

On Windows, open Window's Services and check if the \"Apache Kafka\" service is running.

How to start ZooKeeper and Kafka on all nodes:

On Linux, run the following commands in the Kafka directory:

Copy

# Start Zookeeper on all nodes,
./zookeeper-server-start.sh -daemon ../config/zookeeper.properties
# Start Kafka on all nodes,
./kafka-server-start.sh -daemon ../config/server.properties

On Windows, open Window's Services and start Apache ZooKeeper and Apache Kafka.

If this is a clustered environment with multiple nodes of ZooKeeper and Kafka, you must start all nodes of ZooKeeper first.

Change Journal Health Check

The Change Journal check ensures the Platform Analytics Consumer is healthy. For this check, you must provide a project GUID and a report GUID to test. You are asked to modify the description of the report, which generates a Change Journal log. The test verifies if the Intelligence Server Producer produced the log to the Mstr.PlatformAnalytics.ChangeJournal.CubesReportsDashboard Kafka topic. Then, it tests if the log is processed and written to the Platform Analytics warehouse table lu_object.

If the record is found in both the appropriate Kafka topic and the warehouse, the final line will read Change Journal health check result is healthy.

If you see an error in your check, ensure the feature flag Messaging Service for Platform Analytics is on in the Intelligence server and that the property Telemetry Server enabled is set to True in the Intelligence server.

Suggested Troubleshooting for Change Journal Health Check Errors

Verify the Intelligence Server is Configured to Write Telemetry to Kafka

Using Command Manager, connect to the Intelligence server.
To view the status of the feature flag, run the command:
Copy
```
LIST ALL FEATURE FLAGS;
```
In the results, verify the Messaging Service for Platform Analytics feature flag is ON. If the feature flag is OFF, run the following command to turn it on:
Copy
```
ALTER FEATURE FLAG "Messaging Service for Platform Analytics" ON;
```
To view the status of the Telemetry Server enabled property, run the command:
Copy
```
LIST PROPERTIES FOR SERVER CONFIGURATION;
```
In the results, verify the Telemetry Server enabled property is set to True. If the property is set to False, execute the command below:

Replace <kafka server IP> with your Kafka server IP address.
Copy
```
'ALTER SERVER CONFIGURATION ENABLEMESSAGINGSERVICES TRUE CONFIGUREMESSAGINGSERVICES "bootstrap.servers: <kafka server IP>:9092/batch.num.messages:5000/queue.buffering.max.ms:2000";
```
Restart the Intelligence server.

Verify the Platform Analytics Consumer is Working

On Linux:

Navigate to the folder where Platform Analytics is installed.
```
<Install>/PlatformAnalytics/bin
```
Run the following command:
Copy
```
./platform-analytics-consumer.sh status
```
Start or restart the server with the following command:
Copy
```
./platform-analytics-consumer.sh start
```

If restarting the server does not resolve the issue, check the logs under <Install>/PlatformAnalytics/log/platform-analytics-consumer.log or contact MicroStrategy Technical Support and attach the folder <Install>/PlatformAnalytics/log to your case.

On Windows:

Open Windows Services using services.msc.
Start or restart the Telemetry Store.

Statistics Health Check

This health check ensures the Platform Analytics Consumer can process report statistics. You are prompted to execute the report from the previous health check. This generates a log to the topic Mstr.PlatformAnalytics.IsReportStats. The health check verifies that the Intelligence Server Producer produced the record to Kafka and that it is in the Platform Analytics warehouse.

If the record is found in both the appropriate Kafka topic and the warehouse, the final line will read Statistics health check result is healthy.

If you see an error in your check, ensure Statistics are enabled for the project and that Messaging Services is configured correctly.

Suggested Troubleshooting for Statistics Health Check Errors

Verify Statistics are Enabled for the Project

Using Command Manger, connect to the Intelligence server.
Run the command:

Replace <Project Name> with your project's name.
Copy
```
LIST ALL PROPERTIES FOR PASTATISTICS IN PROJECT "<Project Name>";
```
In the results, verify the property Basic Statistics is set to True. If it is set to False, run the command below:

Replace <Project Name> with your project's name.
Copy
```
ALTER PASTATISTICS BASICSTATS ENABLED DETAILEDREPJOBS TRUE DETAILEDDOCJOBS TRUE JOBSQL TRUE COLUMNSTABLES TRUE IN PROJECT "<Project Name>";
```
Restart the Intelligence server.

Verify the Platform Analytics Consumer is Working

On Linux:

Navigate to the folder where Platform Analytics is installed.
```
<Install>/PlatformAnalytics/bin
```
Run the following command:
Copy
```
./platform-analytics-consumer.sh status
```
Start or restart the server with the following command:
Copy
```
./platform-analytics-consumer.sh start
```

On Windows:

Open Windows Services using services.msc.
Start or restart the Telemetry Store.