Start-Up Health Checks

Multiple services were renamed in the MicroStrategy 2019 release. Because this guide requires modifying the underlying files, it uses the original service name.

The Telemetry Store (i.e. Platform Analytics Consumer) and Identity Telemetry producer (i.e. Usher Metadata Producer) are dependent on and require access to three components to process telemetry logs:

Platform Analytics Repository (i.e. Database Server)
Telemetry Cache (i.e Redis)
Telemetry Server (i.e. Kafka)

All the three components must be in a healthy state for Platform Analytics to successfully process telemetry logs. If any of these components are unavailable, the Telemetry Store consumer and Identity Telemetry producer will stop. Therefore, during start up, both the consumer and producer execute a health check for the three components and generate a detailed report with the results.

At times, one of the components may be in the process of starting and not in a completely ready-state when the health check starts. In such situations, the consumer and producer will perform three consecutive checks with a 60-second delay between each check to confirm if the dependencies are in an unhealthy state.

For more information about Platform Analytics Architecture, see Platform Analytics Architecture and Services.

Naming Conventions and Locations

A health check is performed during start up of both the Platform Analytics Consumer and the Usher Metadata Producer. Therefore, there are two health check reports generated in the Platform Analytics log folder located in the default install path:

Linux: /opt/MicroStrategy/PlatformAnalytics/log
Windows: C:\Program Files (x86)\MicroStrategy\Platform Analytics\log

The name of the file identifies whether the report is corresponding to the consumer or the producer.

For example,

Copy

platform-analytics-consumer-health-check-yyyymmddhhmmss.out
platform-analytics-usher-lookup-producer-health-check-yyyymmddhhmmss.out

Health Check Report Results

Each health check report is structured into four sections:

Health Check
Redis Health Check
Kafka Health Check
Health Check Summary

Each section provides different information on the health of your three components.

Health Check

During the health check, there are two checks being executed:

Can the consumer/producer connect to the database provided during installation and stored in the PAConsumerConfig.yaml configuration file? If not, additional network connectivity testing occurs to diagnose the cause of the issue.
Does the database user have the required privileges? For a full list of installation prerequisites see, Platform Analytics Prerequisites.

The Health Check report provides a list of the privileges and the resulting status. If all the checks are successful, the final line will read Warehouse health check result is healthy.

If any line reads Failed, check your PAConsumerConfig.yaml file and ensure the database has the correct privileges.

Suggested Troubleshooting for Health Check Errors

If you receive any of the following errors in the Health Check, here are suggested workarounds:

Missing Privileges Error

If the database user stored in the PAConsumerConfig.yaml configuration file is missing privilege(s), then INFO [privilege type] privilege: Failed. To resolve this error, the administrator must grant the missing privilege(s) to the database user and restart the consumer.

How to Grant Missing Privileges:

Stop the Platform Analytics Consumer and the Usher Metadata Producer.
Connect to the database server that contains the Platform Analytics Repository. Execute the following command, replacing the 'someuser' and 'somehost' with the customer specified information:
Copy
```
GRANT DROP ON platform_analytics_wh.* TO ‘someuser’@‘somehost’;
```
Restart the Platform Analytics Consumer and Usher Metadata Producer.

Failed Connection Error

If the consumer or producer is unable to connect to the database using the configuration specified in the PAConsumerConfig.yaml configuration file, you may see the following error:

Copy

2018-11-21 21:43:28,793 INFO HealthCheck main - Failed to connect to the database. Retrying after waiting for 60 seconds.
2018-11-21 21:45:31,797 INFO HealthCheck main - Failed to connect to the database. Retrying after waiting for 60 seconds.
2018-11-21 21:47:34,800 ERROR HealthCheck main - Failed to connect to the database using url:jdbc:mysql://XX.Y.Z.1:3306/platform_analytics_wh?rewriteBatchedStatements=true&useLegacyDatetimeCode=false&serverTimezone=UTC. Please double check your connection parameters. 
Communications link failure

The PAConsumerConfig.yaml file is populated based on the database information provided during installation. To resolve this error, connect to the machine hosting Platform Analytics and confirm all fields under the warehouseDbConnection heading are correct in the PAConsumerConfig.yaml file.

Database User Password is Incorrect Error

The consumer or producer will be unable to connect to the database if the encrypted warehouse password is incorrect. To generate a new encrypted password and update the confirmation, see Update the Database User Password Configured to the Platform Analytics Repository .

Database User Created with SSL Enabled Error

Platform Analytics supports MySQL versions 5.6, 5.7, and 8.0. For MySQL 8.0, SSL connection is enabled by default. Currently, Platform Analytics does not support SSL for the database user connecting to MySQL. When you create the database user for the Platform Analytics Consumer or Usher Metadata Producer, specify the SSL/TLS option using the REQUIRE clause.

How to Disable SSL:

Connect to the Platform Analytics Repository and execute the following command:
Copy
```
show variables like '%ssl%';
```
If the result for 'have_ssl' is 'YES' then SSL is enabled. Create the user with mysql_native_password and REQUIRE NONE options to connect without SSL.
Copy
```
CREATE USER 'test'@'%' IDENTIFIED WITH mysql_native_password BY 'password' REQUIRE NONE;
```

Redis Health Check

The Redis Health Check determines if the consumer or producer can successfully connect to the Redis server. The check provides detailed statistics about Redis collected during startup. If all the checks are successful, the final line will read Redis server health check result is healthy.

If you see an error in your check, ensure Redis is running and that your configuration is correct in the PAConsumerConfig.yaml file.

Kafka Health Check

The Kafka Health Check ensures the Telemetry Manager (Apache Zookeeper) and the Telemetry Server (Kafka Server) are started and connected. If all the checks are successful, the final line will read Kafka cluster health check result is healthy.

Since the Telemetry Server is dependent on the Telemetry Manager, the Telemetry Manager must be started first.

If you see an error in your check, ensure ZooKeeper and Kafka are started.

Suggested Troubleshooting for Kafka Health Check Errors

How to check if ZooKeeper servers are running on all nodes:

On Linux, run the following command to run the PID.

Copy

ps ax | grep java | grep -i QuorumPeerMain | grep -v grep | awk '{print $1}'

On Windows, open Window's Services and check if the \"Apache ZooKeeper\" service is running.

How to check if Kafka servers are running on all nodes:

On Linux, run the following command to run the PID.

Copy

ps ax | grep -i 'kafka\\.Kafka' | grep java | grep -v grep | awk '{print $1}'

On Windows, open Window's Services and check if the \"Apache Kafka\" service is running.

How to start ZooKeeper and Kafka on all nodes:

On Linux, run the following commands in the Kafka directory:

Copy

# Start Zookeeper on all nodes,

./zookeeper-server-start.sh -daemon ../config/zookeeper.properties

# Start Kafka on all nodes,

./kafka-server-start.sh -daemon ../config/server.properties

On Windows, open Window's Services and start Apache ZooKeeper and Apache Kafka.

If this is a clustered environment with multiple nodes of ZooKeeper and Kafka, you must start all nodes of ZooKeeper first.

Health Check Summary

If all health checks are successful, the results will be passing. If any of the checks failed, you will get a FAIL for the corresponding component. In failed scenarios, utilize the detailed report list above to investigate possible causes of the failure.