MicroStrategy ONE

MicroStrategy Messaging Services

Messaging Services is a component that is coupled with the Intelligence Server during installations and upgrades. Messaging Services is configured out-of-the-box and runs automatically after the installation is completed.

After installation, you can see the following services are automatically started:

  • Apache Kafka (C:\Program Files (x86)\MicroStrategy\Messaging Services\Kafka\kafka_2.11-0.10.1.0)
  • Apache ZooKeeper (C:\Program Files (x86)\MicroStrategy\Messaging Services\Kafka\kafka_2.11-0.10.1.0)

By default MicroStrategy will still send Intelligence Server diagnostic logs to local disk. Diagnostic logs will be sent to the Messaging Services Server after you perform the following:

Afterwards you will see Kafka log files created in the Kafka installation folder:

C:\Program Files (x86)\MicroStrategy\Messaging Services\tmp\kafka-logs

Different Kafka topics will be created to store data for different MicroStrategy components.

Configuring Messaging Services after upgrading

By default, MicroStrategy Messaging Services are installed along with the Intelligence server upgrade.

Once you have completed the upgrade process, you need to enable MicroStrategy Messaging Services. If not, the Intelligence Server continues to write to the original log.

Messaging Services Workflow for Intelligence Server

  • Intelligence Server is the Kafka Producer and can be deployed a single node or cluster.
  • Kafka Server can be deployed as a single node or cluster.

Enable MicroStrategy Messaging Services

Messaging Services configuration is saved in the MicroStrategy Intelligence Server configuration. It can be enabled or disabled on the fly, without restarting your Intelligence Server.

Command Manager Scripts for Messaging Services

To check if Messaging Services is enabled, execute:

Copy
LIST ALL PROPERTIES FOR SERVER CONFIGURATION;

To enable Messaging Services through Command Manager, execute:

Copy
ALTER SERVER CONFIGURATION ENABLEMESSAGINGSERVICES TRUE CONFIGUREMESSAGINGSERVICES "bootstrap.servers:10.15.208.236:9092/batch.num.messages:5000/queue.buffering.max.ms:2000";

In the example above set:

  • bootstrap.servers: to your Kafka Server IP address and port number.
  • batch.num.messages: to the number of messages to send in one batch when using asynchronous mode.
  • queue.buffering.max.ms: to the maximum time to buffer data when using asynchronous mode.

You can specify more Kafka Producer configuration settings in this command following the same format.

Turn On the Sending Log to Messaging Services Feature

You can turn on the Sending Log to Messaging Services feature using either MicroStrategy Web or Command Manager.

From MicroStrategy Web

  1. Log in using and Administrator account.
  2. Open User Preferences > Project Defaults.
  3. Locate Sending Log to Messaging Services in the Features for Customer Feedback section.
  4. Select On from the drop-down menu.
  5. Click Apply.

From Command Manager

  1. Connect to your project source.
  2. Execute the following:

    Copy
    ALTER FEATURE FLAG "SENDING LOG TO MESSAGING SERVICES" ON;

Modifying Messaging Services Configuration

Apache Kafka Server

The Kafka Server can be configured by modifying the server.properties file found in:

C:\Program Files (x86)\MicroStrategy\Messaging Services\Kafka\kafka_2.11-0.10.1.0\config

Both Apache Kafka Server and ZooKeeper should be restarted after modifying the above configuration file.

MicroStrategy Messaging Services Configuration for Clustered Environments

If you have clustered your Intelligence Servers and want to use a separate machine to run MicroStrategy Messaging Services after upgrading, complete the following steps for each node in the cluster.

The minimum number of nodes for a cluster is 3.

Each node must have the following installed:

  • MicroStrategy Messaging Services
  • Apache Kafka
  • Apache Zookeeper

Configure Zookeeper

  1. Browse to folder C:\Program Files (x86)\MicroStrategy\Messaging Services\Kafka\kafka_2.11-0.10.1.0\config.
  2. Edit file zookeeper.properties by adding following lines:

    Copy
    clientPort=2181
    dataDir=C:\\Program Files (x86)\\MicroStrategy\\Messaging Services\\tmp\\zookeeper
    maxClientCnxns=0
    initLimit=5
    syncLimit=2
    server.1=10.27.20.16:2888:3888
    server.2=10.27.20.60:2888:3888
    server.3=10.15.208.236:2888:3888

    Each server parameter must contain a unique integer identifier as shown above. You attribute the server id to each machine by creating a text file named myid, one for each server, which resides in that server's data directory, as specified by the configuration file parameter dataDir = C:\Program Files (x86)\MicroStrategy\Messaging Services\tmp\zookeeper

  3. Go to folder C:\Program Files (x86)\MicroStrategy\MessagingServices\Kafka\kafka_2.11-0.9.0.1\config\zookeeper.

  4. Create a text file named myid containing the identifying value from the server parameter name in the zookeeper.properties file.

Configure Kafka

  1. Browse to folder C:\Program Files (x86)\MicroStrategy\Messaging Services\Kafka\kafka_2.11-0.10.1.0\config.
  2. Edit file server.properties, add a row zookeeper.connect=10.27.20.16:2181,10.27.20.60:2181,10.15.208.236:2181 to the Zookeeper section.

    Copy
    ############################# Zookeeper #############################
    # Zookeeper connection string (see zookeeper docs for details).
    # This is a comma separated host:port pairs, each corresponding to a zk
    # server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
    # You can also append an optional chroot string to the urls to specify the
    # root directory for all kafka znodes.
    # zookeeper.connect=localhost:2181
    zookeeper.connect=10.27.20.16:2181,10.27.20.60:2181,10.15.208.236:2181
  3. Modify the broker.id value to a unique integer from other Kafka servers (the default value is 0), such as for node 10.27.20.60 we use number 2.

    Copy
    ############################# Server Basics #############################
    # The id of the broker. This must be set to a unique integer for each broker.
    broker.id=2

Start, Stop, Restart, and Check Status of Messaging Services

On Windows installations, open Task Manager > Services to start, stop, restart, and check the status of Messaging Services components.

Messaging Services is a component that is coupled with the Intelligence Server during installations and upgrades. Messaging Services is configured out-of-the-box and runs automatically after the installation is completed.

After installation, you can see the following services are automatically started:

  • Apache Kafka (/opt/mstr/MicroStrategy/install/MessagingServices/Kafka/kafka_2.11-0.10.1.0)
  • Apache ZooKeeper (/opt/mstr/MicroStrategy/install/MessagingServices/Kafka/kafka_2.11-0.10.1.0)

By default MicroStrategy will still send Intelligence Server diagnostic logs to local disk. Diagnostic logs will be sent to the Messaging Services Server after you perform the following:

Afterwards you will see Kafka log files created in the Kafka installation folder:

/opt/mstr/MicroStrategy/install/MessagingServices/Kafka/tmp/kafka-logs

Different Kafka topics will be created to store data for different MicroStrategy components.

Configuring Messaging Services after upgrading

By default, MicroStrategy Messaging Services are installed along with the Intelligence server upgrade.

Once you have completed the upgrade process, you need to enable MicroStrategy Messaging Services. If not, the Intelligence Server continues to write to the original log.

Messaging Services Workflow for Intelligence Server

  • Intelligence Server is the Kafka Producer and can be deployed a single node or cluster.
  • Kafka Server can be deployed as a single node or cluster.

Enable MicroStrategy Messaging Services

Messaging Services configuration is saved in the MicroStrategy Intelligence Server configuration. It can be enabled or disabled on the fly, without restarting your Intelligence Server.

Command Manager Scripts for Messaging Services

To check if Messaging Services is enabled, execute:

Copy
LIST ALL PROPERTIES FOR SERVER CONFIGURATION;

To enable Messaging Services through Command Manager, execute:

Copy
ALTER SERVER CONFIGURATION ENABLEMESSAGINGSERVICES TRUE CONFIGUREMESSAGINGSERVICES "bootstrap.servers:10.15.208.236:9092/batch.num.messages:5000/queue.buffering.max.ms:2000";

In the example above set:

  • bootstrap.servers: to your Kafka Server IP address and port number.
  • batch.num.messages: to the number of messages to send in one batch when using asynchronous mode.
  • queue.buffering.max.ms: to the maximum time to buffer data when using asynchronous mode.

You can specify more Kafka Producer configuration settings in this command following the same format.

Turn On the Sending Log to Messaging Services Feature

You can turn on the Sending Log to Messaging Services feature using either MicroStrategy Web or Command Manager.

From MicroStrategy Web

  1. Log in using and Administrator account.
  2. Open User Preferences > Project Defaults.
  3. Locate Sending Log to Messaging Services in the Features for Customer Feedback section.
  4. Select On from the drop-down menu.
  5. Click Apply.

From Command Manager

  1. Connect to your project source.
  2. Execute the following:

    Copy
    ALTER FEATURE FLAG "SENDING LOG TO MESSAGING SERVICES" ON;

Modifying Messaging Services Configuration

Apache Kafka Server

The Kafka Server can be configured by modifying the server.properties file found in:

/opt/mstr/MicroStrategy/install/MessagingServices/Kafka/kafka_2.11-0.10.1.0

Both Apache Kafka Server and ZooKeeper should be restarted after modifying the above configuration file.

MicroStrategy Messaging Services Configuration for Clustered Environments

If you have clustered your Intelligence Servers and want to use a separate machine to run MicroStrategy Messaging Services after upgrading, complete the following steps for each node in the cluster.

The minimum number of nodes for a cluster is 3.

Each node must have the following installed:

  • MicroStrategy Messaging Services
  • Apache Kafka
  • Apache Zookeeper

Configure Zookeeper

  1. Browse to /opt/mstr/MicroStrategy/install/MicroStrategy/MessagingServices/Kafka/kafka_2.11-0.9.0.1/config.
  2. Edit zookeeper.properties by adding the following lines:

    Copy
    maxClientCnxns=0
    initLimit=5
    syncLimit=2
    server.1=10.27.20.16:2888:3888
    server.2=10.27.20.60:2888:3888
    server.3=10.15.208.236:2888:3888

    Each server parameter must contain a unique integer identifier as shown above.

  3. Go to /opt/mstr/MicroStrategy/install/MicroStrategy/MessagingServices/Kafka/kafka_2.11-0.9.0.1/tmp/zookeeper.

  4. Create a file named myid containing the identifying value from the server parameter name in the zookeeper.properties file.

Configure Kafka

  1. Browse to /opt/mstr/MicroStrategy/install/MicroStrategy/MessagingServices/Kafka/kafka_2.11-0.9.0.1/config.
  2. Edit server.properties, adding zookeeper.connect=10.27.20.16:2181,10.27.20.60:2181,10.15.208.236:2181 to the Zookeeper section.

    Copy
    ############################# Zookeeper #############################
    # Zookeeper connection string (see zookeeper docs for details).
    # This is a comma separated host:port pairs, each corresponding to a zk
    # server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
    # You can also append an optional chroot string to the urls to specify the
    # root directory for all kafka znodes.
    # zookeeper.connect=localhost:2181
    zookeeper.connect=10.27.20.16:2181,10.27.20.60:2181,10.15.208.236:2181
  3. Modify the broker.id value to a unique integer from other Kafka servers (the default value is 0), such as for node 10.27.20.60 we use number 2.

    Copy
    ############################# Server Basics #############################
    # The id of the broker. This must be set to a unique integer for each broker.
    broker.id=2

Start, Stop, Restart, and Check Status of Messaging Services

Kafka Server and Zookeeper have been registered as service on Linux, so we can use service command to start, stop, and check status. The restart command is not supported.

To execute a service command for Kafka Server and Zookeeper, enter:

Copy
/etc/init.d/kafka-zookeeper {start|stop|status}