MicroStrategy ONE
MicroStrategy Messaging Services
- Windows
- Linux
Messaging Services is a component that is coupled with the Intelligence Server during installations and upgrades. Messaging Services is configured out-of-the-box and runs automatically after the installation is completed.
After installation, you can see the following services are automatically started:
- Apache Kafka (
C:\Program Files (x86)\MicroStrategy\Messaging Services\Kafka\kafka_2.11-0.10.1.0
) - Apache ZooKeeper (
C:\Program Files (x86)\MicroStrategy\Messaging Services\Kafka\kafka_2.11-0.10.1.0
)
By default MicroStrategy will still send Intelligence Server diagnostic logs to local disk. Diagnostic logs will be sent to the Messaging Services Server after you perform the following:
Afterwards you will see Kafka log files created in the Kafka installation folder:
C:\Program Files (x86)\MicroStrategy\Messaging Services\tmp\kafka-logs
Different Kafka topics will be created to store data for different MicroStrategy components.
Configuring Messaging Services after upgrading
By default, MicroStrategy Messaging Services are installed along with the Intelligence server upgrade.
Once you have completed the upgrade process, you need to enable MicroStrategy Messaging Services. If not, the Intelligence Server continues to write to the original log.
Messaging Services Workflow for Intelligence Server
- Intelligence Server is the Kafka Producer and can be deployed a single node or cluster.
- Kafka Server can be deployed as a single node or cluster.
Enable MicroStrategy Messaging Services
Messaging Services configuration is saved in the MicroStrategy Intelligence Server configuration. It can be enabled or disabled on the fly, without restarting your Intelligence Server.
Command Manager Scripts for Messaging Services
To check if Messaging Services is enabled, execute:
LIST ALL PROPERTIES FOR SERVER CONFIGURATION;
To enable Messaging Services through Command Manager, execute:
ALTER SERVER CONFIGURATION ENABLEMESSAGINGSERVICES TRUE CONFIGUREMESSAGINGSERVICES "bootstrap.servers:10.15.208.236:9092/batch.num.messages:5000/queue.buffering.max.ms:2000";
In the example above set:
bootstrap.servers
: to your Kafka Server IP address and port number.batch.num.messages
: to the number of messages to send in one batch when using asynchronous mode.queue.buffering.max.ms
: to the maximum time to buffer data when using asynchronous mode.
You can specify more Kafka Producer configuration settings in this command following the same format.
Turn On the Sending Log to Messaging Services Feature
You can turn on the Sending Log to Messaging Services feature using either MicroStrategy Web or Command Manager.
From MicroStrategy Web
- Log in using and Administrator account.
- Open User Preferences > Project Defaults.
- Locate Sending Log to Messaging Services in the Features for Customer Feedback section.
- Select On from the drop-down menu.
- Click Apply.
From Command Manager
- Connect to your project source.
-
Execute the following:
CopyALTER FEATURE FLAG "SENDING LOG TO MESSAGING SERVICES" ON;
Modifying Messaging Services Configuration
Apache Kafka Server
The Kafka Server can be configured by modifying the server.properties
file found in:
C:\Program Files (x86)\MicroStrategy\Messaging Services\Kafka\kafka_2.11-0.10.1.0\config
Both Apache Kafka Server and ZooKeeper should be restarted after modifying the above configuration file.
MicroStrategy Messaging Services Configuration for Clustered Environments
If you have clustered your Intelligence Servers and want to use a separate machine to run MicroStrategy Messaging Services after upgrading, complete the following steps for each node in the cluster.
The minimum number of nodes for a cluster is 3.
Each node must have the following installed:
- MicroStrategy Messaging Services
- Apache Kafka
- Apache Zookeeper
Configure Zookeeper
- Browse to folder
C:\Program Files (x86)\MicroStrategy\Messaging Services\Kafka\kafka_2.11-0.10.1.0\config
. -
Edit file
zookeeper.properties
by adding following lines:CopyclientPort=2181
dataDir=C:\\Program Files (x86)\\MicroStrategy\\Messaging Services\\tmp\\zookeeper
maxClientCnxns=0
initLimit=5
syncLimit=2
server.1=10.27.20.16:2888:3888
server.2=10.27.20.60:2888:3888
server.3=10.15.208.236:2888:3888Each server parameter must contain a unique integer identifier as shown above. You attribute the server id to each machine by creating a text file named
myid
, one for each server, which resides in that server's data directory, as specified by the configuration file parameterdataDir = C:\Program Files (x86)\MicroStrategy\Messaging Services\tmp\zookeeper
-
Go to folder
C:\Program Files (x86)\MicroStrategy\MessagingServices\Kafka\kafka_2.11-0.9.0.1\config\zookeeper
. -
Create a text file named
myid
containing the identifying value from the server parameter name in thezookeeper.properties
file.
Configure Kafka
- Browse to folder
C:\Program Files (x86)\MicroStrategy\Messaging Services\Kafka\kafka_2.11-0.10.1.0\config
. -
Edit file
server.properties
, add a rowzookeeper.connect=10.27.20.16:2181,10.27.20.60:2181,10.15.208.236:2181
to the Zookeeper section.Copy############################# Zookeeper #############################
# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
# zookeeper.connect=localhost:2181
zookeeper.connect=10.27.20.16:2181,10.27.20.60:2181,10.15.208.236:2181 -
Modify the
broker.id
value to a unique integer from other Kafka servers (the default value is 0), such as for node 10.27.20.60 we use number 2.Copy############################# Server Basics #############################
# The id of the broker. This must be set to a unique integer for each broker.
broker.id=2
Start, Stop, Restart, and Check Status of Messaging Services
On Windows installations, open Task Manager > Services to start, stop, restart, and check the status of Messaging Services components.
Messaging Services is a component that is coupled with the Intelligence Server during installations and upgrades. Messaging Services is configured out-of-the-box and runs automatically after the installation is completed.
After installation, you can see the following services are automatically started:
- Apache Kafka (
/opt/mstr/MicroStrategy/install/MessagingServices/Kafka/kafka_2.11-0.10.1.0
) - Apache ZooKeeper (
/opt/mstr/MicroStrategy/install/MessagingServices/Kafka/kafka_2.11-0.10.1.0
)
By default MicroStrategy will still send Intelligence Server diagnostic logs to local disk. Diagnostic logs will be sent to the Messaging Services Server after you perform the following:
Afterwards you will see Kafka log files created in the Kafka installation folder:
/opt/mstr/MicroStrategy/install/MessagingServices/Kafka/tmp/kafka-logs
Different Kafka topics will be created to store data for different MicroStrategy components.
Configuring Messaging Services after upgrading
By default, MicroStrategy Messaging Services are installed along with the Intelligence server upgrade.
Once you have completed the upgrade process, you need to enable MicroStrategy Messaging Services. If not, the Intelligence Server continues to write to the original log.
Messaging Services Workflow for Intelligence Server
- Intelligence Server is the Kafka Producer and can be deployed a single node or cluster.
- Kafka Server can be deployed as a single node or cluster.
Enable MicroStrategy Messaging Services
Messaging Services configuration is saved in the MicroStrategy Intelligence Server configuration. It can be enabled or disabled on the fly, without restarting your Intelligence Server.
Command Manager Scripts for Messaging Services
To check if Messaging Services is enabled, execute:
LIST ALL PROPERTIES FOR SERVER CONFIGURATION;
To enable Messaging Services through Command Manager, execute:
ALTER SERVER CONFIGURATION ENABLEMESSAGINGSERVICES TRUE CONFIGUREMESSAGINGSERVICES "bootstrap.servers:10.15.208.236:9092/batch.num.messages:5000/queue.buffering.max.ms:2000";
In the example above set:
bootstrap.servers
: to your Kafka Server IP address and port number.batch.num.messages
: to the number of messages to send in one batch when using asynchronous mode.queue.buffering.max.ms
: to the maximum time to buffer data when using asynchronous mode.
You can specify more Kafka Producer configuration settings in this command following the same format.
Turn On the Sending Log to Messaging Services Feature
You can turn on the Sending Log to Messaging Services feature using either MicroStrategy Web or Command Manager.
From MicroStrategy Web
- Log in using and Administrator account.
- Open User Preferences > Project Defaults.
- Locate Sending Log to Messaging Services in the Features for Customer Feedback section.
- Select On from the drop-down menu.
- Click Apply.
From Command Manager
- Connect to your project source.
-
Execute the following:
CopyALTER FEATURE FLAG "SENDING LOG TO MESSAGING SERVICES" ON;
Modifying Messaging Services Configuration
Apache Kafka Server
The Kafka Server can be configured by modifying the server.properties
file found in:
/opt/mstr/MicroStrategy/install/MessagingServices/Kafka/kafka_2.11-0.10.1.0
Both Apache Kafka Server and ZooKeeper should be restarted after modifying the above configuration file.
MicroStrategy Messaging Services Configuration for Clustered Environments
If you have clustered your Intelligence Servers and want to use a separate machine to run MicroStrategy Messaging Services after upgrading, complete the following steps for each node in the cluster.
The minimum number of nodes for a cluster is 3.
Each node must have the following installed:
- MicroStrategy Messaging Services
- Apache Kafka
- Apache Zookeeper
Configure Zookeeper
- Browse to
/opt/mstr/MicroStrategy/install/MicroStrategy/MessagingServices/Kafka/kafka_2.11-0.9.0.1/config
. -
Edit
zookeeper.properties
by adding the following lines:CopymaxClientCnxns=0
initLimit=5
syncLimit=2
server.1=10.27.20.16:2888:3888
server.2=10.27.20.60:2888:3888
server.3=10.15.208.236:2888:3888Each server parameter must contain a unique integer identifier as shown above.
-
Go to
/opt/mstr/MicroStrategy/install/MicroStrategy/MessagingServices/Kafka/kafka_2.11-0.9.0.1/tmp/zookeeper
. -
Create a file named
myid
containing the identifying value from the server parameter name in thezookeeper.properties
file.
Configure Kafka
- Browse to
/opt/mstr/MicroStrategy/install/MicroStrategy/MessagingServices/Kafka/kafka_2.11-0.9.0.1/config
. -
Edit
server.properties
, addingzookeeper.connect=10.27.20.16:2181,10.27.20.60:2181,10.15.208.236:2181
to the Zookeeper section.Copy############################# Zookeeper #############################
# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
# zookeeper.connect=localhost:2181
zookeeper.connect=10.27.20.16:2181,10.27.20.60:2181,10.15.208.236:2181 -
Modify the
broker.id
value to a unique integer from other Kafka servers (the default value is 0), such as for node 10.27.20.60 we use number 2.Copy############################# Server Basics #############################
# The id of the broker. This must be set to a unique integer for each broker.
broker.id=2
Start, Stop, Restart, and Check Status of Messaging Services
Kafka Server and Zookeeper have been registered as service on Linux, so we can use service command to start, stop, and check status. The restart command is not supported.
To execute a service command for Kafka Server and Zookeeper, enter:
/etc/init.d/kafka-zookeeper {start|stop|status}