MicroStrategy ONE
Starting in MicroStrategy 2021 Update 4, Hadoop Gateway is no longer supported.
Introduction to the MicroStrategy Hadoop Gateway
Assumptions
- This document is only eligible for MicroStrategy MicroStrategy Hadoop Gateway 10.11 and later versions. Previous versions are not fully covered.
- The Intelligence Server and MicroStrategy Hadoop Gateway version numbers must be identical. Otherwise, data files may not be imported successfully.
- See the Certification Dashboard for a list of certified gateways.
-
Starting with MicroStrategy 10.11, MicroStrategy Hadoop Gateway will be launched with spark-submit.
The application detects Hadoop and Java dependencies and resolves them automatically. MicroStrategy also provides automatic deployment via MicroStrategy Hadoop Gateway Manager. Refer to MicroStrategy Community for more details.
Introduction
MicroStrategy Hadoop Gateway is a new data processing engine, introduced in MicroStrategy 10.6, that can be installed in a Spark environment. This native connector allows analysis of data in Hadoop and provides high-speed parallel data transfer between the Hadoop Distributed File System (HDFS) and the MicroStrategy Intelligence Server.
MicroStrategy Hadoop Gateway is a native connector which was built based on Spark 1.6.x. You can choose one or more data files from Hadoop HDFS and load them into MicroStrategy Intelligence Server. Data files can be published as either an In-Memory Cube or a Live Connect Cube.
MicroStrategy Hadoop Gateway supports three operation modes:
- YARN client
- Standalone (HDP cluster does not support Standalone)
- Local
In YARN client mode, the driver runs in the client process, and the application master is only used for requesting resources from YARN. Running in this mode requires YARN service to be enabled on your Hadoop cluster.
In Standalone mode, MicroStrategy Hadoop Gateway uses Spark Master to coordinate data processing.
In Local mode, no application is deployed in the Spark nodes. All data processing is executed in the MicroStrategy Hadoop Gateway host. This modality is intended for testing and troubleshooting.
Related Topics
How to Deploy the MicroStrategy Hadoop Gateway
How to Start the MicroStrategy Hadoop Gateway