MicroStrategy ONE

Connect to Amazon S3

Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. MicroStrategy Cloud Object Connector provides access to Amazon S3 to quickly browse folders and files and import them into MicroStrategy cubes.

Explore the following topics on this page:

  1. Prepare Connection Parameters
  2. Create a DBRole
  3. Import Data
  4. Limitations

Prepare Connection Parameters

For Cloud Object Connector to successfully browse the Amazon S3 file system, you need an Amazon Web Service account created with proper permissions. IAM entities, such as users, groups, and roles, start without permissions and must be granted them. For more details on granting permissions to IAM entities, please refer to Manage IAM permissions.

MicroStrategy recommends that you create an IAM user for Cloud Object Connector. For an IAM user to access a bucket and the objects within it, the Allow Effect permission should at least be granted to the following two actions:

  • s3:GetObject
  • s3:ListBucket

  • s3:ListAllMyBuckets

After the IAM user is successfully created, please ask an IAM administrator for the access key ID and secret access key.

Create a DBRole

Access the Amazon S3 Cloud Object Connector on MicroStrategy Web or Workstation.

  • Web
  • Workstation
  1. Choose Add Data > New Data.

  2. Find and select the Amazon S3 Cloud Object connector from the data source list.

  3. Next to Data Sources, click New Data Source to add a new connection.

  4. Enter your connection credentials.

    • Connection Name: A name for the new connection.
    • AWS Region: The exact AWS region that your bucket is located.
    • IAM User's AccessKey: The access key ID.
    • IAM User's AccessSecret: The secret access key.

  1. In the left panel, next to Datasets, click Create a new dataset .

  2. Find and select the Amazon S3 Cloud Object connector from the data source list.

  3. Next to Data Sources, click New Data Source to add a new connection.

  4. Enter your connection credentials.

    • Connection Name: A name for the new connection
    • AWS Region: The exact AWS region that your bucket is located
    • IAM User's AccessKey: The access key ID
    • IAM User's AccessSecret: The secret access key

Import Data

Once you've successfully created the connector, you can import data into MicroStrategy.

  1. Select the newly created connection.
  2. In the drop-down, select the bucket and browse the folders or files.

  3. Double-click files or drag them into the right pane.

    In the Preview pane, you can see the sample data and adjust the column type.

  4. Publish the cube to MicroStrategy with your selected data.

Limitations

Supported File Types

Only the following file types are supported:

  • .json
  • .parquet
  • .avro
  • .orc
  • .csv
  • Delta format

Select Folders

When selecting the entire folder, the folder must meet the following requirements:

  • All files under the folder need to have the same file types. A dialog will prompt you to choose the file type
  • All files share the same schema

  • If the folder has sub-folders, the sub-folders should be in a valid partitioned format. The following is an example of a valid folder structure

Spark Limitations

  • Only JSON files with each newline as a complete JSON can be read
  • Parquet files that contain special characters (, ; { } \ = " .) cannot be read
  • Parquet files with column data types as INT32(UINT_8)/(UNIT_16)/(UNIT_32)/(TIME_MILLIS) are not supported
  • Columns with the binary type cannot be published in cube
  • ORC files with field names prefixed with "_col" (e.g., _col0, _col1), where the file schema contains at least one nested structure, array, or map field, cannot be imported

Features

The following features are not supported:

  • Uploading MicroStrategy files connecting to Cloud Object Connector
  • Data wrangling in Data Import
  • Defining geography in Data Import
  • Advanced scheduling for Schedule Cube Publish
  • Group tables in Data Import