Version 2021

Memory Mapped Files for Intelligent Cubes

Memory mapped files (MMF) can be used for effective RAM utilization. For prior releases, either a cube was loaded, residing completely in RAM, or unloaded. Now we look to leverage operating system capabilities to enable more cubes to be ‘loaded’ between RAM and disk.

As shown in the screenshot above, only one cube cache can be loaded in RAM at any given time. With MMF, multiple cube caches can be loaded in RAM.

Requirements

To use MMF, more disk space than system memory is required. The disk space is checked at server startup and logged into DSSErrors.log with a message similar to one of the following errors below. If enabled, it is also checked on subsequent publishings, with additional logging only if the amount of disk drops too low.

  • Enable MMF as the available disk size (25338925056) on /iserver-install/BIN/Linux is not less than the total physical memory size (7890911232)
  • Disable MMF as the available disk size (5338925056) on /iserver-install/BIN/Linux is less than the total physical memory size (7890911232)

The number of descriptors for Linux, using the nofiles limit, should be set to at least 65535 as mentioned in Recommended System Settings for Linux.

File and Folder Structure

When enabled, a file or set of files are created for each managed cube under a CubeGovern folder, with structure \CubeGovern\{ServerDefiniton}\Server_{machine}_P{project}\MMF\{CubeInstanceID}. This folder may contain a single file for non-partition OLAP cubes, or multiple files for partitioned cubes (based on number of partitions and attributes used) and MTDI cubes (based on number of fact tables, along with partitioning). The location of this CubeGovern folder may vary, but can be found from the DSSErrors entry referenced in the Requirements section. These files are deleted if the cube is completely unloaded, and created when the cube is published or loaded again.

Enable or Disable MMF

The usage of memory mapped files is configurable through MicroStrategy Workstation at the configuration and project levels, displayed as Environment and Application within Workstation, with the options for project level shown below. 'Use inherited value’ is not an option for the configuration level. Hovering over these options shows the full text of 'Apply best strategy to maximize performance with given resources' and 'Turn-off the capability without exceptions'.

The first option, Use inherited value, for this setting is the default and its behavior is defined as follows - cubes less than 1GB in size make use of MMF files, with certain governed objects such as element blocks are not included in these MMF files.

The other options determine the feature’s usage for all cubes in the project, regardless of size, with all governed objects into the MMF files if enabled.

Unlike the other options, if any level of this setting is configured to Turn-off the capability without exception, the lower and more granular settings are ignored (e.g., setting this at the configuration level overrides any settings made for the project, or in the future individual cube, level.

Troubleshoot

File descriptor on Linux

When using MMF files on systems with larger numbers of loaded cubes, particularly partitioned cubes, it is possible to exceed the OS-configurable limit of open files per process. This is because when using MMFs, the server process has more files opened for easier access. The following are examples the error you may encounter if this limit is met through usage of this feature against a Linux Intelligence server.

The following error is from publishing a cube in MicroStrategy Web.

(QueryEngine encounter error: MFileSystem:OpenStream: ::fopen failed for '/iserver-install/BIN/Linux/MSIReg.reg_lock'. System Error (EMFILE) --- Too many open files. Error in Process method of Component: QueryEngineServer, Project Nico's MD Population Opt for Cubes, Job 908, Error Code= -214721544.)

The same error is encountered from republishing a cube within the dossier editor in MicroStrategy Workstation.

MMFs are not generated on disk

Please check the available disk space and refer to the previous Requirements section.