Enable Incremental Fetch for a Custom Visualization

Incremental fetch divides your visualization into pages to load your data in batches to improve dashboard performance with large datasets. Incremental fetch is available for out-of-the-box visualizations and, starting in MicroStrategy ONE (December 2024), for custom visualizations as well.

A set of APIs and properties is provided in mstrmojo.CustomVisBase to facilitate incremental data fetching. Use these APIs and properties to enable incremental fetch inside custom visualizations.

• Prerequisites

• API Reference

• Property Reference

• Custom Visualization Developer Guidance

• Sample Usage

• Incrementally Fetched Data Slice

• Important Notes

Prerequisites

To ensure the APIs can be executed correctly, a custom visualization needs to meet the following requirements:

• The custom visualization must be extended from the mstrmojo.CustomVisBase class.

  See the following example that uses a custom visualization named D3WordCloud:

  Copy

  mstrmojo.plugins.D3WordCloud.D3WordCloud = mstrmojo.declare(
    mstrmojo.CustomVisBase,
    null,
    {
      scriptClass: "mstrmojo.plugins.D3WordCloud.D3WordCloud",
      // ...
    }
  );

• The Custom Visualization Editor Model must be extended from the mstrmojo.vi.models.editors.CustomVisEditorModel class.

  See the following example that uses a custom visualization named D3WordCloud:

  Copy

  <!-- plugins\D3WordCloud\WEB-INF\xml\config\visualizations.xml -->
  <visualizations>
    <visualization-list name="ajax" show-in-web="true" view-mode="51">
      <visualization 
        desc="D3WordCloud"
        editor-model="plugins.D3WordCloud.D3WordCloudEditorModel"
        <!-- ... -->
      />
    </visualization-list>
  </visualizations>

  Copy

  mstrmojo.plugins.D3WordCloud.D3WordCloudEditorModel = mstrmojo.declare(
    mstrmojo.vi.models.editors.CustomVisEditorModel,
    null,
    {
      scriptClass: "mstrmojo.plugins.D3WordCloud.D3WordCloudEditorModel",
      // ...
    }
  );

MicroStrategy recommends that you use the Custom Visualization Tool to initialize the new custom visualization. The initialized templates automatically meet these requirements.

For an existing custom visualization, manually make changes to meet the above requirements.

API Reference

fetchRows

Fetch a row set based on the specified result window parameters.

Parameters

fetchOptions: The graphic model to bind to the graphic. This object has the following properties:

memo: An echo object to pass data from fetchRows to onDataFetched. It can be any object type.

Return Value

Boolean. Indicates whether the fetching was successfully triggered.

hasNextRowSet()

Check whether there are more rows set to fetch.

Parameters

None

Return Value

Boolean. Indicates whether more rows need to be fetched.

fetchNextRowSet()

Fetch the next row set with predefined result window parameters. Users can set the result window by configuring Incremental Fetch option in the More Options editor.

Parameters

None

Return Value

Indicates whether the fetch was successfully triggered.

onDataFetched(data, memo)

A callback invoked when data gets fetched by calling the fetchNextRowset or fetchRows API.

CustomVisBase has a default implementation for this API. It merges the visualization data for a single attribute visualization and refreshes the visualization.

Usually, the Custom Visualization Developer needs to override this method to handle the fetched data.

Parameters

data

The fetched node data.

It is an object that has the same properties as visualization.gridData. It has many properties. The main related properties include ghs, gts, gvs, gsi, and rw.

memo

An echo object to pass data from fetchRows to onDataFetched.

It can be any object type.

Return Value

Indicates whether the fetch was successfully triggered.

Property Reference

supportIncrementalFetch

Indicates whether the widget supports incremental fetch or not.

Boolean.

The default is false.

The Custom Visualization Developer must set it to true to enable the incremental fetch functionality.

incFetchStatus

Read-only value. Do not change this.

Value change will be published and Custom Visualization Developer can implement value change handler - onincFetchStatusChange.

The fetch status.

Number.

incFetchError

Read-only value. Do not change it.

Value change will be published and Custom Visualization Developer can implement value change handler - onincFetchErrorChange.

The fetch error. This is useful when the fetch API returns false.

Object.

The code number description.

The initial value is null.

incFetchBlockIndex

Read-only value. Do not change it.

The index of the currently fetched result set. 1 based. It will be reset when data is changed, and the change is not triggered by incremental fetch.

Number.

The initial value is 1.

Custom Visualization Developer Guidance

The steps to enable the Incremental Fetch option on a custom visualization are provided below.

1. Set supportIncrementalFetch by adding the following parameter to the plugin Model class. For example, the D3WordCloud model class could be modified to support incremental fetch by adding the following parameter:

Copy

mstrmojo.plugins.D3WordCloud.D3WordCloudEditorModel = mstrmojo.declare(
  mstrmojo.vi.models.editors.CustomVisEditorModel,
  null,
  {
    scriptClass: "mstrmojo.plugins.D3WordCloud.D3WordCloudEditorModel",
    
    supportIncrementalFetch: true,
    // ...
   }
 );

With this change, the user can see the Incremental Fetch option in the More Options dialog box when editing the custom visualization, as shown below:



A user can enable and disable incremental fetch and adjust the number of elements to return at a time. The visualization retrieves only the specified number of elements when the dashboard is initially opened.

2. Trigger fetch APIs in an event handler, by designing a user interaction to trigger the fetch API.

   See the following code sample that triggers the fetch API when a user clicks a visualization.

   Copy

   mstrmojo.plugins.D3WordCloud.D3WordCloudEditorModel = mstrmojo.declare(
     mstrmojo.vi.models.editors.CustomVisEditorModel,
     null,
     {
       // ...
       onclick: function onclick(e) {
         this.fetchNextRowSet();
       }
       // ...
     }
   );

3. Implement the onDataLoaded API. You need to implement a proper data handler to process the incrementally fetched data by overriding the onDataLoaded API. You can implement incremental rendering in this API.

   See the following example that uses the incrementally fetched data to replace the current grid data and refresh the visualization.

   Copy

   mstrmojo.plugins.D3WordCloud.D3WordCloudEditorModel = mstrmojo.declare(
     mstrmojo.vi.models.editors.CustomVisEditorModel,
     null,
     {
       // ...
       onDataFetched: function onDataFetched(data, memo) {
           const gridData = this.getData();
           mstrmojo.hash.copy(data, gridData);
           this.refresh();
       },
       // ...
     }
   );

4. Implement the error update handler. Implement error handling by implementing the onincFetchErrorChange API.

   Below is an example of an error message on the console.

   Copy

   mstrmojo.plugins.D3WordCloud.D3WordCloudEditorModel = mstrmojo.declare(
     mstrmojo.vi.models.editors.CustomVisEditorModel,
     null,
     {
       // ...
       onincFetchErrorChange: function onincFetchErrorChange() {
           console.error(this.incFetchError && this.incFetchError.message);
       },
       // ...
     }
   );

5. Implement the status update handler. Implement status update handling by implementing the onincFetchStatusChange API.

   Below is an example of printing the fetch status on the console.

   Copy

   mstrmojo.plugins.D3WordCloud.D3WordCloudEditorModel = mstrmojo.declare(
     mstrmojo.vi.models.editors.CustomVisEditorModel,
     null,
     {
       // ...
       onincFetchStatusChange: function onincFetchStatusChange() {
           console.log('fetchStatus: ', this.incFetchStatus);
       },
       // ...
     }
   );

Sample Usage

Equivalent Incremental Fetch

Scenario

In a custom visualization:

• A user can enable incremental fetch by setting elements per page (for this scenario, the user selects 100 elements) in the More Options dialog box.

• When the dashboard is initially opened, the client side fetches and renders the first 100 elements.

• When the user clicks the Fetch button, the client side fetches and renders the next 100 elements.

• When no more data exists to fetch, the Fetch button is hidden.

Custom Visualization Developer Implementation

(function() {
    mstrmojo.plugins.EquivalentIncrementalFetchViz.EquivalentIncrementalFetchViz = mstrmojo.declare(
        mstrmojo.CustomVisBase,
        [],
        {
            // ...
            supportIncrementalFetch: true,
            // Click to fetch next row set
            onFetchButtonClick: function onFetchButtonClick() {
                this.fetchNextRowSet();
            },
            // Handle incremental fetch error
            onincFetchErrorChange: function onincFetchErrorChange() {
                console.error(this.incFetchError && this.incFetchError.message);
            },
            // Track incremental fetch status and update fetch button visibility
            onincFetchStatusChange: function onincFetchStatusChange() {
                // hide the fetch button if there is no more data to fetch
                if (this.incFetchStatus === 2 && !this.hasNextRowSet()) {
                    this.fetchButton.set('visible', false);
                }
            },
            // Handle incremental fetched data - use the data to update the visualization
            onDataFetched: function onDataFetched(data, memo) {
                const gridData = this.getData();
                mstrmojo.hash.copy(data, gridData);
                this.refresh();
            }
            // ...
        }()
    );
});

Freeform Incremental Fetch

Scenario

In a custom visualization:

• A user can enable incremental fetch by setting elements per page (for this scenario, the user selects 100 elements) in the More Options dialog box.

• When the dashboard is initially opened, the client side fetches and renders the first 100 elements.

• When the user maximizes the visualization, the client side fetches and renders all the elements.

Custom Visualization Developer Implementation

(function() {
    mstrmojo.plugins.FreeformIncrementalFetchViz.FreeformIncrementalFetchViz = mstrmojo.declare(
        mstrmojo.CustomVisBase,
        [],
        {
            // ...
            supportIncrementalFetch: true,
            hasFetchedAllData: false,
            // Click maximize to fetch all data
            onMaxize: function onMaxize() {
                // avoid fetching duplicated data
                if (!this.hasFetchedAllData) {
                    this.fetchRows({
                        start: 1,
                        count: -1
                    });
                }
            },
            // Handle incremental fetch error
            onincFetchErrorChange: function onincFetchErrorChange() {
                console.error(this.incFetchError && this.incFetchError.message);
            },
            // Track incremental fetch status and update fetch button visibility
            onincFetchStatusChange: function onincFetchStatusChange() {
                // 0: not started, 1: in progress, 2: completed
                if (this.incFetchStatus === 2) {
                    this.hasFetchedAllData = true;
                }
            },
            // Handle incremental fetched data - use the data to update the visualization
            onDataFetched: function onDataFetched(data, memo) {
                const gridData = this.getData();
                mstrmojo.hash.copy(data, gridData);
                this.refresh();
            }
            // ...
        }()
    );
});

Incrementally Fetched Data Slice

Data Structure

The incrementally fetched data is passed to the custom visualization through the input parameter data. It has the same properties as visualization.gridData below.

When data is fetched incrementally, the sliced parts are gts, ghs, gvs, and the slice info is in rw.

Important Notes

• MicroStrategy does not recommend that you use fetchNextRowSet and fetchRows in combination. You should choose one string for a specific custom visualization otherwise, incremental fetched data might be disrupted.

• Do not change the dashboard instance and fetch data concurrently because an error can occur on the dashboard instance.

• When you switch to another custom visualization, the incremental fetch setting is disabled by default.

• If a user updates to a new version of a custom visualization that supports incremental fetch and configures it accordingly, but then opens the dashboard with an older version of the custom visualization, they initially see partial data. The old version lacks the logic for triggering incremental fetch and the user cannot retrieve any additional data.

• iOS is not supported.

• Documents are not supported.