Strategy ONE

Understand Auto Answers and Best Practices

Understanding the Response

Auto Answers provides responses comprised of natural language text and when valuable, accompanying visualizations. Auto answers supports a range of visualization types, including grids, vertical bar charts, pie charts (ring charts are not supported), line charts, KPIs, heat maps, geospatial service (mapbox), key driver analysis, and insight line charts for trend and forecasting analysis. If you ask a question that requires specific visualizations, ensure that the chosen visualization type is one of the above supported visualization types.

If the Auto Answers response includes many data points, the visualization will display a condensed version with limited data to focus on clarity and readability. This condensed display often applies to bar charts, key drivers, and time series visualizations. You can view the complete set of data points when you maximize the visualization.

When asking questions in Auto Answers, you might notice variations in the narrative or visualization types for the same query, while the numerical data remains accurate. For example, a "Which stores had consistent improvements in revenue?" query may result in slightly different summaries during different sessions but the list of stores are consistent. This variation is due to the non-determinstic nature of the Large Language Model (LLM), which aims to provide the most relevant and accurate response each time you ask a question, even if it has to alter the approach.

Understanding the Question Limits

You can ask a question that is up to 1000 characters at a time.

Auto Answers can answer questions about metrics placed on the current page with panels that are currently selected and all available attributes in a dataset.

  • However, due to the size of the data or the complexity of the dataset (for example, the number of datasets, the number of attributes and metrics, and the object name length), the AI service may reach the computational limit, which cannot be increased. In this case, for every subsequent question, Auto Answers will make the following suggestions in order of priority:
    • Auto Answers will suggest metrics placed in the current page and selected panels, and attributes that are placed in visualizations used in the dashboard.
    • Auto Answers will suggest metrics and attributes placed on the current page with panels that are currently selected.
  • With this logic, while restrictive, you can minimize errors in responses due to computational limitations and maximize your experience.
  • If you clear the chat history, the logic sequence is reset to its default size.

Answering Compound, Multi-faceted Questions

You can receive effective responses to compound, multi-faceted questions that span across multiple datasets in a dashboard. For example, you can ask "In each of the top 3 regions by revenue, which store had the least profit?". This update allows deeper, contextually rich insights drawn from multiple data points within the same question. In the previous example, Auto Answers computes the three regions with the highest revenue growth, then identifies the individual store with the lowest profit in each of the regions. Auto Answers then combines the responses from previous steps to display the final response.

Consider the question "Which 2 categories had the highest Profit growth between Q1 and Q2 2019?". This question unfolds in multiple layers. Initially, Auto Answers pinpoints all categories' profit for both quarters then it calculates the growth rates, and finally, ranks them to identify the top two categories. This methodology is better understood for every question when you expand the Interpretation window, where the multi-step execution is explained.

For more information about Interpretation, see Leverage AI Interpretation.

To expand on Auto Answers's capabilities to answer more complex questions, the following example demonstrates the ability to execute rankings and percentile calculations within groups. If you ask "How do products rank within each category based on their sales volume in 2019 and what percentile does each product fall into?". Auto Answers ranks products within each category by sales volume, then it determines each product's percentile rank, and offers a comprehensive overview of performance within the categories.

This capability empowers business users and analysts to extract multi-dimensional insights with a single question and look deeper into their data to uncover trends, anomalies, and relationships across various datasets. It enhances decision-making efficiency and provides contextually relevant insights in a conversational analytics setting, without extensive data manipulation.

Best Practices

For Authors

  • Do not include an excessive number of objects and long names in datasets.

  • Ensure each dataset, attribute, and metric has a unique name to avoid confusing the AI analysis and for clarity when users select an auto-complete option.

  • Avoid including object names such as "This week" or "Last week". If this name is essential, ensure that you use auto-complete when posing questions to Auto Answers. This helps Auto Answers determine the object name rather than using the current week timeframe.

  • Custom Groups, Consolidations, and Derived Elements are not supported.

  • Avoid using special characters in datasets such as '%', '*', ':', etc. Similarly, accommodate your dataset metric convention in the same manner.

  • Avoid using Time-series attributes in DateTime form for a Time related attribute.

    • You can convert the column separate Date and Time attributes.

  • Avoid multiple Attribute objects from datasets that represent the same business entity. For example, avoid a dataset with an attribute named Week with a MM/DD/YYYY format and a integer datatype named "Week".

  • Compound grid authors may apply identical aliases to multiple objects. However, if the dashboard is intended to be accompanied by Auto Answers, refrain from applying identical aliases and ensure that each object has a unique alias.

For Consumers

  • Ask questions related to the information in the dashboard or dataset. Auto Answers is designed as an AI Assistant to analyze and offer insight specifically from the data in your dashboard. Auto Answers is not equipped to answer questions outside your dataset or dashboard.

  • For accurate and complete responses, ask only one question at a time. Avoid combining multiple questions into one. For example, "What is the total COST for each Quarter? Which Quarter has the highest PROFIT?".

  • Avoid asking repetitive questions.

  • Auto Answers interprets questions with higher accuracy when you list Semantic Graph objects. User-typed questions are enabled with auto-complete suggestions so you can select Strategy objects before you submit a question.

  • When your data spans multiple years, be specific when you ask a question to ensure accurate responses. For example, instead of "What was the total revenue in Q2?", use "What was the total revenue in Q2 2023?".

  • Ensure you use precise element names when constructing your questions. For example, if an element named "Product SKU: ABC123" is in your dataset, you should use that exact element name instead of a variation such as, "Product ABC123" or "SKU:ABC-123". This practice ensures that Auto Answers accurately identifies the data you are referencing and results in more precise and dependable responses.

Common Issues

See the following issues that you can encounter and the best practices to avoid them.

The question's context is too complex to be processed

When you submit a question to Auto Answers, there is a possibility that the data input limit will be reached before returning an answer. If that happens, you receive the following response: The dashboard definition is too large to use Auto, please refine the datasets and dashboard before trying again.

Multiple factors can impact the weight of the question:

  • The number of objects in the dashboard, dashboard page, or datasets.

  • The name of a Semantic Graph. The number of words in an object name impacts the complexity of the question.

Best Practice

  • Authors should review the functionality and performance of Auto Answers prior to release and consider the common business cases and typical questions a user will ask.

  • Authors should simplify their datasets for Auto Answers by reviewing, reducing complexity, and content density. Authors should adopt a clear naming convention and create a lighter version of the dataset to enhance usability with Auto Answers for users.

  • Users can limit the scope of questions to what is displayed on the page.

Multiple Questions

If you ask Auto Answers multiple questions in one questions, the response may only address one question, answer both questions partially, or misinterpret the question.

For example, if you ask "Which Store had the highest Revenue and which product category had the lowest profit margin?", Auto Answers might accurately identify the top revenue store but may not address the profit margin.

Best Practice

Avoid asking multiple questions in one question. Focus on one question at a time, the more specific the question is, the more accurate the result is.

Answering Non-Aggregatable Values

You can occasionally encounter unexpected responses from Auto Answers, even when you use accurately formulated questions. This often occurs due to certain metric definitions and aggregation functions. When metrics are not designed for dynamic aggregation and are queried, the resulting calculation can lead to misleading, erroneous, or null values. For example, when a metric, such as annual population figures, is queried without specifying the necessary dimensions (for example, the year), an inaccurate aggregation of data over time can occur.

For example, if a question does not consider Year, a response from Auto Answers yields skewed results and totals the population value across all years in the dataset. The image below displays the correct value on the left and the incorrect value on the right:

Best Practice

For accurate responses to queries that include non-aggregatable values, use non-aggregate metrics that are based on the Strategy project schema in your dashboard. Non-aggregate metrics, such as inventory calculations, are prevalent in Retail verticals or Banking sectors. In applications powered by the project schema, Auto Answers processes the metrics at the correct level of aggregation and respects non-aggregatable settings, where defined, to provide reliable and precise answers.

For example, if you ask "In December 2020, what was the End on hand inventory level of Discman?", Auto Answers will correctly provide the end-of-month inventory, without summarizing it across the proceeding months.