# Analytics 2.0 add-on
The Analytics 2.0 feature in Virtuoso provides real-time insights into the performance of your tests. Multiple reports are available to help you understand the test execution and the performance of your tests. You can view the reports in the Virtuoso UI and download all the reports in PDF format and individual charts in CSV, Excel, and image format.
To access the Analytics 2.0 feature, click on the Analytics tab in the Project dashboard in Virtuoso UI.
# Available Reports
# 1. Project Execution Dashboard
The Project Execution Dashboard offers a complete view of the test execution status at the project level. It aggregates various key metrics, such as the total number of journey executions, pass/fail rates, execution times, and root cause analysis for failures. This dashboard helps you to evaluate the execution history of your project, monitor trends, and identify problem areas efficiently.
Main Widgets and Reports in the Dashboard:
The dashboard contains several key widgets that help you to monitor project performance. Below are detailed descriptions for each component:
Overall Journey Execution Count: This widget displays the total number of times journeys have been executed in the project. It's a high-level indicator of how active testing has been within the selected time range.
Unique Journeys Executed: This metric provides insight into how many unique journeys (test cases) have been executed. It helps you to gauge the diversity of test cases covered within a given time frame.
Weekly Execution Count: This shows the number of journeys executed in the past week and compares it with the previous week to track the execution trend. This widget can highlight an increase or decrease in testing activity.
Journey Executions Over Time: This line chart presents the number of journeys executed each day. Different colors represent the outcomes (e.g., passed, failed, skipped), allowing you to identify trends and variations in journey execution over time.
Journey Execution Outcome (Pie Chart): This pie chart visually displays the proportion of passed, failed, and skipped journeys. It helps you to quickly assess the success rate of the project's test executions.
Failed Journey Contribution (Sunburst Chart): The Failed Journey Contribution (Sunburst Chart) visually breaks down journey failures by goal, allowing you to drill down into specific journeys that failed under each goal.
- Inner Circle: Represents the different goals in the project. Each segment corresponds to a particular goal.
- Outer Segments: Displays the specific journeys that failed under each goal. By hovering over these segments, you can see the failure count and the percentage of failures within that journey.
- Hover Functionality: Hovering over an outer segment highlights a specific journey and shows its failure details, such as the number of failures within that journey, the percentage of total failures it represents, and the contribution of that journey to failures within the associated goal.
Execution Operating System and Browser/Device: This chart shows which operating systems and browsers were used for journey executions, helping ensure that the tests are being run across the intended environments. It can also highlight any environment-specific issues.
Reasons for Journey Execution Failures: Provides an analysis of the primary causes for failed journey executions, categorized by failure reason. This chart can help teams focus on specific areas that need improvement, whether it's a test configuration or infrastructure issue.
Execution Outcome Per Month (Raw Data) chart: This bar chart presents a month-by-month breakdown of journey execution outcomes, categorized by passed, failed, and skipped statuses. It provides a historical view of testing trends, allowing you to quickly identify fluctuations or patterns in test performance.
Journey Execution Outcome Per Month (Percentage): Similar to the previous chart, but shown as percentages, this widget helps you to understand the success rate and how it's changed over time.
Goal Pass and Fail Rate: This bar chart provides a detailed comparison of pass and fail rates across all test goals within the project. Each bar represents a test goal, color-coded to indicate success (green) and failure (red) rates. The chart helps you to quickly spot underperforming areas that may require further investigation or remediation, while also highlighting goals that are achieving higher success rates.
Goals Summary (Table): This table offers an interactive view of each goal’s pass, fail, and skip rates, as well as the number of executed journeys. Clicking on a goal filters all related visualizations on the page, giving a detailed, goal-specific breakdown across the project’s key metrics.
Goal Execution Count: This widget presents a detailed breakdown of how many times each goal within the project has been executed. Each bar represents the total count of executions for a specific goal, helping you quickly identify which goals have received the most focus during testing. The percentage column on the right represents the share of each goal's executions relative to the entire project.
- Sortable columns: The columns (Goal, Count, and Percentage) can be sorted in different ways using the arrows next to the column headers. Sorting can be done alphabetically (for goals) or numerically (for execution count and percentage). This allows you to see the goals in various orders, such as from highest to lowest execution count.
- Cross-filtering: Clicking on a specific goal filters other charts on the page. For instance, selecting "Plans" will filter the rest of the dashboard to show data only related to that goal. This is particularly useful when comparing execution data across different metrics.
Percentage breakdown
This chart displays data at the project level, meaning the percentage breakdown is based on the overall executions within the project. For example, "Apple - Global" executions account for 96.433% of the total journey executions in the Apple docs project.
Team Member Execution Count: One of the most interesting features is how the Team Member Execution Count chart below interacts with this table. When you click on a specific goal in the Goal Execution Count chart, the Team Member Execution Count chart below dynamically updates to show who has executed journeys within that selected goal. This gives you insights into which team members are most active in testing particular goals, revealing data that would otherwise remain hidden.
# 2. Execution Details
The Execution Details Report allows you to view detailed insights into the execution of test journeys. This feature enables you to analyze the performance of test journeys, track execution outcomes, and take appropriate actions such as rerunning failed journeys.
Main Widgets and Reports in the Dashboard:
The dashboard contains tables that help you to monitor your project's performance. Below are detailed descriptions for each component:
Journey Execution Details Table: Provides comprehensive details on each journey execution, including the execution date, outcome, user actions, and additional metadata. This table enables you to perform in-depth analysis of specific executions, facilitating troubleshooting and performance assessment.
Execution Duration by Journey: Offers a detailed breakdown of journey execution times, showing the minimum, maximum, and average durations. This table allows you to assess performance consistency and identify potential outliers or performance bottlenecks across different test journeys.This table includes fields like Execution Status, Journey Name, Execution Outcome, and Execution Duration.
# Filter Options
The available reports offer various filtering options, allowing you to refine the data presented. These filters include, but are not limited to, the following:
- Time Range (Last day, week, month, quarter, or year): This enables the user to adjust the reporting period and see data for the selected time frame.
- Execution Outcome (Pass, Fail, Skipped): You can choose to focus on executions with specific outcomes.
- Goal archived and Journey draft options: Filters help narrow down the displayed goals and journeys based on their archived status or whether they are still in draft mode.
# Report Download Options
You can export any of the charts or tables into formats such as CSV, Excel, or Image. The PDF of the entire report can also be downloaded using the download button present beside the name of the report in the left panel.
Chart/Widget Download: Click on the three dots present on the top right corner of the chart and select the desired format to download the chart.
Entire Report Download: Click on the download button present beside the name of the report in the left panel to download the entire report in PDF format.
Note
The processing of the entire report download may take some time depending on the data and length of the report. Please wait for the download to complete. Do not refresh the page or navigate to other pages during the download process.
# Important Points to Note
- Live Authoring Executions: Executions triggered via the "I'm done" button during Live Authoring journeys are not reflected in dashboards for current data. They are, however, recorded in historical data, allowing you to review them for past activity but not in current data. We are working on having this for current data to be available soon.
- Active Goal and Published Journey in Historical Executions: Historical execution data assumes that each execution was executed as an active goal and a published journey. This applies only to historical data, not to new executions, the new executions account for the goal being archived and an journey being draft.