Xray Virtuoso App

Is Xray part of your workflow? If so, we have just the right thing for you.

The Xray Virtuoso App automatically synchronizes your Virtuoso projects, goals, journeys, test steps, and executions to your Xray Cloud installation.

Xray Test Management for Jira is a test management tool that "provides the structure to organize, plan, and report with accuracy on the progress of testing as well as the readiness to deploy" [1].

For more information on Xray, visit the product page of Xray.

Current limitations

• The Xray Virtuoso App does not support required custom fields on Jira issues, nor on Xray Test Steps.
• While we support multiple Jira instance configurations, a Virtuoso project can only be associated with one Jira instance at a time;
• Library checkpoints are currently not supported by our Xray integration and may cause issues when synchronizing the Virtuoso test steps, if enabled;
• Goal executions that simultaneously include journeys with and without data tables may fail to be imported into Xray.

Setup requirements

To set up the Xray Virtuoso App, ensure you have either a Jira Cloud instance equipped with an active Xray Cloud installation or a Jira Data Center / Server instance configured with an active Xray Data Center / Server installation. If you do not have Xray installed, you can follow the Getting Started guide provided by Xray (use the specific guide for your instance).

Xray Issue Types

For Xray to work properly, some Issue Types need to be created and mapped with JIRA. All the info can be found in their Quick Setup.

Virtuoso requires the following information to be able to connect to your team's Jira and Xray instances:

Cloud instance

• Your Jira instance URL;
• The email associated to a Jira account;
• An API token of that same Jira account;
• Specifying that the Jira Instance is Cloud;
• The client ID of the Xray installation;
• The client secret of the Xray installation.
• The region where your Xray data is located. If you're unsure, select None.

To generate an API token for a Jira account, please use the following guide: Manage API tokens for your Atlassian account. To create an Xray API key, please follow this guide: Global Settings: API Keys.

Data Center / Server instance

• Your Jira instance URL;
• The email associated to a Jira account;
• A personal access token of that same Jira account;
• Specifying that the Jira Instance is Server;
• The email associated to a Jira account (can be the same as above);
• A personal access token of that same Jira account (can be the same as above);
• For this instance, the region can be left blank (or select None);

To generate a personal access token (PAT) for a Jira account, please use the following guide: Using personal access tokens. XRay Data Center / Server makes use of the same personal access token as Jira.

Supported Jira and Xray Data Center / Server versions

Officially, Virtuoso only supports Jira Data Center / Server version 8.20.15 and higher, and XRay Data Center / Server version 7.4.1 and higher

Xray data regions

Your Xray data can be in a specific region, and not selecting the correct region will make the integration fail. Contact your Xray administrator to know where your data is located. This is only relevant for Xray Cloud instances.

Jira permissions and ownership

The user account used for setting Xray Virtuoso App up will be associated with the operations executed, for instance, the app will only be able to list projects that are visible to that user.

Dedicated user in Jira

We recommend that you create a dedicated user in Jira with administrative privileges on the projects you want to see listed.

Xray permissions and ownership

We recommend that you create an Xray API key for the same user as the Jira API token, or use the same PAT (Personal access token) in the case of Data Center / Server.

Application setup

To install the Xray Virtuoso App follow the steps described in Installing an app.

User permissions

Although it is not mandatory, it is strongly advised that you create a new Virtuoso user dedicated to this application: Virtuoso projects shared with this application user will have the application enabled.

The Virtuoso Xray App supports more than one Jira Instance. To start, click on Add another instance and fill out the form on the right:

1. Fill in the Jira URL field with the URL to your team's Jira instance;
2. Fill in the Jira user email field with the email of the account you want to link to the operations;
3. Fill in the Jira API token field with an API token of the Jira account you want to link, or user PAT (Personal access token) for the case of Data Center / Server;
4. Select the Jira Instance Type from the supported instance type dropdown list. By default it uses Cloud;
5. Fill in the Xray client ID field with the client ID of the generated API key, or user email for the case of Data Center / Server;
6. Fill in the Xray client secret field with the client secret of the generated API key, or user PAT (Personal access token) for the case of Data Center / Server.

Enabling project integration

The synchronization of Virtuoso to Xray is configured on a project's basis on a first step, and then goal by goal. To enable the integration for a project, navigate to it and then click on the context menu to the left of the "Project dashboard" title. You should be able to see and click an action called Manage Xray integration.

If the action is not displayed

Please check if the project is shared with the Xray Virtuoso application user, and refresh the page.

This action opens a modal form with the following fields:

• Enable Xray Integration: enables or disables the integration for the selected project;
• Jira instance: this field lists the Jira instances added to the application's configuration and lets the user choose the one they wish to synchronize to;
• Jira project: this field lists all Jira projects available to synchronize to, filtered by user input;
• Test type: this field lists all Xray test types available to use. Currently, Virtuoso only supports Xray Manual Test types;
• Sync manual executions: if enabled, every future execution will be imported into Xray, otherwise only executions from Execution Plans will be sent;
• Sync test steps: if enabled, the created Xray tests will include the journeys' test steps.

Fill out the form to link the Virtuoso project to a Jira project of your choice. To unlink a project, disable the "Enable Xray Integration" option and submit the form. If the integration was successful, a remote link shall appear near the project name in the organization dashboard that will take you to the linked Jira Project.

Disabling project integration

Please beware that disabling a project's integration will not delete the issues in Jira, only the links between Virtuoso's entities and their counterparts in Jira/Xray; if you link the project again, Virtuoso will create new issues (Test Sets, Tests, etc.).

Project configuration errors

If there were changes in either Virtuoso's Xray configuration or the Jira/Xray configuration itself, the integration might be unable to properly communicate, and an error message will appear. If this error persists, try disabling and re-enabling the project integration. Be mindful that this will create new issues, as mentioned above.

Enabling goal integration

To enable the app for a goal, click on the context menu in its card in the project dashboard, or navigate to it and click the context menu to the left of the title "Goal: goal name". You should be able to see and click an action called Manage Xray Test Set integration.

This action opens a modal form with the following fields:

• Enable Xray Integration: enables or disables the integration for the selected goal;
• Link to existing Test Set: if enabled, the goal will be linked to the Test Set provided in the next field, which means that every Test created will be added to a provided Test Set. Leave blank to create a new Test Set;
• Test Set: lists the existing Test Sets, filtered by user input, and allows the user to pick one for the goal to be linked to.

Fill out the form to link the Virtuoso goal to a new Test Set, or to an existing Test Set of your choice. To unlink a goal, disable the "Enable Xray Integration" option and submit the form. If the integration was successful, a remote link shall appear under the goal in the project view, that will take you to the linked Test Set in Xray.

Disabling goal integration

Please beware that disabling a goal's integration will not delete the issues in Jira, only the links between Virtuoso's entities and their counterparts in Jira/Xray; if you link the goal again, Virtuoso will create new issues (Test Sets, Tests, etc.).

After enabling the integration for a goal, the Xray Virtuoso App will start synchronizing the goal with Jira and Xray by creating a Test Set (if the user did not choose to link to an existing Test Set), and by creating tests for each of the goal's journeys.

Test synchronization time

It may take a while for Tests to be visible in Jira if you are linking a goal with a large number of journeys and/or test steps.

Tips and FAQ

How are Virtuoso entities represented in Xray?

Virtuoso	Xray
Project	Project
Goal	Test Set
Journey	Test
Test Step	Test Step
Goal Execution	Test Execution
Journey Execution	Test Run

What happens in Xray when...?

Virtuoso	Xray
Goal integration is enabled	A Test Set is created for the Goal, and Tests are created for each of the Goal's Journeys.
Goal name changes	The Test Set's summary for the Goal is updated.
Journey is published	A Test is created for the published Journey.
Journey name changes	The Test's summary for the Journey is updated.
Test step is created	If test step synchronization is enabled, the test step is added to the Test issue.
Test step is changed	If test step synchronization is enabled, the test step is updated in the Test.
Test step is deleted	If test step synchronization is enabled, the test step is deleted from the Test.
Execution finishes	Creates a Test Execution issue with Test Runs for each Test.
Data-driven execution finishes	Creates a Test Execution issue with Test Runs for each Test. If test step synchronization is enabled, it includes a dataset of the values used for each iteration. The Test Run result will be a failure if any of the iterations failed.