# Frequently Asked Questions (FAQ)
# Table of contents:
- When trying to paste a checkpoint, why is the clipboard pasted into the "Checkpoint title" field when I press CTRL+V?
- I have an assertion that does not have "continue on failure" enabled, why is it still continuing the execution when it fails?
- Can I speed up the execution of my tests?
- Why does it take too long to click an element?
- My test step should be pressing keys in some input field/element but is not.
- My application requires multiple windows, how do I interact with multiple windows in Virtuoso?
- Can I run Virtuoso tests using custom data from an external location?
- Virtuoso is entering text in a text input field in a strange way?
- My test fails because an element was too slow to load, so it was not found by the test step. How can I delay the test step until the element has loaded?
- Using natural language I am unable to target the element that I want, how do I use selectors to modify which field is used?
- Are parallel explorations and executions supported by Virtuoso?
- I edited the journeys, and they disappeared after an exploration. Did I lose my work?
- What are differences between the element name and test step selector hint in the test step properties?
- I added a test step through natural language but it failed or Virtuoso interacted with the wrong field. How can I overcome this problem?
- A test step inside a checkpoint failed, why are the remaining checkpoints skipped?
- I added an invalid email to a form with a write test step. Was it not supposed to fail?
- Why am I seeing duplicate pages in my application graph?
- My application uses CAPTCHA or Two-factor Authentication (2FA). How can I run automated tests against my application?
- How can I validate that my Two-factor-Authentication (2FA) mechanism is working?
- How can I distinguish traffic coming from Virtuoso bots on my application?
- I cannot find an answer to my question, how do I contact support?
- The application graph of my goal seems to be outdated, how can I update it?
# When trying to paste a checkpoint, why is the clipboard pasted into the "Checkpoint title" field when I press CTRL+V?
In order to paste a checkpoint, there needs to be no editable test step open. Having an editable test step will cause the clipboard to paste into the title of a test.
# I have an assertion that does not have "continue on failure" enabled, why is it still continuing the execution when it fails?
An assertion will continue if it is able to successfully execute, but the condition of the assertion does not match. This is so that you can make many validations without stopping test execution, and being able to reach the end.
The "continue on failure" is only used when a step fails to execute. For example, in
assert "element" contains "text", if "element" is not found, the "continue on failure" is evaluated, but if the element is found but the value does not contain "text", it will mark the journey as failed, but will continue.
# Can I speed up the execution of my tests?
Tests with too many steps can execute slowly, especially if they have to locate elements on the screen. See how long each step takes when running against your application, and discover which checkpoints/steps take the most time. This information can help you understand how to reduce the duration of the execution.
# Why does it take too long to click an element?
Virtuoso waits for a page to stop loading before proceeding further with a step. Often a click will involve navigating to a different URL, which usually calls your backend. It is likely that your application's backend is taking too long to respond after the click. Note that this delay is something your users will be facing as well. In short, it is not Virtuoso that is taking too long to perform an action, but rather, the target application that is taking too long to respond after the action was taken. If you think you have come across an unusual waiting time that is not related to your application, please contact our support.
# My test step should be pressing keys in some input field/element but is not.
By default, the
press command will execute on the whole page, or the last active element. To execute some key in a specific element, use
press KEY in "identifier" (e.g.,
press ENTER in "Search").
# My application requires multiple windows, how do I interact with multiple windows in Virtuoso?
Virtuoso supports multiple frames and tabs using the switch command, and this can simulate using multiple windows. To open a new tab, use the
navigate command (e.g.,
navigate to "http://www.spotqa.com" in new tab. To change to the next tab, the test command
switch to next tab will open a new tab. The default tab can be accessed using
switch to previous tab or
switch to tab "0".
# Can I run Virtuoso tests using custom data from an external location?
There are few ways you can achieve this:
- Use an extension to fetch the data from your backend and make it available to your journey.
- Make your journey data-driven, and use the Virtuoso API to update the data (e.g., import a new CSV).
- Execute your journey with custom initial data using the API.
# Virtuoso is entering text in a text input field in a strange way?
xxxx-xxxx-xxxx-xxxx can modify the order from that which was originally entered).
write test steps, one for each part, to enter the entire credit card number, ensuring that each test step appends to the input field.
# My test fails because an element was too slow to load, so it was not found by the test step. How can I delay the test step until the element has loaded?
By default, Virtuoso will not wait a long time for elements to appear. This is expected behaviour and will not change. However, using the
wait command, it is possible to delay interaction with an element until it has loaded. For example, to wait 15 seconds for "your element",
wait 15 for "my element" as a previous test step would ensure that the element is loaded before continuing. This implicitly waits for the element to load, such that if it loads in 2 seconds, Virtuoso will not wait 15 seconds.
# Using natural language I am unable to target the element that I want, how do I use selectors to modify which field is used?
You can add custom selectors with the
add selector option when modifying a test step. When selecting an element to interact with, the selectors will be used in order of top-to-bottom. If a selector is not working, it may be worth placing a custom selector above it.
# Are parallel explorations and executions supported by Virtuoso?
Yes, Virtuoso supports parallel explorations and executions with simple restriction:
This means that it is possible to have multiple jobs active in a project as long as they are using different goals, and many executions for one goal can run simultaneously.
# I edited the journeys, and they disappeared after an exploration. Did I lose my work?
If you explore a goal from scratch, your existing data will be removed in the new version. However, do not worry if you did this by mistake. You can rollback to the previous version. To do this, open the goal, navigate to an earlier version you want to rollback to, and then from the options, select
Rollback goal to this version.
Once committed, the Version store all the information added by the user so the journeys that you created/edited are still listed on one of the previous versions. You can fork the previous version to a new project to continue augmenting those tests if the new exploration is not needed, or select and copy the test steps that you need to a new version after the most recent exploration.
If you wanted to instead explore further from a point in your goal, you can use the explore further feature on the goal view (exploration page).
# What are differences between the
element name and test step selector
hint in the test step properties?
When creating a test step, Virtuoso uses the
element name introduced and creates a selector
Hint to find the element on the page (this selector can be seen by clicking the eye icon and opening the test step properties).
Hint selector, changes the field/element that Virtuoso Executes the test on, while
element name only affects how the test step is represented. Because test steps may not have a
Hint selector, Virtuoso allows you to edit the
element name to customize the test step making it more readable.
For example, you can write the following test step:
click on "Sign in"
Virtuoso creates a test step with the that text and a hint selector
Then, you notice that the button is called
Sign In (capitalization change) and update the selector so that Virtuoso can find the element.
If you could not edit the
element name, you would have to delete and recreate the test step or the text would be out of sync with the selectors you are using. By editing the test step
element name you can now change the text of the test to present:
click on "Sign In".
# I added a test step through natural language but it failed or Virtuoso interacted with the wrong field. How can I overcome this problem?
When you add a test step through Natural language, Virtuoso generates a
Hint selector based on the text introduced. This text is case-sensitive so you should pay special attention to the case of the letters in the HTML. For example, if an application uses CSS to change the text to small caps variant, the text will appear to be capitalized yet in the HTML it may only have the first letter with caps.
If this still does not solve the problem, you should use another type of selector that you know that produces a unique result for the page you are in. Again, for example, a login form page can have two intractable elements with the text
Sign In if you do the following steps once you reach that page:
write "[email protected]m" in "Email" write "dfg123" in "Password" click on "Sign In"
Virtuoso will click on the first
Sign In element, which can be the link at the top navigation bar to access the login form and you end with an empty form.
You can use another selector type or try guiding Virtuoso in finding the correct element by creating a more specific test step:
click on button "Sign In"
# A test step inside a checkpoint failed, why are the remaining checkpoints skipped?
Unlike journeys, the Execution of checkpoints is performed sequentially. Consider the following scenario:
Checkpoint 1: Navigate to www.google.com Latin version Checkpoint 2: Search for "ipsum lorem"
If the first checkpoint fails because Virtuoso was not able to access the specific version of the website, it does not make sense to attempt to perform the search as the browser can be in a completely different state than what was expected.
If the checkpoints are independent, for example they do not have side effects on the browser state, you can create a new journey with the second checkpoint and have them run independently and in parallel.
# I added an invalid email to a form with a
write test step. Was it not supposed to fail?
write test steps, Virtuoso fills the specific form element with the text provided (see examples in Virtuoso tests → Natural language syntax). If it was able to find an element and input the text, the test step will pass independently of any form validations that may happen after the text is written.
Without this behavior, you could not assert the error messages that are expected when introducing invalid data.
# Why am I seeing duplicate pages in my application graph?
Web applications can be complex in their structure, with multiple paths and routes to the same page. Virtuoso automatically evaluates each page it encounters during exploration, and attempts to ascertain whether it has been seen before using a combination of important page features and a history of the pages previously discovered.
Occasionally, the dynamic nature of the page contents might result in duplicates making their way into the application graph. As Virtuoso interacts with your site like a user, following web application design best practices can help avoid these inaccuracies. For example, ensuring that URLs are meaningful, and using distinct & appropriate title tags for each page can help ensure pages are more likely to be disambiguated.
You can also manually edit which states can be considered the same or distinct from the application graph view.
Improved state discovery and manual flagging of duplicates
Virtuoso has encountered hundreds of thousands of web pages during explorations and discoveries, and will continue to improve in matching accuracy to ensure duplicates are rare.
# My application uses CAPTCHA or Two-factor Authentication (2FA). How can I run automated tests against my application?
Our general recommendation based on testing best practices is to ask your development team to provide a mechanism to disable verification of the CAPTCHA / 2FA. Here are some suggestions you can pass on to them:
- Create a specific testing environment in which CAPTCHA/2FA disabled.
- Create and use specific accounts for which CAPTCHA/2FA is disabled (e.g., disable CAPTCHA/2FA for certain test users, or to validate that the value of the submitted captcha matches a fixed value for these users).
- Accept a special cookie: disable CAPTCHA or ignore the value of the submitted captcha when a known cookie is present and its value matches a specific value that is known by the backend (you can set cookie values in Virtuoso test steps).
- Accept a special value: Create a special CAPTCHA or 2FA value which is always accepted. For example, see the I'd like to run automated tests with reCAPTCHA. What should I do? page on Google's suggested approach.
- IP Whitelisting: We can set Virtuoso to always target your application from a fixed IP, which you can then use to whitelist (i.e., disable captcha/2FA for our IPs). Contact our support if you want to proceed with this option. Note that the previous options are generally preferred over this, since they improve testability of your application regardless of whether or not you use Virtuoso.
CAPTCHA on a login page
If you are using a Captcha on your login page, our recommendation is to only require a captcha submission after a number of failed login attempts (as common across the industry, for example, Google, Microsoft, etc.).
This creates a better experience for users, ensures that automated tests using the correct credentials will not face a captcha, and maintains the security safety net to avoid bruteforce attempts.
# How can I validate that my Two-factor-Authentication (2FA) mechanism is working?
If disabling 2FA for testing purposes is not an option, you can attempt to solve them during the journey, based on the type of 2FA method. For example:
- Email based Token/Link: You can navigate to your email provider, login, and then retrieve the token/link to proceed from. WARNING: DO NOT use this against Gmail, Microsoft, Yahoo, etc. mail providers since they often have bot prevention tooling that will automatically block access to your account, thinking your account may have been compromised.
- SMS token: You can use the natural language extension (script) feature of Virtuoso to call the API of your sms-service provider (e.g., Twilio), to retrieve the token.
# How can I distinguish traffic coming from Virtuoso bots on my application?
By default, all requests made using Virtuoso's default browser contain the following suffix on the user agent:
Virtuoso - spotqa.com/virtuoso - If this request is unexpected or unwelcome, please contact [email protected]
You can look for
Virtuoso in the browser user agent to identify our bots. Alternatively, you can use the custom HTTP headers capability when executing tests using the Virtuoso API.
# I cannot find an answer to my question, how do I contact support?
If you have any questions, doubts, concerns or suggestions you can email us at [email protected].
# The application graph of my goal seems to be outdated, how can I update it?
If you have a goal that was created before October 2020, you need to first Synchronize it, and then Capture checkpoint snapshots to synchronize the graph. Once a successful capture is performed the graph will be updated and, from that point onwards, it will be synchronized with authoring changes performed.