# Virtuoso Tests

Tests on Virtuoso are presented in natural language and can be augmented in a plain English syntax.

Test steps that target page elements use selectors to find the element Virtuoso should interact with. Currently the following selectors are available: Hint, Xpath, ID, CSS. Read more about how to write advanced test targets.

# Natural language syntax

In this section we will list the different commands that can be used, along with examples of different phrases that the parser can understand.

# Available commands:

To navigate to an URL, the commands browse, go, navigate and open can be used.

Examples of valid phrases:

go to https://google.com
open https://amazon.com
browse http://spotqa.com
navigate to https://duckduckgo.com

You can also start the navigation in a new tab by adding in new tab.

navigate to "https://google.com" in new tab
open "https://amazon.com" in new tab

# Click

To click on an element, use the click command. It must be followed by an element identifier with or without location hints.

Examples of valid phrases:

click on bottom "Submit"
click "search"
click in "Name"
click $variableTarget

Tip: you can use variables (e.g., $name) to target elements dynamically. This works for every command that has elements. For example: click $name

# Write

To write values in input fields or text areas, use type, enter or write, the value to write, and the element to write in.

Examples of valid phrases:

write "Joe" in "First name"
type "24" in "Age"

# Select

To select a value from a drop down list, use select or pick. The element identified accepts location hints.

Examples of valid phrases:

select "March" from "Month"
pick "March" from "Month"

You can also select options by their index either through ordinals up to ten (e.g., pick third option), last item (e.g., pick last option), or to specify it (e.g., pick option five, pick option 42). Note that dropdown indexes start at 1, and negative indexes indicate reverse selection order (-1 means last item).

pick second option from "list"
pick last option from "Month"
pick option 7 from "Addresses"

# Wait

To pause a given time period or to wait for an element to appear on a page, use the wait or pause command. This is quite versatile, and will understand various forms of input. The intervals can be specified in milliseconds, seconds, minutes or hours, and will be internally converted to the canonical time unit (seconds).

Examples of valid phrases:

wait 1
wait one second
pause two minutes
wait 500ms
wait three seconds for "Logged in!"
wait for "Welcome"

Default and maximum timeout

Waiting for an element without specifying the time causes Virtuoso to wait for up to 20 seconds or until the element appears. The maximum wait time Virtuoso can execute is currently 10 minutes (if you enter a value higher than this your test will fail during its execution).

# Mouse actions

To perform mouse actions, use mouse, the action to perform, and the element to interact with. Valid actions are split into two categories: actions requiring an element, and actions requiring a position. Actions up, down, enter, over (short hand command available hover), out, leave, click, right click and double click require an element to perform the respective action on. The action move requires coordinates. The action drag can take either coordinates or an element.

Examples of valid phrases:

  • Targeting elements:
mouse double click "Element"
mouse right click "Element"
mouse enter "Name"
mouse over "Categories"
hover "Categories"
  • Targeting coordinates:
mouse move to 100, 400
mouse move by 10, 40
  • Without elements (use current position of the mouse):
mouse click
mouse right click
mouse double click

Mouse coordinates

move to accepts absolute x and y integer values to move the mouse to, while move by accepts relative x, y values. x refers to horizontal movement, while y refers to vertical movement.

Negative coordinates are accepted for the case of relative movement. For example move by -10, 40, moves the mouse backwards on the horizontal axis and forward on the vertical axis.

Click vs Mouse click

click and mouse click are separate commands, where the former performs a natural user click with several intelligent mechanisms such as popup auto-dismiss, fallback to alternative mechanisms, etc. while the latter performs a simple mouse click on the position.

Prefer to use the click command for general cases of clicking elements, and only use mouse click when you wish to cover a special interaction such as clicking on specific mouse coordinates.

# Drag and drop

You can drag and drop using the commands drag to and drag by. The former drags the mouse from its current position to either a target element or coordinate, and the latter drags the element by relative coordinate values.

mouse drag to "target element"
mouse drag to 240, 402
mouse drag by 0, -50

To drag and drop from one element (A) over another (B), you can achieve this like the following:

mouse over "A"
mouse drag to "B"

Mouse coordinates

Similarly to move to and move by, drag to is absolute movement and drag by is relative movement. drag to also accepts negative coordinates.

iOS support

Drag and drop is not currently supported on iOS

# Store

To store the details and content of an element, or an exact value into a variable, use the store command. Note that a variable must start with a $ and can contain letters, numbers and underscore, but must always start with a letter.

Examples of storing the contents of an element:

store element text of "password" in $user_pass
store element details of top link "login" in $login_link

And storing literal values in a variable:

store value "Hello World" in $myVariable
store text "John Smith" in $user_name

Store element details

Store element details allows you to make sure you use the element on further test steps. For example using the $login_link above:

assert $login_link.url ends with "/login"
assert $login_link.text contains "login"
click on $login_link

You can also use this store to check specific properties of the element:

Store element details of button "add more" in $button
Assert $button.disabled equals "true"

Available element details can be seen by executing the journey with the store test step, and looking at variable contents in the side effect panel associated to the test step.

Read more about using variables →

# Frames and tabs

To change the active frame (<iframe>) or browser tab, use switch, followed by an order integer index, or: for frames id, target element specification or parent; for tabs next or previous. Note that by switch to parent frame you return to the immediate parent frame of the current frame and not the top-level frame.

Examples of switching frames:

switch frame to "search"
switch iframe to id "menu-bar-frame"

And examples of switching tabs:

switch to previous tab
switch to next tab
switch tab to 3
switch to tab 3

# Scroll

To scroll the page to a certain element, coordinate, page position, or to scroll relative to where you are, use the scroll command. When a target is provided, the target element will be scrolled to the center of the window.

Examples of valid phrases:

scroll to page top
scroll to page bottom
scroll to "search"
scroll to 20, 40
scroll by 20, 40

Scroll input

scroll to accepts absolute x and y integer values to scroll the browser window to, while scroll by accepts relative values

# Upload

To upload files via file input fields, use upload, providing the URL of the file to upload, and the element to interact with. The URL should be a direct link to any file that is publicly accessible online, and the target element should be either the file input box, an element near the file input box, or the area that a file should be dragged to to start the upload.

Examples of valid phrases:

upload "http://www.mypersonalpage.com/resume.pdf" in "Résumé:"
upload "https://www.myhomepage.com/me.jpg" to "Drop image here"

Upload compatibility

Due to browser limitations, upload is not supported in the following environments:

  • Edge 15
  • All Safari versions
  • Mobile devices

To manipulate browser cookies, use cookie. You can add and remove one or more cookies.

Examples of valid phrases:

cookie "login" set to "username"
cookie create "login" as "username"
cookie remove "login"
cookie wipe all

# Window

To resize and manipulate the active window, use window.

Examples of valid phrases:

window resize to 640, 480

# Execute (extension execution)

You can execute a language extension script you have previously created using the extensions manager.

You can provide arguments to your extension either as exact values (e.g., "some data" as inputName), or use variables defined in the context (e.g., $fullName as fullname). The as varName denotes which function argument would receive the data.

You can also store the result of your script execution (e.g., the result of an ajax call) into a variable in the context by adding returning $nameOfVariableToStoreOutputIn.

For example:

execute "my script name"
execute scriptName using "Jon Snow" as name, 42 as counter and $age as age returning $result

You can also call extensions simply by their names (without specifying execute), and provide their input arguments in order:

checkBrokenLinks
validateElement ("#my .element-selector")
sumNumbers ($price, 42) returning $total

Extensions can also be implicitly defined without using the extension manager. You can simply write your script in-line, or via the properties panel. See example below for details.

execute "return document.querySelectorAll(mySelector).length" using "div.specialSelector" as mySelector returning $numberOfElements
execute "document.querySelector('h1').textContent = count" using $numberOfElements as count

Read more about extensions →

# Dismiss

To dismiss browser's alerts, confirms and prompts, use dismiss. Note that it's important to distinguish in the command whether the native alert is a basic alert or a confirm(Ok/Cancel) or a prompt (provide some text or cancel).

Examples of valid phrases:

dismiss alert
dismiss confirm reply ok
dismiss confirm respond cancel
dismiss prompt respond "text to write"
dismiss prompt reply cancel

# Press keyboard keys

To press individual or groups of keyboard keys, use press.

Note that keys separated by underscore _ indicate that they'll be held down together (this is only possible with keys such as CTRL, SHIFT, ALT).

You can also supply pieces of text to be entered in one sequence, by separating them by space character " ", e.g., S p o t SHIFT_Q A for writing SpotQA. In order to write the space character, you need to explicitly provide SPACE, for example Sp ot SPACE QA becomes Spot QA.

Examples of valid phrases when targeting elements:

press ENTER in "Search"
press SHIFT_A in "target"
press 'H E L L O SPACE W O R LD' in "Element"
press $variable

Note that special keys (e.g., ENTER, SHIFT, CTRL, SPACE, UNDERSCORE) need to added in ALL CAPS case.

Examples of valid phrases when targeting active element:

press TAB
press CTRL_SHIFT_X
press X
press '1 2 3 4'
press "Hello SPACE world"
press 'TAB First Name TAB Last N a m e TAB ENTER'

notice that you can use any keys including Tab to switch to following elements.

Default target

Press commands that do not specify a target will target the active element, to perform a press on a field you have to specify it. If you wish to specifically target the body, you can pass the body selector as target element, e.g., press F1 in 'body'.

Expressions: Press command currently does not support expressions. To use expressions first store value ${expression} in $var and then use the variable in the press command.

# Assertions

There is a large subset of assertions that can be used to verify that items or values on a page are as expected. These are described below:

# Assert Exists

This command will verify if an element exists on a page. look for and assert exists are ways to invoke this command.

Examples of valid phrases:

look for "Submit"
see "Submit"
assert exists "First name"
see element $var

Tip: You can assert elements dynamically by using variables, for example assert exists $foo

# Assert Not Exists

This command will verify that an element does not exist on a page.

Examples of valid phrases:

assert not exists "Please confirm you are not a robot"
assert not exists element "Request failed"

# Assert Equals

This command will verify that an element on a page has a given value.

Examples of valid phrases:

assert "First name" equals "John"
assert that "Last name" is equal to "Smith"
assert that "Age" == 25
assert "age" = 25

# Assert Not Equals

This command will verify that an element on a page does not have a given value.

Examples of valid phrases:

assert "First name" not equals "John"
assert that "Last name" is not equal to "Smith"
assert that "Age" != 25
assert "age" <> 25

# Assert Less Than

This command will verify if an element on a page contains a value less than the indicated value.

Examples of valid phrases:

assert "age" less than 18
assert "age" < 18
assert that "age" is less than 25

# Assert Less Than Or Equal

This command will verify if an element on a page contains a value less than or equal to the indicated value.

Examples of valid phrases:

assert "age" less than or equal to 18
assert "age" <= 18
assert that "age" is less than or equal to 25

# Assert Greater Than

This command will verify if an element on a page contains a value greater than the indicated value.

Examples of valid phrases:

assert "age" greater than 18
assert "age" > 18
assert that "age" is greater than 25

# Assert Greater Than Or Equal

This command will verify if an element on a page contains a value greater than or equal to the indicated value.

Examples of valid phrases:

assert "age" greater than or equal to 18
assert "age" >= 18
assert element "age" >= 18
assert that "age" is greater than or equal to 25

# Assert Matches

This command will verify if the value of an element matches a given regular expression. The regular expression is indicated by placing it between '/' characters.

Examples of valid phrases (no claim is made that the regular expressions are any good):

assert that "First name" matches /[a-zA-Z]*/
assert "Email" matches /^[a-zA-Z0-9._%+-][email protected][a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$/
assert "//title" matches "My page title"
assert "#parent > .selector" matches /Price:\s*42.11/

# Assert Selected

This command will verify if a given drop down list has the specified value selected.

Examples of valid phrases:

assert that "March" is selected in "Month"

# Assert Checked

This command will verify if a check box or radio button is checked.

Examples of valid phrases:

assert "Male" is checked
assert "I agree" ticked

# Assert Variable

Enables you to validate data stored in variables. Different comparison options are available: equals, not equals , contains, starts with, ends with, less than, less than or equal, greater than, greater than or equal, matches (for regular expression comparison).

Examples of valid phrases:

assert $result equals "expected value"
assert $foo is less than 4
assert $bar is less than or equal 100
assert $pi is greater than 3.14
assert $var starts with "expected start value"
assert $sample contains "some text"

You can also use expressions in your assert statements, for example:

assert ${1 + 2} equals "3"
assert ${$var1 == $var2} equals "true"
assert ${42 + $var1 * $total == $price} equals "true"
assert ${document.title} contains "Virtouso"

Additional examples can be seen in Using expressions to compare variable values.

# Comments

You can leave a comment on your tests by starting a test step with //. With this you can leave informative text to help contextualize the surrounding steps (for yourself or other users that visit/author the checkpoint). The comment text is limited to 500 characters, will never change the executions' outcome, and steps starting with TODO and FIXME are highlighted.

Examples:

// FIXME: add some assertion to make sure the login modal is shown
Click on "Login"
Write $username in field "username"
// TODO Button implementation is not ready yet
// Click on button "Continue"

Emphasize specific comments

You can emphasize your comments starting them with TODO or FIXME, this highlights the comment.

# Defining target elements

As mentioned above, test steps take multiple type of selectors (e.g., css, xpath, guess) for their target elements. Selectors are used by the order they are presented and the first selector that returns a valid element is used by Virtuoso.

# Guess (hint) selectors

Given a case-sensitive, guess selector, Virtuoso attempts to intelligently find an element to interact with. This can be as simple as writing: click on "Login"

Selectors on natural language tests

Each test created through natural language, that targets a page element (e.g., click), will automatically create a Hint selector based on the target's element text.

You can improve the accuracy of your hint by providing more context to Virtuoso.

# Relative position

You can provide top, right, left, bottom hints (or a mix such as top right) to distinguish between multiple elements. For example: Click on top "Login"

# Element type

When multiple elements of different types can match your query, you can provide more context to Virtuoso to make the better decision. Available element types are:

  • link: Links such as <a> elements
  • button: Buttons such as <button> or <input type="submit"> elements
  • input: Input elements such as <input> or <textarea> elements
  • image: Image elements such as <img> elements
  • dropdown: Dropdowns such as <select> elements
  • file: File input elements such as <input type="file"> elements or file drop zones

Position ordering

Note that in the natural language, the type always comes after the position. For example: click on top right button "Login"

# Standard selectors: CSS and Xpath

When writing a natural language test step, you can use the standard commonplace selectors as before. For example:

click on ".btn .btn-large"
click "/html/body/div[2]/div[5]/a"
click on "#elementId"
click '[href="https://spotqa.com/contact-us"]'

You can also guide Virtuoso by adding more selectors to the target elements.

Test step properties

You can open test step properties by clicking on the eye icon while viewing an exploration or editing test steps. This will show more detail associated with the test step, e.g., active selectors of the target element.

# Using variables in your tests

Test steps that take a value also optionally accept variables. Variables enable you to store and re-use values across your test steps in a journey.

Variables are denoted with a dollar symbol (like $fullName), and you can define them in different ways, by storing them:

  • Explicitly: store value "hello world" in $message
  • or Implicitly: write $message with default "hello world" in "input"

For example, you can store a variable in one page, and use it later on another page:

write $fullname with default "Jon Snow" in "Enter full name:"
click on button "Submit"
assert ".titleElement" matches $fullname

If a variable contains JSON data, you can use a JavaScript-like lookup syntax to lookup the inner data:

GET ("https://api.url/path") returning $data
write $data.user.firstName in "First Name"
click "Submit"
assert exists $data.user.firstName

# Clicking using variables

When executing the Click command, it is possible to provide a variable to control what is clicked on. To do this, first create a test step and provide a descriptive name for the element to be clicked:

Click "New Document"

Next, open the Test step properties for this test step, and click on More hint options (underneath selectors). The following modal will appear:

More hint options popup

Inside the Variable field, type the name of the variable storing the hint to be used by Virtuoso to find the element to interact with, you do not need to include $.

Now when the test executes, the value inside the Hint field will be replaced with the value of the given variable.

# Data driven element selection

Given that you can identify an element in a page using variables, and you can assign variables in data driven tests from test data tables, it is possible to identify and locate elements in a page using a test data table. To see how to make a test data table, see Data driven testing.

To do this, assuming that you have a table column productTitle, you can write:

click $productTitle

An alternative approach is to do this by explicitly assigning data table attributes. We need the following test steps:

Store value "foo" in $var
// if you want to have a default value for the variable
Click "New Document"
// OR otherwise
click $var

Next, open the Click test step properties , and click on More hint options (underneath selectors).

Inside the Variable field, type the name of the variable storing the hint to be used by Virtuoso to find the element to interact with, you do not need to include $.

Now we need to associate the test data: click on the context menu next to the journey name and press "Manage test data", select the appropriate test data table, and tick the checkbox next to the Store test step.

From this point when the journey is executed data-driven, the variable will be assigned to the values of the test data table, and the Click hint selector each value on the variable.

Default value of dynamic elements

Since the guess selector of an element is data-driven, note that the hint's value becomes the value if the variable is empty or missing.

# Variables with default value

Sometimes you may wish to have use a variable value if available, and if not available, use a default.

The with default and as keywords indicate to the bot to use the variable if available, if not, use the default text value provided.

Note: that if the variable was not defined, the default value will be set on the variable as well..

For example, take the following two test steps:

1. Write $var with default "foo" in "text1"
2. Write $var with default "bar" in "text2"

Line 1 will observe that var does not exist, and place the default text "foo" inside the variable var. Then, on line 2, var does exist so instead of typing "bar" in text2, the value of $var ("foo") will be used instead.

As another example, assume we want to type the current timestamp into a text field called time. This would need the following two test steps:

1. Execute "return new Date().getTime();" returning $currentTime
2. Write $currentTime with default "default value" in "time"

Because the variable currentTime is defined on line 1, line 2 uses the value stored inside currentTime instead of the hard coded default value.

Tip: Variable with default shorthand

As a shorthand for $var with default "default value you can write "default value" as $var.

# Variable containing downloaded files

If a file is downloaded as a side effect of an executed test step (e.g., clicking a download link), then this file will automatically be stored in Virtuoso to allow easy access for validation purposes. A new variable called LAST_DOWNLOADED_FILE is created and allows access to the file in any test step where a URL can be used.

File download restrictions

File download is only supported by the Virtuoso Chrome browser (not available for cross-browser testing), and the download is limited to 5 MB files.

To learn more about how to use file downloads in Virtuoso and methods of using the LAST_DOWNLOADED_FILE variable, take a look at our Example extensions.

# Using expressions in your tests

Test steps that take a value also optionally accept expressions in similar fashion as variables work. Expressions allow you to use multiple variables, to access the browser context and to combine both.

Expression are denoted with a dollar symbol followed by braces containing a Javascript expression (like ${$variable + $anotherVar.toLowerCase()}), The expression might refer to any defined variable, by using its name preceded by $.

The expression is evaluated in the current page's context, so you can access its DOM and properties. Examples of valid expressions are:

write ${'user' + $userName} in "login"
select ${new Date().getDate()} from "day"
write ${'name-' + new Date().getTime().toString(36)} in "intput"
click ${document.querySelector('.user').innerText}

# Using expressions to compare variable values

Using expressions you can compare the values of two or more variables. Using the following test steps with expressions:

// compare two numbers to be equal
assert ${ $number1 == $number2 } equals "true"

// assert $num1 is smaller than $num2
assert ${ $num1 < $num2) } equals "true"

// validate variable $text1 contains all the text in $text2
assert ${ $text1.includes($text2) } equals "true"

// validate that $var2 starts with text in $var1 (also trims whitespace on the texts)
assert ${ $var2.trim().startsWith($var1.trim()) } equals "true"
Last Updated: 9/14/2021, 12:30:39 PM