YUI Test Selenium Driver
The YUI Test Selenium Driver is a utility to enable automated execution of JavaScript tests written in YUI Test through the use of Selenium. In order to use the Selenium Driver, you must have access to a Selenium Remote Control Server or a Selenium Grid Server (you can download the server software from SeleniumHQ).
The YUI Test Selenium Driver works by contacting a Selenium Remote Control Server and specifying which test page URLs to load in which browsers. When the test page URL is loaded, the JavaScript tests are executed, and once complete, the results are retrieved and then output into files. The files can then be picked up by a build system, such as Hudson, in order to allow reporting.
Installation
The YUI Test Selenium Driver requires Java 5 (1.5) or higher and a library provided by Selenium. To begin, download the latest Selenium Remote Control distribution. Unzip the distribution into a local folder and copy the selenium-java-client-driver.jar
file to your JRE's /lib/ext/
directory (the location of the JRE is system dependent).
IMPORTANT: In order to use this tool, you'll need to use the following YUI Test versions:
- YUI Test Standalone 1.0.0+
- YUI Test for YUI 2.8.1+
- YUI Test for YUI 3.1.1+
Other versions of YUI Test are not supported.
General Usage
There are several steps to using the YUI Test Selenium Driver:
- Write JavaScript tests using one of the compatible versions of YUI Test.
- Create a page to run your JavaScript tests. The tests should be run automatically on page load.
- Setup a Selenium server.
- Use YUI Test Selenium Driver to run your tests against various browsers via Selenium.
Test Pages
A test page is an HTML page containing the code to be tested as well as YUI Test Test to execute the tests. In order to ensure that the test page runs correctly, follow these steps:
- Include the YUI Test JavaScript library file.
- Add test suites/test cases to the
TestRunner
object (YUITest.TestRunner
for standalone,YAHOO.tool.TestRunner
in YUI 2.x, andY.Test.Runner
in YUI 3.x. - Run the tests automatically as Selenium Driver will not start them for you. Ensure that
TestRunner.run()
is executed either duringwindow.onload
or while the page is loading.
Command Line
Usage: java -jar yuitest-selenium-driver-x.y.z.jar [options] [test files]
Global Options -h, --help Displays this information. --browsers <browsers> Run tests in these browsers (comma-delimited). --conf <file> Load options from <file>. --coveragedir <dir> Output coverage files to <dir>. --erroronfail Indicates that a test failure should cause an error to be reported to the console. --host <host> Use the Selenium host <host>. --port <port> Use <port> port on the Selenium host. --resultsdir <dir> Output test result files to <dir>. --silent Don't output test results to the console. --tests <file> Loads test info from <file>. -v, --verbose Display informational messages and warnings.
The options are:
- The
--browsers
option specifies a comma-delimited list of Selenium browsers to test. By default, this is set to*firefox
. You can use any number of browsers, such as*firefox,*iexplore,*opera
. This can be set in a configuration file using the keyselenium.browsers
. - The
--conf
option specifies a properties file that can specify most of the command line options. - The
--coveragedir
option specifies the output directory for code coverage results. - The
--erroronfail
option specifies that if a test fails, it should cause an error such that the Selenium Driver exits immediately with a code of 1 instead of continuing on to other tests and exiting with a code of 0. This can be set in a configuration file by setting the keyconsole.erroronfail
to1
. - The
--host
option specifies the Selenium or Selenium Grid host on which to run the tests. The default value islocalhost
. This can be set in a configuration file using the keyselenium.host
. - The
--port
option specifies the port on the Selenium or Selenium Grid host on which to run the tests. The default value is4444
. This can be set in a configuration file using the keyselenium.port
. - The
--resultsdir
option specifies the output directory for test results. - The
--silent
option specifies that you dont want test results output to the console. By default, test results are output onto the console. This can be set in a configuration file setting the keyconsole.mode
tosilent
. - The
--tests
option specifies a XML file with test information (see below for format). When a tests file is specified, you need not pass in any test URLs on the command line. The tests file can be either a local file or a web address. Omitting this option means that test URLs must be passed on the command line.
In all cases, command line options override properties read in via --conf
.
Examples
The following runs the driver for a single URL:
java -jar yuitest-selenium-driver.jar http://www.example.com/tests/test_ui
The following runs the driver for two URLs on two browsers (Firefox and Internet Explorer):
java -jar yuitest-selenium-driver.jar --browsers *firefox,*iexplore http://www.example.com/tests/test_ui http://www.example.com/tests/test_util
The following runs the driver against tests specified in an XML file:
java -jar yuitest-selenium-driver.jar --tests tests.xml
Tests File
The --tests
option can be used to specify an XML file containing test information. This contains information about the tests to run and has the following general format:
<?xml version="1.0"?>
<yuitest>
<tests base="http://www.example.com/tests/" timeout="10000">
<url>test_core</url>
<url timeout="30000">test_util</url>
<url>test_ui</url>
</tests>
</yuitest>
There are only three possible elements to use in the file:
-
<yuitest>
document element. -
<tests>
second level element, indicates a group of tests with common settings. Thebase
attribute is used to indicate the base location for these tests and can be safely omitted if the tests dont share the same base. Thetimeout
attribute indicates the maximum time (in milliseconds) that the Selenium Driver should wait for the tests to complete. If the test doesnt complete in that amount of time, an error is thrown. You can also specify aversion
attribute on this element to override the one provided on<yuitest>
. There can be any number of<tests>
elements under<yuitest>
. -
<url>
third level element, indicates the URL to hit to run tests. If there is abase
attribute on the parent<tests>
element, then that value is prepended. Each<url>
element inherits values forversion
andtimeout
from its ancestors but also can override them individually if necessary. There can be any number of<url>
elements in each<tests>
element.
Config File
You can use a config file to store commonly used configuration options in order to avoid putting them on the command line. A config file is a simple Java properties file comprised of key-value pairs. The following keys are supported:
-
selenium.host
the hostname of the Selenium or Selenium Grid server. -
selenium.port
the port of the Selenium or Selenium Grid server. -
selenium.browsers
a comma-delimited list of browsers on which to run the tests. The browser names are those used by Selenium and so any valid browser name to Selenium is also valid here. -
results.format
the format in which to save the test results. These correspond to the test formats provided by YUI Test. There are three possible values:YUITestXML
,TAP
, andJUnitXML
(default). -
coverage.format
the format in which to save the coverage information. These correspond to the coverage formats provided by YUI Test. Possible values areLCOV
(default),GCOV
andHTML
. -
results.outputdir
the directory in which to save the test results. -
coverage.outputdir
the directory in which to save the coverage information. -
results.filename
the filename format for test results. There are three variables you can specify in the filename format:{browser}
inserts the Selenium browser name,{name}
inserts the test page name, and{date:<format>}
inserts the date in the specified format, where is a SimpleDateFormat -
console.mode
the console output mode. Possible values arenormal
andsilent
. This setting determines whether or not test results are output to the console. -
console.erroronfail
indicates if the Selenium Driver should exit with a status code of 1 when a test fails.
Troubleshooting
Class not found error
You see this:
Exception in thread "main" java.lang.NoClassDefFoundError: com/thoughtworks/selenium/Selenium
at com.yahoo.platform.yuitest.selenium.YUITestSeleniumDriver.main(YUITestSeleniumDriver.java:146)
This means that you've not copied the Selenium Java client driver into the appropriate JRE location. Please see Installation for more details.
Failed to start new browser session
[ERROR] Could not start Selenium session: Failed to start new browser session: java.lang.RuntimeException: Firefox 3 could not be found in the path!
Please add the directory containing ''firefox-bin' or 'firefox'' to your PATH environment
variable, or explicitly specify a path to Firefox 3 like this:
*firefox3/blah/blah/firefox-bin
This sometimes occurs in RHEL environments. The problem is that Selenium (not the Selenium Driver) cannot find Firefox on the system and therefore can't start it up. The error message actually describes the exact steps you should follow: either add the full path of the Firefox executable to your PATH
environment variable or specify the full path of the browser using the selenium.browsers
configuration parameter in the configuration file:
selenium.browsers = *firefox /usr/bin/firefox
Timeout issue
By default, Selenium will only wait for 30 seconds for an action to complete. In the case of the Selenium Driver, this means a page must complete all tests within 30 seconds. If you need longer, you'll need to set this up when launching the Selenium server:
java -jar selenium-server.jar -timeout 120
More information: http://release.seleniumhq.org/selenium-core/1.0/reference.html#setTimeout