Using the TPTP automatable services
TPTP provides several automatable services which can be invoked from outside
the context of the eclipse workbench. The services include test execution and
test result interrogation.
This help topic explains how to use the built-in TPTP automatable services to
do the following:
Test Execution Service
The test execution service is provided for the
programmatic launch, execution and generation of test results. The test execution
service wraps existing test execution harness functionality
and obsoletes the previous standalone test execution harness methods. Using the test execution service, tests can
be launched using a headless Eclipse from Ant scripts, shell scripts
and arbitrary programs. Multiple tests can be run at the same
time given enough memory on the executing machine (each service
execution currently requires a separate Eclipse host instance). Even if
tests are running on remote machines using TPTP remote test execution
features and deployment, the service consumer's machine is the machine that
runs the Eclipse host instances.
Service properties
Refer to the reference material on the supported test
execution service properties -- the same properties are valid across client
automation adapters. For Ant scripts, a set of simple Ant tasks are
provided to set service properties and execute the service. For
shell scripts, the properties are specified in the expected command line
paradigm (using dashes for properties followed by space and then a value
for instance). For Java programs, the automation
client jars will be referenced and used for setting properties
and controlling the service execution. Some client adapters
might have additional options that are required for that particular client
environment (or offered as extra features from that adapter).
Automatable service framework packaging
The TPTP automatable service components are
packaged within the org.eclipse.hyades.execution plug-in represented
with three jar files, some XML in the plugin.xml and some entries in the
MANIFEST.MF file.
-
tptp-automation-client.jar (the
lightweight bit of code needed to establish the client side of the service
bus, launch Eclipse if needed and execute headless services, this jar will
be referenced when running tests from the command line and shell scripts as
well as when running tests from arbitrary Java code).
-
tptp-automation-server.jar (the TPTP
automation server, loaded on-demand as services are interrogated and executed
using the framework, implements the core Eclipse applications extension point,
used as the broker and service locator)
-
ant-tptp.jar (this jar has the same
contents as the tptp-automation-client.jar but is renamed to be
consistent with other Ant task libraries)
Running tests from Ant scripts
Ensure Ant 1.6.5 or higher is properly installed if tests are
to be run outside of a running Eclipse instance -- refer to the Ant
documentation to set this up properly. In addition to the standard Ant
installation, also copy the ant-tptp.jar into the Ant lib directory alongside
the other tasks. If running inside of Eclipse, Ant is already installed
although be sure to include the ant-tptp.jar in the additional tasks tab of the
Ant launch configuration, or in the Global Entries section of the Ant Classpath
preference page. The embedded sample Ant script below is an
example of an Ant script named "test.xml" that executes a test named testA
within the unit-tests folder of the test project. The test project is
located in the specified workspace of "d:\development\workspace" and the Eclipse
host instance is specified to be "d:\development\eclipse". Refer to the
reference list of supported properties for other valid combinations of
properties.
The example below uses the TPTP test task although it is also
possible to execute a test using the TPTP automation task.
The more general automation task is an Ant task for executing arbitrary
TPTP automatable services which contrasts the more specific TPTP test task. The test
task is a type-safe task specifically
created for launching the test execution service. More Ant script examples are included in the
org.eclipse.hyades.execution plug-in's src-automation-client source folder's org.eclipse.hyades.automation.test package.
<project name="TPTP Ant Tasks Test Script"
default="test"
xmlns:tptp="antlib:org.eclipse.hyades.automation.client.adapters.ant">
<property name="tptp.automation.eclipse" value="D:\development\eclipse"/>
<description>
This Ant script tests the TPTP automation Ant task client adapters
and also can be used to test the underlying execution of TPTP automatable
services via the TPTP automation server
</description>
<!-- Execute TPTP test suites -->
<target
name="test" description="Executes TPTP Test Suites using default TPTP Ant
Tasks">
<!--
Execute a TPTP test suite using the test task and providing the workspace
-->
<tptp:test
workspace="D:\development\workspace"
project="tests"
suite="unit-tests/testA.testsuite"
synchronicity="synchronous"/>
</target>
</project>
Running multiple tests and interrogating their results from Ant scripts
Ensure that you have properly configured Ant to run TPTP automatable services
(refer to the
above section on running tests from Ant.) The example below shows how
to the use the tptp:execution task to execute multiple tests, and how to use the
tptp:interrogation task to interrogate the results of those test executions.
Both of these Ant tasks accept multiple Ant filesets and/or
filelists, allowing you to use wildcards or to directly
specify multiple files as inputs to the tasks. In this example, we execute
all of the test suites in the TPTPJUnitTests project, and the set of
all resulting .execution files is returned to us as a filelist
in the Ant variable that we supply (in this case "tptp.test.results".) We then
pass that filelist as input to the tptp:interrogation service,
which will interrogate each of those execution files to determine an overall
verdict and will assign it to the Ant variable that we supply (in this case
"tptp.test.verdict".)
<project name="TPTP Automatable Services Tutorial"
default="main" xmlns:tptp="antlib:org.eclipse.hyades.automation.client.adapters.ant">
<!-- Define common properties for all automatable services -->
<property name="tptp.test.workspace" value="C:\eclipsecon\eclipse\workspace"/>
<property name="tptp.test.project" value="TPTPJUnitTests" />
<property name="tptp.test.connection" value="tptp:rac://localhost:10002/default"/>
<property name="tptp.automation.eclipse" value="C:\eclipseconwb_birt\eclipse" />
<!-- Define script-local properties, that vary by installation -->
<property name="project.dir" value="${tptp.test.workspace}\${tptp.test.project}"/>
<target name="main" depends="test-execution, test-results-interrogation"/>
<!-- Execute test suites using default results name -->
<target name="test-execution">
<echo message="Executing test suite..." />
<tptp:execution resultsrefid="tptp.test.results">
<fileset dir="${project.dir}">
<include name="**/*.testsuite"/>
</fileset>
</tptp:execution>
</target>
<!-- Interrogate test suite results for verdict -->
<target name="test-results-interrogation">
<echo message="Interrogating test suite results..." />
<condition property="tptp.test.success">
<tptp:interrogation verdictproperty="tptp.test.verdict">
<filelist refid="tptp.test.results"/>
</tptp:interrogation>
</condition>
<echo message="The overall test result verdict is: '${tptp.test.verdict}'" />
</target>
</project>
Running tests from the command line and shell scripts
Tests can be executed directly from the command line and
shell scripts with the test properties taking the form of command line
arguments. The TPTP automation client adapters provide a .bat and
.cmd file for use within Windows and a .sh file provided for use
within the Linux operating system. It is possible to specify long
sets of arguments and/or frequently repeated arguments in a configuration
file to minimize the amount of typing necessary to run the test execution
service from the command line. The shell automation client adapter
manifests as a text file (for the batch file or script) and a complementary Java
component -- both must exist to execute tests from the command line.
All the TPTP provided automation client adapters can be found in the
tptp-automation-client.jar although the ant-tptp.jar replicates some of the
adapter code in the form of an Ant jar (this was done in order to keep the
TPTP task jar named consistently with the other available Ant
tasks).
The test execution service's properties are set by prefixing the
property names with a hyphen making sure to also specify the Eclipse home or
root directory.
Running tests from Java applications and plug-ins
Tests can be launched from any Java application and in
fact, the other client automation adapters such as the Ant and shell adapters,
use the Java adapter internally to hook into the service bus. The only
requirement for executing tests from some Java code is to make sure the
tptp-automation-client.jar is properly referenced (in Eclipse this translates to
having the library in the Java project's build path) and of course all the
typical TPTP requirements apply as well, such as the agent controller must
be running.
The Java automation client adapter provides a very
simple API for executing tests with all specifiable properties being contained
in a standard Java properties object. An example follows for executing the
same test as introduced in the previous Ant script example. Note that the
service identifier passed in to the execute method is the service
identifier for the TPTP test execution service (this service supports all the
base TPTP test types) -- it is also possible to execute other automatable
services, if available, but varying the identifier and properties configured in
the code below.
// Create the Java adapter associated with the specified Eclipse home
AutomationClientAdapter automation = new AutomationClientAdapter
("d:\\development\\eclipse");
// Create and configure a properties object
Properties properties = new Properties();
properties.setProperty("workspace", "D:\\development\\workspace");
properties.setProperty("project", "tests");
properties.setProperty("suite", "unit-tests/testA.testsuite");
// Execute the service named below using the configured properties
automation.execute("org.eclipse.hyades.test.tools.core.execute", properties);
ASF services can also be invoked from within an Eclipse application, and
an in-process strategy is provided with ASF so that services invoked from within
Eclipse need not spawn a new Eclipse instance to be executed. If the services
you invoke are using the same workspace as your running eclipse instance, simply
use the Java AutomationClientAdapater with the no-arg constructor to use the
in-process strategy, as shown in the code below.
// Create the Java adapter using the in-process strategy when already in
// an Eclipse application
AutomationClientAdapter automation = new AutomationClientAdapter();
// Create and configure a properties object
Properties properties = new Properties();
properties.setProperty("workspace", "D:\\development\\workspace");
properties.setProperty("project", "tests");
properties.setProperty("suite", "unit-tests/testA.testsuite");
// Execute the service named below using the configured properties
automation.execute("org.eclipse.hyades.test.tools.core.execute", properties);
Related concepts
Overview of the automatable services framework
Related tasks
Launching tests from scripts and applications
Related references
Supported test execution service properties
