Starting expecco via Command Line/en

Introduction

expecco can be started manually via the shell command line, via batch/shell scripts or automatically via the crontab (Unix), scheduled tasks (Windows) mechanism of the operating system. Thus, you can have tests to execute unattended and without GUI "overnight". This is also the method to integrate expecco with other quality tools, such as Jenkins, HP-Quality Center, Polarion, expeccoNET etc.

Command line options allow for test suites to be loaded, executed and result files being generated without requiring user interaction. (the expeccoNET automation environment assumes that expecco is already running on each slave host, and each has been started with a "--server" command line argument. It uses SOAP to communicate with those expecco slaves by default).

Command Line

The simplest command line to execute a test suite is:

expecco --execute <testsuite filename> ...

<testsuite filename> is the name as saved from within the expecco GUI application. This should be a saved ".ets" testSuite file. If more than one test suite filename is given, each is processed in sequence. All top-level test plans as found in the test suite(s) are executed (see below for more options).

By default, the overall execution outcome is returned by the command's return value (exit code), which is

0 - all PASSED

non-zero - any ERROR, FAILURES or INCONCLUSIVE items.

The return value is useful if expecco is started via a makefile, or from jenkins for example. If the caller needs a different exit code, use one of the exitCodeOnXXX options as described below. This is useful to avoid a make or jenkins process from stopping.

Command Line Options

Command line options are handled at two places: the runtime system (Smalltalk/X VM) and the expecco application itself. When expecco is started, the VM is first to look for command line arguments and processes any of the VM options, until a non-VM option is encountered. It passes the remaining options to expecco's main entry. Thus, any option which is given after the first non-VM option will not be handled by the VM, but by expecco. Therefore, no VM options should be placed after the first non-VM option. VM options are marked as "VM option" in the list below.

Also, some options are specific to the operating system, on which expecco is executed. These are marked as "Windows", "Linux" etc.

Recommended Command Line Options

To start expecco on an existing test suite, use:

expecco <testsuite filename>

this is also the way expecco is started, when a test suite file is double-clicked on the Linux or Windows desktop. The above opens the expecco GUI and loads the given test suite initially. It does not execute any test.

To execute a suite, add a "--execute" argument, as in:

expecco --noBanner --noTray --verdicts --execute <testsuite filename>

or, if you want to use a special parameter-file:

expecco --noBanner --noTray --verdicts --parameter <param-file> --execute <filename>

To execute a suite without GUI, add a "--noWindow" argument.

To compose "scatter-gather" test cases from possibly multiple suites, found in a directory use:

expecco --suiteDirectory <dirName> --testCaseNames "T01,T02,T05" --execute

or, to explicitly specify the suite-files where test cases are taken from, use:

expecco --testCaseNames "T01,T02,T05" --execute <suiteFile1>.ets <suiteFile2>.ets ...

Finally, to get a report, add a "--xxxReportFile <fileName>" argument, as in:

expecco --noBanner --noWindow --noTray --verdicts --parameter <param-file> --pdfReportFile result.pdf --execute <filename>

All options are described in more detail below.

Option Description

Information/Debugging/Logging

• --help
  Prints an up-to-date list of possible options.

• --version
  Prints the expecco release identification and exits.

 

• --verbose (VM option)
  Additional startup, execution and result info is sent in human readable format to the stderr output. That may be too much of printout; try "--verdicts" instead.

 

• --noBanner (VM option)
  Suppress the splash startup banner.

 

• --console (VM option, Windows only)
  Open a debug console window for Stdout and Stderr.

 

• --debug
  Turns on startup debugging. By default, if an error occurs during startup, expecco terminates itself. With this option enabled, a command line debugger is opened instead. This is probably only useful for plugin developers.

 

Test Selection

the following options allow for individual test plans to be selected from a suite which contains multiple test plans:

• --testPlanName <namePattern>
  Only execute test plans with a matching name.
  For backward compatibility, "--testSuiteName <namePattern>" is still supported and has the same effect.

• --testPlanTag <tagPattern>
  Only execute test plans with a matching tag.
  For backward compatibility, "--testSuiteTag <tagPattern>" is still supported and has the same effect.

• --testPlanID <uuid>
  Only execute that particular test plan (repeat to execute multiple test plans).
  Starting with release 2.7, the following shortcut can also be used: --testSuiteID "<uuid1>,<uuid2>,...,<uuidN>".
  For backward compatibility, "--testSuiteID <uuid>" is still supported and has the same effect.

Scatter Gather Test Suite Composition

these options allow for a test plan to be composed out of a single or multiple of individual suite files. When composing, a new test plan is temporarily created, which contains the specified test cases. This is in contrast to the above options, where preexisting test plans are selected for execution. Typically, this kind of composition is used when expecco is called from a QA system, such as polarion or HP Quality Centre. (These options are only available in release ≥ 2.7)

• --suiteDirectory <dirName>
  defines the directory, where test suite files are searched for the composed test plan. Test cases as sepcified in one of the options below must be found in one of the files present there. Notice, that the first file which contains a requested test case action will be chosen. Thus, you should ensure, that only one version of each suite is present there. Typically, an up-to-date test suite directory as checked out from the revision control system is specified here. If this argument is not given, test cases are searched in the list of explicitly listed suite files (".ets" arguments).

• --suiteTitle <string>
  defines the name of the new test plan, which is created dynamically. This is used to control the labels in the generated report files.

• --testCaseIDs <id-list>
  defines the test cases to be included in the test by version ID. id-list is a list of comma-separated UUIDs of the individual test actions.

• --testCaseFIDs <id-list>
  defines the test cases to be included in the test by function ID. id-list is a list of comma-separated UUIDs of the individual test actions.

• --testCaseNames <name-list>
  defines the test cases to be included in the test by name. name-list is a list of comma-separated names of the individual test actions.

Test Execution

• --parameter <fileName>
  Load parameter set from filename. A parameter set contains values for a test suite's top environment. This is in XML format and can be generated with the "save-parameter-set" menu function, edited manually, or provided by a QM system (such as expeccoNET, HP Quality Center or similar), or be generated by another controlling program.

• --scriptFile <fileName>
  read and execute a script (see scripting below) from a file.

Reporting

• --verdicts
  Send execution and result info in human readable format to the stderr output. (see example below.)

• --logFile <fileName>
  Used to change the name of the generated log file, where stderr is written to.
  This is only useful if running in server mode.

• --exitCodeOnFail <n>
  Defines the exit code in case any test ends with a test-failure status.
  The default is 0 (zero), so that when you run expecco out of make, jenkins or other tools which check for the exit status, the calling program does not interpret this status as if expecco itself failed to execute. This option also changes the exitCodeOnError and exitCodeOnInconclusive values (see below), unless explicitly defined by those options.

• --exitCodeOnError <n>
  Defines the exit code in case any test ends with a test-error status.
  The default is 0 (zero), so that when you run expecco out of make, jenkins or other tools which check for the exit status, the calling program does not interpret this status as if expecco itself failed to execute. This option also changes the exitCodeOnInconclusive values (see below), unless explicitly defined by that option.

• --exitCodeOnInconclusive <n>
  Defines the exit code in case any test ends with an inconclusive status.
  The default is 0 (zero), so that when you run expecco out of make, jenkins or other tools which check for the exit status, the calling program does not interpret this status as if expecco itself failed to execute.

• --elfReportFile <elfFilename>
  Writes a report in expecoo's native XML logfile format. The report contains a lot of detail information and could be processed by an XSLT processor or a similar tool. It also contains enough additional information about test plans and actions so that expecco can redisplay the activity log tree from this file. The generated file can be opened directly via the command line ("expecco <file>.elf") or by double clicking on the elf-file in the operating system's desktop.

• --csvReportFile <csvFilename>
  Writes a summary report in CSV format. Each row consists of:
  
  • TESTSUITE - the filename
  • TESTPLAN - the test plan
  • TESTCASE - the test case
  • VERDICT - the test result; one of SUCCESS, FAILED, ERROR or INCONCLUSIVE
  • INFO - additional info in case of failure

Additional pseudo verdicts such as STARTTIME, ENDTIME etc. are written. A typical CVS output file can be found in the examples.

• --textReportFile <textFilename>
  Writes a summary report in a human readable textual format. A typical text output file can be found in the examples.

• --xmlReportFile <xmlFilename>
  Writes a report in XML format. The report contains a lot of detail information and could usually be processed by an XSLT processor or simular tool. A typical XML output file can be found in the examples.

• --pdfReportFile <pdfFilename>
  Writes a summary report in pdf format.

• --htmlReportFile <htmlFilename>
  Writes a summary report in html format.

• --junitReportFile <htmlFilename>
  Writes a summary report in a junit compatible XML format. Useful, when expecco is to be started by jenkins or similar driver programs.

• --polarionReportFile <htmlFilename>
  Writes a summary report in an XML format which is especially suited for the Polarion QM tool. The structure of the generated XML is the same as with the junit format above, but a slightly different time format is used.

• --reportTemplateName <name>
  Use the given report template (must be an element inside the test suite) for the following "--xxxReportFile" argument. If no such argument is given, the default template as defined in the suite is used.

Operation

Startup

• --noTray (Windows only)
  Disable the tray control icon (server and execute modes only).

• --noWindow
  Do not open the main window (useful when scripting or executing only).

• --noPlugins
  Disable plugin loading. This makes startup slightly faster. You can still force individual plugins to be loaded using the "--loadPlugin" option.

• --loadPlugin <pluginName>
  Force loading a specific plugin. This is only useful if a "--noPlugins" option was given before. pluginName is the name of the folder in the "plugin" folder (under the expecco installation folder).

• --noUpdateCheck
  If configured by the user's settings, expecco automatically checks for updates and patches when started (by checking the exept web-server for the presence of new patch files). This option disables that check, even if configured in the settings. Notice, that this is usually controlled by the user settings; however, in situations where settings are shared among multiple machines (network drive) and a test host has no (or should not have) internet access, it may be useful to disable this from the command line.

• --requireLogin
  Opens expecco in a (lightweight) multi-user mode, in which a username must be entered before the actual interaction with expecco begins. This user name will be used to load custom per-user settings (named ".expeccoSettings_<name>") and will also be used in the generated reports. Also, the "File" menu will contain an additional "Logout" menu item, which brings back the initial login dialog.
  This mode is useful, if expecco is used on a machine by multiple users, and expecco should remain opened (i.e. as a test stand). For example in a production floor, where tests are executed in multiple shifts by multiple operators.

• --settings <fileName>
  Force using <fileName> instead of the default "~/.expeccoPreferences" to be used for the settings file name. Useful if you want different setups to be linked to different desktop icons, or to start expecco with particular settings from a batch or shell script. To generate a preferences file, open the "File - Settings" dialog change the settings as required, and press the "Save As..." button.

Services

• --server
  Starts expecco in server mode. In this mode, HTTP and SOAP requests can be used to remote-control expecco's execution; especially, to execute tests, and upload results. The most obvious use for this is for expecco to act as an execution host for an expeccoNET or Polarion server.
  However, because standard protocols are used, this may also be used to interface the execution engine to many other QM management or automation systems (e.g. HP Quality centre).

• --port <portNr>
  Used to change the HTTP and SOAP port numbers, when running in server mode.
  The default is 9090 (server mode only).

• --info
  Provide an additional info service which allows for monitoring and control of an expecco server via a web browser (server mode only).

• --scripting <portNr>
  Enable scripting (remote control) on portNr (see more below).

• --allowHost <hostName>
  Allow hostName to connect to the scripting port. Repeat for multiple hosts (see more below).

Utility Functions

• --diff <file1> <file2>
  Print differences between two test suites. This can be used eg. to automatically generate log messages for a revision control system.
  (This function is only available in release ≥ 2.7)

Command Exit Status

• 0 - all tests passed

• >0 various other errors (command line, file not found, etc.)

• 101 - at least one test failed (FAIL status). This exitCode can be overwritten by a command line argument.

• 102 - at least one test was inconcusive (INCONCLUSIVE status). This exitCode can be overwritten by a command line argument.

• 103 - at least one test lead to an error (ERROR status). This exitCode can be overwritten by a command line argument.

The diff option returns 0 if the compared suites are the same, or 200+n, (where n is the number of differences) otherwise.

Sample Outputs

The following output files were generated by executing the test suite from the file:

"C:\Programme\exept\expecco\testsuites\projects\Example.ets"

which contains a single test plan named:

"Example Testplan"

which contains three test cases named:

"Test1"

"Test2"

"Test3-Should fail"

of which the first two are supposed to PASS and the last one generates an ERROR.

All timestamp information is written in ISO8601 format.

CSV File Example ("--csvReportFile" option)

By default, entries are separated by ";"-characters. Additional double quotes (") are used to delimit strings with embedded ";" or spaces. Any double quote within such a string is doubled.

TESTSUITE;TESTPLAN;TESTCASE;VERDICT;INFO
;;;StartTime;2008-10-31T08:57:13.689
"C:\Programme\exept\expecco\testsuites\projects\Example.ets";"Example Testplan";Test1;PASSED;  
"C:\Programme\exept\expecco\testsuites\projects\Example.ets";"Example Testplan";Test2;PASSED;  
"C:\Programme\exept\expecco\testsuites\projects\Example.ets";"Example Testplan";"Test3 - Should fail";ERROR;   
;;;EndTime;2008-10-31T08:57:18.746

XML File Example ("--xmlReportFile" option)

<SummaryReport>
  <StartTime>2008-10-31T08:56:49.774</StartTime> 
  <TestSuite>
    <File>C:\Programme\exept\expecco\testsuites\projects\Example.ets</File> 
    <TestPlan>
      <Name>Example Testplan</Name> 
      <TestCase>
        <Name>Test 1</Name> 
        <Verdict>PASSED</Verdict> 
      </TestCase>
      <TestCase>
        <Name>Test 2</Name> 
        <Verdict>PASSED</Verdict> 
      </TestCase>
      <TestCase>
        <Name>Test 3 - Should fail</Name> 
        <Verdict>ERROR</Verdict> 
      </TestCase>
    </TestPlan>
  </TestSuite>
  <EndTime>2008-10-31T08:56:55.142</EndTime> 
</SummaryReport>

Textual File Example ("--textReportFile" option)

Test Execution Summary
  Start: 2008-10-31 08:56:12.751
  TestSuite: "C:\Programme\exept\expecco\testsuites\projects\Example.ets
    TestPlan: Example Testplan
      Test 1 ....................... PASSED
      Test 2 ....................... PASSED
      Test 3 - Should fail ......... ERROR
  Finish: 2008-10-31 08:56:18.009

Verdicts Example

the "--verdicts" option generates a simple trace on the standard error (to be redirected into a file):

C:\selftest\commandLineArgs> expecco --noBanner --noTray --verdicts 
--parameter paramesForTest.xpar --execute parameterTest.ets

Expecco [info]: start testSuite "parameterTest.ets"...
Expecco [info]: using parameters "paramesForTest.xpar"...
Expecco [info]: start testPlan "Test Parameters From CommandLine"
Expecco [info]: start testCase "Validate Parameters"
Expecco [info]: done testCase "Validate Parameters"; STATUS=PASSED
Expecco [info]: done testPlan "Test Parameters From CommandLine"; STATUS=PASSED
Expecco [info]: done testSuite "parameterTest.ets"; STATUS=PASSED

Integration with Quality Managament and Automation Tools

Integration with Jenkins

To have expecco tests being executed automatically by Jenkins, take the following example as a guideline. This is a real world setup, taken from our own setup within eXept.

1. Create a test job, which is triggered by a successful build (or a change in the test suite repository)
2. Check the "Block Build while prerequisite Jobs are Active" flag
3. Check either the "Trigger from Source Code Management" or the "Trigger from other Job" flags
4. Check "Abort build if it is Stuck" (just to make sure, that a faulty suite does not block Jenkins)
5. Check "Fail the Build if Stuck" and give it a reasonable elastic Time-out Strategy (we use 300% of the last 10 builds, to make sure a longer execution time due to more test cases will not be interpreted as a stuck) and an extra timeout of 60 minutes. The timeout is very generous for the expected execution time of a few minutes.
6. Check "Run xvnc during Build" to allow for monitoring, what is going on
7. Define the "Build Procedure" as "Execute Shell Script" (Unix/Linux) or "Execute Batch Job" (Windows)
8. Enter a build procedure similar to the ours, which is:

   cd exept\expecco\application

   del *.junit

   expecco.com --noBanner --exitOnInternalError --debug --noTray \

   --jUnitReportFile selfTestResult.junit \

   --execute ..\projects\expecco\eXpeccoSelfTest.ets

   expecco.com --noBanner --debug --noTray \

   --jUnitReportFile CommunicationWithExpeccoNETResult.junit \

   --execute ..\projects\expecco\CommunicationWithExpeccoNET.ets

   The details (especially pathes and names) will of cause be different in your setup. The above uses the relative pathes as it first changes the current directory to where the previous build has compiled the executable, which will of course not be the case in your setup.

9. Define a "Post-build Action"
10. Check "Publish Junit Test Results"
11. Enter a path expression to declare the location of the resulting reports. Here, we used:

    exept\expecco\application\*.junit

12. Check the "Publish Test Attachments" flag, so the results will be presented in the Jenkins management console

Thats it! Because expecco has been told to generate jUnit compatible report files, we can use Jenkin's existing jUnit support. However, the jUnit xml format does not support the full range of information which is present in the native expecco result files (especially the activity log and pin value information is not included). Therefore, it is a good idea to also generate a regular ".elf" (expecco log file) and add it to the archived build artefacts.

Integration with expeccoNET

--- to be written

Integration with HP Quality Center

This is described in the HP Quality Center Plugin document.

Integration with Polarion

This is described in the Polarion Plugin document.

Scripting

Expecco can be scripted either by reading a script from a file or remote controlled via a telnet like command-line interface. This is useful to execute complex test scenarios and meta tasks, to create or manipulate test suites automatically and for self-testing expecco by using another expecco to control its GUI. It can also be used to execute and automate more complex scenarios without using expeccoNET.
Example uses are:

• Sequentially load and/or execute all suites found in a folder, reimport libraries, save the updated suite
• Construct new suites by merging/selecting tests from other suites
• Nulk renaming, commenting, attributing etc.
• Change attachments inside a number of suites
• Controlling an application or web interface (via a GUI plugin) and saving a sequence of screen shots

By default, the scripting interface expects expressions in JavaScript syntax (but with a Smalltalk object model), similar to the language used for elementary block definition. However, the language can be changed to Smalltalk syntax via a "~" directive.

Script File Example

To control expecco via a scriptfile, use the "--scriptFile <filename>" command line argument, where <filename> must be a file containing JavaScript expressions.

For example, the following script file named "run3Suites.js" may contain the code:

    // Example for an automatic expecco script.
    // (loads and executes a number of test suites)
    //

    // to enable debugging in case of an error.
    // STXScriptingServer.errorDebugging(true);

    var files = [
        "foo.ets",
        "bar.ets",
        "baz.ets",
    ];


    Expecco::ExpeccoPreferences.current.automaticReimportOnLoad(true);
    Expecco::ExpeccoPreferences.current.checkForReimportableImportsOnLoad(true);
    Expecco::ExpeccoPreferences.current.checkForAnchestorOnAutomaticReimport(false);
    // Expecco::ExpeccoPreferences.current.libraryPath(importPathes);

    // --------------------

    function executeTestPlan ( aTestPlan ) {
       Stderr.nextPutLine("executing testplan "+aTestPlan.name()+"...");
       resultList = aTestPlan.execute();
       Stderr.nextPutLine("done.");
     }

    function runSuite(suiteFile) {
        var loadedProject;

        Stderr.nextPutLine("loading "+suiteFile+"...");
        expecco.loadProjectFromFile(suiteFile);
        loadedProject = expecco.project();

        if (loadedProject == null) {
            Stderr.nextPutLine("failed to load library "+suiteFile+".");
            Smalltalk.exit(1);
        } else {
            loadedProject.allTestPlansDo( executTestPlan );
        }
    };

    files.do(runSuite);

And can be executed with:

   expecco --scriptFile run3Suites.js

for windowless operation, use the "--noWindow" command line argument. To speed up the startup, you can turn off the loading of plugins, with a "--noPlugins" argument. Finally, enable debug messages with a "--debug" argument. Thus, the recommended options for scripting are:

   expecco --debug --noPlugins --noWindow --scriptFile run3Suites.js

of course, this command line can be put into a shell (or batch) script file, and then executed with a single word command.

Telnet Scripting Example

The remote scripting service is enabled when a "--scripting" command line argument is given. By default, the service only accepts connections from the local host (for security reasons). Using the "--allowHost" option, more hosts can be gained access to the scripting feature - but be aware, that this opens a potential security hole, as all of the internal functions can be accessed via the scripting system (and this includes the possibility to write files and call operating system services with the access rights of the one who started the scripting server).

When used with a remote script connection (telnet), additional connection- and execution related commands are introduced by lines starting with the "~" (tilde) escape character (similar to telnet). For example, "~." (tilde-period) is used to close the connection. Press "~?" (tilde-question) for help.

The following is a trace of a typical scripting session (user input in bold):

c:\programs\expecco> expecco --scripting 8008
c:\programs\expecco>     ... expecco now running in the background with GUI ...
...
c:\programs\expecco> telnet localhost 8008
Welcome to Expecco (Type ~? for help)
> println(expecco);
an Expecco::Browser
> expecco.menuNewProject();
...expecco performs the new-function from its menu...
> expecco.loadProjectFromFile("C:\Users\cg\testsuites\webedition\projects\BearerTest.ets");
...expecco loads the testSuite...
> expecco.close();
> ~.
Lost Connection.
c:\programs\expecco>

You can also start a scripting session without an initial expecco browser window:

c:\programs\expecco> expecco --noWindow --scripting 8008
c:\programs\expecco>     ... expecco now running in the background without GUI ...
...
c:\programs\expecco> telnet localhost 8008
Welcome to Expecco (Type ~? for help)
> println(expecco);
an Expecco::Browser
> expecco;
> ~v+ // turn result-printing on
> expecco
an Expecco::Browser
> ~v- // and off again
> expecco.loadProjectFromFile("~/SomeTest.ets");
...expecco loads the testSuite...
> var blk = expecco.project.elementWithName("My Action");
> blk.name("My Fun Action");
> expecco.saveProjectToFile("~/ModifiedTest.ets");
> expecco.close();
>~.
Lost Connection.
c:\programs\expecco>

Scripting API Overview

A few objects are predefined by name; among them are "expecco", which refers to the browser itself (i.e. the UI) and "environment", which refers to a collection of variable bindings (this is a dictionary of interpreter variables - not to be confused with a variable environment as known inside an expecco activity block).
New variables can be defined with a "var <name>;" statement. For example:

    var ws;

    ws = new ExpeccoWorkspaceApplication;
    ws.open();
    ws.close();

expecco Object

The variable named "expecco" refers to the opened GUI-object, which represents the expecco application itself.

Expecco Functions

• project()
  the current project
• currentApplication()
  the current page (initially, there is only one)
• currentTreeApplication()
  the current tree sub-application

• selectModelItem( anItem )
  select an item and open an editor for it
• selectedModelItems()
  returns a collection of selected items

 

Menu Functions

These execute the underlying menu functions (as if clicked by the user).

• menuNewProject()
  creates a new testSuite
• menuOpen()
  opens a dialog requesting a testSuite
• menuExit()
  closes the expecco application

• loadProjectFromFile(fileNameString)
  load a testSuite
• saveProjectToFile(fileNameString)
  save the testSuite
• selectTestPlanWithName(testPlaneNameString)
  select a testplan item by name
• selectTestPlanWithFunctionalId(uuid)
  select a testplan item by uuid

• testSuite()
  retrieves the testSuite object (see below)

 

Test Suite (Project) Functions

the project as returned by the above "project()" function provides access to its elements:
(see also Expecco API#Test Suite Project Functions)

• authorName()
  get the author name string
• authorName(aString)
  set the author name string
• versionString()
  get the version string
• versionString(aString)
  set the version string
• projectsWorkingDirectory()
  the directory, where attachments etc. are found
• ensureEnvironment()
  an instantiated environment (concrete values). See below.
• elementWithName( nameString )
  returns a matching element or null if not found
• elementWithId( aUUID )
  returns that element or null if not present. (create the UUID with "<stringConstant>.asUUID" )
• elementWithFunctionalId( aUUID )
  returns that element or null if not present

 

Environment Object

Environment objects are used to store key-value bindings for expecco variables. Environments form a hierarchy, where values are searched for in an environment's parent environment, if not present. This hierarchy follows the containing compound activity chain up to the project (= test suite) environment. Additional temporary environments are kept by the executor and the browser, to keep temporary values for a single execution's or a single session's lifetime.

• at(key)
  retrieves a value, given the name of the entry. Raises and error if no variable exists by that name.
• at_ifAbsent(key, default)
  like above, but returns default if no variable exists by that name (instead of raising an error).
• includesKey(key)
  true if a variable by that name exists, false otherwise.
• at_put(key, value)
  store value as variable named key. Raises an error, in an attempt to change a readOnly variable
• environmentDescription()
  returns the environment's description. This describes types, default values and read/write attributes per entry.

 

EnvironmentDescription Object

This is a collection of entries which describe variables of an environment. It does only hold the description, not the bindings themself; i.e. it contains the information required to instantiate an environment. EnvironmentDescriptions inherit from OrderedCollection and therefore entries are accessed via "at:" usinf a numeric index from 1 to the environmentDescription's size.

Entries (as per variable) provide the following protocol:

• name()
  the name of the variable
• isReadOnly()
  true if readonly
• defaultValue()
  the initial value, assigned when the environment is created
• datatype()
  the type of the variable
• comment()
  description

 

Remote Controlling Expecco

Expecco testruns can be automated by various scripting mechanisms:

• via command line arguments (described in "Command Line" above)

• via script files (see "Script File Example" above)

• via the telnet interface (see "Telnet Scripting Example" above)

Or, by standardized service interfaces, which are meant for programmatic clients (Quality Management and other scheduling tools). Our own companion product, ExpeccoALM and third party tools, such as Polarion also use these service interfaces.

• via a SOAP service interface

• via a Rest service interface

The service interface is especially meant for unmonitored slave machines in the network, among which expecco test execution jobs are scheduled by another program. For this, the basic operations download of a suite, executing a suite and requesting execution results are available via service calls.

Both SOAP and Rest services are available, if expecco is started with the "--server" argument. An optional "--port" argument allows for the HTTP port to be changed from the default (9090) on which expecco is listening. Alternatively, the service can be controlled via expecco's main menu (in the "Extras" - "Web-Services" - "Start / Stop" menu items).

These services use HTTP requests as transport layer.

Expecco SOAP Service Interface

This accepts SOAP requests, via the URL "/expeccoService". I.e. the default URL is "host:9090/expeccoService". A WSDL spec can be aqcuired from the running service via "host:9090/wsdl/ExpeccoSOAPService.wsdl".

The service entries are:

Loading and Executing a Suite

The execute request is the main service entry. It passes the suite and also optional parameters.

The test suite can be bade available to expecco either by passing it with the request (as base64 CDATA), or by passing a URL from which expecco can fetch the suite iteslf.

The execute call contains the following fields:

<message name="expecco_executeProject_v5_Request">
   <part name="id" type="xsd:string"/>
   <part name="testSuiteId" type="xsd:string"/>
   <part name="dataSource" type="xsd:string"/>
   <part name="dataSourceUri" type="xsd:string"/>
   <part name="environment" type="xsd:string"/>
   <part name="testplans" type="xsd:string"/>
   <part name="resources" type="xsd:string"/>
   <part name="optionalParameters" type="xsd:string"/>
 </message>

an responds with:

 <message name="expecco_executeProject_v5_Response">
   <part name="result" type="xsd:string"/>
 </message>

This call starts execution, and immediately returns a response, which consists of an identifier. This identifier can later be used to ask expecco about the execution status. As a test may take a long time to finish, it is useful to poll via status requests from time to time, to see if the execution has finished and to get progress status.

The parameters are:

• id - mandatory; a ticket ID; this should be a UUID-string, to avoid possible conflicts
• testSuiteId - mandatory; the UUID of the suite to execute. See below for details about download optimizations
• dataSource - optional; if present, this should contain the whole ets-file's contents as a Base64 encoded CDATA string
• dataSourceUri - optional; if present, this should be the URI from which expecco shall fetch the suite. See below for details
• environment - optional; top-level environment variable values. This can be used to provide alternative initial values for variables in the top level variable environment (the "Project-Environment")
• testplans - optional; a list of UUIDs of testplans to execute. If not present, all top-level testplans are executed
• resources - optional; a list of resources to be used by the test (see expeccoALM resource description)
• optionalParameters - optional; a dictionary containing additional parameters

The reply consists of the ticket-id, which has to be passed in further status requests.

Loading Only

It is also possible, to download a suite without execution, via the "download" request:

 <message name="expecco_downloadProject_Request">
   <part name="testSuiteId" type="xsd:string"/>
   <part name="dataSourceBase64" type="xsd:string"/>
   <part name="dataSourceUri" type="xsd:string"/>
   <part name="optionalParameters" type="xsd:string"/>
 </message>

 <message name="expecco_downloadProject_Response">
   <part name="result" type="xsd:string"/>
 </message>

Download optimization

Expecco remembers downloaded suites in a temporary folder, to avoid repeated transfers of possibly huge test suites. If a URI is passed (but no base64 encoded data), expecco checks if the suite is already present on the test host, and will not fetch it again, if already present. If not present, expecco will fetch the suite via a HTTP request from the URI.

If no URI is passed (i.e. both data and dataURI fields are empty), expecco will either respond with an error reply, if not present, or with an ok response, if already present.

Thus, if you have to pass the suite as data (i.e. there is no file service available, from which expecco could fetch the suite), it s recommended to first check if a suite is already present (by sending a download request without data) and check for an error return. If no error is reported, the suite is already there and no further download action is required. If there is an error response, the client should send the suite with another download request, this time with a non-empty data field.

After any successfull download (either via download, or via execute request), the suite can be executed by an execute request without data.

Status Request

This query returns status information about an ongoing execution. Its single parameter field must contain a ticketId as described in the above execute request.

 <message name="expecco_getExecutionInfo_Request">
   <part name="ticketId" type="xsd:string"/>
 </message>

 <message name="expecco_getExecutionInfo_Response">
   <part name="result" type="xsd:string"/>
 </message>

A client may have to send multiple "getExecutionInfo" requests, until a "finished" status is returned. Expecco will keep the generated reports and log files in a temporary folder, until fetched via a getResult request. This allows for both expecco and the client to be turned off and restarted, without loosing any status information.

Result Request

Once finished, reports and log files should be fetched from expecco.

 <message name="expecco_getResult_Request">
   <part name="ticketId" type="xsd:string"/>
   <part name="removeResultBoolean" type="xsd:string"/>
 </message>

 <message name="expecco_getResult_Response">
   <part name="result" type="xsd:string"/>
 </message>

After the "getResult" request, corrsponding files are automatically removed from the temporary folder by expecco.

Terminate/Abort Request

Use this, to abort and terminate an ongoing test execution.

 <message name="expecco_killExecution_Request">
   <part name="ticketId" type="xsd:string"/>
 </message>

 <message name="expecco_killExecution_Response">
   <part name="result" type="xsd:string"/>
 </message>

Remove Ticket Request

Use this, to tell expecco, that no further interest exists in a ticket. If the test is still running, it is aborted. If it is about to be started, it will not be. Any temporary files which might have already been created due to this execution are removed.

 <message name="expecco_killExecution_Request">
   <part name="ticketId" type="xsd:string"/>
 </message>

 <message name="expecco_killExecution_Response">
   <part name="result" type="xsd:string"/>
 </message>

Ping

This request answers with information about the host, architecture, disk usage and other status about the machine on which expecco is running. Of course, it is also useful to see if the expecco service is ready and the communication works as expected.

 <message name="ping_Request">
 </message>

 <message name="ping_Response">
   <part name="result" type="xsd:string"/>
 </message>

Cleanup of Old Result/Log Files

This request can be used to remove all temporary files, especially leftover report- and log files. This should be used, if a client has crashed, and lost track of

 <message name="expecco_cleanupTempFiles_Request">
 </message>

 <message name="expecco_cleanupTempFiles_Response">
   <part name="result" type="xsd:string"/>
 </message>

Expecco Rest Service Interface

This accepts Rest requests, via the URL "/expeccoService/rest". I.e. the default URL is "host:9090/expeccoService/rest". A description of the supported Rest calls be aqcuired from the running service via "host:9090/expeccoService/rest/protocolInfo".

 

Back to Online Documentation

---

Copyright © 2014-2024 eXept Software AG