ElementaryBlock Element/en

Aus expecco Wiki (Version 2.x)
Wechseln zu: Navigation, Suche

Inhaltsverzeichnis

Introduction

Elementary blocks provide the basic low-level functionality of an expecco testsuite. They are used for operations which are not broken down into more elementary subactivities, and which are therefore not representable as an activity diagram ("Compound action"). Typically, operations described as elementary action block are numeric calculations, low level string, stream or collection operations, interfaces to protocols or to external devices. Especially domain specific low-level interfaces to local or remote processes, document converters, databases and hardware devices are implemented as elementary block.

Also, elementary actions are typically executed much faster than compound (diagram) actions. Therefore, performance critical operations are sometimes written as elementary blocks.

Languages

The behavior of an elementary block is defined by one of the following:

  • a piece of textual program code, which executes inside expecco, and is written in one of the directly supported and builtin programming languages (currently: Smalltalk or a JavaScript dialect)
  • a piece of code, written in Groovy, which executes in a Java Virtual Machine (JVM) on the local system, a remote system or inside the system under test (SUT) itself (requires extra Java plugin)
  • a piece of code, written in JavaScript, which executes in a Node.js interpreter on the local or a remote system (bridged or script)
  • a piece of code, written in Python (IronPathon), which executes in a .NET Virtual Machine (CLR) on the local system, a remote system or inside the system under test (SUT) itself (requires extra DotNET plugin)
  • a piece of code, written in C# (CSharp), which executes in a .NET Virtual Machine (CLR) on the local system, a remote system or inside the system under test (SUT) itself (requires extra DotNET plugin)
  • a piece of code, written in VisualBasic Script, which executes in a separate script interpreter process (requires extra VB plugin)
  • a piece of code, which is executed by an external scripting language interpreter (shell, batch, ruby script, gnuplot or any other scripting language)
  • a gnuplot script to generate plots and graphs
  • an 'R' language script; both for statistical analysis and to generate plots and graphs
  • a call to a DLL (shared library) function (which is often, but not required to be) written in C or C++
  • a call to an external program
  • a remote procedure or service call (SOAP, REST, XML-RPC, Sun RPC)

Builtin Languages

Expecco provides built in language interpreters and compilers for two scripting languages: the builtin Smalltalk and a builtin JavaScript-like scripting language. When such an elementary block is executed, the script code runs inside the expecco process. These support high level symbolic debugging including single-step, breakpoints or inspection of variables. The debugger even allows for code changes to be done to the running program (for example, while the program is stopped at a breakpoint). These internal scripts are transparently and automatically compiled into machine code when first executed ("hot swap" and "Just in Time" (JIT) compilation). Changed code will be executed when the function/block is called the next time; thus, there is a limitation, in that a suspended function's code has to either proceed and return or be aborted for the new code to become effective (but only that particular function/elementary block - not the whole execution).

These builtin languages can use the full power of the underlying class library, (part of) which is described in "Expecco_API".

Additional builtin scripting languages (Ruby and Scheme) are being prepared and planned for future releases.

Bridged Actions

In addition to script execution (described below), expecco also allows "Bridged Execution" of code which is loaded into and executed in an external language interpreter. In contrast to script execution, the bridge-partner is a running program which communicates with expecco. Any state created there will remain alive as long as the bridge-partner and the communication channel to it are still alive. Thus, code can be defined in expecco to be execute inside a running "Java", "C#" or "Node.js" program. Bridged action blocks provide a pin-interface similar to the above described builtin actions, where values are passed to and from the action. However, not all bridges support all datatypes; some bridge-language specific limitations may apply. Also, notice, that any objects which were allocated (instantiated) in a bridge-partner will become void, when either the bridge partner is terminated or the connection to it is broken. Of course, this also means, that bridge objects cannot be recovered from the execution log (only the printed representations will be stored in it).

Groovy Language

With the Java Bridge extension plugin, elementary blocks can also be written in Groovy, which is a scripting language with a syntax similar to Java, and which is transparently (and dynamically) compiled to Java bytecode, and executed inside the SUT or on the expecco host or on any other such configured machine in the local network. Groovy blocks are very useful to define Java callbacks, subclasses of existing Java classes or to create and manipulate complicated Java objects inside a tested Java application. When such a block is executed, the script code runs inside a Java VM, which can be either a local one, dedicated to script execution, or on the target system (even inside the SUT, if it is written in Java). More detail is found in the Expecco API Documentation.

IronPython and C# Languages

With the DotNET Bridge extension plugin, elementary blocks can also be written in IronPython or C#. Similar to the above Groovy code, this is transparently (and dynamically) compiled, and executed inside the SUT or on the expecco host or on any other such configured machine in the local network. DotNET blocks are useful to access internals of the SUT or to interface to systems for which DotNET bindings (libraries) are available. More detail is found in the Expecco API Documentation.

VBScript Language

With the VisualBasic Plugin, elementary blocks can be written in the VisualBasic Script syntax and are executed in an external VisualBasic interpreter. This may run on the local host (the one on which expecco runs) or optionally on a remote host. The host on which the script engine executes must be a Windows host. For expecco users working under Unix/Linux or OSX, this means that at least a virtual (or real) machine with a Windows system must be setup and reachable in the network, and that the Windows machine serves as a script execution server for the Unix machine.

Script Languages

Other scripting languages (Ruby, Perl, TCL, Python, Gnuplot etc.) can be called via external script interpreters. It is also possible to call existing command line scripts (Shell under Unix or Windows+cygwin, Batch under Windows).

Code Editing

The source code is usually edited in the code editor. It is interpreted according to the used language’s syntax at runtime. For the builtin languages Smalltalk and JavaScript, it is possible to switch the syntax pattern for already implemented blocks, causing the code to be automatically converted to that particular syntax style. The values of data pins can be directly accessed (read and write) in the code using getters/setters on the pin's name. The external interface of an elementary block (e.g. number of pins) is defined in the schema editor.

Notice that the standard library as delivered already contains a number of useful elementary blocks for a wide range of applications. So that the need to define your own elementary blocks is often limited to special interfacing or specialized processing of data. Depending on the testing domain (technical/business/UI-testing), many expecco users even do not need to write any elementary actions and can realize their tests using diagrams only - this is especially the case in the area of UI testing. On the other side, others may depend heavily on the ability to interact directly with deep internals of the SUT, for example to call Java functions with complex object interfaces. Expecco tries hard to provide a convenient framework for both ends of this spectrum, by integrating elementary code development seamless into the test development process.

Smalltalk and JavaScript Blocks

The following examples are meant as tutorial and introductory examples. They are written to be easy to understand, not to use the full power of the underlying languages (eg. some could be written much shorter, by using more sophisticated existing library functions).

Don't worry: you will not need to go down to that low level programming to get started with expecco. All of the functionality presented below is already present in the Standard Library and can be used "out of the box". However, it may be good to know, that there are virtually no limitations in what you can do in expecco, and that whenever you need an additional function which the Standard Library did not foresee, it can be added in a few seconds.

Example: Simple String Processing

The first example simply takes a string and cuts off some characters at either side, and returns some middle part of it. To make the example more interesting, the parts to be cut off shall be the first 5 characters at the left, and everything after the last separator (blank character or tab). We first define the external interface of that action block to consist of an input-pin named "inString" and an output-pin named "outString". Both pins are declared as being of "String"-datatype, this ensures, that when placed into a diagram, only other String-typed pins can be connected to it. The so called "schema" definition of the action block is therefore:

Datei:String example elementaryBlock.png

Then, write the code, either in Smalltalk as:

execute
    |inStringValue|

    "/ fetch the data value from the pin
    inStringValue := inString value.

    "/ find the last separator in the string
    indexOfLastSeparator := inStringValue lastIndexOfSeparator.

    "/ extract the middle part
    middlePart := inStringValue copyFrom:6 to:(indexOfLastSeparator-1).

    "/ write to the output pin
    outString value: middlePart

The same operation written in JavaScript as:

execute() {
    var inStringValue;

    // fetch the data value from the pin
    inStringValue = inString.value();

    // find the last separator in the string
    indexOfLastSeparator = inStringValue.lastIndexOfSeparator() + 1; // see below

    // extract the middle part
    middlePart = inStringValue.copyFrom_to(6, indexOfLastSeparator-1);

    // write to the output pin
    outString.value(middlePart);

There is one possible pitfall, when using JavaScript: JavaScript array-indexing is 0-based, whereas Smalltalk uses 1-based indices. This applies to all standard JavaScript functions (of which "indexOf" is one). However, when Smalltalk functions are called from JavaScript (as is the case with "copyFrom_to" above), we have to adjust the index.


You should always provide a little test and demo of your elementary actions in the Test/Demo page. There, place at least one instance of the block as a step, give it its required inputs and execute it there. For example:

Datei:String example elementaryBlock test.png

Example: Number Reading

The next example takes a string and reads three numbers from it, which it returns as a 3 element vector (tuple). This could be useful as part of a reader for test data, or to process user input. The three numbers will be separated by spaces or tabs. It has a String-typed input pin named "inString" and an Array-typed output pin named "numberVector". It will report an error if not exactly 3 numbers are contained in the string:

execute
    |inStringValue inStream n1 n2 n3|

    "/ fetch the string from the pin
    inStringValue := inString value.

    "/ get a read-stream on the string's characters
    inStream := ReadStream on:inStringValue.

    "/ read the first number
    n1 := Number readFrom:inStream onError:[ self error:'error reading the first number'].

    "/ and so on...
    n2 := Number readFrom:inStream onError:[ self error:'error reading the second number'].
    n3 := Number readFrom:inStream onError:[ self error:'error reading the third number'].

    "/ check, if the end of the stream has been reached (e.g. nothing more is there)
    "/ but allow additional spaces
    inStream skipSeparators.
    inStream atEnd ifFalse:[
        self error:'garbage after the third number'
    ].

    "/ create a vector and write it to the output pin
    outString value: { n1 . n2 . n3 }

The same operation written in JavaScript as:

execute() {
    var inStringValue, inStream, n1, n2, n3;

    // fetch the data value from the pin
    inStringValue = inString.value();

    // get a read-stream on the string's characters
    inStream = ReadStream.on(inStringValue);

    // read the numbers
    n1 = Number.readFrom_onError(inStream, function() { this.error("error reading the first number");});

    n2 = Number.readFrom_onError(inStream, function() { this.error("error reading the second number");});

    n3 = Number.readFrom_onError(inStream, function() { this.error("error reading the third number");});

    // check, if the end of the stream has been reached (eg. nothing more is there)
    // but allow additional spaces
    inStream.skipSeparators();
    if (! inStream.atEnd) {
        this.error("garbage after the third number");
    }

    // create a vector and write it to the output pin
    outString.value( [ n1 , n2 , n3 ] );

Please notice the inner function argument in the above code, which is the error call back argument to the "Number readFrom_onError" call. In Smalltalk, this is simply written as a block "[...]", whereas in JavaScript the ugly "function() {...}" syntax is needed. This is one of the examples, where Smalltalk code is much easier to write and understand (at least to those who have not been conditioned to C/Java style language syntax).

Example: Minimum and Maximum of an Arbitrary Function

Here is a block to compute the minimum and maximum values of a mathematical function over a range of x-values. It is especially flexible, as the name of the function can be passed as an argument (for example: 'sin', 'cos' or 'arcTan'). The block's definition is:

Min max elementaryBlock.jpg

and its code could be written in Smalltalk as:

execute
    |fn minY maxY y|

    fn := functionName value asSymbol.

    (x0 value) to:(x1 value) by:(dX value) do:[:x |
        y := x perform:fn.
        minY := minY isNil ifTrue:[y] ifFalse:[ minY min:y ].
        maxY := maxY isNil ifTrue:[y] ifFalse:[ maxY max:y ].
    ].

    min value:minY.
    max value:maxY.

or in JavaScript as:

execute() {
    var fn, minY, maxY, x, y;

    fn = functionName.value().asSymbol();
    for (x = x0.value(); x <= x1.value(); x += dX.value()) {
        y = x.perform(fn);
        minY = (minY.isNil()) ? y : minY.min(y);
        maxY = (maxY.isNil()) ? y : maxY.max(y);
    }
    min.value(minY);
    max.value(maxY);
 }

(notice the use of the "perform"-function, to call a function by name)


Of course, as usual, there are multiple ways to write the same function; an experienced Smalltalker would write:

execute
    |fn minMax|

    fn := functionName value asSymbol.

    minMax := ((x0 value) to:(x1 value) by:(dX value))
                  collect:[:x | x perform:fn])
                      minMax.
    min value:minMax first.
    max value:minMax second.

and a JavaScript programmer might prefer:

execute() {
    var fn, minY, maxY, x, y;

    fn = functionName.value;
    for (x = x0.value; x <= x1.value; x += dX.value) {
        y = x.perform(fn.asSymbol());
        minY = (minY == null) ? y : Math.min(minY,y);
        maxY = (maxY == null) ? y : Math.max(maxY,y);
    }
    min.value(minY);
    max.value(maxY);
 }


The block could be used to calculate the min/max of the sine function in the range 0..1 as follows:

Min max in use with sine.jpg

Example: Multi-Setter with Variable Number of Input Pins

The following defines a block to set multiple key-value associations in a dictionary object. In the schema, the last two input pins were defined as a variable-pin-group (using the second to last pin's "Special Attribute" menu). Therefore, the last two pins can be replicated an arbitrary number of times in a step. The block's elementary code must of course be prepared for this, and loop over the step's number of input pin values.
The block's definition is:

MultiSetter elementaryBlock.png

and its code could be written in Smalltalk as:

execute
    |dictionary|

    dictionary := dictionaryInPin value.
    1 to: keyPin numberOfVariablePins do:[:index |
        |eachKey eachValue|

        eachKey := keyPin valueAt:index.
        eachValue := valuePin valueAt:index.
        dictionary at:eachKey put:eachValue.
    ].
    dictionaryOutPin value:dictionary.

or in JavaScript as:

execute() {
    var dictionary;

    dictionary = dictionaryInPin.value;
    for (var index = 1; index <= keyPin. numberOfVariablePins(); index++) {
        var eachKey, eachValue;

        eachKey = keyPin.valueAt(index);
        eachValue = valuePin.valueAt(index);
        dictionary[eachKey] = eachValue;
    }
    dictionaryOutPin.value( dictionary );
}

A step to set 5 key-value pairs would look as follows (input pin group replicated 4 more times):

MultiSetter use with 5 groups.png

Bridge Action Blocks vs. Script Action Blocks

Both bridge actions and script actions are executed in an external program or language interpreter. In contrast to script actions which are described below, a bridge partner is started once (or an already running program), and expecco communicates with this running program via a communication mechanism (typically a socket connection). The bridge partner is kept alive between bridge action calls.

Script actions on the other side are independent actions, where the language interpreter is in a fresh state for every call.

Thus, bridge actions can refer to state and definitions which were created in previous calls, whereas script actions cannot.

Groovy Blocks

A Groovy action is a bridge action and its code is executed in a Java Virtual Machine (JVM), either locally, on a remote system or inside the system under test (SUT). It is possible to define new classes, new functions and to create objects which persist between Groovy calls. It is also possible to define callbacks and other functions which lead back into expecco. Groovy actions are described in more detail in the Expecco API documentation on Groovy .

C# Action Blocks

Similar to Groovy actions, these are executed inside a .NET Virtual Machine (CLR), either locally, on a remote system or inside the system under test (SUT). It is possible to define new classes, new functions and to create objects which persist between C# calls. It is also possible to define callbacks and other functions which lead back into expecco. C# actions are described in more detail in the Expecco API documentation .

Node.js Blocks (Bridged)

A Node.js block's code is executed in a Node.js interpreter; either locally or on a remote system. These are similar to Groovy actions, but the function is executed by a Node.js interpreter. You have to ensure that an appropriate node interpreter is installed on the machine (it is not part of the expecco delivery package).

The node interpreter path can be either be left unspecified, or specified in the expecco settings ("external tools"), or via an expecco environment variable named "NODEJS_SHELL"). The environment variable can be the global project environment, or the global test suite environment. If left unspecified, or without an absolute path, the node interpreter should be found along the PATH.

The Node.js API is described in the Expecco API documentation on Node.js.

Notice, that both bridged Node.js actions and scripted Node.js actions are supported by expecco.

VisualBasic Blocks

A VisualBasic Script block's code is executed by a scripting host (see Microsoft documentation on Visual Basic). It is described in more detail in the Expecco API documentation on VisualBasic and the VisualBasic Plugin documentation.

Script Action Blocks

Script actions are executed by an external language interpreter. Beside the basic Shell- and Batch-Script (Windows) actions, a number of script languages are specially supported. Although any external script in any script language can be called by using appropriate command line arguments (i.e. using a Shell-Script action with a command which calls the language interpreter), these specially supported languages present themselves with a special icon, support syntax highlighting, and provide better error diagnostics (in particular: error-line highlighting).

A demo suite with various script actions is found in the examples/demos folder named "d45_ExternalScriptElementaryActionsDemo.ets".

Script Execution in General

The following sections describe the mechanism of script blocks in general, all of this is valid for Shell, Batch and all the other external scripting language action blocks (i.e. python, ruby, tcl, etc.).

Standard Input / Standard Output Handling

Script blocks provide the standard output (stdout) and standard error (stderr) of the executed command at corresponding output pins as text, so that it can be read and analyzed further by other expecco actions. These pins can be configured to either provide the whole output as one big string, or likewise.

Buffered vs. Unbuffered Stdout and Stderr

The stdout pin is buffered by default. This means that all of the output is collected and only passed to the "stdout" pin if the command finishes with an "OK" execution status (i.e. a zero exit status). Of course, this has the consequence that expecco waits for the command to finish and only writes that pin at the end, and only if the exit status is 0 (i.e. "No Error").

This may be inconvenient in two situations:

  • if the called program/script interpreter returns an invalid exit status (some programs return a non-zero status even if everything worked out ok)
  • if you are interested in the output, either during the execution, or in the error case.

One typical situation when you want to read the output, is when a server program is started, which takes a while to start and send some "I am ready" prompt after a while, and the stdout text is processed by other actions.

Then you should change the "Buffered" attribute to "Unbuffered". The output will then be read linewise and passed on to the output pin whenever multiples of the NLines parameters are received. With an NLines value of 1 (which is the default), every line will be sent individually.

By default, the stderr pin is unbuffered, with an NLines value of 1.

End of Line Conventions (Windows Operating System Specific)

Windows uses a different End-of-Line (EOL) convention than Unix/Linux: every line sent to stdout/stderr is terminated with a CRNL (carriage-return + newline) sequence, whereas under Unix, a single NL is used as line terminator. If your stdout/stderr pins are configured to generate likewise output, there is not difference; however, if they have an NLines parameter of zero, the whole output is presented as one big string. If executing under Windows, this string is processed and CRNL sequences are replaced with NL, so the generated string will be compatible among different operating systems.

There may be rare situations, when you want the "raw" (i.e. unprocessed) output. For this, add the special pin "rawOutput" and give it a true value.

Be reminded that this has no effect on Unix/Linux/OSX systems.

Showing stdin/stderr on the Transcript

In addition, the settings dialog contains a "Show Stdout/Stderr on Transcript" flag setting in the "Execution-Tracing" section. If this flag is set, all output from external script execution is also shown in the Transcript window (after version 2.10.1, there is an additional flag named "Show Executed Commands", which logs the commands as they are executed, but not their output).

Notice, that there is another flag setting in the "Execution-Logging" section, to enable log-entries in the activity log (which will be also in the generated report).

Matching stdin/stderr against a Pattern

Starting with expecco 18.1, two additional pins are provided for script actions: "prompt_optional" and "prompt_detected". The "prompt_optional" input pin, can provide a matchpattern (GLOB pattern), which will be matched against every line of stdout and stderr. When a matching line is detected, the "prompt_detected" output will receive that line. This can be used to wait for a server program to come up and indicate its ready status via a prompt (in earlier expecco versions, you had to write a little filter for this).

Command Line Arguments

To pass command-line arguments into the script, you can list them in a StringCollection and connect it to the "args" input pin. The individual arguments are then accessible with the usual means of your script interpreter, e.g. in a shell/bash as $1, $2 etc. For shell- and batch actions, arguments can also be provided by the "arguments" field (found in the action's code editor). That value is used, if no args pin is present or it has no value (technically, this creates a pin-freeze value with the value from the editor).

Execution Directory

By default, the scripting interpreter is executing in the executor's temporary "execution directory". The script finds any attachments in a folder named "$(Attachments)" or "%(Attachments)" and refer to attachments by "%(Attachments)/file" (or "%(Attachments)\file" on windows), where "file" is the attachment's filename (not the attachment's name). It is not a bad idea, to use the same name for both.

If the script must run somewhere else, you can provide the path at the block's "execDir" input pin (or for shell scripts, you can also specify it in the editor, which is effectively defining the directory as a freeze value).

The execution directory pathname may contain placeholders in the form "$(xxx)", where "xxx" is either a shell environment variable, or one of the special builtin placeholder variables:

  • $(AttachmentsDirectory) - the directory, where all attachments of the suite are located.
  • $(Attachments) - the same, but a little shorter
  • $(ExecutionDirectory) - a temporary directory, which vanishes after the execution
  • $(ExpeccoInstallationDirectory) - the directory, where expecco was installed
  • $(ExpeccoPluginDirectory) - the directory, where expecco's plugins are installed
  • $(ProjectDirectory) - the temporary project directory
  • $(HomeDirectory) - the user's home directory
  • $(TmpDirectory) - a temporary directory (subfolder of the system's tmp directory)
  • $(CurrentDirectory) - the "current" directory (see below)
  • $(DocumentsDirectory) - the "documents" directory
  • $(DesktopDirectory) - the "desktop" directory
  • $(LoginName) - your login name (username under Unix/Linux/OSX)
  • $(UserName) - your full name (if known, otherwise the login name)

Especially "$(AttachmentDirectory)" is useful, if your script refers to other scripts or files which are attached to the test suite. Then, the script sees all attachments in its current directory. The attachment files are NOT write protected, so be careful to not overwrite or remove attachment files in the script (unless you want to create new attachments and add them to the suite later).

If the script is executed outside the expecco folder hierarchy (i.e. if you specify another execution directory via the "execDir" input pin), the script can refer to the attachments via "%(AttachmentsDirectory)/foo.dat" (unix) or "%(AttachmentsDirectory)\foo.dat" (windows).

Variables are searched in any block-local environment, the test-suite environment, or the shell environment. Thus, you can also refer to shell variables such as "HOME" via "$(HOME)".

  • Notice, if you have imported many scripts which refer to attached expecco configuration or data files without a directory prefix, you can change the execution directory globally (i.e. for all scripts) via the "Settings" - "Execution" - "Execute Scripts in Attachment Folder" flag.

However, as this is a global flag, it may break imported script actions, which are not prepared for this.

  • Also notice, that the above $-expansion of the "execDir" value is a speciality of the "execDir" pin. Other pins which expect filenames will not perform this expansion. However, inside scripts, those values are expanded and can be referred to via %(...).

Environment (Shell-) Variables

You can define shell environment variables for the script by providing a Dictionary containing key-value pairs. These are added or replace the values from the current shell environment. (i.e. these are merged with expecco's own shell environment). To pass a variable as empty (i.e. unset), place a nil value into the dictionary.

In addition to any bindings-Directory provided at the "environment" pin, the above listed directory values and additional pin values are also present in the script's shell environment. Thus, you can add a pin named "foo" and refer to its value inside the shell script with "$foo".

Script Expansion

You can add additional input pins, and their values will be inserted into the script via "%(<pinName>)".
For example, the following (unix-) shell script will list the contents of the folder defined by a pin named "folderName":

ls -l %(folderName)

This script expansion is applied to all scripting languages (shell, ruby, python, etc.) and especially useful with gnuplot and 'R' scripts.

Shell Script Blocks

Under Unix operating systems (incl. Linux, Solaris, MacOSX etc.) or Windows with Cygwin or a similar shell installed, you can also execute shell-script blocks. For example, here is a script to read its input, filter all lines beginning with some pattern, sort them, and pass them on as output:

grep "^xx" | sort

Of course, arbitrary commands can be called (even programs with their own GUI).

Under Windows, make sure that you have installed an appropriate shell emulator, and that it is either found along the PATH, or that you have specified the sh-path in the expecco-settings dialog (in "External Script Interpreters" - "Shell Path"). You can also define an environment variable named "SHELL", containing the path to a shell interpreter. The environment variable can be in the global project environment, the test suite environment, or a compound action's variable in the scope of the executed shell action.

This is convenient, if the shell path is to be determined dynamically, or if another language interpreter is to be used. For example, by defining a local environment variable (in the surrounding compound action), and setting it to "/bin/ksh", the script will be passed to a "ksh" instead of the default "bash", which is useful if you have to run shell scripts written by different teams for different shell interpreters (although, we'd consider that "bad style")

Batch Script Blocks

These are similar to shell scripts, but use the Windows-batch file syntax. These cannot be executed on Unix systems, so your tests become OS-dependent, if you use them. The example code below would check out some source code from a CVS repository and start a build:

REM
REM batch file script (Windows only)
REM
echo Checking out...
cvs co myProgram
cd myProgram
echo Building...
make

Like Shell Script action blocks, Batch Script blocks provide the standard output and standard error at corresponding output pins as text, so you can to read and analyze that further. Also, the above sections on buffering, argument expansion and command line arguments also apply to Batch Script actions. For details and examples, please refer to the "Script Execution in General" section above and read the paragraph on buffering the output, the execution directory, the environment and prompt filtering.

Ruby Script Blocks

These are similar to shell scripts, but the script is executed by a ruby interpreter, instead of the sh (or bash). You have to ensure that an appropriate ruby interpreter is installed on the machine (it is not part of the expecco delivery package).

Similar to shell scripts, the ruby interpreter path either be left unspecified, or specified in the expecco settings ("External Tools"), or via an expecco environment variable named "RUBY_SHELL"). The environment variable can be the global project environment, the global test suite environment, or a compound action's variable in the scope of the executed ruby action. If left unspecified, or without an absolute path, the ruby interpreter should be found along the PATH.

Ruby Script blocks provide the standard output and standard error at corresponding output pins as text, so you can to read and analyze that further. For details and examples, please read to the "Script Execution in General" section above.

Ruby can be downloaded from eg. https://www.ruby-lang.org

Python Script Blocks

These are similar to shell scripts, but the script is executed by a python interpreter, instead of the sh (or bash). You have to ensure that an appropriate python interpreter is installed on the machine (it is not part of the expecco delivery package).

Similar to shell scripts, the python interpreter path can be either be left unspecified, or specified in the expecco settings ("external tools"), or via an expecco environment variable named "PYTHON_SHELL"). The environment variable can be in any environment which is visible in the scope of the python action. If left unspecified, or without an absolute path, the python interpreter should be found along the PATH.

Python Script blocks provide the standard output and standard error at their output pins as text, so you may have to read and analyze that further. For details and examples, please refer to the "Script Execution in General" section above.

Python can be downloaded from www.python.org/downloads.

Special Note on Python Releases

Due to some bad planning in the python community, there were a number of changes between python2.x and python3.x, which may (and usually do) make scripts written for either version incompatible with the other. For example, the following print expression from python2:

print "Hello World from a Python2 script\n"

needs parentheses in python3:

print ("Hello World from a Python2 script\n")

and will lead to a syntax error if executed in python3.

Thus, it may be necessary to specify an explicit python interpreter version to be used for python script actions. If all of your actions depend on the same python interpreter version, it is sufficient to ensure that either the "PYTHON_SHELL" variable is set correctly, or the "Python Path" is defined in the settings to point to the correct interpreter. However, if you have mixed version scripts (for example, when importing python actions written by another team, which used a different version), you have to specify the python interpreter explicitly for some of the actions.

For this, expecco contains 3 setting fields for the python interpreter: "Python Path" in the settings or "PYTHON_SHELL" in the shell environment, which are used when no explicit version is required, or if all of your script require the same python version, "Python2 Path" in the settings (or "PYTHON2_SHELL"), which are used for scripts which have been marked as explicitly requiring python2 and "Python3 Path" in the settings (or "PYTHON3_SHELL") for scripts which are marked to require python3.

The python version requirements are defined in the environment of the suite/imported library in which the action is defined: a variable named "PYTHON_VERSION", which should be an integer with either "2" or "3" as value specifies the version to be used for ALL of the python actions within that library. Notice, that the environment of the project from which the action was imported is relevant here - not the active environment at execution time, or the suite's top environment. This version number then determines, which of the above setting is to be used (i.e. PYTHON2_SHELL/path or PYTHON3_SHELL/path).

Using that double-indirect mechanism, it is still possible to:

  • import multiple libraries with different python-versions requirements
  • specify the paths independent of where the python interpreters are installed (i.e. the machine on which the final suite is executed may have different installation paths than the machine on which the imported libraries were developed).

A concrete demo importing libraries which require different python interpreter versions is found in "d46_Suite_using_both_Python2_and_Python3_actions.ets" in the demo folder.

Node.js-Script Blocks

These are similar to shell scripts, but the script is executed by a Node.js interpreter, instead of the sh (or bash). You have to ensure that an appropriate node interpreter is installed on the machine (it is not part of the expecco delivery package).

Similar to shell scripts, the node interpreter path can be either be left unspecified, or specified in the expecco settings ("External tools"), or via an expecco environment variable named "NODEJS_SHELL"). The environment variable can be the global project environment, or the global test suite environment. If left unspecified, or without an absolute path, the node interpreter should be found along the PATH.

Node.js-Script blocks provide the standard output and standard error at their output pins as text, so you may have to read and analyze that further. For details and examples, please read the "Script Execution in General" section above.

Notice that in older expecco (pre 18.1) versions, a shell/batch script action block, with appropriate command and command line options can be used to execute node.js scripts.

Node can be downloaded from nodejs.org.

Plot/Graph Action Blocks

With expecco rel18.1, a new action type was added, which makes it very easy and convenient to generate graphs of measured or generated statistical data. It is now trivial to add graphs of execution times, response times, quality data etc. to the generated report.

As a prequisite, you need to install gnuplot (-> Gnuplot on Sourceforge) and it must be found along your shell-path, or in an environment variable named "GNUPLOT_SHELL" or configured in the "External Tools" settings.

The "general information" on script actions also applies to gnu plot actions. In addition, more detailed information, step-by-step guidance and examples are found in "Using Gnuplot Action Blocks". Concrete examples are found in the library "Plot_Graph_Library.ets", which is found in the gnuplot plugin or the demos folder.

R Action Blocks

With expecco rel18.1, another new action type was added for 'R' action blocks. These are evaluated using an 'R' language script interpreter, a language especially suited for statistical analysis, bulk data processing and graph plotting. Similar to Plot/Graph actions, these can also generate figures and graphs to be attached to the final report and/or log.

As a prequisite, you need to install R and it must be found along your shell PATH or configured in the "external Tools" settings. Notice that the 'R' interpreter is named 'r' (lowercase) on some systems, and 'R' (uppercase) on others.

The The "general information" on script actions also applies to RScript actions. More information, step-by-step guidance and examples are found in "Using R Action Blocks". Concrete examples are found in the library "R_Demo_Suite.ets", which is found in the examples folder.

PowerShell Script Blocks

As a prerequisite, install powershell and ensure that it is either found along your shell PATH or configured in the settings (powershell is now also available for MacOSX and Linux) Notice, that the command is named either "powershell" or "pwsh", depending on the operating system you use.

TCL, Perl and Go Script Actions

Same as above.

Any other Scripting Language via Shell Blocks

Any external interpreter for any scripting language can be called via a regular shell/batch block. For this, either place your script into a well defined folder or add it as attachments to the suite. Then define corresponding compound blocks, which consist of a simple shell/batch call to the interpreter, and pass the script file's name plus additional parameters as command-line arguments.

Although now somewhat outdated due to the now available Python script action blocks, an example to call a python script via a shell block is found in the demo suite "d43_Call_Python_Script_via_Shell.ets", which looks like:

  CallPythonViaShellExample.png  

and a python script (in the "test.py" attachment):

print( "Hello World from a python script (1)" )

DLL-Calls

A DLL-call block defines an interface to a function which is contained inside a DLL (Dynamic Link Library), also called "Shared Library" or "Shared Object" in the Unix world). Typically, these are C or C++ functions. For a dll-call to be possible, some of the function's attributes have to be entered into the dll-call editor.

These are:

  • name of the dll (for example: "user32.dll"). If the DLL-name has no extension (".dll"/".so"), an OS-specific extension is added by expecco. For portability of your suite, it is better to only provide the base name here and let expecco choose the extension.
  • the name of the function within the dll
  • Windows operating system only: the calltype (WINAPI or C-Call).
  • the number and types of the arguments
  • the type of the return value.

In addition, it is possible to specify that the dll-call is to be executed by a separate API-call thread. This avoids a blocking of the expecco user interface and other executing activities, for long running dll-calls, or called functions which stay in a blocking wait or a busy polling loop.

If your testsuite is planned to be portable across operating systems (i.e. it should run under both Windows- and Unix test machines), you will need both a windows DLL ("foo.dll") and a Unix shared library object ("foo.so" or "foo.dylib"). Because the filename extensions are OS-specific, it is recommended to enter the basename without suffix into the "DLL name" field. Expecco will fill in the extension as used by the OS (however, if your dll uses a non-standard extension, you will have to provide it).

If the shared library file names are different among different operating systems, you can either define appropriate dll-name mappings as described below, or alternatively define separate call blocks as per operating system, and choose dynamically at runtime, which one to call. (e.g. by connecting the trigger input of each to an action which checks for the OS-type at runtime).

In seldom situations, the name of the dll is different in a 64bit operating system. Under WIndows, the 64bit variant sometimes has a '64' suffix. If this is the case, and your suite needs to run on both 32 and 64 bit operating systems, specify the 64-bit name int the extra "DLL (64bit)" field.

Argument and Return Types

Before the call, incoming argument values are read from the input pin(s) and converted to corresponding C arguments, as specified by the "argumentTypes" field in the DLL-call editor. The argument type list consists of a comma-separated list of C types, the return type of a single individual type name, each of which being one of:

   char, short, int, long, longlong,
   uchar, ushort, uint, ulong, ulonglong,
   float, double,
   pointer, HANDLE,
   char*, short*, int*, long*, uchar*, ushort*, uint*, ulong*,
   float*, double*

Notice that the exact argument type is not absolutely required for the call to succeed. For example, the callout mechanism (which is based on the ffi foreign function call interface) does not really need the type of structure or union to which a pointer points to. Thus, the generic "pointer" type is usually all you need to specify, when a structure or array or character pointer is passed to a C function.

However, some argument types might be passed in specialized registers (especially: floats, doubles and structs), and wrong argument values might be passed if the declaration is wrong.

For a comprehensive list, consult the "ExternalLibraryFunction >> ffiTypeSymbolForType:" method.

Calling Conventions

The term "calling convention" refers to the way arguments and the return value are passed to/from the called function. For most function calls, the default (C-API) is to be used. However, for historic reasons, functions in Windows32 may be defined in two flavours: WINAPI and CDECL (C-API)

The WINAPI (or API-) calltype specifies that a different argument passing scheme is to be used. This is specific to MS-Windows 32bit DLLs, and even there, only to a subset of all libraries.

Unless you have the C-header file parser plugin/extension installed, expecco has no builtin mechanism to figure out, which call convention is to be used for a particular DLL function. On Windows, if you do not have the source code of your called function at hand, consult the C header file, where the called function's declaration is found. There, look for a "WINAPI" or "API" prefix. In general, the WINAPI interface is a historic leftover from ancient times, when Windows was written in Pascal. However, they are still used (for backward compatibility) in many system DLLs - especially in the Kernel, User and GDI dlls.

On Unix- and 64bit Windows systems, the calltype is ignored and the C-calling convention is used. Thus, unless the name or parameter types are different, the same DLL-call action can be used for both architectures (given that corresponding DLLs are available).

C++ Name Mangling

If the called function is a C++ function or C++ wrapper function, the compiler may have arbitrarily mangled the function's name (-> Wikipedia: "name mangling"); usually a prefix or suffix, or both are added to the plain function's name. As this name mangling is very OS and compiler-specific, no automatic name generation can be done by expecco, and you will have to find out the name yourself, and paste it into the "C++ Name" field. Use any of your system's symbol-table listing tool ("nm" on Unix and "dllWalker" on Windows) to find out that name.

It is recommended to use C-wrapper functions ("extern C" declaration) as entry points which forward calls to corresponding C++ functions. This gives you constant entry points and thus reduces the amount of maintenance later, when a different compiler or different version mangles names differently.

DLL Mapping

Expecco (actually Smalltalk) maintains a DLL-name mapping feature, which translates DLL-library names on an individual per-DLL basis. Thus, it is possible to control which DLL is actually used by DLL call blocks. The mapping is very useful, if multiple versions of a DLL are present (e.g. use "foo" in the DLL call block, and define a mapping of "foo" to "foo.vsnX") or to control from which location a library is to be loaded (e.g. define a mapping from "foo" to "/mylibs/libraries/foo.so").

This mapping can be defined in the "Settings" -> "Project Management" -> "DLL Mapping" dialog, and is stored in your local user settings.

In addition, the StandardLibrary contains a block "Define DLL Mapping", which can be used to modify this mapping table dynamically by the suite itself. However, DLL-mapping is an issue depending on the underlying execution machine - not of the test suite. In order to get portable suites, it is recommended to define the mapping in the user-preferences of the execution machine, and leave any dependencies on paths or DLL-file names out of the suite.  

DLL Path

In addition, expecco needs to know the location of the dll. You can define this as an absolute path or a relative path (in the DLL-call action block). You may use absolute paths if the dll is part of the tested system and the folder in which the dll is found is well known and relatively stable. It is of course your responsibility to ensure that the dll is present when the action block is invoked.

Alternatively, you can give only the name of the file (without the folder name prefix) in the DLL-call action block, and either place the dll as attachment right into the test suite, or alternatively define the dll search path in the expecco settings dialog.

The dll-path settings are saved as part of your personal (per system and per user) settings - not in the ets file as project parameters.

DLL Example

You should find an example folder containing a simple DLL (source plus makefiles) and an expecco suite which calls functions from that DLL in the "expecco/projects/examples/sampleDLL" folder.

SOAP-Calls

These blocks define a remote procedure call via the SOAP protocol. SOAP blocks are normally generated automatically, by importing a WSDL service description. For more information, please refer to the WSDL Import Plugin description.  

REST-Calls

These blocks define a remote procedure call via the REST protocol. REST blocks are created in the navigation tree via the "New Action"-"Services"-"REST Action" menu item. They are to be configured with a service endpoint (i.e. URL) and a sub-path to address/identify the object under that endpoint.

REST actions have a predefined scheme, which looks like:

REST scheme sample.png

Part of its parameters are given dynamically by input pins, whereas others are predefined in the declaration tab:

REST declaration sample.png

The final HTTP URL is constructed by concatenating these path components (endpoint URL and object locator sub-path), of which either may be empty. I.e. it is possible to specify the resulting full path completely statically via the declaration and leave the pin unconnected, or to specify it completely dynamically, by leaving the declaration empty. In addition, the endpoint URL can be specified by a separate input pin named "__url__", or an environment variable named "REST_URL".

In most situations you would define the service endpoint in the environment variable (the target host plus REST-service prefix), and the object-path via a possibly frozen input pin. Of course, it is also possible to provide the object locator from another action block. The best way depends on your concrete setup and/or personal preferences.

The "__decode__" input specifies if the response text should be decoded (as JSON- or XML-DOM object) and provided at the responseObject output pin. If the REST service does provide one of them as answer, it is convenient to get the object already decoded at this output. Otherwise, you will have to fetch the value from the "responseText" output pin, and decode the value yourself (in this case, it will usually be plain text or encoded in a non-standard format, for which you will have to provide a decoder yourself).

The "httpBody" input should receive a string to be sent with the request, if the REST-service requires so. Typically, this is a JSON- or XML-encoded object to be stored or passed as argument (e.g. in a PUT/POST request). However, there are also REST calls, which do not require any input data. The value passed to this pin should be a string and thus already encoded from whatever object is to be sent. If required, use on of the JSON-encode or XML-generate action blocks. Or generate the string using one of the String-format or String-concatenation blocks.

The "encodeUTF8" and "decodeUTF8" flags are new with expecco 2.10.1. In previous releases, the given httpBody and returned responseData where transmitted unchanged (i.e. you had to encode/decode them explicitly, by using one of the EncodeUTF/DecodeUTF actions. Now, this can be done automatically for you, by checking one of those flags. For backward compatibility, the default is "false"; if your suite already provides UTF-8-encoded data (or there is no need to do so, because it only contains ASCII text), it will run unchanged in 2.10.1.

When structured objects are returned (as JSON or XML), it is recommended to create another compound action, which performs the required decoding and isolates the details from the rest of the suite. For example, if a XML-DOM is returned, a wrapper action may extract relevant fields and look like:

REST wrapper sample.png


Take a look at the examples in the "w09_RESTcallDemo.ets" file, which is found in the examples folder. More details on REST actions are also found in ElementaryBlock Element RESTCall/en .

XML-RPC-Calls

These blocks define XML-RPC remote procedure calls. XML-RPC is a very light-weight and low-overhead XML-based protocol, which is supported by many service oriented applications (among them, expecco and expecco ALM themselves). For an XML-RPC call, the following information is mandatory:

There are 3 alternative ways, to specify the URL:

  • hardwired as a string in the blocks specification (not recommended)
  • via an environment variable
  • via a pin value (freeze or dynamic)
Hardwired Service URL

Because this is very inflexible, it should be only used for very constant URLs, or while exploring a service interface. For this, type the URL into the block's specification form.

Specifying the Service URL via an Environment Variable

Use a macro-replacement pattern such as "%(RPC_HOST)" in the URL field (as above). This will be replaced by a corresponding variable name from the reachable environment variable named accordingly. You can either specify a full URL (i.e. host, port and path) or a partial URL using this mechanism. For example, you can also provide the URL via 3 separate environment variables, by entering: "%(RPC_HOST):%(RPC_PORT)/%(RPC_PATH)". A runtime error will be reported, if the variable expansion leads to an invalid URL. However, if a missing expansion leads to a valid URL, no error is reported. This allows for optional fields to be added to the URL, for example as in: "%(JIRA_HOST)%(OPTIONAL_RPC_PORT)/jira/rpc/xmlrpc".

Providing the URL via an Input Pin

Add an input pin with the (obligatory) name "url", and deliver it a string value. This is the most flexible way, as the URL value can be dynamically generated by any other block, or be provided via a pin-freeze value. If a url-pin is present, but not connected, it is ignored and the above variable mechanism is used. Otherwise, the pin-value takes precedence over any value in the specification.


See Also

For a list of already existing blocks, see Standard Library
For the programmer API for the built in languages see: Expecco API
For the scripting API for the built in languages see: Expecco Scripting API
For information on the functions available in the built-in classes, please refer to the Smalltalk/X Online Documentation and especially the Class Reference. For example, some of the mathematical functions are described here and String processing functions are found here and here.



Copyright © 2014-2018 eXept Software AG