Glossary/en

Action, Step and Activity

In expecco, an "Action Block" (or "Action" or "Block" for short) refers to the definition of how an action is done. It is therefore also called occasionally "Action Definition". In the application, action definitions are found in a tree like organization at the left of the window. Actions can be specified in multiple ways:

• as an activity diagram, with interconnected steps
• as a list of sequentially executed steps (either as keyword list action or in a test plan)
• as a piece of code in a programming language
• as a remote procedure call (service call)

Actions with an activity diagram are called "Compound Actions", and their elements are called "Steps".

Actions with a sequence list are called "Keyword Actions". Test plans are also a kind of sequence list, but provide additional control for execution (looping, skipping, selecting etc.). Although their elements are also technically steps, they are typically named "Test Steps", "Test Cases" or "Test Plan Items".
All other actions are called "'Elementary Actions".

A step is an action being placed (instantiated) in either an activity diagram or a sequence list. Notice that the same action can be placed into multiple other compound actions, keyword actions or test plans. Thus, an action definition can be resused (possibly with different parameters) in multiple places.

When a step receives its required input values, it starts executing. For this, a so called "Activity" is created, which executes the step's action. Conceptionally, a thread (lightweight process) is created, which executes the step's action with the given parameters (1). Also notice, that multiple activities can be active in parallel, even executing the same step at the same time.

Notice that in the literature, the meaning of "Action", "Step" and "Activity" is often unclear, and the exact definition is often unclear.

See also: Action Block in the Tree Elements documentation.

1) technically a thread pool is used, and a free one is chosen to execute it.

Black Box Test

In a "black box test", the test-machine and test-application are completely separated from the System Under Test (SUT). No knowledge about the internals (data structures, object model etc.) of the SUT is known when testing. The test only operates against official external interfaces, such as input/output, data and configuration files, command line interfaces, GUI interfaces or communication protocols.

The opposite being a "White Box Test", which executes inside the SUT and has full access to all internal details (i.e. functions, data objects etc.).

Expecco supports both, "Black Box" by various libraries which provide interfaces to commonly used external interfaces, and "White Box" via code injection mechanisms, such as "Groovy" or "VisualBasic".

Mixed mode testing is called "Grey Box Testing".

Bridged Actions

Expecco uses "bridges" to implement action blocks written in different programming languages. A bridge has two sides, one inside expecco, another inside the external scripting language interpreter, which typically communicate via a socket interprocess communication mechanism. Whenever a bridged action block is to be executed, expecco packs the arguments and additional call information into a message and sends it over to the other bridge side. There, the function's code is executed and a result returned as message back to expecco.

The bridge remains alive, as long as expecco is alive, or the bridge connection is closed explicitly (by the testsuite or via a menu function).

All bridges support code injection, which means that action blocks can be edited inside expecco and are forwarded and installed automatically, when first called. This makes the handling of bridged actions almost transparent and identical to the way in which internal actions (Smalltalk and JavaScript) are handled. For some languages, expecco even includes a debugger with breakpoint-, single step and data inspection features. Currently, bridges are provided for Java/Groovy, Node.js, DotNET/CLR, Python and C/C++.

Compound Block

An action block which is defined by an activity diagram. Its execution semantics are defined by a network of interconnected steps. These themselves can be compound- or elementary steps.
See Compound Block in the Tree Elements documentation.

Compound Step

A step inside an activity diagram, whose action definition is a compound block. When triggered, such a step will execute another activity diagram.
See Compound Block in the Tree Elements documentation.

".csf" File

"Connection Settings Files" are used by the Mobile Testing and Selenium WebDriver interfaces. The file is a text file contain lines with key-value pairs in the format:

key: value

each pair specifies an attribute or required capability of the connection. Typically, these are generated via the GUI browser's "Connection" dialog and kept as attachments inside the suite.

".csv" File

"Comma Separated Values Files" are files as generated by Excel and other Spreadsheet programs, by databases and many other record-oriented programs for data interchange. These files represent data records as textual (i.e. human readable) text lines, where the columns (fields) are usually separated by commas (although often semicolon is used as field separator). The very first line of a "csv" file contains the column names. Expecco can read and write "csv" files, eg. for parameter sets or via action blocks. It is also possible to give a CSV file to the command line, to preset top-level environment variables from it.

DLL (Dynamic Link Library)

Are dynamically loadable programs or libraries. In the Unix world, these are typically called "shared objects". Expecco can call into such libraries using elementary DLL-call blocks. These should be used to call driver or utility functions inside a dll, especially if you do not have access to the function's source code or header definition files.
See more in Elementary Blocks - DLL Calls.

DUT (Device Under Test)

See System Under Test in this document.

Elementary Block

An non-compound action block. I.e. any action which is not defined by an activity diagram or keyword list, but either builtin (well known to expecco), a call to an external function or application, or defined programmatically as a piece of textual program code in one of the supported programming languages.
See Elementary Block in the Tree Elements documentation.

Elementary Step

A step inside an activity diagram, whose action definition is an elementary block. When triggered, such a step will execute either a builtin function, an existing function inside a DLL, another program or a piece of code written in one of the programming languages supported by expecco.
See Elementary Block in the Tree Elements documentation.

Environment

Variables are bundled together in so called Environments, which are technically collections of key-value pairs (Dictionary/HashTable). Environments can be attached to individual compound actions, to individual testcases, to a testplan and are also attached to the suite. There are also temporary environments attached to the current execution, the current browser window and the current expecco session.

During execution, environments are linked to form a parent-child hierarchy, and variables are fetched along this hierarchy. Thus, variables declared at the block level may shadow variables at the testplan-level, which may shadow variables declared in the current testsuite.
Typically, the top-level suite's environment (so called "ProjectEnvironment") is used most of the time.

Environment Freeze Value

A variable attached to an action step's input pin. Instead of being provided via a connection from another output pin, this provides a value from an environment variable.

".elf" File

"elf" stands for "Expecco Log File" and is the native format in which result files are written. "elf" files contain all of the execution, trace and data-flow information and the original suite as executed. "elf"-files can be reloaded at any time into the expecco UI, values of that run be inspected and it is even possible to reexecute the original test. Also, any of the other reports can be regenerated, possibly with different report options (i.e. detail). Technically, "elf" files are zip-containers containing traces, logs and the original test suite in XML-format, and any attachments (screen shots, measurement data etc.) in whatever format they were written (images usually as PNG). They can be read and processed by third party tools (i.e. unzipped, and then processed with an XSL/XSLT ruleset to generate custom reports or to transport the result to any other XML-capable QM system).

".ets" File

"ets" stands for "Expecco Test Suite". These are technically zip-containers, containing the test suite in XML-format and any attachments in whatever format they where imported or generated. In addition, an index (manifest) is present, which allows for fast access to certain information without a need to read the whole zip archive. Also, signatures ensure that the manifest and remaining contents are in sync. Therefore, ".ets" files should not be manipulated manually, as the consistency of the suite may be harmed.

Freeze Value

A fix (static) value attached to an action step's input pin. Instead of being provided via a connection from another output pin, this provides a constant value to an action step's input. Notice that starting with expecco release 19.2, variable references can be embedded inside a String-Freeze-Value using the "$(xxx)" notation, where "xxx" is the name of a variable or input pin of the enclosing compound action. See also Environment Freeze Value in this document.

Functional ID (FID)

A UUID attached to every action block inside expecco. This identifier is assigned once and never changed again. Inside expecco, actions are referred to via their FID or Version ID, independent of their name. Actions with the same FID are considered to be (possibly different versions) of the same action.

Gray Box Test

Some knowledge about internal interfaces of the System Under Test are known. For example, data structures or function entries inside the SUT are exploited and used. The categorization of a test as white- or gray box test is vague - there is often no clear line of separation between them.
See "Grey Box Testing" in Wikipedia.

Groovy

A language interpreter which runs on top of a Java Virtual Machine (JVM), with a language very similar and almost compatible to Java. Used in expecco to call into Java interfaces or define required callback and listener classes, in case the SUT or the interface to the SUT require those.

Groovy code is compiled to bytecode inside the JVM, which is further Just-in-Time compiled to fast machine code within the target JVM. Thus, it executes at full Java speed.

Expecco allows for Groovy code to be executed both on the local machine (the one on which expecco itself is running) and on remote machines which are reachable from the local machine via a TCP/IP connection. Multiple of these connections are possible and handled simultaneously in parallel, making it possible to interact with multiple hosts in a distributed system or when testing both ends of a communication protocol.
See "Groovy Elementary Blocks" in the Expecco API Document and Elementary Block in the Tree Elements documentation.

Groovy Elementary Block

An action block which is defined by an execute function which is written in the Groovy language. The block's code will be executed by a Groovy interpreter on a Java Virtual Machine (JVM). The JVM may run either on the local or on a remote machine. Expecco uses code injection techniques to implant the code into the SUT dynamically at run time. Except for sealed systems, it is normally not required to specially instrument or otherwise recompile the code in the SUT.
You have to install "java" from the official website (due to legal reasons, we cannot provide a java interpreter with the expecco installer).
See "Groovy Elementary Blocks" in the Expecco API Document and Elementary Block in the Tree Elemens documentation.

Inconclusive State / Test Outcome

See "Verdict" in this document.

IronPython Elementary Action Block

Bridged IronPython actions are supported by expecco. IronPython is a python interpreter which runs inside a DOTNET CLR (Common Language Runtime) environment, and has therefore full access to any .NET framework/assembly. It can be used if .NET interfaces are needed, both under native Windows and under Mono (linux/unix/osx).
You have to install "IronPython" from the official website.
See "Bridged Python Elementary Blocks" in the Expecco API document.

JavaScript Elementary Action Block

An action block which is defined by an execute function which is written in a JavaScript like language. The block's code will be executed inside expecco itself (as opposed to Node-Action-Blocks, which are executed by an external interpreter).
See "JavaScript Elementary Blocks" in the Expecco API Document and Elementary Block in the Tree Elements documentation.

JSON ".json" File

JSON is a common format to exchange structured data, which originated in JavaScript but is now very common and supported by many applications. JSON can represent simple (non recursive) objects as textual (i.e. human readable) text, organized as a nested object hierarchy with key-value associations. Expecco can read and write "json" files, eg. for parameter sets or via action blocks. It is also possible to give a JSON file to the command line, to preset top-level environment variables from it. Because JSON is often used in many other apps, expecco also provides the JSON representation in its data inspector, for easy copy-paste.

Jython Elementary Action Block

Bridged Jython actions are supported by expecco. Jython is a python interpreter which runs inside a JVM, and has therefore full access to any Java framework. It can be used as an alternative to Groovy, if Java interfaces are needed.
You have to install "jython" from the official website.
See "Bridged Python Elementary Blocks" in the Expecco API document.

Node Elementary Action Block

Node.js (or "NodeJS" or simply "Node") is a language interpreter for JavaScript, which is used in cloud computing, Alexa, IOT and other internet applications. Node is rapidly gaining interest and many packages are already available, covering a wide range of protocols, file formats and interfaces. Both bridged and scripted Node.js actions are supported, and expecco includes good support for bridged Node.js action blocks, including breakpoints and debugging with single stepping.
You have to install "node" and "npm" from the official website: https://nodejs.org/en/download.
See "NodeJS Elementary Blocks" in the Expecco API document and Elementary Block in the Tree Elements documentation.

Pin - Consuming Pin

An input pin of a step, which does consumes its input value when the step starts to execute. Technically, this takes the next input value from the queued values pending in the input-basket. Consuming pins are used when an input pin gets its value from another step's output pin.

Pin - Parameter Pin

An input pin of a step, which does not consume its input value and which does not trigger the step. Used for constants and parameters, especially if the step is to be used in a loop.

Pin - Triggering Pin

An input pin of a step, which triggers the execution of a step. A step will only start to execute, when all of its triggering pins have received a value.

Python Elementary Action Block

Both bridged and scripted Python actions are supported by expecco, and also both Python2.x and Python3.x can be used inside the same suite.
You have to install "python" and "pip" from the official website: https://www.python.org/downloads.
See "Bridged Python Elementary Blocks" in the Expecco API document.

Script Actions

Script actions are actions written in one of the supported scripting languages, including Shell, Batch, Powershell, Node.js, Python, Ruby, GnuPlot, R, TCL, Go and others. In contrast to "bridged actions", these are called as host-commands, where a new interpreter is started for every action invokation. This has the advantage, that any existing script can be easily used/embedded into a testsuite, but the disadvantage that the action's execution is somewhat slowed down due to the startup/shutdown times of the external interpreter, and that no state (open files, connections, objects etc.) can be passed from one action to another.

Shell/Batch Script Elementary Block

An action block which is defined by a shell (or batch) script which is written in a shell command-line-interpreter language. The block's code will be executed outside expecco, by a script interpreter. See "ShellScript Elementary Blocks" in the expecco elementary block documentation.

Smalltalk Elementary Block

An action block which is defined by an execute function which is written in the Smalltalk language. The block's code is usually executed inside expecco itself, but it is also possible to define bridged Smalltalk actions, which execute inside another Smalltalk process (local or remote). See "Smalltalk Elementary Blocks" in the Expecco API Documentation.

Smalltalk/X

Smalltalk/X ("ST/X") is the brand name of a Smalltalk programming language environment from exept / Claus Gittinger. Smalltalk/X is an ANSI compatible Smalltalk programming environment, and the base on which expecco and expeccoALM are built.

Stderr

Short for "Standard Error". This is any program's standard error output stream. If running in a console window, this is typically displayed there. If running without a console window it may be lost or redirected into a logfile. If expecco starts other programs, you have the option of redirecting their stderr to the expecco Transcript (which is the expecco console). There, this will be prefixed by "2:" and colorized in red (to indicate that this is text from stderr, as opposed to stdout). The setting is found in "Extras" → "Settings" → "Execution" → "Tracing" → "Show Stdout/Stderr in Transcript".

Stdout

Short for "Standard Output". This is any program's standard output stream. If running in a console window, this is typically displayed there. If running without a console window it may be lost or redirected into a logfile. If expecco starts other programs, you have the option of redirecting their stdout to the expecco Transcript (which is the expecco console). There, this will be prefixed by "1:" and colorized in blue (to indicate that this is text from stdout, as opposed to stderr). The setting is found in "Extras" → "Settings" → "Execution" → "Tracing" → "Show Stdout/Stderr in Transcript".

SUT (System Under Test)

The tested system.

Expecco interacts with this either in a pure black box (only talking to external interfaces), or gray to white box fashion (some or a lot of knowledge about internals of the SUT are known).

Technically, black box tests interact with the SUT via (more or less official) external interfaces, such as messages, documents, communication interfaces, GUI interaction etc. White box tests can access data structures and interfaces internal to the SUT.

In expecco, White Box Tests are done by implanting code into the SUT via code injection techniques. Depending on the type of technology and environment used, this injection (or instrumentation) may be possibly dynamically (Java or .NET) or has to be done statically by linking expecco-support libraries to the program (C, C++).

Some companies (especially hardware developers) use the alternative term "DUT" (Device under Test) instead.

Syntactic Sugar

A feature of (typically) a programming language, which does not give you more semantic features, but instead exists for the convenience of the user. Typically "syntactic sugar" designates a feature which is easier to use or shorter to write, but which could also be defined in terms of other features of the language - albeit possibly more clumsy. In expecco, a number of syntactic sugar constructs are available, which could also be described in terms of regular action blocks. For example, the "attachment step", "shell step" and even "Groovy" steps could be also defined in terms of regular elementary blocks which read a file, start a shell or talk to a JVM respectively.

Tag

A token (word) attached to an element. Multiple tags can be attached and are used to search, group or otherwise mark tree items, steps and other elements of the test suite.

Tagged Value

A tuple consisting of a token plus value, and an optional type. Similar to tags, these can be attached to an element. Expecco does not use tagged values. They are created when suites are imported from some external tools, stored with the suite and passed back transparently to the tool.

Testplan Element

A "Test Plan" or "Testplan" is an item in the project tree, which contains a list of test case items to be executed in sequence. A test plan's items are called "Test Cases" and each is implemented by an action (elementary or compound). It is also possible to drag another test plan as item into another test plan, to combine multiple tests plans into master test plans. A suite may therefore contain multiple test plans, which can be chosen for execution via the command line or via a remote service request.

Testsuite Element

"Test suite" or occasionally "Testsuite" is the term used for a packaged test project. It contains a collection of Tree Elements like testplans, blocks and datatypes as well as optional resources, attachments and documentation.

Test suites can be imported into other test suites and can therefore also be used as libraries for reuse. When a suite is imported into another suite, all of the imported elements are visible and usable inside the importing suite.

Typically, suites are organized in a hierarchical fashion, and low-level functions (device control and access, protocols, data format handling etc.) are packaged into separate libraries, which are used by higher level component test and support actions, which are then further reused by system-, acceptance or end-to-end test scenarios.

Transcript

The "Transcript Window" is the expecco console window, which can be opened via "Extras" - "Tools" - "Transcript". If open, both information messages from expecco and explicit Transcript action messages will appear there. The standard library provides a number of action blocks eg. to send messages and to clear the transcript window. Messages can also be sent to it from elementary code (both internal and bridged).
If not open, those messages will be sent to "stderr" instead.

Verdict

The execution's summary-outcome-status from running a test plan, test case or action. In addition to detail information (traces, data, execution time, log-messages), every execution in expecco will always return one of 4 values as a summary status:

• PASSED (green)
  if all went well, and the execution did not encounter any problems
• FAIL (red)
  the execution detected an error in the System Under Test (SUT). For example, it measured a wrong/unexpected value, detected invalid behavior, invalid protocol behavior etc.
• ERROR (dark red)
  the test itself ran into some error. This means that during the test's execution, an error occurred inside expecco or one of the user-defined action blocks. For example, if a file containing test-data could not be found or a division by zero or an index out of bounds error occurred etc.
• INCONCLUSIVE (gray)
  the test had no chance to perform its testing operations. For example, if a machine could not be reached in the network, if it was not powered up or if a resource (operator, measurement device, test-device etc.) was not available, etc.

The distinction is very useful as it should be kept in mind, that a test-manager/tester has to react differently depending on the reason for a test not resulting in a PASSED state:

• for a FAIL, (s)he has to inform the developer of the SUT, to fix the problem in the product.
• for an ERROR, (s)he has to inform the test-developer/analyst, to fix the problem in the test-case
• for INCONCLUSIVE, (s)he has to check for the availability of required devices and possibly inform the network administrator or test-lab-manager. After that, the test can be rerun without a need to contact either the product-development, nor the test-development team.

Notice, that many test systems/frameworks only provide PASS/NOT PASS information as final verdict. If expecco is to be integrated into such a framework, FAIL, ERROR and INCONCLUSIVE will all be treated as NON-PASS (e.g. when running under the control of Jenkins, QC, Polarion etc.)

VersionID (VID)

Every item within expecco (action, type, testcase, testplan, the suite itself etc.) has an associated unique identifier, which is guaranteed to be world-wide unique (a so called "UUID"). This guarantees, that these objects can be uniquely identified, even if renamed, or if the same name is used for an action, type etc. in two different test suites. Thus, two suites can always be merged and suites can always be imported without name conflicts.. expecco will internally always refer to any object by its ID, never by its name. The versionID is updated with every change made to an object. Thus it is guaranteed, that no two different objects have the same ID. And vice versa, that the equality (actually identity) can always be checked via the ID.

VisualWorks

VisualWorks ("VW") is the brand name of a Smalltalk programming language environment from Cincom, an US based company. Starting with expecco 20.2, VisualWorks bridged blocks and VisualWorks GUI tests are supported by expecco.

White Box Test

Designates a test which executes within the System Under Test and uses or refers to (but also depends on) the internals of the System Under Test. The test directly creates or manipulates data objects inside the SUT and/or calls internal functions. Most unit tests are typically implemented as white box tests (but not required to be so).

White Box Tests have both advantages and disadvantages.

• + because they know the internals, more of internal details can be tested, and more of existing internal utilities can be used as utility in the test. For example, data definitions, data structures, functions and interfaces etc. can be accessed from the test system. Depending on the type of test and the available frameworks which can be "white-boxed", this may save a lot of test-development time, because those interfaces need not be reimplemented in the test system.

• - because they know the internals, they are also more reliant on them, and may also suffer from bugs in the internal frameworks. For one, if data structures and internal interfaces change, the white box test usually has to change with it. Whereas external interfaces usually remain more stable - especially if they are based on communication or data format standards. The other downside is that if internal interfaces of the SUT are heavily used, bugs in them are also injected into the test system. This may make some of the bugs invisible and undetectable to the test system. For example, if a communication data structure (say, a message or document format) is always accessed via a white-boxed SUT interface, but never actually tested physically, any bug in that framework will go unnoticed to both the SUT and to the test system. Both the SUT and the test system may see perfect data, while the data sent to the outside world may still be completely bogus.

In a real world test system, you would usually use a combination of whitebox and blackbox tests. The level of insider-knowledge of the whitebox test depends on the type of SUT, the risk analysis and trust in the frameworks being used.

See also: Black Box Test

Workspace (or Notepad)

This is a tool which is found in "Extras" - "Tools" - "Notepad". The name is a bit misleading and this should not be confused with the Windows "Notepad" application (therefore, it is also named "Workspace" in the documentation, although this again may lead to confusion, as "workspace" is often used with a different meaning in other contexts, such as the Eclipse IDE).

In expecco, a Workspace is a little text editor, which can also evaluate snippets written in either Smalltalk or JavaScript. It also provides a number of bulk string processing operations in its edit menu.

Whenever you have to manipulate text, a workspace is a useful edit-tools to perform such tasks (of course, you can use any other editor, if you prefer so, but startup times of external editors are often much longer, than the time it takes expecco to open its internal tools).

Workspaces and all other editors in expecco (incl. the FileBrowser, Data Inspector and Elementary Action code editors) are all based on a common editor base class and have common behavior. Take a look at the popup menu to see what functionality is provided: beside the usual stuff, it can sort text, by columns, filter lines, transform lines and many more.

Especially useful are the "Open FileBrowser" and "Open URL" menu functions (found deeper down in the "Tools" submenu). Whenever you find a filename or URL in some text, these will quickly open the document for you.

Abbreviations

Common used abbreviations in this documentation Wiki are:

• ALM - Application Lifecycle Management
• BD - (Action-) Block Description
• CB - Compound Block
• ".csf" - Connection Settings File (file suffix)
• ".csv" - Comma Separated Values File (file suffix)
• DCE - Distributed Computing Environment (an RPC mechanism)
• DLL - Dynamic Link Library (a synonym for "SO" = Shared Object); also a file suffix ".dll"
• EB - Elementary Block
• ".elf" - Expecco Log File (file suffix)
• ".ets" - Expecco Test Suite (file suffix)
• FID - Functional ID
• GUI - Graphical User Interface; also sometimes simply "UI"
• QM - Quality Management
• RPC - Remote Procedure Call
• ".png" - a bitmap image file format (file suffix)
• SO - Shared Object (a synonym for DLL); also as file suffix ".so"
• SUT - System Under Test
• UI - User Interface (often actually meaning: Graphical-UI)
• URL - Unified Resource Locator (typically used with web-Browsers)
• UUID - Universal Unique Identifier
• VID - Version ID

---

Copyright © 2014-2024 eXept Software AG