Glossary/en: Unterschied zwischen den Versionen

Aus expecco Wiki (Version 2.x)
Zur Navigation springen Zur Suche springen
Zeile 59: Zeile 59:
== Syntactic Sugar ==
== Syntactic Sugar ==
A feature of (typically) a programming language, which does not give you more semantic features, but instead exists for the convenience of the user. Typically "''syntactic sugar''" designates a feature which is easier to use or shorter to write, but which could also be defined in terms of other features of the language - albeit possibly more clumsy. In expecco, a number of syntactic sugar constructs are available, which could also be described in terms of regular action blocks. For example, the "attachment step", "shell step" and even "groovy" steps could be also defined in terms of regular elementary blocks which read a file, start a shell or talk to a JVM respectively.
A feature of (typically) a programming language, which does not give you more semantic features, but instead exists for the convenience of the user. Typically "''syntactic sugar''" designates a feature which is easier to use or shorter to write, but which could also be defined in terms of other features of the language - albeit possibly more clumsy. In expecco, a number of syntactic sugar constructs are available, which could also be described in terms of regular action blocks. For example, the "attachment step", "shell step" and even "groovy" steps could be also defined in terms of regular elementary blocks which read a file, start a shell or talk to a JVM respectively.

== Verdict ==

The execution's summary-outcome-status from running a test plan, test case or action. In addition to detail information (traces, data, execution time, log-messages), every execution in expecco will always return one of 4 values as a summary status:
- PASSED<br>if all went well, and the execution did not encounter any problems
- FAIL<br>the execution detected an error in the System Under Test (SUT). For example, it measured a wrong value, detected invalid behavior of the SUT, invalid protocol behavior etc.
- ERROR<br>the test itself ran into some error. This emans that during the test's execution, an error occurred inside expecco or one of the user-defined action blocks. For example, if a file containing test-data could not be found, if a division by zero occurred etc.
- INCONCLUSIVE<br>the test had no chance to perform its testing operations. For example, if a machine could not be reached in the network, if a resource (operator, measurement device, test-device etc.) was not available, etc.

The distinction is very useful as it should be kept in mind, that a test-manager/tester has to react differently depending on the reason for a test not resulting in a PASS:
- for a FAIL, he has to inform the developer of the SUT, to fix the problem in the product.
- for an ERROR, he has to inform the test-developer/analyst, to fix the problem in the test-case
- for INCONCLUSIVE, he has to check for the availability of required devices and possibly inform the network administrator or test-lab-manager. After that, he can rerun the test without a need to contact either the product-development, nor the test-development team.


== White Box Test ==
== White Box Test ==

Version vom 8. September 2016, 10:38 Uhr

Please keep this list alphabetically sorted!

Black Box Test[Bearbeiten]

In a "black box test", the test-machine and test-application are completely separated from the System Under Test (SUT). No knowledge about the internals (data structures, object model etc.) of the SUT is known when testing. The test only operates against official external interfaces, such as input/output, data and configuration files, command line interfaces, GUI interfaces or communication protocols.

The opposite being a "White Box Test", which executes inside the SUT and has full access to all internal details (i.e. functions, data objects etc.).

Expecco supports both, "Black Box" by various libraries which provide interfaces to commonly used external interfaces, and "White Box" via code injection mechanisms, such as "Groovy" or "VisualBasic".

Mixed mode testing is called "Grey Box Testing".

Compound Block[Bearbeiten]

An action block which is defined by an activity diagram. Its execution semantics are defined by a network of interconnected steps. These themselfes can be compound- or elementary steps. See Compound Block in the Tree Elements documentation.

Compound Step[Bearbeiten]

A step inside an activity diagram, whose action definition is a compound block. When triggered, such a step will execute another activity diagram. See Compound Block in the Tree Elements documentation.

Elementary Block[Bearbeiten]

An non-compound action block. I.e. any action which is not defined by an activity diagram, but either builtin (well known to expecco), a call to an external function or application, or defined programmatically as a piece of program code in one of the supported programming languages.

Elementary Step[Bearbeiten]

A step inside an activity diagram, whose action definition is an elementary block. When triggered, such a step will execute either a builtin function, an existing function inside a DLL, another program or a piece of code written in one of the programming languages supported by expecco.

Gray Box Test[Bearbeiten]

Some knowledge about internal interfaces of the System Under Test are known. For example, data structures or function entries inside the SUT are exploited and used. The categorization of a test as white- or gray box test is vague - there is often no clear line of separation between them. (see "Grey Box Testing" on wikipedia).

Groovy[Bearbeiten]

A language interpreter which runs on top of a Java Virtual Machine (JVM), with a language very similar and almost compatible to Java. Used in expecco to call into Java interfaces or define required callback and listener classes, in case the SUT or the interface to the SUT require those.

Groovy code is compiled to bytecode inside the JVM, which is further Just-in-Time compiled to fast machine code within the target JVM. Thus, it executes at full Java speed.

Expecco allows for Groovy code to be executed both on the local machine (the one on which expecco itself is running) and on remote machines which are reachable from the local machine via a TCP/IP connection. Multiple of these connections are possible and handled simultaneously in parallel, making it possisble to interact with multiple hosts in a distributed system or when testing both ends of a communication protocol. See "Groovy Elementary Blocks" in the Expecco API Document and Elementary Block in the Tree Elements documentation.

Groovy Elementary Block[Bearbeiten]

An action block which is defined by an execute function which is written in the Groovy language. The block's code will be executed by a Groovy interpreter on a Java Virtual Machine (JVM). The JVM may run either on the local or on a remote machine. Expecoo uses code injection techniques to implant the code into the SUT dnamically at run time. Except for sealed systems, it is normally not required to specially instrument or otherwise recompile the code in the SUT. See "Groovy Elementary Blocks" in the Expecco API Document and Elementary Block in the Tree Elements documentation.

JavaScript Elementary Block[Bearbeiten]

An action block which is defined by an execute function which is written in a JavaScript like language. The block's code will be executed inside expecco itself. See "JavaScript Elementary Blocks" in the Expecco API Document and Elementary Block in the Tree Elements documentation.

Shell/Batch Script Elementary Block[Bearbeiten]

An action block which is defined by a shell (or batch) script which is written in a shell command-line-interpreter language. The block's code will be executed outside expecco, by a script interpreter. See "ShellScript Elementary Blocks" in the Expecco elementary block documentation.

Smalltalk Elementary Block[Bearbeiten]

An action block which is defined by an execute function which is written in the Smalltalk language. The block's code will be executed inside expecco itself. See "Smalltalk Elementary Blocks" in the Expecco API Documentaton.

SUT (System Under Test)[Bearbeiten]

The tested system.

Expecco interacts with this either in a pure black box (only talking to external interfaces), or gray to white box fashion (some or a lot of knowledge about internals of the SUT are known).

Technically, black box tests interact with the SUT via (more or less official) external interfaces, such as messages, documents, communication interfaces, GUI interaction etc. White box tests can access data structures and interfaces internal to the SUT. In expecco, this is done by implanting code into the SUT via code injection techniques. Depending on the type of technology and environment used, this injection (or instrumentation) may be possibly dynamically (Jaba or DotNet) or has to be done statically by linking expecco-support libraries to the program (C, C++)

Syntactic Sugar[Bearbeiten]

A feature of (typically) a programming language, which does not give you more semantic features, but instead exists for the convenience of the user. Typically "syntactic sugar" designates a feature which is easier to use or shorter to write, but which could also be defined in terms of other features of the language - albeit possibly more clumsy. In expecco, a number of syntactic sugar constructs are available, which could also be described in terms of regular action blocks. For example, the "attachment step", "shell step" and even "groovy" steps could be also defined in terms of regular elementary blocks which read a file, start a shell or talk to a JVM respectively.

Verdict[Bearbeiten]

The execution's summary-outcome-status from running a test plan, test case or action. In addition to detail information (traces, data, execution time, log-messages), every execution in expecco will always return one of 4 values as a summary status: - PASSED
if all went well, and the execution did not encounter any problems - FAIL
the execution detected an error in the System Under Test (SUT). For example, it measured a wrong value, detected invalid behavior of the SUT, invalid protocol behavior etc. - ERROR
the test itself ran into some error. This emans that during the test's execution, an error occurred inside expecco or one of the user-defined action blocks. For example, if a file containing test-data could not be found, if a division by zero occurred etc. - INCONCLUSIVE
the test had no chance to perform its testing operations. For example, if a machine could not be reached in the network, if a resource (operator, measurement device, test-device etc.) was not available, etc.

The distinction is very useful as it should be kept in mind, that a test-manager/tester has to react differently depending on the reason for a test not resulting in a PASS: - for a FAIL, he has to inform the developer of the SUT, to fix the problem in the product. - for an ERROR, he has to inform the test-developer/analyst, to fix the problem in the test-case - for INCONCLUSIVE, he has to check for the availability of required devices and possibly inform the network administrator or test-lab-manager. After that, he can rerun the test without a need to contact either the product-development, nor the test-development team.

White Box Test[Bearbeiten]

Designates a test which executes within the System Under Test and uses or refers to (but also depends on) the internals of the System Under Test. The test directly creates or manipulates data objects inside the SUT and/or calls internal functions. Most unit tests are typically implemented as white box tests (but not required to be so).

White Box Tests have both advantages and disavantages.

  • + because they know the internals, more of internal details can be tested, and more of existing internal utilities can be used as utility in the test. For example, data definitions, data structures, functions and interfaces etc. can be accessed from the test system. Depending on the type of test and the available frameworks which can be "white-boxed", this may save a lot of test-development time, because thise interfaces need not be reimplemented in the tet system.
  • - because they know the internals, they are also more reliant on them, and may also suffer from bugs in the internal frameworks. For one, if data structures and internal interfaces change, the white box test usually has to change with it. Whereas external interfaces usually remain more stable - especially if they are based on communication or data format standards. The other downside is that if internal interfaces of the SUT are heavily used, bugs in them are also injected into the test system. This may make some of the bugs invisible and undetectable to the test system. For example, if a cmmunication data structure (say, a message or document format) is always accessed via a white-boxed SUT interface, but never actually tested physically, any bug in that framework will go unnoticed to both the SUT and to the test system. Both the SUT and the test system may see perfect data, while the data sent to the outside world may still be completely bogus.

In a real world test system, you would usually use a combination of whitebox and blackbox tests. The level of insider-knowledge of the whitebox test depends on the type of SUT, the risk analysis and trust in the frameworks being used.

See also: Black Box Test



Copyright © 2014-2024 eXept Software AG