Selftest plugin

The plugin runs selftest scripts on acitions of a library. Each action has associated to it a test script which specifies the inputs and expected behavior of the action. The script's syntax is very simple and scripts can easily be created from a requirements or certification document.

Its main use is as input for a certification suite of expecco's standard library.

Running Selftests

The plugin adds another tab to every action block, in which a script is defined for that particular action. A run button to execute the script is shown in the editor's toolbar.

Also, a tab is added to the project's tree item, which executes all existing self tests of the whole library and provides the outcome of each test in a list view. Double click on a list item there, to navigate to the corresponding individual action's self test script.

Script Syntax

The script consists of individual commands, each containing a textual command.

Lines beginning with ";" or "#" are comment lines. Text after a ";" is ignored (end-of-line comments). (notice that "#" is not recognized as end-of-line comment unless occurring at the beginning, because "#" is a valid syntax element for symbol-literal constants.)

Currently, there are 5 different commands:

• "in <expr1> . <expr2> ..."
  specifies the input value(s) to be given to the action. Any number of expressions (incl. zero) can be given.
• "out" <expr1> . <expr2> ..."
  specifies the expected output value(s). Any number of expressions (incl. zero) can be given.
• "run" <expectedOutcome>"
  runs the test setup (given by previous in & out commands) and verifies the outcome. The outcome must empty (expect success) or one of "fail", "error", "inconclusive".
• "prerequisite" <st-expression>
  a prerequistite for the action, to be executed before the next test run. "st-expression" must be a valid Smalltalk expression (which is given to eval).
• "cleanup" <st-expression>
  a cleanup action to be executed after the test run. "st-expression" must be a valid Smalltalk expression (which is given to eval).

"in" Command

Specifies input values as Smalltalk expressions separated by a period ("."). For unconnected inputs, individual expressions can be left empty. For example:

• in 1 . 2 . 3
  Provides 3 integer input values to the first 3 input pins, with values "1", "2" and "3" respecitvely.
• in 1 . . 'abc'
  Provides the integer "1" to the first input pin, no values to the second and third input pin, and the string "abc" to the fourth input pin.
• in
  no input values are provided

"out" Command

Similar to the above "in" command, this defines the expected output values. For pins where no output is expected, an empty expression must be given.

"run" Command

Executes the test with the previously define input values, and verifies the expected outputs. Also, the expected success/fail/error state is given by a keyword after the "run" command. This can be one of:

• <empty>
  expect success
• "fail"
  expect failure
• "error"
  expect an error
• "inconclusive"
  expect an inconclusive outcome

"prerequisites" Command

Defines a Smalltalk expression to be evaluated BEFORE the test run is executed. A typical use is the creation of a temporary file (eg. when testing the "File [exists]" action block.

"cleanup" Command

Defines a Smalltalk expression to be evaluated AFTER the test run is executed. A typical use is the deletion of a temporary file (eg. when testing the "File [exists]" action block.

Sample Scripts

The following sample script could used with the "Arith [ Sum ]" action:

in        ; no inputs
out       ; no outputs expected
run error ; expect an error (no inputs given)
 
in 1 . 2  ; two inputs given
out 3     ; expected output
run       ; no error expected

The next sample script could used with the "File [ Exists ]" action:

prerequisite '/tmp/testfile' asFilename contents:'hello'
cleanup      '/tmp/testfile' asFilename delete
in           '/tmp/testfile'
out       ; no output
run       ; expect no error
 
cleanup '/tmp/testfile' asFilename delete
in      '/tmp/testfile'  
out       ; no output
run fail  ; expected outcome

---

Copyright © 2014-2024 eXept Software AG