DiagramElements-Probe/en

Aus expecco Wiki (Version 2.x)
(Weitergeleitet von DiagramElements-Probe)
Wechseln zu: Navigation, Suche

Introduction

Probes can be placed into a diagram (via the diagram editor's pin pop-up menu) to monitor values generated by a step's output pin. They can be configured to check those pin values against a wide number of attributes (values are in a range, values are increasing, decreasing, below or above a limit etc.)

The parameters (i.e. limits or range bounds) can either be defined statically, or recorded by a prototypical "good" run through the testsuite.

For this, probes can be set into one of three modes:

  • inactive - probes are disabled completely
  • recording - probes will record incoming values and adjust their check-parameter(s)
  • checking - probes will monitor incoming values and check them against their check-parameter(s)

The operation mode of individual probes, probe groups or all probes can be changed via menu items, or via action blocks found in the standard library.


Probe Types

Different probe types are available, which either check individual values for being valid, or check whole value sequences (for example, for being increasing, decreasing, etc.). Finally, there are probes which only record their incoming values without any value checks. These non-checking probes are useful to collect data for graph generation (i.e. using Plot/Graph actions).

Probe types are:

Single Value Probes

These probe types check each individual value independent of previous values.

Value must be Equal

given a single constant setpoint value, this probe checks that every incoming value is equal. The probe generates a FAIL, as soon as a non-equal value is encountered. This probe can compare numbers, vectors, strings, etc.

Value may NOT be Equal

given a single constant setpoint value, this probe checks that every incoming value is not equal. The probe generates a FAIL, as soon as an equal value is encountered. This probe can compare numbers, vectors, strings, etc.

Value must/may not Match a Pattern

given a single constant setpoint value, this probe checks that every incoming value is matching or not matching that String pattern. The match can be defined as GLOB-pattern match, regular expression match, string-includes, string-start or string end match, and also to optionally ignore upper/lower case differences. The probe generates a FAIL, as soon as a matching/non-matching value is encountered. This probe can be used with strings only.

Value must be Greater/Greater or Equal/Less/Less or Equal

given a single constant setpoint value, this probe checks that every incoming value conforms to the particular magnitude relation. The probe generates a FAIL, as soon as a non-conforming value is encountered. This probe can compare numbers, and strings; for strings, the unicode encoding of individual characters defines the order.

Value must be in Interval

given two constant setpoint values which define the minimum and maximum of a numeric interval, this probe checks that every incoming value is within that interval (i.e. min <= value <= max). The probe generates a FAIL, as soon as a non-conforming value is encountered. This probe can compare numbers, and strings; for strings, the unicode encoding of individual characters defines the order.

Value may NOT be in Interval

similar to the above, this FAILs if an incoming value is WITHIN the defined interval

Value must be one of the Following

Every incoming value must be member of a given set of ok-values.

Value may NOT be one of the Following

Every incoming value must NOT be member of a given set of bad-values.

Value must be Roughly Equal

this FAILS if a value is not within an interval defined as setpoint +/- tolerance-fraction. The tolerance fraction is given as 0..1. For example, to check that a value is 100 +/- 5%, use 0.05 as tolerance and 100 as setpoint.

Expression must Evaluate to True

this allows for a Smalltalk expression to be defined as check-condition. The check can therefore perform arbitrary checks (and also more complex combinations of checks)

Sequence (Remembering) Probes

All of the above probe types do checks on individual values and are therefore stateless. The following probes perform checks based on previous incoming value(s). This means that they remember previous values and keep this as an internal state for followup checks. The state is held in the probe itself, and independent of which outer action block was responsible for its activation. Those probes may need a reset, when a followup test sequence is executed, which involves the same probes There are both a UI-menu entries and action blocks for resetting - for individual probes, for probe-groups and for all probes.

All Values must be the Same

All incoming values must be the same as the very first incoming value (i.e. the first appearing value defines the setpoint for all following)

All Values must be Different

No incoming value may appear twice.

Values must be form the same Sequence

The sequence of incoming values must be the same as the setpoint sequence.

Values must be strict/non-strict Increasing/Decreasing

Every incoming value must be >, >=, < or <= the previous value.

Delta to Previous Value must be within an Absolute or Relative Limit

Computes the delta between an incoming value and the previous value and ensures that it lies within a limit.

Average of Deltas to Previous Value must be within an Absolute or Relative Limit

Computes the floating average of all recorded values and ensures that this average always lies within an interval.

Expression must Evaluate to true for the recorded sequence

With every incoming value, that is added to the recorded sequence, and a Smalltalk expression is evaluated on this sequence. Every such expression evaluation must return true.

Value Independent Probes

The previous probe's checks were based on the incoming values. The following probes perform value independent checks.

Any Value must Appear

FAILS if no value appeared ever during the execution of a test case. This probe is resetted at the beginning of a test case (an item in a test plan) and checked at the end of a test case.

The Same Number of Values Must Appear

FAILS if the number of values which appeared during the execution of a test case is not the given setpoint This probe is resetted at the beginning of a test case (an item in a test plan) and checked at the end of a test case. (it may fail early, if more values appear during the run)

At Least/at Most that Number of Values Must Appear

FAILS if the number of values which appeared during the execution of a test case is less/greater than the given setpoint This probe is resetted at the beginning of a test case (an item in a test plan) and checked at the end of a test case. (the "at Most" probe may fail early, if more values appear during the run)

Maximum/Minimum TimeDelta between Incoming Values

This probe ensures that the time-deltas between incoming values are within/outside a given setpoint.

Recording Only Probes

These probes only record incoming values and do not perform any checks. They are useful to generate statistics (average, mean, deviation) or graphics (using a plot/graph action) at the end of a run.

Collect all Values

Collects all incoming values sequentially. Does not perform any check on the values.

Collect all Distinct Values

Collects all incoming values as a set. If a value appears which is already recorded, that value is only remembered once. Does not perform any check on the values.

Collect Counts of all Distinct Values

Collects all incoming values in a bag. For every incoming value, the count of how often it appears is recorded. Does not perform any check on the values.



Copyright © 2014-2018 eXept Software AG