Test Editor/en

Aus expecco Wiki (Version 2.x)
Zur Navigation springen Zur Suche springen
Test Editor after a run

Notice: starting with release 1.8, this editor has been split into separate editor and execution tabs:

Test/Demo
Definition of the network under test. This is used to define a test or demo for the edited action. As such, it combines features of the network editor.
Run
Shows information of the current or last execution of the network under test.

This should not be used for "official" test-runs; instead, use it for the "private" execution of a single block (i.e. unit tests for the test-suite itself) or as a place for demonstration or "executable documentation".

To the right, you see a test editor after the successful execution of a test-block.

Buttons[Bearbeiten]

  • Icon Run.png "Run" - Start execution. All auto started steps will be triggered. Unless enabled in the execution settings, no debugger is opened when an error occurs; instead, the execution is stopped with a failed status (shown in red)

  • Icon Run Debug.png "Debug Run" - Start the test execution in debug mode causing the debugger to open on exceptions (regardless of what is specified in the execution settings).

  • Icon Run Single Step.png "Step" - Start the test execution in single step mode or continue single stepping (when at a breakpoint).

  • Icon Run SingleStepInto.png "Step Into" - Like the above, but steps into other compound actions.

  • Icon Pause.png "Pause" - Pause the current test run. To proceed or single step, click on the according button.

  • Icon Stop.png "Stop" - Stop the execution of the current test run. Exception handling & cleanup (post actions) will be performed.

  • Icon InterruptRun.png "Interrupt" (new in 18.1 aka "2.12") - Interrupt the execution of the current test run and open a debugger. Useful to debug endless loops or to inspect the execution state in a long running elementary action. In the debugger, the execution can be proceeded, single stepped or aborted.

  • Icon PauseOnError.png "Pause on Error" - This toggle controls how the executor behaves when an error is encountered. The default is to stop the test execution and to mark the test case as "FAILED" or "ERROR". However, especially during the test development process itself, it is often useful to be able to proceed after an error. For example, when the test consists of a web-browser's input field values being checked and validated, you may want to continue after a validation error. With this toggle set, the executor goes into the pause-state whenever an error is encountered, and execution can be continued by pressing the "Run" or "Single Step" button again (i.e. it behaves as if "Pause" was pressed).

  • Icon Hard Stop.png "Hard Stop" - Hard-Terminate the execution of the current test run, without doing any exception/cleanup handling. This is useful e.g. for recursion hang-ups, or if a block's cleanup action leads to another blocking situation.


  • Icon Print Report.png Generate a report from the log data


  • Icon Follow Execution.gif Follow execution: auto-select the currently active block in the log (while running).
    In recent expecco versions, this button got replaced by a drop-down box, to selecte the number of levels, which should be followed. The button's icon image is slightly different.

  • Icon FindNextError.png In the log, select the next activity which has finished with an error.

  • Icon Next Inconclusive.png In the log, select the next activity which has finished with an inconclusive state.

  • Icon FindNextActive2.png In the log, select the next activity which is currently executing.
  • Icon BackToPreviouslyVisited.png In the log, select the previously visited activity (i.e. back in visited-history).

The remaining edit-related and breakpoint buttons are described in the network editor.


Views[Bearbeiten]

Activity Tree[Bearbeiten]

The activity tree displays a hierarchical view of the activities that are invoked during the test run, each with a single entry in the tree. If an activity spawns sub activities, these sub activities are enlisted in the branch of the spawning activity, and you can expand or collapse the branch with the node browsing item. Only the first level is expanded automatically when the test run is started. Activities that perform elementary actions never have sub activities, while those that perform compound actions always have at least one sub activity (else they fail due to lack of executable steps inside). Activities on the same level are sorted chronologically by invocation time.

The symbol in front of the activity indicates the state of the activity. This can be either in progress or one of the test results (verdicts): "PASSED", "FAIL", "INCONCLUSIVE" or "ERROR".

Context Menu

A context menu is available in the activity tree view, relating to the Activity that is currently selected:

Find Next Unhandled Error
This option switches the selection in the tree from the currently selected activity to the next, failed activity for which no error handling was performed (and which was not ignored). The search is in the order of the tree hierarchy (i.e. deep-first search, starting at the current selection).
Find Next Error
Similar to the above, but this menu function will find both handled and unhandled errors and failures.
Find Previous Unhandled Error
Similar to the above, but searches backward for the previous unhandled failure or error.
Find Previous Error
Similar to the above, but searches for any error (both handled and unhandled)
Generate Report from here...
This option generates a report of the selected activity and all nested activities.
Open page on selected Item
This opens a new page on the block corresponding to the selected activity.

Network View[Bearbeiten]

This view is shown for compound activities. It displays the activity diagram and highlights steps as they are executed. A step's color reflects its execution state, as described below in "State Colors".

Note that in this view, editing is not possible. You can, however, press Enter after selecting a step, causing a new browser page to open up, showing the selected step's diagram.

Log View[Bearbeiten]

This view shows the log created by the activity. The log consists of the messages (and possibly attached data or screenshots) as generated by the activity (and subactivities) via the "logInfo" / "logWarning" / "logError" and "logFailure" calls (either from elementary code or via the corresponding action steps in a compound action).

The toolbar buttons show or hide different message types. The checkbox named "Sublogs" shows or hides the logs of subactivities.

Pin Data View[Bearbeiten]

This view is available for all types of activities. It shows two lists, the input pin value list to the left, and the output pin value list to the right. In each of the lists, three columns arrange the information. The first field of each row displays the pin name, the second one displays the value, and the third one displays the sender, or the receiver, respectively.

Note that input information is not changing after an activity was spawned, while output values can still be produced or changed during the execution of the activity. In each list, only one row can be selected at a time. According to the selected row, the above icon buttons can be used:

Icon Sender.png
Jump to the sender (the source) of the value. Only available for input pins when a sender object is available. This causes the selection in the activity tree to change to the activity that sent the selected value to this activity.
Icon Receiver.png
Jump to the receiver (the sink) of the value. Only available for output pins when a receiver object is available. This causes the selection in the activity tree to change to the activity that received the selected value from this activity.
Icon Inspect Value.png
Inspect the value object.

State Colors[Bearbeiten]

When a compound action's diagram is executed, a read-only version of the diagram editor is showing the state of the execution. There, steps are colored according to their execution state. The color setting is part of the user settings and can be changed according to your personal taste (or, if you suffer from red-green color blindness, for example). The defaults are:

  • grey/original color
    the step was not yet executed
  • blue
    the step is about to be executed. This means that it has all of its input values (and possibly any required resources) are available and it is ready to run. However, its code or network is not yet being executed - either because other steps are still running and the maximum execution parallelity has been reached, or because the diagram is single stepped, and waiting for you to press the "Step Next" button.
  • bright green
    the step is being executed
  • normal green
    the step has finished execution with success
  • yellow
    the step has finished execution with success, but a warning was generated during its execution. Select the step and select the "Log" tab for details.
  • normal red
    the step finished execution with an error
  • dark red
    the step finished execution with a failure
  • dark grey
    the step was skipped or its execution was aborted. Either due to the user initiating an explicit abort ("Stop" or "Hard Stop" button), or because another step forced the containing diagram's execution to stop

Tasks[Bearbeiten]

Running a Trial Test of the Block[Bearbeiten]

To invoke the test wrapper, click on the start button in the editor's tool bar. The network is executed as usual, by starting with steps marked as autostart. If logging is enabled, the lower monitoring panel monitors the execution status. If logging is not enabled, the monitoring panel will be cleared and remains empty. Logging is initially enabled.

The monitoring panel is split vertically, showing the activity tree on the left, and the observation panel to the right, which displays the current state of the selected activity. Select any activity to see the details in the right area. The depth of the activity tree can be limited by setting a log depth limit. See settings.

If the "follow execution" toggle(Icon Follow Execution.png) has been checked, the selection in the tree automatically follows the currently executing activity.

Debugging[Bearbeiten]

The "normal" reaction to errors during a testplan-run is to finish the test plan with a FAIL or ERROR status. This behavior is OK for test execution; however, during test development, it is helpful to get a symbolic debugger which gives more detail about what happened, the current state (i.e. variables) and how the program got there (the back-trace). The debugger can be enabled via the global settings dialog. These settings are also used if you press the "Run"-button in this test-block runner. However, the additional "Run with Debug" button starts executing with the debugger enabled, ignoring the global settings.

Modifying the Program while Executing[Bearbeiten]

It is possible to change the running program. This may sound a weird thing to do at first, but is very useful when test cases are created in an explorative manner, or changes have to be made to a long running test, and you do not want (or cannot) restart the test from the beginning.

You can change the program at any time (simply edit the action in another tab or window and accept). However, any action being already executed continues execution until finished in the old action definition. Only then (when invoked for the next time) will any changed definitions become effective.

This means:

  • if an elementary block is currently being executed, the changed program text will be effective when the elementary block is called the next time
  • if a compound action is changed, it will use the new activity diagram, when invoked the next time.

To make code and diagram changes an atomic action, all changes are first done to a temporary edit-copy, which is finally installed as an atomic action when the "Accept" button is pressed. If multiple actions are to be edited, any ongoing execution should be paused first, and proceeded once all changes have been done.

Single Stepping and Breakpoints[Bearbeiten]

Steps in an Activity Diagram[Bearbeiten]

Either place a breakpoint on a step and proceed by pressing the "Single Step" button, or start the execution right from the start via the "Single Step" button. There is also a "breakpoint"-action, which can be placed into the network. By conditionally enabling these steps, conditional breakpoints can be added. When a breakpoint is reached, you can:

  • continue single stepping (by pressing the "Step" button),
  • cancel execution (press the "Stop" or "Hard Stop" button), or
  • continue with normal execution (press the "Run" button).

Notice that there are now two different single step buttons; one is showing steps as executed in the currently visible network, the other ("Step In") also halts before any step in any subnetwork is about to be executed.

Code Lines in an Elementary Action[Bearbeiten]

In a JavaScript action, place a line such as:

this.halt();

or

halt();

(in Smalltalk code, write "self halt") somewhere in your elementary block's code. When executed, a debugger will be opened, highlighting the line containing the code-breakpoint.

As an alternative, place a line breakpoint by double-clicking beside the code line in the left side-bar area (a red brakpoint sign will be shown then). Notice that line breakpoints can only be placed at expressions starting in that very line, whereas the above described halt can be placed anywhere, where an expression is allowed.

In the debugger, press one of

  • "Next Line" to single step to the next code line
  • "Step" to single step over the next expression
  • "Send" to step into the next expression
  • "Continue" to proceed.
  • "Abort" to stop the execution

The debugger also shows the calling chain (how did I get there) in the top pane, and the activity's local variables and temporary variables in the lower pane.

The debugger is described in detail in http://live.exept.de/doc/online/english/tools/debugger/TOP.html.

Premature Stop of a Test[Bearbeiten]

Occasionally, it might be required to stop a test even though it has not completed. For example, if you detect an error or malfunction which is not detected by the test, or the test is caught in an endless loop due to an unexpected situation. In this case, press the "Stop" button. This button's function is to ask the test-executor for a controlled termination of the run. Controlled means, that any cleanup actions (post-execution actions) are to be executed. This is the preferred way to stop test execution, because those cleanup actions are meant to free any acquired resources, remove temporary files or to turn off hardware equipment which was acquired and configured by the test. I.e. to bring the state of your equipment back to a well known (initial) state. However, sometimes, the cleanup actions themselves might encounter trouble. For example, if your test requires a login procedure to a remote machine, and the cleanup tries to perform a clean logout, this may block if for example the connection is lost during the test. In this case, the cleanup action will not be able to perform its logout and may even block forever.

To handle this situation gracefully, expecco will wait for some pre-configured time for all the cleanup actions to complete (this is a configurable settings value found in the "extra"-"execution settings"-dialog). After that, a dialog window will appear, asking if the cleanup should be aborted or if you want to continue waiting for the cleanup.

If you are certain that no cleanup action is required, or if you know in advance, that it will fail or block, use the "Hard Stop" button, which terminates execution without even attempting to perform any cleanup actions. Use this with care, as it may leave allocated resources and devices in a non-determined state. It may also lead to file streams or sockets being left open. You can repair this (i.e. close the streams) via the "Extra" - "Debugging" menu, where you fild menu items to find and/or close those streams.



Copyright © 2014-2024 eXept Software AG