Manueller Test Tutorial/en: Unterschied zwischen den Versionen

Aus expecco Wiki (Version 2.x)
Zur Navigation springen Zur Suche springen
(Weiterleitung zur neuen Seite)
Markierung: Neue Weiterleitung
 
(4 dazwischenliegende Versionen von einem anderen Benutzer werden nicht angezeigt)
Zeile 1: Zeile 1:
#WEITERLEITUNG [[Manual_Test_Tutorial/en]]
Note: sorry for the German screenshots. We'll update ASAP (but things should be clear despite this ;-)

= Manual Tests =

At first sight, it may seem strange to cover manual tests in the context of "automation". However, there are many situations in which manual operations are required - even in an automated test. For example: power on a device, readout of measured values, visual inspection etc.

In addition, expecco helps you to migrate: if you are already performing manual tests, and you can provide machine readable requirements- or test specifications, then expecco can import those and run them as manual tests.

Although still executed manually, this will give you better reports and - probably even more of significance - the guaranty that the steps where executed properly.

You can also migrate incrementally from fully manual tests, via partially automated tests to fully automated tests, by starting with manual test steps and replacing them with automated actions one by one.

Even these partial automated tests will save time and money, as repeated actions (like filling out forms , confirming dialogs or reading measurement values) run much faster when automated and thereby reduce the overall execution time.

If you are impatient, skip right to the [[#Manual Test Example (Step by Step Tutorial) | "'''Step-by-Step Tutorial'''"]] further down, and return to the details later.

== Combining Manual Test Steps with Automated Steps ==

You can arbitrarily combine manual steps with any other action step inside a diagram or test.
For example, a manual step might ask the user to turn on power, to plug a cable, to perform a visual inspection or to perform any other operation involving human interaction.

Also, partial checks which are not (yet) automated can be included this way. If you put these into separate wrapper actions, these can later be replaced by automated ones, without a need to change other parts of your suite.

Especially remember that such actions can be palced into a separate imported expecco action library, so that later, a simple reimport operation is required to switch to the automated version.

= The Manual Test Wizard =

If any manual test step is executed by expecco, a dialog window (the so called "''Manual Test Wizard'') is opened, which guides the user and provides hints about the test to be performed.

The tester will be guided in a step-by-step fashion by the wizard, who can add comments, data or screenshot as documentary of the test run, and confirm each step as ''successful'' (possibly with additional notes), as ''failed'' or as ''inconclusive''. The "'íncionclusive''"outcome means that the test step could not be performed (for example because a device was offline), and therefore, no statement about the proper operation of the tests system could be made. Such tests have to be repeated later.
The wizard's user interface consists of 3 main areas:
# Toolbar area
# general informationen on the current step
# Orders on what do do in the current test step


[[Datei:Manual_Test_1.png|Manual Test GUI]]

== The Toolbar ==
The toolbar provides functions functions to set the outcome of the current step, to add comments and to attach data files or screenshots to the report.

Of special relevante are the "Next"buttons (triangular right arrows); these confirm the current step and mark it as "''Successful''" (green), "''Failed''" (red) or "''Inconclusive''" (gray).
The yellow button also stands for "''Success''", but asks for a comment to be added as warning to the final report and log file.

The other buttons are used to attach files or screenshots.

== Informationen on the current step ==
This area show the current step's name, the test case, of which it is part of, and the test plan which is currently running. Some of this depends o whether the action is executed as part of a test plan, or as a single-action run (in the "test/demo"tab of the editor):

[[Datei:Manual Test 2.png|Manual Test Info Bar]]
Whether displayed as TEST-CASE or TEST-STEP depends on corresponding tags of the current action. Although this has no semantic meaning w.r.t. the execution of actions it is useful to organize and structure the suite into layers.

Testates are identified by having a tag named TEST-CASE or TEST_CASE. Either of them should be present in tags found in the documentation area of the test case's action definition.
If you create such action via the toolbar above the project tree, these tags will be automatically set. But you can cahnge these tags any time later if required.

[[Datei:Manual Test 3.png|Manual Test Side Bar]]

Test steps are marked with TEST-STEP tag. Likewise, this is to be found in the documentation editor at the bottom. This tag is also automatically applied if the step is created via the toolbar.

[[Datei:Manual Test 4.png]]

Actions marked as TEST-CASE- or TEST-STEP-Tags are shown with special icons in the tree. This is for them to be easier to find and to be somewhat standing out of other actions (but as already mentioned: has no other semantic meaning).

[[Datei:Manual Test 5.png]]

== Orders on What to Do in Test Step ==
The wizard's lower area present the test step's description.

Depending on how the test step was defined, this can be presented as simple plain text, or in a more appealing HTML presentation. See more on this in the ''Standard Teststep'' doc below.

Simple Plaintext presentation:
<br>[[Datei:Manual Test 6.png|300px]]

HTML Contents:
<br>[[Datei:Manual Test 7.png|300px]]

== Manual Test Action Library ==

Manual test sequences are usually created by placing individual test steps via drag&drop from the Manual Test Library into a diagram.

In this library you'll find the 3 most important actions to execute manual test steps, in addition to support actions eg. to open/close the wizard etc).

Usually you will not need those, as the wizard opens automatically when the first manual test step is executed. However, they are useful to open the wizard at a particular screen position or with a different size, minimized to not cover the screen, or full screen.

In addition, the library contains examples with extended dialogs which can be shown inside the wizard (eg. with multiple Input fields). You can even define your own layouts and use the examples there as a starting point for your design.

[[Datei:Manual Test 8.png]]

=== Manual Test Wizard – Start ===

This action opens the wizard with individual values for position, size and behavior.

If this step is not present, the wizard will be opened with default values when the first manual test step is executed.

InputPin Parameter:
* origin:<br>defines the upper-left corner of of the wizard window in pixel coordinates on the screen. It should be given as a Point instance with x/y coordinates.<br>Example: 20@40 means 20 pixel from the left, 40 pixel from top origin of the screen.
* extend:<br>defines the size of the window given as a Point instance with width/height values.<br>Example: 600@400 means 600 pixel width and 400 pixel height.
* wizardWindowMode:<br> also defines the size of the window; it can be one of:
** Normal<br>the wizard window opens as specified by the above values (or defaults).
** Minimize<br>the wizard wind gets the specified or default values, but will be minimized initially.
** fullScreen<br>size and position are ignored; the window will take the full screen.
** fullHeight<br>use the specified width but the whole screen's height.
** fullWidth<br>use the specified height but the whole screen's width.
** fullWidthMinimize<br>minimized in full width. 
* wizardDecorationMode:<br>defines the window's decoration:
** full<br>as usual incl. minimize, maximize and close controls.
** None<br>no window controls.
** Dialog<br>with control to close.
* allowInconclusive:<br>to hide the button for "''Inconclusive''". If off, the tester can only give success/fail results.

<br>

=== Manual Test Wizard – Preset Values ===

You can preset the layout before the wizard is opened the first time with this action.
The meaning of the values is as described in the above “<CODE>Manual Test Wizard – Start</CODE>” action; these will be used when the first manual test step is executed.

=== Manual TestStep (API) ===

You'll find a actions for different uses in the library. The 3 most heavily used are described below.

be reminded, that the presented texts are not required to be defined as constants. They might of course be generated by another step. For example, they could could be read from a database or a file, or be generated dynamically from text fragments.

==== Manual TestStep (Text) ====

This action presents the texts as simple text.

The input pin "''testStepDescription''" expects the text. This plain text will be shown "as-is" in the step and the same in the wizard.

[[Datei:Manual Test 9.png]]

==== Manual TestStep (HTML) ====

This action presents the text in the wizard as HTML formatted text.

The input "''testStepDescription''" expects an HTML string. You schuld have some experience with HTML, as the input must contain valid HTML syntax. However, you may also put the text into an attachment or a document element and use the document editor to enter HTML.

Notice: the embedded HTML viewer only supports a subset of the full HTML. For example, the style element is unsupported and tables do not have the full set of features).

The HTML-viewer in the wizard will present the text formatted.

[[Datei:Manual Test 10.png]]

==== Manual TestStep (SimpleWikiStyle) ====

A text format commonly used in Wikis (so called "Wikistyle") can be used to format simple bullet lsits, headlines etc. without a need for HTML. For that, text in Wiki-style can be internally translated to HTML and presented as such.

The input "''testStepDescription''" expects the Text in simple WikiStyle syntax.
The formatting rules are simple:
* headlines
:lines starting with "=" are presented as headline. Subchapters with '==' and '===':
: &#61; &lt;h1&gt;
: &#61;&#61; &lt;h2&gt;
: &#61;&#61;&#61; &lt;h3&gt;
* Horizontal Line
:lines starting with or containing only a single "-"(minus) will be presneted as a horizontal separating line:
<hr>
* Lists
:lines starting with &#42; are presented as &lt;li&gt; ("Bullet List").
:All other lines are unformatted. In addition, you may embed HTML tags (for example "<br>").

Here as well, the text is presented in the wizard in an HTML viewer.

= Manual Test Example (Step by Step Tutorial) =

== Step 1: Create a New Test Suite ==

Open expecco and create a new test suite. You may of course also add manual test steps to any other suite. Then import the Manual Test Library (which is done automatically, as soon as you create manual test step or test case).

[[Datei:Manual Test 11.png]]

== Step 2: Create a Test Case ==

Next we have to create a test case item in the tree. The easiest way is via the toolbar menu, which creates an action with the TEST-CASE tag already attached, and it appears with its own icon in the tree.

Remark: the TEST-CASE tag is irrelevant, regarding the later execution; it is only a tag to affect its presentation, and makes it easier for you to spot test cases, test steps as oppose to the helper action. Other attributes of the appearance are specific colors or icons; you can define them as to your personal preferences to enhance the "readability" of your suite.

[[Datei:Manual Test 12.png]]

Note: please always give your test cases (and other items) reasonable and descriptive names. Usually, test cases should include a reference number, which refers to the corresponding items or chapter in your test- or requirements specification. In addition, append a short Description after the number so the humans also see what the individual steps do.
Example: "TC-4711 Login with valid Name".
Note: it has turned out to be a best practice to name test cases as "TC_xxx" and test steps which naht be reused in other scenarios as "TS_xxx". Other helper actions should have their function in the name.
Also please add a short documentation (Documentation-Tab). This helps to find actions and is also useful as these will be shown as tooltip ("fly by help") when you hover the mouse over a tree item or step in a diagram.

Alternative:<br>You can also mark actions afterwards as test case, by adding the corresponding tag later. Click first on "New Compound Action":

[[Datei:Manual Test 13.png]]

then navigate to the "''Documentation''" tab of the action and add the tag "''TEST-CASE''" to the list of tags at the bottom of the editor. Confirm by pressing "Accept" in the editor pane on the right side.

The icon shown in the project tree changes to "test case".

[[Datei:Manual Test 14.png]]

Thus, by adding the tag, any action may later be marked as being a test case or test step. You can also use the tree's popup-menu to add this tag to multiple actions at once.

== Schritt 3: Create a Test Step for a Test Case ==

navigate to the Diagramm tab of your test case.
Initially, this is empty and has to be filled with individual steps. (sorry, in the picture below this is still named "New Action").

Teure are various options to add a step:
* by drag & drop from the project tree,
* by copy-paste from the project tree or from another diagram
* by searching among existing actions (dialog), directly in the diagram (menu: "''New Step...''").
* by creating a new action (dialog), directly in the diagram (menu: "''New Action...''").

In the following, new steps are added via the diagram's popup menu (right click). The menu item "New Action" opens a dialog which creates a new action and adds it as step into the diagram:

[[Datei: Manual Test 15.png]]

This opens a little input dialog.
Enter the name of the new test step, honor the rules as above.

Note:
If you intent to reuse the test step in multiple test cases later, the name should not contain step-numner or test case number. Use names like "''TS - Open Application''".

After confirming the dialog with ÖK", a cursor appears to let you specify the position of the new step in the diagram. Place the step and confirm the changes with "Accept" to save the editor's changed diagram in the tree.

The project tree now shows the new step below the test case.

[[Datei:Manual Test 16.png]]

[[Datei:Manual Test 17.png]]

Sorry, but actions created this way did not automatically get the right tag (in previous expecco versions). Therefore, you may have to navigate to the test step's definition in the tree, got to the documentation tab and add the "TEST-CASE" tag (or better yet, a "MANUAL-TEST-CASE" tag. As usual, click on "Accept" when done.

== Step 4: Define What is to be Shown in the Wizard ==

We now define what the wizard should show when the "TS - Open Application" Step is executed.

Up to now, we did not actually define whether this step will be a manual or automated test step. This is done now, by placing an action from the Manual Test Library into our action step.

Notice:
:in theory, we could have added the manual test step right into the test case. However, this has the disadvantage that this might require more rework if we ever want to change this step to a fully automated one. We would have to change the test case and probably all places were the test step is also used.
:By wrapping the actual wizard action into separate action, enables use to change the inside at any time later, without a need to change all uses of that step. We'll thus get more maintainability and cost reduction in the future.
:It is a good practice to place any action which might be changed in the future into a separate compound action / even if it seems strange to have such simple diagrams which only cnsist of a single step.

Now choose, how the text should be presented to the test and how you want to enter it (i.e. Text/HTML/Wiki). For that, select the test step action in the tree, go to the diagram and choose "''New Step...''" from the popup menu.<br>(this time not "New Action", as we want to select an existing action instead of creating a new one):

[[Datei:Manual Test 18.png]]

Choose the action to be placed into the diagram as step. If the list is too long, filter by listong only individual libraries (in this case: "Manual Test Library") or by name (you might afilter for "HTML" or "Wiki" here) or by tag.

In this example, we choose: "<CODE>Manual Test Template (WikiStyle)</CODE>" (the action shown in the screenshot is somewhat obsolete - therefore different).

[[Datei:Manual Test 19.png]]

Confirm with "OK" to add the step to the diagram.

[[Datei:Manual Test 20.png]]

Doubleclick on the input pin named "''testStepDescription''" to open an editor for the text. Alternatively, choose "''Freeze as String''" from the pin's context menu to open the text editor.

[[Datei:Manual Test 21.png]]

Type the text which is to be shown in the wizard eg. according to WikiStyle rules.

[[Datei:Manual Test 22.png]]

Again confirm with "OK". The text is now attached to the input pin as so called "''Freeze Value''".

[[Datei:Manual Test 23.png]]


Finally, click on "''Accept''" to make your changes permanent in the project tree.

== Schritt 5: Testfall einzeln ausführen und Anzeige im Wizard verifizieren ==

Wählen sie den den Testfall im Projektbaum aus und &ouml;ffnen sie den Tab "''Netzwerk''".

Drücken Sie den "Play"-Button in der Toolbar des Netzwerkeditors.
Das Wizard-Fenster erscheint ge&ouml;ffnet und der Testschritt wird angezeigt:

[[Datei: Manual Test 24.png]]

Die Testausführung nun auf Ihre Eingaben im Wizard. Wenn Sie im Wizard das grüne Play-Symbol drücken, schaltet dieser weiter und stoppt nach der Ausführung des letzten Schrittes (in diesem Fall, einem Einzeltest, gleich nach dem Ersten).
Die Testausführung und der Testfall werden als "''Passed''" markiert (sofern Sie "grün" gedrückt haben).

Der Wizard bleibt geöffnet und kann nun von Hand geschlossen werden.

== Schritt 6: Erstellen eines Testplans, und automatisches Öffnen und Schließen des Wizards ==

Bisher wurde der Testfall im Einzeltestmodus innerhalb des Editors gestartet. Dies ist die normale Vorgehensweise: testen Sie Ihre Aktionen immer in kleinen Abschnitten, damit Sie im Fehlerfall nicht jedes mal eine lange Kette von anderen Aktionen durchklicken müssen.

Anmerkung:
:Erst dann sollten Sie Ihre Testpläne, in denen dann mehrerer Testfälle hintereinander ablaufen, in einem Gesamtlauf testen. In unserem Beispiel ist nicht mit Problemen zu rechnen; wenn Sie aber z.B GUI-Tests erstellen, sollte jeder Testfall das System (die Oberfläche) in einem Zustand hinterlassen, in dem der nächste Testfall weiter laufen kann. Typische Fehler in GUI Tests sind z.B. offen gelassene Dialoge oder Popup Fenster, falsche Selektion in Listen etc.

: Daher versichern Sie sich im Einzeltest, dass solche Situationen nicht entstehen.

=== Erstellen eines Testplans ===

[[Datei:Manual Test 25.png]]

Wählen Sie den neuen Testplan im Projektbaum aus.
Ziehen Sie dann die Aktion des Testfalls mittels Drag & Drop in die Liste der Testfälle rechts im Testplan Editor.
Wie Sie bemerken, kann ein Testfall in mehreren Testplänen wiederverwendet werden. Daher ist die Position im Projektbaum irrelevant für den Testplan: Sie können den Projektbaum beliebig nach Ihren Vorstellungen organisieren, und ebenso die Listen in den Testplänen unabhängig davon beliebig sortieren.

[[Datei:Manual Test 26.png]]

[[Datei:Manual Test 27.png]]

Wenn Sie möchten, können Sie jetzt obige Schritte mehrfach wiederholen, und mehrere unterschiedliche Testfälle in den Testplan ziehen.

=== Definieren von Pre- und Postaktionen ===

Jetzt soll noch eine Aktion erstellt werden die als Pre-Aktion vor der Ausführung des Testplans ausgeführt werden soll; in unserem Fall: Start des Wizards mit definierter Grösse. Zur Erinnerung: wenn Ihnen der Wizard in seiner Defaultgrösse passend ist, brauchen sie dies nicht; wir möchten hier aber z.B. eine Fullscreen Darstellung einstellen.

Nach der Ausführung soll der Wizard in einer Post-Aktion auch wieder automatisch geschlossen werden.

Es gibt dafür mehrere mögliche Lösungen, wobei wir die dritte unten empfehlen:
* (1) Open und Close Aktionen als Testfall in die Liste im Testplan
* (2) Open als Pre-Aktion; Close als Post-Aktion (Felder finden Sie unten im Testplan Editor)
* (3) Wrapper Aktionen Definieren, eine mit nur der Open-Wizard Aktion, eine nur mit Close-Wizard, und diese als Pre/Post Aktionen definieren.

(1) ist schlecht da im Fehlerfall die Abarbeitung des Testplans abgebrochen werden könnte und dann ein Wizardfenster offen bliebe. Das ist kein Problem, wenn Sie vor dem Schirm sitzen und ihn schliessen können; jedoch wird damit eine Vollautomatisierung u.U. Probleme haben, und Fenster auf dem Schirm stehen bleiben.
(2) ist besser, da die Post-Aktion immer ausgeführt wird; auch wenn die Ausführung des Testplans abgebrochen wird.
(3) ist noch besser, da Sie nun mit der eigen definierten Post-Aktion einen Ort haben, an dem später weitere "Aufräumarbeiten" untergebracht werden können (z.B. temporäre Dateien löschen, Strom ausschalten etc.), und Sie später weniger Wartungsaufwand haben, diese in möglicherweise mehreren Testpläne nachzutragen.
Wir werden also zwei neue Aktionen definieren, diese mit "PreAction"und "PostAction" benennen und als solche auch im testplan definieren.

Fügen Sie zur PreAction die "''Manual test Wizard -Start''" Aktion mittels "''New Step''" oder Drag & Drop hinzu;
<br>und dann platzieren Sie die "''Manual test Wizard -Stop''" Aktion in der PostAction.

So könnte eine mögliche Konfiguration aussehen:

[[Datei:Manual Test 28.png]]

Zuletzt definieren wir diese Aktionen im Testplan als Pre- und Post Aktionen.
Öffnen Sie dafür den Testplan Editor, und ziehen die jeweilige Aktion in der Felder unten im Editor.

[[Datei:Manual Test 29.png]]

Drücken Sie "''Übernehmen''" um die Änderungen zu speichern.

Führen Sie nun den Testplan aus, wird der Wizard entsprechend der Konfiguration im Startblock geöffnet,
der Testfall ausgeführt und am Ende die Postaction den Wizard wieder schließen.

= Weitere Beispiele =

In der Manual Test Bibliothek selbst finden Sie einen Beispieltestplan mit mehreren manuellen Schritten.

Auch finden Sie in den mitgelieferten Demobeispilene die Testsuite "''ManualTestExample.ets''". Diese enthält ein gebrauchsfertiges Beispiel mit einem Setup ähnlich dieser Beschreibung.

Aktuelle Version vom 22. August 2023, 10:48 Uhr

Weiterleitung nach:


Copyright © 2014-2024 eXept Software AG

Abgerufen von „https://doc.expecco.de/w2.x/index.php?title=Manueller_Test_Tutorial/en&oldid=28637“