Expecco GUI Tests Extension Reference/en

The "expecco GUI Tests Extension" (GUI Browser) provides a common base for the development of test sequences for graphical user interfaces (GUIs) in expecco. The tool integrates seamless into the common expecco UI philosophy. It is used as a common interface for various diferent GUI technologies (Qt, Java/Swing, Android, iOS, MFC, etc). In addition to plain recording & replay and UI-structure inspection (widget tree and attributes), it is possible to try and experiment with individual actions, partial and complex sequences, which can be developed interactivly and added to the suite, when completed ("explorative Test Development").

Main Features

• Seamless integration into the expecco user interface
• One common tool for various different UI technologies
• Hierarchical visualisation of the user interfaces structure (component structure)
• Informationen on state and attributes
• Feedback and convenient localization of interface components through highlighting, screen shots and mouse-over feedback
• Filtering and presentation of function blocks which match the corresponding UI technology
• Creation of test sequences via "Drag &Drop" and/or automatic capture, or import of formal and non-formal descriptions from external documents (XMI, UML, XML, Word, Excel, CSV)
• Experimental try and compose of actions, partial sequences and access pathes
• Incremental creation of complex tests from shorter sequences in the test project

General Design Philosophy

Like any other action in expecco, GUI-interaction and verification is also performed through the execution of actions within activity diagrams.

Thus, for test execution, actions are placed into a test sequence, which control and verify the GUI to be tested or to be manipulated as part of the test. It should be emphasized, that such GUI interaction is not limited to testing: it is also a valuable tool to automate common tasks such as initialization, configuration or deployment proceedures, in situations where manual user interface interaction is required. Such automation may otherwise be difficult even for very simple tasks. For example if the operating system opens dialog boxes for warnings or user confirmations (as often happens in eg. the Windows operating system).

Although similar in many ways, different UI technologies (eg. JavaSwing vs. Android vs. Qt) may provide different operations when automated. Depending on the set of widget elements, the way the UI technology is interfaced and possibly even the way it is configured, some operations depend on the concrete technology. Therefore, each technology is supported by its own GUI interface library, which provides action blocks for the particular technology.

Inside a test suite, multiple such libraries can be imported, and interaction blocks be placed in any order - it is therefore possible and often suseful, to interact with different devices using different technologies even within a single test sequence. For examle, it is quite common to trigger som operation on eg. a mbile device (using actions from the appium interface library) followed by actions to verify some fields in a Java GUI (using actions from the JavaSwing interface library). Of course, it is even possible, to add totally different actions to the same sequence, which would perform database or protocol requests.

Locating UI Elements

When UI interaction blocks are executed, they typically need a way to specify which element is to be manipulated. For exameple, a "Send Button Press" action will need to know, which UI element the event should be sent to. This information is typically named "locator", and different technologies may use different locator styles.

Challenges

Most locators are based on the xPath syntax, which specifies UI elements using a path from the top element (typically the application window or the screen) down to the destination element. Here, a compromise is needed in that we want both deterministic, exact and unique addressing of the UI element, and at the same time the most possible flexibilityy in case of structural or visual changes of the user interface.

As a concrete example, let us assume that a button element is located inside a hierarchy of 3 other panels, which care for the subelement's geometries. The testsuite should be as much as possible independent from both the layout (i.e. the actual position of the button), and also be independent from any change in the hierarchy (i.e. if the next version of the app has the same button embedded in a different number of panels or in another panel.

Whether such independance is possible or not in a particular technology and for a particular application often depends on how the application was built, and whether enough information is available to the test tool.

For example, if the button contains a unique identifier (widgetID) in its attributes, it can be uniquely identified and addressed by this, and the best xPath locator for it would be "/**/button[id=XYZ]". This xPath is completely independent of the hierarchy. This being the optimal situation - in practice, app developpers often fail to be nice to testers and fall into one of the following error-traps: - use of dynamically generated ids - changing ids between versions/releases - changing ids between national language variants - not providing ids at all and not providing any unque alternative

If all of the above are given (an in real world testing, they often are), the xPath must depend on the hierarchy; for example, we could specify it to be the 3rd button in the second panel as: "/**/panel/panel/button[3]". Such an xPath would work as long as no other button gets placed before the target button, and as long as the button is somewhere inside two nested panels.

Solutions

Expecco offers a number of solutions to this problem (but if you can talk to your developper, the first option is always to talk to them and ask them to use constant identifiers, which do not change between versions, and which are independent of national languages or sessions).

Scenario 1: Developers can provide information

It is often possible, that the app-developper can provide widget ID information. Especially, if the UI is generated via a WindowBuilder tool, it is often possible to export an XML or CSV containing such information. The test could then import this file, and generate a mapping table (into an expecco variable, or into anumber of variables) at test startup. Then, instead of using hard-wired xPath strings, those variable values will be used as locators.

Scenario 2: dynamic session-ids

If all sub-element ids can be constructed from a session id, a similar strategy is possible, by remembering this sessionID during startup (in an expecco variable), compute the sub-ids into a table or individual variables, and use those a locators.

Scenario 3: non of the above

This is the worst case fallback. If no information is available at all, you can fall back to either geometric locator information (i.e. specify that the button is in a particular bounding rectangle) or image locator (eg. specifying the bitmap image of the button, and having expecco search for the image in the UI). Of course, these are highly sensible to changes in the application, and it is therefore absolutely recommended, that such geometry information is kept separate from the action blocks (eg. in an attachment or in a file), so that the suite does not need to be changed at many places, whenever the look of the app changes.

GUIBrowser's Role in Test Development

The GUIBrowser helps to deal with the above challenges. You can interatively explore the UI widget hierarchy, find out which attributes are present and what values are found there, try different xPath expressions to locate elements, and collect actions into subsequences interactively. You can at any time switch between the GUIBrowser's view of the application and the test diagram - even in the middle of a stopped or paused test run, change the diagram or parameters on the fly, and possibly proceed or restart test sequences.

Description

Opening a concrete UI

The "expecco GUI Tests Extension" (or "GuiBrowser" for short) provides a common base for the interaction with graphical user interface applications. Various different UI technologies are supported, in that the concrete interface is realized by technology specific plugin extensions and action block libararies. Currently available are interfaces for Qt, Java/Swing, MFC, DevExpress, Android, iOS, Windows Mobile and VNC.

Therefore, in order to use the "expecco GUI Tests Extension", at least one concrete UI-technology plugin is required.

One the other hand, it is possible to handle complex test scenarios in which multiple technologoes are involved (for example, Qt plus Android plus Java-Swing). Any combination of interfaces can be controlled simultaneously, within a single test scenario and even execute in parallel.

Thus, complex integration test scenarios, End-to-End tests and other multi-technology tests are easily created, in which both mobile and web-based and classical user interfaces are to be controlled and verified.

To open the "expecco GUI Tests Extension" within expecco, click on the "GUI Browser" icon in the toolbar.

User Interface

Fig.1: Main View

Fig.2: Usage

Overview on the layout, usage and the functions of the UI. Fig.1 Shows the main view. Fig.2 shows the control flow.

Connection Setup

Provides a selection list with available UI technologies.

After a click, a connection setup dialog opens, which lets you choose among connected devices (i.e. if multiple mobile devices are connected) or applications (if multiple applications are present inside the device or running on a machine). Once connected to a concrete technology/device and application, the components of the user interface are presented in the hierarchical tree on the left.

UI Component Hierarchy (left view)

This view displays both connected and disconnected applications. Underneath each connection, the component (widget-) hierarchy of the application is shown. When selecting a disconnected application, the GUI Browser will try to reconnect and search or a running application, in case the connection was lost or never started.

When an element is selected, matching operations (i.e. actions which are applicable to that element) are shown in the block selection dialog (in the middle), and the component's attributes are shown in the parameter view.

The xPath to the component (the locator) is shown at the bottom. This is an automatically generated xPath, which is usually too specific and inflexible (as described in "challenges" section above). Thus, you may shorten the xPathm and replace intermediate components in it by wildcards ("*" or "**"), and reverify that the optimized xPath is still valid (i.e.referes to exactly the one selected component). You may need a little experience and gut-feeling to get to the optimum - and also often some interaction with app developers, so they know, on what hierarchical details your test suite depends upon.

Tools

A number of functions are available as toolbar buttons or on the selection menu. Most of them are context menu functions and provide operations specific to the selected element.

Disconnect All

Closes all connections of all technologies.

Disconnect Selected

Closes the selected connection (or the connection used by the selected widget element).

Refresh

Reloads the widget hierarchy from the device/application. Some technologies can do this automatically; however, on many, this would take too long and slow down or even interfere with the applications operation. For example, a refresh of an Android app can take a few seconds, depending on the number of UI items, and the speed of the USB connection, and is thereore not done automatically.

Use this when the UI changed without user interaction (normally, the tree is refreshed when you click or enter keyboard input. But sometimes, changes come late or without any user action.)

Reload Child Elements

Only reloads the child elements of the selected element. Notice that not all UI technologies support a partial reload, and these will do a (slow) full reload.

Highlight

Emphasizes the widget's display area (usually by drawing a rectangle around it).

Follow Pointer

When active, the element under the mouse pointer is automatically selected whenever moved over an item in the UI. Not all UI technologies support this function.

Start Recording (on all)

Starts recording your interaction with all conneted devices. I.e. you can interact with any application or mobile device and have all those recorded in one recording sequence. If there is already a partial recording in the sequence view, the new interactions are appended to the end of the sequence.

Stop Recording (for all)

Stops the recording.

Attribute View

Shows the attributes of the selected element. The list of attributes depends on the underlying technology (ie. a Java Swing element may show more attribtues than eg. an Android UI element) and the type of element (ie. a Button will have different attributes than eg. a Combolist). When an attribute is selected, corresponding action blocks are shown in the upper action/attribute list, so that only blocks are presented, which are applicable to the selected element and selected attribute.

Action-Library

Shows action blocks which are applicable to the selected element. These blocks are user-interaction blocks, which generate click, press, release, keyboard or other user events. The list will only show the set of action for a particular kind of element. For example, if an input-field element is selected, actions for text entry will be shown, whereas these are not seen when a panel or label is selected.

Whenever an action block is selected, it is copied to the the test-pane and any locator/path input pin is set to refer to the selected UI element. I.e. it will be presented as it would be used within a test sequence. You can now either execute the action, or copy it to the end of the sequence (by presing the "Add to Sequence" button in the top-right. O course, you can also edit the network in this test view and add any other action block or modify the pin parameters.

If you double click on an action block, it is imemdiately added to the recorded sequence.

Test Network and Sequence Views

As already mentioned above, the "Test" pane shows the selected block from the action/attribute list. You can edit and change the pin setup. Also, the block can be executed and the results are shown in the lower right execution log window. Finally, the block is added to the end of the recorded sequence via the "Add to Sequence" button.

The "Sequence"-View presents the test sequence as recorded so far. This is a regular diagram editor as described elsewhere. So any of the well known editong features can also be used here. Especially, it is possible to insert other blocks from the library, to change the execution order of recorded steps, or to replace them by other steps (without a need to re-record the sequence). The sequence view also allows for the whole sequence to be replayed in whole or in part, to be single stepped, breakpointed etc. Finally, when complete, you can create a new action block in your project, which contains the recorded sequence..

Path Display

Shows the XPath (locator) of the selected UI element. By default, a full explicit path is automatically generated and added as freeze value to the locator pins (in the recorded sequence). However, it is often desirable to shorten this path via placeholders. This will make you test less affected by changes in the widget hierarchy. Therefore, it is useful to reduce the path as long as it uniquely identifies the desired element. The details depend on the way the UI was programmed; for example, if each element has a unique ID, no path is needed at all, and the locator can be something like "/*[ID=xxx]". In contrast, if no ID or other unique attribute is present, a full path which enumerates the parent hierarchy might be required. The "Check Path" button will apply the entered path to the element hierarchy and give an OK-response if the element is still uniquely identified by it.

Component Preview

Display a preview (snapshot image) if the selected UI element. To see the image in original dimensions, click on it.

Bedienelementpfad

Der "Bedienelementpfad" dient als Referenz auf Bedienelemente innerhalb der Benutzeroberfläche. Er ist vergleichbar mit der XPath-Notation im XML Umfeld. Komponentenpfade werden bei der Testentwicklung festgelegt und später, bei der Testausführung, verwendet um UI-Komponenten zu referenzieren. Der Pfad eines ausgewählten Bedienelements wird unten im "Bedienelement-Pfad" Feld angezeigt, und bei Auswahl eines Elements automatisch aktualisiert. Umgekehrt kann er an dieser Stelle auch geändert und überprüft werden, welche(s) Element(e) durch einen gegebenen Pfad referenziert werden. Der Pfad muss nicht vollständig sein - er kann dynamische Teile enthalten. Dies erhöht die Flexibilität und macht die Testsequenz robuster gegenüber einer möglichen späteren Änderung der getesteten Benutzeroberfläche. Insbesondere gegenüber Änderungen der Hierarchie, z.B. wenn weitere Containerelemente eingefügt werden.

Hinweis: Viele Benutzeroberflächen ändern während der Ausführung die Struktur dynamisch. Oft werden Container- und Bedienelemente während dem Lauf hinzugefügt oder entfernt. In solchen Fällen sind dynamische Teile im Pfad sehr hilfreich.

Pfade Definieren

Ein typischer Pfad kann wie folgt aussehen:

/frame[@name='Notepad']/root pane/layered pane/panel/panel/panel/tool bar/push button[2]

Dieser Pfad beschreibt die vollständige Adressierung des zweiten Buttons in einer Toolbar, welche ihrerseits in einer verschachtelten Containerhierarchie (panels) liegt.

Der Pfad wird beschrieben durch eine Liste von Identifikatoren, getrennt durch und beginnend mit "/". Jeder einzelne Identifikator beschreibt eine Unterkomponente des bereits durch den linken Teilpfad beschriebenen Elements. "/frame[@name='Notepad']" referenziert das Hauptfenster der Anwendung. "root pane", "layered pane", "panel" etc. referenzieren jeweils das korrespondierende Kindelement. Zur Referenzierung einer einzelnen konkreten Komponente wird der Pfad beginnend mit dem obersten Bedienelement schrittweise verfolgt.

Im Beispiel wird zuerst versucht, das Fenster mit Namen "Notepad" zu finden. Das Fenster hat ein oder mehrere Kinder, dessen Identifikator zu "root pane" passt. "root pane" wiederum hat Kinder die zum Identifikator "layered pane" passen und so weiter.

Die Syntax eines jedes individuellen Identifikators ist:

/<KnotenIdentifikator><[Optionaler Selektor]>

KnotenIdentifikator

Der KnotenIdentifikator beschreibt den Typ des Bedienelementes das gesucht wird.

Mögliche Identifikator:

Der Typ des Bedienelements

Das Ergebnis sind eine oder mehrere Bedienelemente die dem angegebenen Typ entsprechen.

Der "Jeder" Identifikator ("*")

Löst den Rest des Pfades unter Verwendung alle Kindelemente, unabhängig vom Typ des aktuell aufzulösenden Identifikators auf.

Beispiel:
Vollständiger Pfad:

/frame[@name='Notepad']/root pane/layered pane/panel/panel/panel/tool bar/push button[2]

Der selbe Pfad mit "*" Identifikatoren:

/*[@name='Notepad']/*/layered pane/*/panel/*/tool bar/push button[2]

Diese zwei Pfade adressieren in obigem Fall das selbe Bedienelement. Jedoch ist die zweite Variante unempfindlicher gegenüber geänderten Komponenten in der Hierarchie. Zum Beispiel könnten die Entwickler der zu testenden Anwendung eine panel-Komponente durch eine box-Komponente ersetzen. Der erste Pfad würde dann im Testlauf zu einem Fehler wegen nicht auflösbarem Pfad führen. Zu beachten ist, dass der "*"-Identifikator zur als Platzhalter einer einzelnen Komponente dient. Um den Pfad auch robust gegenüber zusätzlich in die Hierarchie eingebrachte Komponenten zu machen, sollte die "//"-Notation (siehe unten) verwendet werden.

Selektor

Der Selektor ist Optional. Er wird benötigt um Bedienelemente mit gleichem Typ zu unterscheiden. Gibt es beispielsweise 3 Buttons als Kinder eines Bedienelements, so kann mit dem Selektor der zu referenzierende Button festgelegt werden. Der Selektor wird in eckigen "[ ]" Klammern angegeben. Der Selektor wird nach der Auflösung des KnotenIdentifikators auf das Ergebnis angewendet. Gibt es zum Beispiel ein Bedienelement mit zwei Buttons und zwei Checkboxen als Kinder, und der Identifikator liefert als Ergebnis 2 Buttons, so wird der Selektor auf diese 2 Buttons angewendet.

Mögliche Selektoren:

Der "Index" Selektor

Wählt das Bedienelement anhand des Index. Der Index ist 1 basiert (i.e. [1] bezieht sich auf die erste Komponente).

Der "Schlüssel-Wert" Selektor (engl. "Key-Selector")

Wählt das Bedienelement dessen Schlüssel den angegebenen Wert hat. Die Menge der möglichen Schlüssel ist abhängig von der jeweiligen UI-Technologie, der betroffenen Komponente und deren Attribute. Typische Schlüssel sind "name", "id", "label", "value" etc. Bei einigen Komponenten sind auch dynamische Schlüssel möglich, so dass diese erst während der Laufzeit angelegt werden.

Beispiel (Index Selector):

/*/tool bar/push button[2]

Wählt den zweiten Button innerhalb der Toolbar aus.

Beispiel (Schlüssel Selector):

/*/tool bar/push button[@name='Save As...']

Wählt den Button aus dessen Attribut "name" (= Schlüssel) den Wert "Save As..." hat.

Platzhalter Schreibweise

Die Platzhalterschreibweise ("//") kann Pfade stark verkürzen und flexibel gegenüber zusätzlich eingefügten Hierarchieebenenen machen. "//" ersetzt dabei ganze Teile des Pfades. Der Vergleich findet nicht nur auf Kinder eines Bedienelementes statt, sondern auf ganze Teilpfade.

Beispiel:
Angenommen der vollständige Pfad eines Buttons wäre folgender:

/frame[@name='Notepad']/root pane/layered pane/panel/panel/panel/tool bar/push button[@name='Save As...']

Außerdem angenommen, die Anwendung hat nur einen Button mit dem Namen "Save As...", welche sichtbar ist. Dann kann der Pfad mittels Platzhalterschreibweise wie folgt verkürzt werden:

//push button[@name='Save As...']

Noch kürzer ginge es, wenn nur ein Bedienelement mit Namen "Save As..." existiert. Dann wäre folgender Pfad möglich:

//*[@name='Save As...']

Auch ist es möglich, wie im folgenden Beispiel, nur Teile des Pfades mit Platzhaltern zu versehen:

/*[@name='Notepad']//panel//tool bar/*[@name='Save As...']

Hinweis: Die verkürzten Pfade in den oben gezeigten Beispielen sind wesentlich unanfälliger gegenüber Änderungen im Layout der Benutzeroberfläche. Jedoch erhöht es je nach Anwendung und Benutzeroberfläche auch die Wahrscheinlichkeit, dass mehrere passende Bedienelemente gefunden werden. Dies wird dann ebenfalls zur Laufzeit zu einem Fehler führen ("Nicht eindeutiger Komponentenpfad"). In obigem Beispiel könnte dies passieren, wenn im UI neben dem Button auch ein Menueintrag mit dem selben Text existiert. In der Praxis wird die Auswahl des geeigneten Pfades immer einen Kompromiss aus Flexibilität (möglichst kurz) und Eindeutigkeit darstellen.

---

Copyright © 2014-2024 eXept Software AG