Expecco GUI Tests Extension Reference/en: Unterschied zwischen den Versionen

Aus expecco Wiki (Version 2.x)
Zur Navigation springen Zur Suche springen
Zeile 29: Zeile 29:
-->
-->


The "expecco GUI Tests Extension" ("''GUI Browser''") provides a common base for the development of test sequences for graphical user interfaces (GUIs) in expecco. The tool integrates seamless into the common expecco UI philosophy. It is used as a common interface for various different GUI technologies (Qt, Java Swing, Android, iOS, MFC, VNC, etc). In addition to plain recording & replay and UI-structure inspection (widget tree and attributes), it is possible to try and experiment with individual actions, partial and complex sequences, which can be developed interactively and added to the suite, when completed ("explorative Test Development").
The "expecco GUI Tests Extension" ("''GUI Browser''") provides a common base for the development of test sequences for graphical user interfaces (GUIs) in expecco. The tool integrates seamless into the expecco UI philosophy. It is used as a common interface for various different GUI technologies (Qt, Java Swing, Android, iOS, MFC, VNC, etc). In addition to plain recording & replay and UI-structure inspection (widget tree and attributes), it is possible to try and experiment with individual actions, partial and complex sequences, which can be developed interactively and added to the suite, when completed ("explorative Test Development").


=Main Features=
=Main Features=

Version vom 18. Oktober 2018, 09:24 Uhr


The "expecco GUI Tests Extension" ("GUI Browser") provides a common base for the development of test sequences for graphical user interfaces (GUIs) in expecco. The tool integrates seamless into the expecco UI philosophy. It is used as a common interface for various different GUI technologies (Qt, Java Swing, Android, iOS, MFC, VNC, etc). In addition to plain recording & replay and UI-structure inspection (widget tree and attributes), it is possible to try and experiment with individual actions, partial and complex sequences, which can be developed interactively and added to the suite, when completed ("explorative Test Development").

Main Features[Bearbeiten]

  • Seamless integration into the expecco user interface
  • One common tool for various different UI technologies
  • Hierarchical visualization of the user interfaces structure (component structure)
  • Information on state and attributes
  • Feedback and convenient localization of interface components through highlighting, screen shots and mouse-over feedback
  • Filtering and presentation of function blocks which match the corresponding UI technology
  • Creation of test sequences via "Drag &Drop" and/or automatic capture, or import of formal and non-formal descriptions from external documents (XMI, UML, XML, Word, Excel, CSV)
  • Experimental try and compose of actions, partial sequences and access paths
  • Incremental creation of complex tests from shorter sequences in the test project

General Design Philosophy[Bearbeiten]

Like any other action in expecco, GUI-interaction and verification is performed through the execution of actions within activity diagrams.

Thus, for test execution, actions are placed into a test sequence, which control and verify the GUI to be tested or to be manipulated as part of the test. It should be emphasized, that such GUI interaction is not limited to testing: it is also a valuable tool to automate common tasks such as initialization, configuration or deployment procedures, in situations where manual user interface interaction is required. Such automation may otherwise be difficult even for very simple tasks. For example if the operating system opens dialog boxes for warnings or user confirmations (as often happens in e.g. the Windows operating system).

Although similar in many ways, different UI technologies (e.g. Java Swing vs. Android vs. Qt) may provide different operations when automated. Depending on the set of widget elements, the way the UI technology is interfaced and possibly even the way it is configured, some operations depend on the concrete technology. Therefore, each technology is supported by its own GUI interface library, which provides action blocks for the particular technology.

Inside a test suite, multiple such libraries can be imported, and interaction blocks be placed in any order - it is therefore possible and often useful, to interact with different devices using different technologies even within a single test sequence. For example, it is quite common to trigger some operation on e.g. a mobile device (using actions from the appium interface library) followed by actions to verify some fields in a Java GUI (using actions from the Java Swing interface library). Of course, it is even possible, to add totally different actions to the same sequence, which would perform database or protocol requests.

Locating UI Elements[Bearbeiten]

When UI interaction blocks are executed, they typically need a way to specify which element is to be manipulated. For example, a "Send Button Press" action will need to know, which UI element the event should be sent to. This information is typically named "locator", and different technologies may use different locator styles.

Challenges[Bearbeiten]

Most locators are based on the XPath syntax, which specifies UI elements using a path from the top element (typically the application window or the screen) down to the destination element. Here, a compromise is needed in that we want both deterministic, exact and unique addressing of the UI element, and at the same time the most possible flexibility in case of structural or visual changes of the user interface.

As a concrete example, let us assume that a button element is located inside a hierarchy of 3 other panels, which care for the subelement's geometries. The testsuite should be as much as possible independent from both the layout (i.e. the actual position of the button), and also be independent from any change in the hierarchy (i.e. if the next version of the app has the same button embedded in a different number of panels or in another panel.

Whether such independence is possible or not in a particular technology and for a particular application, often depends on how the application was built, and whether enough information is available to the test tool.

For example, if the button contains a unique identifier (widgetID) in its attributes, it can be uniquely identified and addressed by this, and the best XPath locator for it would be

"/**/button[id=XYZ]"

This XPath is completely independent of the hierarchy. This being the optimal situation - in practice, app developers often fail to be nice to testers and fall into one of the following error-traps:

  • use of dynamically generated ids
  • changing ids between versions/releases
  • changing ids between national language variants
  • not providing ids at all and not providing any unique alternative

If all of the above are given (and in real world testing, they often are), the XPath must depend on the hierarchy; for example, we could specify it to be the 3rd button in the second panel as:

"/**/panel/panel/button[3]"

Such an XPath would work as long as no other button gets placed before the target button, and as long as the button is somewhere inside two nested panels.

Solutions[Bearbeiten]

Expecco offers a number of solutions to this problem. However, if you can talk to your developers, the first thing to do is to talk to them and ask them to use constant identifiers, which will not change between versions, and which will be independent of national languages or sessions. Most GUI frameworks do allow for such identifiers to be added.

Scenario 1: Developers can provide information[Bearbeiten]

It is often possible, that the app-developer can provide widget ID information. Especially, if the UI is generated via a WindowBuilder tool, it is often possible to export an XML or CSV containing such information. The test could then import this file, and generate a mapping table (into an expecco variable, or into a number of variables) at test startup. Then, instead of using hard-wired XPath strings, those variable values will be used as locators. Use one of the XML- or CSV reader actions to parse the data, and place the element-IDs into either a dictionary or into the environment and use those values in the test.

Scenario 2: dynamic session-ids[Bearbeiten]

If all sub-element ids can be constructed from a session id, a similar strategy is possible, by remembering this sessionID during startup (in an expecco variable), compute the sub-ids into a table or individual variables, and use those a locators.

Scenario 3: none of the above[Bearbeiten]

This is the worst case fallback. If no information is available at all, you have to use possibly sensitive XPaths. With the VNC plugin) you can also fall back to either geometric locator information (i.e. specify that the button is in a particular bounding rectangle) or image locator (e.g. specifying the bitmap image of the button, and having expecco search for the image in the UI. VNC is supported with expecco 18.1 (= 2.12).

GUIBrowser's Role in Test Development[Bearbeiten]

The GUIBrowser helps to deal with the above challenges. You can interactively explore the UI widget hierarchy, find out which attributes are present and what values are found there, try different XPath expressions to locate elements, and collect actions into subsequences interactively. You can at any time switch between the GUIBrowser's view of the application and the test diagram - even in the middle of a stopped or paused test run, change the diagram or parameters on the fly, and possibly proceed or restart test sequences.

Description[Bearbeiten]

Opening a concrete UI

The "expecco GUI Tests Extension" (or "GuiBrowser" for short) provides a common base for the interaction with graphical user interface applications. Various different UI technologies are supported, in that the concrete interface is realized by technology specific plugin extensions and action block libraries. Currently available are interfaces for Qt, Java Swing, MFC, DevExpress, Android, iOS, Windows Mobile and VNC.

Therefore, in order to use the "expecco GUI Tests Extension", at least one concrete UI-technology plugin is required.

One the other hand, it is possible to handle complex test scenarios in which multiple technologies are involved (for example, Qt plus Android plus Java-Swing). Any combination of interfaces can be controlled simultaneously, within a single test scenario and even execute in parallel.

Thus, complex integration test scenarios, End-to-End tests and other multi-technology tests are easily created, in which both mobile and web-based and classical user interfaces are to be controlled and verified.

To open the "expecco GUI Tests Extension" within expecco, click on the "GUI Browser" icon in the toolbar.

User Interface[Bearbeiten]

Fig.1: Main View (pre 2.10 version)
Fig.2: Usage (pre 2.10 version)

Overview on the layout, usage and the functions of the UI. Fig.1 Shows the main view. Fig.2 shows the control flow. In the 2.10 version, the look has changed slightly changed, and an additional preview has been added.

Connection Setup
Provides a selection list with available UI technologies.

After a click, a connection setup dialog opens, which lets you choose among connected devices (i.e. if multiple mobile devices are connected) or applications (if multiple applications are present inside the device or running on a machine). Once connected to a concrete technology/device and application, the components of the user interface are presented in the hierarchical tree on the left.

UI Component Hierarchy (left view)
This view displays both connected and disconnected applications. Underneath each connection, the component (widget-) hierarchy of the application is shown. When selecting a disconnected application, the GUI Browser will try to reconnect and search or a running application, in case the connection was lost or never started.

When an element is selected, matching operations (i.e. actions which are applicable to that element) are shown in the block selection dialog (in the middle), and the component's attributes are shown in the parameter view.

The XPath to the component (the locator) is shown at the bottom. This is an automatically generated XPath, which is usually too specific and inflexible (as described in "challenges" section above). Thus, you may shorten the XPath and replace intermediate components in it by wildcards ("*" or "**"), and reverify that the optimized XPath is still valid (i.e.refers to exactly the one selected component). You may need a little experience and gut-feeling to get to the optimum - and also often some interaction with app developers, so they know, on what hierarchical details your test suite depends upon.

Tools
A number of functions are available as toolbar buttons or on the selection menu. Most of them are context menu functions and provide operations specific to the selected element.
Disconnect All
Closes all connections of all technologies.
Disconnect Selected
Closes the selected connection (or the connection used by the selected widget element).
Refresh
Reloads the widget hierarchy from the device/application. Some technologies can do this automatically; however, on many, this would take too long and slow down or even interfere with the application's operation. For example, a refresh of an Android app can take a few seconds, depending on the number of UI items, and the speed of the USB connection, and is therefore not done automatically. Also, a refresh may close currently open popup menus or change the keyboard focus in some technologies, and would change the state of the application, which is another reason for not doing this automatically.

Use this when the UI changed without user interaction (normally, the tree is refreshed when you click or enter keyboard input. But sometimes, changes come late or without any user action.)

Reload Child Elements
Only reloads the child elements of the selected element. Notice that not all UI technologies support a partial reload, and these will do a (slow) full reload.
Highlight
Emphasizes the widget's display area (usually by drawing a rectangle around it).
Follow Pointer
When active, the element under the mouse pointer is automatically selected whenever moved over an item in the UI. Not all UI technologies support this function.
Start Recording (on all)
Starts recording your interaction with all connected devices. I.e. you can interact with any application or mobile device and have all those recorded in one recording sequence. If there is already a partial recording in the sequence view, the new interactions are appended to the end of the sequence.

For some technologies (eg. Android and iOS), a separate recorder window is opened, which presents the screen, and records user input. Thus, you do not need to have the physical device on your desktop (which is especially useful, if the device is located in the server room...)

Stop Recording (for all)
Stops the recording.
Attribute View
Shows the attributes of the selected element. The list of attributes depends on the underlying technology (i.e. a Java Swing element may show more attributes than e.g. an Android UI element) and the type of element (i.e. a Button will have different attributes than e.g. a Combolist). When an attribute is selected, corresponding action blocks are shown in the upper action/attribute list, so that only blocks are presented, which are applicable to the selected element and selected attribute.
Action-Library
Shows action blocks which are applicable to the selected element. These blocks are user-interaction blocks, which generate click, press, release, keyboard or other user events. The list will only show the set of action for a particular kind of element. For example, if an input-field element is selected, actions for text entry will be shown, whereas these are not seen when a panel or label is selected.
Whenever an action block is selected, it is copied to the the test-pane and any locator/path input pin is set to refer to the selected UI element. I.e. it will be presented as it would be used within a test sequence. You can now either execute the action in isolation, or copy it to the end of the sequence (by pressing the "Add to Sequence" button in the top-right. Of course, you can also edit the network in this test view and add any other action block (drag & drop from the suite or via the context menu) or modify the pin parameters.
If you double click on the action block, it is immediately added to the recorded sequence.
Test Network and Sequence Views
As already mentioned above, the "Test" pane shows the selected block from the action/attribute list. You can edit and change the pin setup. Also, the block can be executed and the results are shown in the lower right execution log window. Finally, the block is added to the end of the recorded sequence via the "Add to Sequence" button.
The "Sequence"-View presents the test sequence as recorded so far. This is a regular diagram editor as described elsewhere. So any of the well known editing features can also be used here. Especially, it is possible to insert other blocks from the library, to change the execution order of recorded steps, or to replace them by other steps (without a need to re-record the sequence). The sequence view also allows for the whole sequence to be replayed in whole or in part, to be single stepped, breakpointed etc. Finally, when complete, you can create a new action block in your project, which contains the recorded sequence (by pressing on the "Define Action Block" button at the top right).
Path Display
Shows the XPath (locator) of the selected UI element. By default, a full explicit path is automatically generated and added as freeze value to the locator pins (in the recorded sequence). However, it is often desirable to shorten this path via placeholders. This will make your test less affected by changes in the widget hierarchy. Therefore, it is useful to reduce the path as long as it uniquely identifies the desired element. The details depend on the way the UI was programmed; for example, if each element has a unique ID, no path is needed at all, and the locator can be something like "/*[ID=xxx]". In contrast, if no ID or other unique attribute is present, a full path which enumerates the parent hierarchy might be required. The "Check Path" button will apply the entered path to the element hierarchy and give an OK-response if the element is still uniquely identified by it.
Component Preview
Display a preview (snapshot image) of the selected UI element or the application's main window/screen, with the element's bounds emphasized.


Locator Path (XPath)[Bearbeiten]

The locator addresses elements within a user interface. It is based on the XPath notation as known in the XML world. Locator paths are specified during test development, and used later, at test execution time, to refer to UI elements. The path of a selected UI element is shown in the lower right "Path" field, and is updated when another element is selected. Vice versa can the path be changed in that input field, in order to check which element(s) would be addressed by a given path. The path does not need to and should not be specified in full detail. It can contain wildcards and dynamic parts, which makes it more flexible and more robust against possible future changes in the test application; especially against changes in the hierarchy, when new container elements are inserted (which is one of the most common reasons for test cases to fail on new releases).

Notice: Many user interfaces change their structure dynamically during the execution. Often container and interaction elements are added or removed during the run. Especially in these situations, the flexibility in the XPath notation is very useful.

Defining Paths[Bearbeiten]

A typical path may look like:

/frame[@name='Notepad']/root pane/layered pane/panel/panel/panel/tool bar/push button[2]

This path describes the full explicit path to the second button inside a toolbar, which itself is contained in a nested hierarchy of containers (panels).

The path consists of a list of identifiers, beginning and separated by "/". Each individual identifier describes a subcomponent of the element which is already described by the left part.

"/frame[@name='Notepad']"

refers to the top-level application window. "root pane", "layered pane", "panel" etc. refer to corresponding child elements. To get to the final destination, each component is found along the path starting with the top level window.

In this concrete example, expecco will first look for a top-level window which has the name "Notepad". This window will have one or more child elements, which match the identifier "root pane". Each such "root pane" will have children, matching the identifier "layered pane", and so on. At the end, there should be exactly one element, which matches all of those "constraints", and this will be the target element. If no element matches, or of multiple elements match, a runtime error will be triggered (at test execution time), and the normal exception handling mechanism of expecco applies (i.e. the test will finish with a failure, or an exception handling action is triggered).

The syntax of each individual identifier is:

/<NodeIdentifier><[Optional Selector]>

Node Identifier[Bearbeiten]

The Node Identifier defines the type of UI element, which is to be searched

Possible Identifiers:

The type of the UI element (eg. "button", "panel" etc.)
matches all elements of that type (can be zero, one or multiple elements).
The "Any" identifier ("*")
matches all child elements, independent of their type. Further matching must be done via the selector or via child element matches (e.g. the right rest of the XPath).

Example:
Full path:

/frame[@name='Notepad']/root pane/layered pane/panel/panel/panel/tool bar/push button[2]

the same path using Any ( "*" ) identifiers:

/*[@name='Notepad']/*/layered pane/*/panel/*/tool bar/push button[2]

Both paths refer to the same UI element. However, the second version is less sensible to changed types of the intermediate components. For example, the app developer might replace a "panel" component by a "box" component in a new release. The first path would then fail to find the UI component, as no element will then match the "panel" identifier. Notice that the "*"-identifier is a wildcard for only one component (i.e. the hierarchy must still be as deep as before). To make the path robust against hierarchy changes, use the "//"-Notation (as described below).

Selector[Bearbeiten]

The selector is optional. It is used to select among same-typed elements. For example, if a component contains 3 buttons as child elements (eg. child elements of the same type), the selector can be used to specify which of the 3 buttons is to be matched. The selector is specified in square "[ ]" brackets after the node identifier, and is applied during the match process after matching the node identifier. For example, if an elements contains 2 buttons plus 2 checkboxes as child elements, and the node identifier gives 2 buttons as result, the selector will only be applied to the 2 buttons.

Possible Selectors:

the "Index" selector
chooses an element by its index, which is the position within the list of children (of the matching type). The index is 1-based (i.e. [1] refers to the first child).
the "Key" selector (or "Key-Value" selector)
chooses the element, which has a particular value associated to a given attribute key. The set of attribute keys depends on the concrete UI technology, the component type and possibly its current state. Typical keys are "name", "id", "label", "value" etc. but also "visible", "enabled" or "isActive". Some components have dynamic attributes (and therefore keys which show up dynamically), which are created during the execution time.

Example (Index Selector):

/*/tool bar/push button[2]

Chooses the second button inside the toolbar.

Example (Key Selector):

/*/tool bar/push button[@name='Save As...']

Chooses the button which has the value "Save As..." stored under its "name" attribute.

Be careful with attribute values which depend on the national language. If attributes are present, which are language independent, (e.g. "id", "refID", "uuid" etc.) those are to be preferred. If not, you should keep the language texts separate from the action block (i.e. attachment, external file or expecco variable) and construct the XPath using those values. This makes it much easier to later support multiple languages or changed button labels in your test suite.

"Super Wildcard"[Bearbeiten]

The super wildcard ("//") can dramatically shorten paths, and also makes them robust against additional inserted or removed hierarchy levels. "//" matches arbitrary subpaths: children are not only matched in the immediate sub-child level, but also in any level below.

Example:
Given the full path of a button as:

/frame[@name='Notepad']/root pane/layered pane/panel/panel/panel/tool bar/push button[@name='Save As...']

also assuming, that the application contains only one single button by the name "Save As...". Then, the path can be rewritten (using the super wildcard) as:

//push button[@name='Save As...']

even short, if there is only one single element by that name (i.e. no other non-button exists, which has that name attribute). Then, the following path is also possible:

//*[@name='Save As...']

The following example shows, that you can also use the super wildcard for partial hierarchies:

/*[@name='Notepad']//panel//tool bar/*[@name='Save As...']

this matches a component by the name "Save As...", which is inside a toolbar, which is arbitrarily deep inside a panel, which is arbitrarily deep inside the top window named "Notepad".

Notice: The short paths as shown above are dramatically more robust against changes in the layout and hierarchy of the user interface. However, at the same time, they may increase a risk to match multiple elements inside the GUI, which would also lead to an error being reported at test execution time ("non specific component path"). In the above example, this might happen if the UI contains another menu item by the same name. In real world testing, the choice of path will always be a compromise between flexibility (as short as possible) and unambiguousness.

Technology Specific Features[Bearbeiten]

Firefox, Edge, Internet Explorer, Chrome etc.[Bearbeiten]

See Selenium WebDriver Plugin/en

Android and iOS[Bearbeiten]

These are documented in the separate document: "Mobile Testing Plugin"

VNC[Bearbeiten]

These are documented in the separate document: "VNC Testing Plugin"



Copyright © 2014-2024 eXept Software AG