VNC Plugin Reference/en
Inhaltsverzeichnis
Introduction[Bearbeiten]
The VNC plugin interfaces to local or remote applications via the RFB protocol, which enables interaction with a remote screen on any other machine in the network.
Using VNC has one big advantage, and a number of disadvantages:
- [+] it allows interaction with and the testing of almost any application, and no instrumentation or recompilation of the tested application is needed, given that a VNC-server is running and reachable on the target system. VNC servers are available or already installed on almost any operating system or device, even for mobile devices, embedded control systems, head units in cars, etc.
- [-] whilst all other UI testing plugins interact with the tested application directly, via a protocol which enables expecco to reflect (query) on internals of the app, the VNC interface can only show the screen, and can only send events based on screen coordinates. No information about the widget element hierarchy or internal attribute can be interchanged via VNC.
Thus, with VNC, expecco cannot ask the application about the position of GUI elements and cannot ask for any internal attribute values.
- [-] the VNC plugin cannot provide the values of internal attributes of widget elements (text, labels, enable state etc.)
- [-] VNC is very sensible to changes in the layout and look of the application.
- [-] the geometry information (i.e. the location of widget elements) is kept inside expecco, and cannot be queried from the application.
In view of those limitations, you should use the VNC plugin, if no other plugin is usable with the application. i.e. if one of the following applies:
- it is written in C/C++ and cannot be instrumented or recompiled
- it is written in Java and the JVM cannot be started with additional jars being loaded or the JVM provides no debugging interface (i.e. embedded Java)
- it runs under Windows and does not support the Accessability API
- it is an old style (hand-written) X-windows application (Xt or non-widget library)
- the target app or operating system of the target opens additional windows/dialogs, which cannot be controlled by other mechanisms and which need to be closed or confirmed. Examples are operating system warnings ("disk full") or keyboard entry windows on mobile devices.
Philosophy[Bearbeiten]
To allow for a maximum of flexibility, the VNC plugin uses description objects, which describe where and how to find UI elements. These descriptions are called "Screenplays" (or "Storyboards") and are typically stored in attachments inside the test suite.
Each screenplay describes the ui elements of an application. It is structured into multiple "Scenes", which correspond to pages/tabs or states of the application. Whenever the application changes its widget layout (i.e. changes to another UI-hierarchy or layout), a new stage-description is activated. Finally, each stage contains "Players" (UI-elements), and descriptions on how to find them.
The testsuite refers to UI-elements by scene- and element (player) identifiers, and the current active scene provides the information of where and how to find them on the screen.
Screenplays, Scenes and Players are described in a human-readable, textual format in attachments. The GuiBrowser can read, modify and write such attachments. They can also be imported/exported in some standard (XML) format. Import is especially useful, if app-developers can provide this information from their IDE/Window builder files. Exepcco uses a format similar to the FXML format, so it may be easy to exchange such description files with app developers.
Preparation of the System Under Test[Bearbeiten]
You need a VNCserver to be installed and running on the target system. Any freely available or commercial VNC/RFB server should work out of the box. Is is known to work (i.e. we tried) the following VNC servers (as target system):
- Android (Samsung tablet): alpha VNC server (Beta version) (no rooting req'd)
- OS-X (MacBook Pro/Yosemite 10.10.5): Vine Server 4.01
- xvnc (Gentoo Linux)
Recording in the GUIBrowser[Bearbeiten]
The VNC plugin is well integrated into the expecco GUIBrowser framework. It now seamlessly integrates the definition of geometric information (screenplay, stage & player definition) with the recording process. The following gives a sample session, where a login procedure is recorded on a remote (OS-X) system.
Opening the GUIBrowser[Bearbeiten]
Open the GuiBrowser by clicking on its icon in the toolbar.
Opening the VNC Connection[Bearbeiten]
Click on the "Connect"-button and choose "VNC Gui Automation".
A connect dialog will appear, asking for the hostname (or ip-address) of the target system, a port-number (which is 5900 for screen 0, 5901 for screen 1, etc.) and some other optional values. For now, you can ignore those.
when connected, the guiBrowser's tree is updated and an initial "Main Scene" is appearing. Also, a screenshot is shown in the lower left area, if you have the "Preview" button enabled. In this session, the OS-X login screen is shown in the preview:
Now, you have two options: either open a recorder, which combines interaction, recording and screenplay definition, or define the screenplay in the preview.
Both have similar functionality and allow the same operations; however, the recorder displays a live image of the target system, whereas the preview shows a static "snapshot", which needs to be updated manually. Depending on the bandwidth of your network/usb connection and/or the speed of your target system, either may be a better choice. Also, because the preview shows a frozen snapshot, it may be easier to define elements there, if the application changes its look dynamically (i.e. to specify the bounds of dynamically disappearing menus or popup dialogs).
Locators[Bearbeiten]
Inside the suite, elements are referred to by their elementID (inside the current scene). For example, a "Button-Press" action takes the elementID (a string), the mouse-button number and a position relative to the wigdet's origin as argument. Expecco will then find the element's geometry in the current scene, translate the coordinates to screen coordinates, and send a VNC button-press to the screen.
Currently, two kinds of locators are supported:
- bounds
- bitmap
Bounds Locator[Bearbeiten]
With bounds, the origin+corner (i.e. the rectangular bounds) of the element are stored in the scene, and this is used to locate the element - independent of what is actually shown on the screen. This kind of locator is useful, if you can get the geometry information from the widget developers, or when the position of the elements remains constant, but the look is changing often.
Bitmap Locator[Bearbeiten]
Here, a "prototype bitmap" is kept in the scene information, and the bitmap is "searched" on the screen. This makes the test less affected by changes in the exact position of the element, but more affected by changes in the look (i.e. changed icons or changed labels). The bitmap to be searched can be kept inside the testsutie itself (as attachament) or in external bitmap files. In general, it is recommended to keep them inside the testsuite; however, if you can get the set of images from the app development team (from their source repository), it may also make sense to keep them separate.
Combined Bitmap+Bounds Locator[Bearbeiten]
This is similar to the bitmap locator described above, but here, the bitmap to be searched defines one of the four corners of the actual bounds, and the bounds are translated according to where the bitmap was found. This kind of locator is useful, if an input field is always located to the right or below some known bitmap pattern.
