Grafical Elements in GUI Technologies: Unterschied zwischen den Versionen
Matilk (Diskussion | Beiträge) (Die Seite wurde neu angelegt: „= Introduction = The GUI technologies generally provide information about available elements inside their applications. By a locator (usually an XPath) you can…“) |
Matilk (Diskussion | Beiträge) |
||
| Zeile 6: | Zeile 6: | ||
Following technologies support an additional screenplay so far: [[Selenium_WebDriver_Plugin|Web Test (Selenium)]], [[Mobile_Testing_Plugin|Mobile Testing (Appium)]], [[WindowsAutomation_Reference_2.0|WindowsAutomation 2]], and [[Java_GUI_Plugins|Java]]. |
Following technologies support an additional screenplay so far: [[Selenium_WebDriver_Plugin|Web Test (Selenium)]], [[Mobile_Testing_Plugin|Mobile Testing (Appium)]], [[WindowsAutomation_Reference_2.0|WindowsAutomation 2]], and [[Java_GUI_Plugins|Java]]. |
||
= Defining Screenplays = |
|||
= Define a Screenplay = |
|||
The easiest way to define a screenplay is using the [[Expecco_GUI_Tests_Extension_Reference/en|GUI Browser]]. First connect to the application you want to use. In its context menu you will find then the entry ''Screenplay'' with several options, including to create a new screenplay. The elements inside the screenplay are divided into scenes. They can be used for structuring, but you can use just one scene as well. The screenplay will be saved as an attachment to the current project, as well all image files that are created to define the elements. |
The easiest way to define a screenplay is using the [[Expecco_GUI_Tests_Extension_Reference/en|GUI Browser]]. First connect to the application you want to use. In its context menu you will find then the entry ''Screenplay'' with several options, including to create a new screenplay. The elements inside the screenplay are divided into scenes. They can be used for structuring, but you can use just one scene as well. The screenplay will be saved as an attachment to the current project, as well all image files that are created to define the elements. |
||
= |
= Use Screenplays = |
||
To use a screenplay in a Test, it has to be loaded first. Some technology libraries contain a block to do this. In the Common GUI Library for example, that can be used in any case, it is called ''[UI] Set Screenplay''. Place the attachment defining the screenplay to your compound block and set its output type to ''Contents as String'', so you can connect the pin to the screenplay block. The defined elements can be used with the blocks from the technology libraries or the Common GUI Library, but not all actions may work. |
|||
= |
= Elements = |
||
There are different kinds of elements or players, that can be defined in a screenplay. In the GUI Browser you can create a new element by selecting the screenplay or a scene and choosing the desired type of element under ''Define New Element ("Player" in Scene)'' in the context menu. |
|||
== |
== Graphical Elements == |
||
Graphical elements are all elements that can be found by the screenshot. Elements which are found like that are independent from the used technology. You can only execute basic actions, like click or get text (by OCR). Although these actions are executed using the respective technology, actions that need a true technology element (e.g. get property value) are not possible with these elements. There are different options to define such an element. |
|||
Grafische Elemente sind alle Elemente, die über den Screenshot erkannt werden. Die Elemente, die so gefunden werden sind unabhängig von der Technologie. Mit ihnen können nur grundlegende Aktionen ausgeführt werden, wie Klicken oder den Text auslesen (über OCR). Auch wenn diese Aktionen trotzdem über die jeweilige Technologie ausgeführt werden, sind mit diesen Elementen keine Aktionen möglich, die ein echtes Technologie-Element benötigen (z.B. Attribut-Werte auslesen). Es gibt verschiedene Möglichkeiten, ein solches Element zu definieren. |
|||
=== Position === |
=== Position === |
||
The simplest option is to just provide a position. This doesn't even require the screenshot to be searched, but it classifies as graphical element nevertheless, as the possible action with such an element are the same. |
|||
Die einfachste Möglichkeit ist es, einfach eine Position anzugeben. Hierfür muss noch nicht einmal der Screenshot durchsucht werden, trotzdem handelt es sich hierbei um ein grafisches Element, da die möglichen Aktionen mit einem solchen Element dieselben sind. |
|||
=== |
=== Image Search === |
||
Elements can be defined by one or more images, which will then be searched in the current screenshot. There are several subcategories of this. The element can be defined directly by the image or at a position relative to the image. Furthermore it is possible to define an area bordered by two images. |
|||
Elemente können mit einem oder mehreren Bilder definiert werden, die dann im aktuellen Screenshot gesucht werden. Hiervon gibt es verschiedene Unterkategorien. Das Element kann direkt durch das Bild definiert sein oder eine Position relativ zum Bild haben. Es ist außerdem möglich, einen Bereich zu definieren, der von zwei Bildern aufgespannt wird. |
|||
=== Text |
=== Text Search === |
||
If OCR support is configured in expecco, it is possible to search for text inside the image. Be aware that the success obviously depends on the OCR result. A higher contrast may be an advantage, dark font on a light background is usually better recognized than vice versa. Icons or other graphical content might be interpreted as characters, which can cause problems if you search for a word next to it. |
|||
Wenn die OCR-Unterstützung in expecco konfiguriert ist, kann auch nach Text innerhalb des Bildes gesucht werden. Hier ist zu beachten, dass der Erfolg natürlich vom OCR-Ergebnis abhängt. Dabei ist hoher Kontrast in der Regel von Vorteil, dunkle Schrift auf hellem Grund wird meist besser erkannt als umgekehrt. Icons oder andere grafische können als Zeichen erkannt werden, was zu Problemen führen kann, wenn ein Wort direkt daneben gesucht wird. |
|||
== XPath |
== XPath Elements == |
||
You can also define elements by XPath in the screenplay. This allows a path to be assigned a name, so to speak. On the one hand this contributes to the readability of the test, on the other hand this will make it easier to replace paths. Elements found like this are in the same way true technology elements as if you had written the path directly at the pin, i.e. they can be used for any action that is possible for the technology. |
|||
Im Drehbuch können auch Elemente mit XPath definiert werden. Dadurch kann einem Pfad quasi ein Name zugewiesen werden. Dies trägt zum einen zur besseren Lesbarkeit des Tests bei, zum anderen können Pfade so einfacher ausgetauscht werden. Elemente, die so gefunden werden, sind genauso echte Technologie-Elemente wie wenn man den Pfad direkt am Baustein verwendet, das heißt mit ihnen können alle Aktionen ausgeführt werden, die für die Technologie möglich sind. |
|||
= |
= Notation = |
||
The @-sign is used to mark an element-ID when using an element from the screenplay at a block. E.g. ''@OkButton''. There are other notation when using a screenplay with VNC or on local screen, but they are not allowed when using it with the other GUI technologies, as it must be distinguishable if a locator is an XPath or a screenplay element. |
|||
Um ein Element aus dem Drehbuch an einem Baustein zu verwenden, wird das @-Zeichen verwendet, um eine Element-ID zu kennzeichnen. Zum Beispiel ''@OkButton''. Bei der Verwendung eines Screenplays mit VNC oder auf dem lokalen Bildschirm gibt es daneben noch andere Schreibweisen, die bei den anderen GUI-Technologien nicht zulässig sind, da man hier unterscheiden können muss, ob es sich um einen XPath oder um ein Screenplay-Element handelt. |
|||
Besides it is possible to combine several elements in one path. The path will be parsed from left to right and only the previously found part will be used to search further. Each part has to be separated by"/". E.g. ''@leftArea/@OkButton''. It is allowed to combine XPath of the technology with Element-IDs, but it has to be noted, that no Element-ID of a graphical element can be left of an XPath. |
|||
Version vom 20. Dezember 2022, 10:18 Uhr
Inhaltsverzeichnis
Introduction[Bearbeiten]
The GUI technologies generally provide information about available elements inside their applications. By a locator (usually an XPath) you can then define at a block, for which element a certain action shall be executed. But in some cases in can be difficult or even impossible to find such a path. Either because the structure is not static, there are no suitable properties, or there even is no element available by the technology, but only its surrounding element.
By using a screenplay it is possible, to define elements that can be found using the screenshot. The pronciple is the same as for the automation on the local screen or using VNC server, but can be used in parallel with the normal form of addressing elements.
Following technologies support an additional screenplay so far: Web Test (Selenium), Mobile Testing (Appium), WindowsAutomation 2, and Java.
Defining Screenplays[Bearbeiten]
The easiest way to define a screenplay is using the GUI Browser. First connect to the application you want to use. In its context menu you will find then the entry Screenplay with several options, including to create a new screenplay. The elements inside the screenplay are divided into scenes. They can be used for structuring, but you can use just one scene as well. The screenplay will be saved as an attachment to the current project, as well all image files that are created to define the elements.
Use Screenplays[Bearbeiten]
To use a screenplay in a Test, it has to be loaded first. Some technology libraries contain a block to do this. In the Common GUI Library for example, that can be used in any case, it is called [UI] Set Screenplay. Place the attachment defining the screenplay to your compound block and set its output type to Contents as String, so you can connect the pin to the screenplay block. The defined elements can be used with the blocks from the technology libraries or the Common GUI Library, but not all actions may work.
Elements[Bearbeiten]
There are different kinds of elements or players, that can be defined in a screenplay. In the GUI Browser you can create a new element by selecting the screenplay or a scene and choosing the desired type of element under Define New Element ("Player" in Scene) in the context menu.
Graphical Elements[Bearbeiten]
Graphical elements are all elements that can be found by the screenshot. Elements which are found like that are independent from the used technology. You can only execute basic actions, like click or get text (by OCR). Although these actions are executed using the respective technology, actions that need a true technology element (e.g. get property value) are not possible with these elements. There are different options to define such an element.
Position[Bearbeiten]
The simplest option is to just provide a position. This doesn't even require the screenshot to be searched, but it classifies as graphical element nevertheless, as the possible action with such an element are the same.
Image Search[Bearbeiten]
Elements can be defined by one or more images, which will then be searched in the current screenshot. There are several subcategories of this. The element can be defined directly by the image or at a position relative to the image. Furthermore it is possible to define an area bordered by two images.
Text Search[Bearbeiten]
If OCR support is configured in expecco, it is possible to search for text inside the image. Be aware that the success obviously depends on the OCR result. A higher contrast may be an advantage, dark font on a light background is usually better recognized than vice versa. Icons or other graphical content might be interpreted as characters, which can cause problems if you search for a word next to it.
XPath Elements[Bearbeiten]
You can also define elements by XPath in the screenplay. This allows a path to be assigned a name, so to speak. On the one hand this contributes to the readability of the test, on the other hand this will make it easier to replace paths. Elements found like this are in the same way true technology elements as if you had written the path directly at the pin, i.e. they can be used for any action that is possible for the technology.
Notation[Bearbeiten]
The @-sign is used to mark an element-ID when using an element from the screenplay at a block. E.g. @OkButton. There are other notation when using a screenplay with VNC or on local screen, but they are not allowed when using it with the other GUI technologies, as it must be distinguishable if a locator is an XPath or a screenplay element.
Besides it is possible to combine several elements in one path. The path will be parsed from left to right and only the previously found part will be used to search further. Each part has to be separated by"/". E.g. @leftArea/@OkButton. It is allowed to combine XPath of the technology with Element-IDs, but it has to be noted, that no Element-ID of a graphical element can be left of an XPath.
