Graphical Elements in GUI Technologies/en - Versionsgeschichte

Matilk: Matilk verschob die Seite Grafical Elements in GUI Technologies/en nach Graphical Elements in GUI Technologies/en: wrong spelling

2022-12-22T14:18:36Z

Matilk verschob die Seite Grafical Elements in GUI Technologies/en nach Graphical Elements in GUI Technologies/en: wrong spelling

← Nächstältere Version	Version vom 22. Dezember 2022, 14:18 Uhr
(kein Unterschied)

Matilk: /* Image Search */ link to vnc

2022-12-22T14:02:25Z

Image Search: link to vnc

← Nächstältere Version		Version vom 22. Dezember 2022, 14:02 Uhr
Zeile 49:		Zeile 49:
	\|}		\|}
	Even if an application renders the same element, it can happen, that it is not to the pixel identical in the screenshot. To compensate the inaccuracies due to this effect, there are several search parameters. In the context menu of the element you can open an editor to edit these parameters (''Locator (Image Search) Parameters...''). There you can set for example the percentage how much the reference image may differ from the original or have only a grayscale or black-and-white search be done.		Even if an application renders the same element, it can happen, that it is not to the pixel identical in the screenshot. To compensate the inaccuracies due to this effect, there are several search parameters. In the context menu of the element you can open an editor to edit these parameters (''Locator (Image Search) Parameters...''). There you can set for example the percentage how much the reference image may differ from the original or have only a grayscale or black-and-white search be done.

			Image locators are described in more detail in [[VNC_Plugin_Reference/en#Locators\|the VNC plugin reference documentation]].

	=== Text Search ===		=== Text Search ===

Matilk: /* Elements */ Relative Elements

2022-12-22T13:40:08Z

Elements: Relative Elements

← Nächstältere Version		Version vom 22. Dezember 2022, 13:40 Uhr
Zeile 59:		Zeile 59:
	== XPath Elements ==		== XPath Elements ==
	You can also define elements by XPath in the screenplay. This allows a path to be assigned a name, so to speak. On the one hand this contributes to the readability of the test, on the other hand this will make it easier to replace paths. Elements found like this are in the same way true technology elements as if you had written the path directly at the pin, i.e. they can be used for any action that is possible for the technology.		You can also define elements by XPath in the screenplay. This allows a path to be assigned a name, so to speak. On the one hand this contributes to the readability of the test, on the other hand this will make it easier to replace paths. Elements found like this are in the same way true technology elements as if you had written the path directly at the pin, i.e. they can be used for any action that is possible for the technology.

			== Relative Elements ==
			Each of the screenplay elements described above can also be created relative to a technology element. In this case, the technology element will be searched first and further searching will then be done with its screenshot. An element is automatically created relatively, if a technology element is selected when defining it. Relative elements are displayed in the tree with a yellow tint.

			Alternatively to this method, you can simply combine paths, see section [[#Notation\|Notation]].

	= Notation =		= Notation =

Matilk: /* Using Screenplays */

2022-12-22T12:02:08Z

Using Screenplays

← Nächstältere Version		Version vom 22. Dezember 2022, 12:02 Uhr
Zeile 10:		Zeile 10:
	The easiest way to define a screenplay is using the [[Expecco_GUI_Tests_Extension_Reference/en\|GUI Browser]]. First connect to the application you want to use. In its context menu you will find then the entry ''Screenplay'' with several options, including to create a new screenplay. The elements inside the screenplay are divided into scenes. They can be used for structuring, but you can use just one scene as well. The screenplay will be saved as an attachment to the current project, as well all image files that are created to define the elements.		The easiest way to define a screenplay is using the [[Expecco_GUI_Tests_Extension_Reference/en\|GUI Browser]]. First connect to the application you want to use. In its context menu you will find then the entry ''Screenplay'' with several options, including to create a new screenplay. The elements inside the screenplay are divided into scenes. They can be used for structuring, but you can use just one scene as well. The screenplay will be saved as an attachment to the current project, as well all image files that are created to define the elements.

	= ~~Use~~ Screenplays =		= Using Screenplays =
			If a screenplay is attached to a connection, it can be used with it. In the context menu of an application in the GUI Browser you can in addition to creating a new screenplay also load an already existing one from an attachment or a file.
⚫	To ~~use~~ a screenplay in a ~~Test,~~ it ~~has~~ to ~~be loaded first~~. Some technology libraries contain a block ~~to do this. In~~ the Common GUI Library ~~for example~~, ~~that~~ can be used in any case, it is called ''[UI] Set Screenplay''. Place the attachment defining the screenplay to your compound block and set its output type to ''Contents as String'', so you can connect the pin to the screenplay block~~. The defined elements can be used with the blocks from the technology libraries or the Common GUI Library, but not all actions may work~~.

		⚫	To load a screenplay in a test there are appropriate blocks. Some technology libraries contain such a block, in the Common GUI Library, which can be used in any case, it is called ''[UI] Set Screenplay'' for example. Place the attachment defining the screenplay to your compound block and set its output type to ''Contents as String'', so you can connect the pin to the screenplay block.

			The elements defined in the screenplay can be used with the blocks from the technology libraries or the Common GUI Library, but not all actions may work. Read in the section [[#Notation\|Notation]] further below how to write the elements at the input pins.

	= Elements =		= Elements =

Matilk: /* Graphical Elements */ search parameters

2022-12-22T09:55:46Z

Graphical Elements: search parameters

← Nächstältere Version		Version vom 22. Dezember 2022, 09:55 Uhr
Zeile 44:		Zeile 44:
	\| [[Datei:MenuBetweenTwoLocator.png]] \|\| '''Area Defined by Two Bitmaps''' \|\| In contrast to the previous option the images are not part of the element, but are anchors for the top left and bottom right corner of it.		\| [[Datei:MenuBetweenTwoLocator.png]] \|\| '''Area Defined by Two Bitmaps''' \|\| In contrast to the previous option the images are not part of the element, but are anchors for the top left and bottom right corner of it.
	\|}		\|}
			Even if an application renders the same element, it can happen, that it is not to the pixel identical in the screenshot. To compensate the inaccuracies due to this effect, there are several search parameters. In the context menu of the element you can open an editor to edit these parameters (''Locator (Image Search) Parameters...''). There you can set for example the percentage how much the reference image may differ from the original or have only a grayscale or black-and-white search be done.

	=== Text Search ===		=== Text Search ===

Matilk: /* Elements */

2022-12-22T09:12:29Z

Elements

@@ Zeile 21: / Zeile 21: @@
 === Position ===
 The simplest option is to just provide a position. This doesn't even require the screenshot to be searched, but it classifies as graphical element nevertheless, as the possible action with such an element are the same.
 === Image Search ===
 Elements can be defined by one or more images, which will then be searched in the current screenshot. There are several subcategories of this. The element can be defined directly by the image or at a position relative to the image. Furthermore it is possible to define an area bordered by two images.
 === Text Search ===
 If OCR support is configured in expecco, it is possible to search for text inside the image. Be aware that the success obviously depends on the OCR result. A higher contrast may be an advantage, dark font on a light background is usually better recognized than vice versa. Icons or other graphical content might be interpreted as characters, which can cause problems if you search for a word next to it.
 == XPath Elements ==

Matilk: Die Seite wurde neu angelegt: „Deutsche Version | '''English Version''' = Introduction = The GUI technologies generally provide information about a…“

2022-12-20T16:03:20Z

Die Seite wurde neu angelegt: „Deutsche Version | '''English Version''' = Introduction = The GUI technologies generally provide information about a…“

Neue Seite

[[Grafische_Elemente_in_GUI-Technologien|Deutsche Version]] | '''English Version'''
= Introduction =
The GUI technologies generally provide information about available elements inside their applications. By a locator (usually an XPath) you can then define at a block, for which element a certain action shall be executed. But in some cases in can be difficult or even impossible to find such a path. Either because the structure is not static, there are no suitable properties, or there even is no element available by the technology, but only its surrounding element.

By using a screenplay it is possible, to define elements that can be found using the screenshot. The pronciple is the same as for the automation on the [[Local_Screen_GUI_Plugin_Reference|local screen]] or using [[VNC_Plugin_Reference|VNC server]], but can be used in parallel with the normal form of addressing elements.

Following technologies support an additional screenplay so far: [[Selenium_WebDriver_Plugin/en|Web Test (Selenium)]], [[Mobile_Testing_Plugin/en|Mobile Testing (Appium)]], [[WindowsAutomation_Reference_2.0/en|WindowsAutomation 2]], and [[Java_GUI_Plugins/en|Java]].

= Defining Screenplays =
The easiest way to define a screenplay is using the [[Expecco_GUI_Tests_Extension_Reference/en|GUI Browser]]. First connect to the application you want to use. In its context menu you will find then the entry ''Screenplay'' with several options, including to create a new screenplay. The elements inside the screenplay are divided into scenes. They can be used for structuring, but you can use just one scene as well. The screenplay will be saved as an attachment to the current project, as well all image files that are created to define the elements.

= Use Screenplays =
To use a screenplay in a Test, it has to be loaded first. Some technology libraries contain a block to do this. In the Common GUI Library for example, that can be used in any case, it is called ''[UI] Set Screenplay''. Place the attachment defining the screenplay to your compound block and set its output type to ''Contents as String'', so you can connect the pin to the screenplay block. The defined elements can be used with the blocks from the technology libraries or the Common GUI Library, but not all actions may work.

= Elements =
There are different kinds of elements or players, that can be defined in a screenplay. In the GUI Browser you can create a new element by selecting the screenplay or a scene and choosing the desired type of element under ''Define New Element ("Player" in Scene)'' in the context menu.

== Graphical Elements ==
Graphical elements are all elements that can be found by the screenshot. Elements which are found like that are independent from the used technology. You can only execute basic actions, like click or get text (by OCR). Although these actions are executed using the respective technology, actions that need a true technology element (e.g. get property value) are not possible with these elements. There are different options to define such an element.

=== Position ===
The simplest option is to just provide a position. This doesn't even require the screenshot to be searched, but it classifies as graphical element nevertheless, as the possible action with such an element are the same.

=== Image Search ===
Elements can be defined by one or more images, which will then be searched in the current screenshot. There are several subcategories of this. The element can be defined directly by the image or at a position relative to the image. Furthermore it is possible to define an area bordered by two images.

=== Text Search ===
If OCR support is configured in expecco, it is possible to search for text inside the image. Be aware that the success obviously depends on the OCR result. A higher contrast may be an advantage, dark font on a light background is usually better recognized than vice versa. Icons or other graphical content might be interpreted as characters, which can cause problems if you search for a word next to it.

== XPath Elements ==
You can also define elements by XPath in the screenplay. This allows a path to be assigned a name, so to speak. On the one hand this contributes to the readability of the test, on the other hand this will make it easier to replace paths. Elements found like this are in the same way true technology elements as if you had written the path directly at the pin, i.e. they can be used for any action that is possible for the technology.

= Notation =
The @-sign is used to mark an element-ID when using an element from the screenplay at a block. E.g. ''@OkButton''. There are other notation when using a screenplay with VNC or on local screen, but they are not allowed when using it with the other GUI technologies, as it must be distinguishable if a locator is an XPath or a screenplay element. An element ID may not contain the characters '@', '$', and '/'.

Besides it is possible to combine several elements in one path. The path will be parsed from left to right and only the previously found part will be used to search further. Each part has to be separated by"/". E.g. ''@leftArea/@OkButton''. It is allowed to combine XPath of the technology with Element-IDs, but it has to be noted, that no Element-ID of a graphical element can be left of an XPath.