VNC Plugin Reference/en - Versionsgeschichte

Matilk: /* Typical Problems */

2023-10-11T10:50:41Z

Typical Problems

← Nächstältere Version		Version vom 11. Oktober 2023, 10:50 Uhr
Zeile 331:		Zeile 331:

	== Typical Problems ==		== Typical Problems ==
	All OCR programs have problems in that some characters are too similar to be recognized correctly, depending on the font. Most typical is ~~the~~ the letter "O" looks too similar to the digit "0", or that both the digit "0" and "%" are too similar to the digit "8" (some fonts have a diagonal line inside the zero).		All OCR programs have problems in that some characters are too similar to be recognized correctly, depending on the font. Most typical is that the letter "O" looks too similar to the digit "0", or that both the digit "0" and "%" are too similar to the digit "8" (some fonts have a diagonal line inside the zero).
	Also "l" (lower case "L") often looks like the digit "1" (one) or ~~lower~~ case "I" in non-serif fonts.		Also "l" (lower case "L") often looks like the digit "1" (one) or "I" (upper case "i") in non-serif fonts.

	To solve this, you can give additional constraints to the OCR program, when text is extracted:		To solve this, you can give additional constraints to the OCR program, when text is extracted:

Cg: /* When to use VNC or Local Screen Automation */

2022-03-20T10:04:05Z

When to use VNC or Local Screen Automation

← Nächstältere Version		Version vom 20. März 2022, 10:04 Uhr
Zeile 23:		Zeile 23:
	* it is written in C/C++ and cannot be instrumented or recompiled		* it is written in C/C++ and cannot be instrumented or recompiled
	* it is written in Java and the JVM cannot be started with additional jars being loaded and it cannot be accessed via the Windows Accessibility Protocol, or the JVM provides no debugging interface (i.e. embedded Java)		* it is written in Java and the JVM cannot be started with additional jars being loaded and it cannot be accessed via the Windows Accessibility Protocol, or the JVM provides no debugging interface (i.e. embedded Java)
	* it runs under Windows and does not support the Accessibility API		* it runs under Windows and does not support the Accessibility API, and the other lightweight automation methods (autoIt and Windows Access Library) do not work for you
	* it is an old style (hand-written) X-windows application (Xt or non-widget library)		* it is an old style (hand-written) X-windows application (Xt or non-widget library)
	* the target app or operating system of the target opens additional windows/dialogs, which cannot be controlled by other mechanisms and which need to be closed or confirmed. Examples are operating system warnings ("disk full") or keyboard entry windows on mobile devices.		* the target app or operating system of the target opens additional windows/dialogs, which cannot be controlled by other mechanisms and which need to be closed or confirmed. Examples are operating system warnings ("disk full") or keyboard entry windows on mobile devices.

Cg: /* Introduction */

2022-03-20T10:02:25Z

Introduction

← Nächstältere Version		Version vom 20. März 2022, 10:02 Uhr
Zeile 7:		Zeile 7:
	Using VNC (or local screen automation) has one big advantage, and a number of disadvantages:		Using VNC (or local screen automation) has one big advantage, and a number of disadvantages:

	* [++] it allows interaction with and the testing of almost any application, and no instrumentation or recompilation of the tested application is needed, given that a VNC-server is running and reachable on the target system, or the target's screen can be shown on the local screen somehow.<br>VNC servers are available or already installed on almost any operating system or device, even for mobile devices, embedded control systems, head units in cars, CNC machines etc.<br>In addition, screen-viewers are available for other applications, for example emulators or screen-replicators.		* [++] it allows interaction with and the testing of almost any application, and no instrumentation or recompilation or UI automation support of the tested application is needed, given that a VNC-server is running and reachable on the target system, or the target's screen can be shown on the local screen somehow.<br>VNC servers are available or already installed on almost any operating system or device, even for mobile devices, embedded control systems, head units in cars, CNC machines etc.<br>In addition, screen-viewers are available for other applications, for example emulators or screen-replicators.

	* [-] whilst all other UI testing plugins interact with the tested application directly (via a protocol which enables expecco to reflect (query) on internals of the app), the screen interfaces only see the screen as a pixel array, and can only send events based on screen coordinates. No information about the widget element hierarchy or internal attribute can be interchanged via VNC.<br>Thus, with VNC, expecco cannot ask the application about the position of GUI elements and cannot ask for any internal attribute values.<br>Expecco must rely on bitmap comparison algorithms in order to detect UI elements.		* [-] whilst all other UI testing plugins interact with the tested application directly (via a protocol which enables expecco to reflect (query) on internals of the app), the screen interfaces only see the screen as a pixel array, and can only send events based on screen coordinates. No information about the widget element hierarchy or internal attribute can be interchanged via VNC.<br>Thus, with VNC, expecco cannot ask the application about the position of GUI elements and cannot ask for any internal attribute values.<br>Expecco must rely on bitmap comparison algorithms in order to detect UI elements.

Cg: /* Introduction */

2022-03-20T10:00:12Z

Introduction

← Nächstältere Version		Version vom 20. März 2022, 10:00 Uhr
Zeile 3:		Zeile 3:
	Local screen automation and VNC automation behave the same, except for the underlying mechanism for screen image fetching and event handling. Therefore, the following describes both mechanisms.		Local screen automation and VNC automation behave the same, except for the underlying mechanism for screen image fetching and event handling. Therefore, the following describes both mechanisms.

	The VNC ~~and Local Screen~~ automation ~~plugins~~ ~~interface~~ to local or remote ~~applications~~ via the [https://en.wikipedia.org/wiki/RFB_protocol RFB] protocol, which enables interaction with the screen or a remote screen on any other machine in the network.		The VNC automation plugin interfaces to a local or remote application via the [https://en.wikipedia.org/wiki/RFB_protocol RFB] protocol, which enables interaction with the screen or a remote screen on any other machine in the network. The Local Screen automation plugin interfaces to the local screen only.

	Using VNC (or local screen automation) has one big advantage, and a number of disadvantages:		Using VNC (or local screen automation) has one big advantage, and a number of disadvantages:

Cg: /* Typical Problems */

2022-03-20T09:57:28Z

Typical Problems

← Nächstältere Version		Version vom 20. März 2022, 09:57 Uhr
Zeile 331:		Zeile 331:

	== Typical Problems ==		== Typical Problems ==
	All OCR programs have problems in that some characters are too similar to be recognized correctly, depending on the font. Most typical is the the letter "O" looks too similar to the digit "0", or that digit "0" and "%" are too similar to the digit "8". ~~Also~~ ~~"l"~~ ~~(lower~~ ~~case~~ ~~"L")~~ ~~often~~ ~~looks like~~ the ~~digit~~ ~~"1" or locate case "I" in non-serif fonts.~~		All OCR programs have problems in that some characters are too similar to be recognized correctly, depending on the font. Most typical is the the letter "O" looks too similar to the digit "0", or that both the digit "0" and "%" are too similar to the digit "8" (some fonts have a diagonal line inside the zero).
			Also "l" (lower case "L") often looks like the digit "1" (one) or lower case "I" in non-serif fonts.

	To solve this, you can give additional constraints to the OCR program, when text is extracted:		To solve this, you can give additional constraints to the OCR program, when text is extracted:
	* character constraints (eq. ~~all~~ digits)		* character constraints (eg. recognize only digits)
	* word lists (set of ~~wellknown~~ words to be expected)		* word lists (give a set of well known words to be expected)
			* national language

	Be prepared, that OCR readout is not without problems, and that misreadings are quite common.		Be prepared, that OCR readout is not without problems, and that misreadings are quite common.

Cg: /* Automatic Element Definition on Click */

2021-12-06T12:37:35Z

Automatic Element Definition on Click

← Nächstältere Version		Version vom 6. Dezember 2021, 12:37 Uhr
Zeile 370:		Zeile 370:
	When clicking into the recorder window,		When clicking into the recorder window,
	expecco will try to find the image under the click position		expecco will try to find the image under the click position
	and verify that it is found exactly once on the screen.		and verify that it is found exactly once on the screen.

			Notice that this automatism may present a strange (big) search image iff eg. a clicked icon is found multiple times on the screen. The algorithm will then extend the image until there is exactly one match on the screen.<br>This happens especially, when the recorder itself currently shows the image.

	A dialog will appear presenting this image, ask for the name of the element,		A dialog will appear presenting this image, ask for the name of the element,

Cg: /* Menu and Toolbar Items */

2021-08-30T19:51:22Z

Menu and Toolbar Items

← Nächstältere Version		Version vom 30. August 2021, 19:51 Uhr
Zeile 383:		Zeile 383:

	The VNC recorder offers the following items in its toolbar:		The VNC recorder offers the following items in its toolbar:
	* [[Datei:VNCRecorder_RecordButton.png]] - ~~toggle~~ recording		* [[Datei:VNCRecorder_RecordButton.png]] - resume recording
	* [[Datei:VNCRecorder_PauseButton.png]] - pause recording<br>events will still be sent to the remote application, but not recorded		* [[Datei:VNCRecorder_PauseButton.png]] - pause recording<br>events will still be sent to the remote application, but not recorded
			* [[Datei:VNCRecorder_StopButton.png]] - stop and close recorder
	* [[Datei:VNCRecorder_SelectInTree.png]] - select in tree		* [[Datei:VNCRecorder_SelectInTree.png]] - select in tree
	* [[Datei:VNCRecorder_HighlightElementUnderCursor.png]] - highlight the element under the cursor		* [[Datei:VNCRecorder_HighlightElementUnderCursor.png]] - highlight the element under the cursor

Cg: /* Menu and Toolbar Items */

2021-08-30T19:41:27Z

Menu and Toolbar Items

← Nächstältere Version		Version vom 30. August 2021, 19:41 Uhr
Zeile 384:		Zeile 384:
	The VNC recorder offers the following items in its toolbar:		The VNC recorder offers the following items in its toolbar:
	* [[Datei:VNCRecorder_RecordButton.png]] - toggle recording		* [[Datei:VNCRecorder_RecordButton.png]] - toggle recording
			* [[Datei:VNCRecorder_PauseButton.png]] - pause recording<br>events will still be sent to the remote application, but not recorded
	* [[Datei:VNCRecorder_SelectInTree.png]] - select in tree		* [[Datei:VNCRecorder_SelectInTree.png]] - select in tree
	* [[Datei:VNCRecorder_HighlightElementUnderCursor.png]] - highlight the element under the cursor		* [[Datei:VNCRecorder_HighlightElementUnderCursor.png]] - highlight the element under the cursor

Cg: /* Menu and Toolbar Items */

2021-08-30T19:40:30Z

Menu and Toolbar Items

← Nächstältere Version		Version vom 30. August 2021, 19:40 Uhr
Zeile 387:		Zeile 387:
	* [[Datei:VNCRecorder_HighlightElementUnderCursor.png]] - highlight the element under the cursor		* [[Datei:VNCRecorder_HighlightElementUnderCursor.png]] - highlight the element under the cursor
	* [[Datei:VNCRecorder_HighlightAllEements.png]] - highlight all clickable elements		* [[Datei:VNCRecorder_HighlightAllEements.png]] - highlight all clickable elements
	* [[Datei:VNCRecorder_RecordAsScreenAction.png]] - record as screen action		* [[Datei:VNCRecorder_RecordAsScreenAction.png]] - record as screen action<br>The mouse or keyboard action will be sent to the absolute screen coordinate without any element image search
	* [[Datei:VNCRecorder_AddScene.png]] - add a new scene		* [[Datei:VNCRecorder_AddScene.png]] - add a new scene
	* [[Datei:VNCRecorder_AddElement.png]] - add a new element		* [[Datei:VNCRecorder_AddElement.png]] - add a new element

Cg: /* Typical Problems */

2021-08-30T19:38:17Z

Typical Problems

← Nächstältere Version		Version vom 30. August 2021, 19:38 Uhr
Zeile 335:		Zeile 335:
	To solve this, you can give additional constraints to the OCR program, when text is extracted:		To solve this, you can give additional constraints to the OCR program, when text is extracted:
	* character constraints (eq. all digits)		* character constraints (eq. all digits)
	* word lists (set of ~~well known~~ words to be expected)		* word lists (set of wellknown words to be expected)

	Be prepared, that OCR readout is not without problems, and that misreadings are quite common.		Be prepared, that OCR readout is not without problems, and that misreadings are quite common.