Difference between revisions of "Environment Editor/en"

From expecco Wiki (Version 2.x)
Jump to navigation Jump to search
 
(8 intermediate revisions by the same user not shown)
Line 45: Line 45:
 
 
 
==== Constant ====
 
==== Constant ====
:a constant value, such as a port number, counter values, host name or other simple (unstructured) values. This value is used "as is" except for variable expansion: you may refer to other variables by name or the operating system's shell environment variables. Use "$(Name)" to refer to the value of the variable named "Name". The parenthesis are required. Thus if your environment contains a variable "Foo", with value "Hello", the value "$(Foo)World" will expand to "HelloWorld" whereas "$Foo World" remains as is.<p>Be careful when referring to variables from within the same environment, because they are initialized top-to-bottom, and the values of variables further down the list will not yet be setup.
 
  +
:a constant value, such as a port number, counter values, host name or other simple (unstructured) values.
  +
: This value is used "as is" except for variable expansion: you may refer to other variables by name or the operating system's shell environment variables.
  +
: Use "$(Name)" to refer to the value of the variable named "''Name''". The parenthesis are required. Thus if your environment contains a variable "Foo", with value "Hello", the value "$(Foo)World" will expand to "HelloWorld" whereas "$Foo World" remains as is.<p>Be careful when referring to variables from within the same environment, because they are initialized top-to-bottom, and the values of variables further down the list will not yet be setup.
   
If the variable is a compound action block's local variable (i.e. not a project-global or testplan/testitem variable), then the action's pin values are accessible as "$(pinName)".
+
: If the variable is a compound action block's local variable (i.e. not a project-global or test plan / test item variable), then the action's pin values are accessible as "$(pinName)".
 
In addition to the shell's variables, such as "$(USERNAME)" and "$(PATH)", a few additional pseudo variables are accessible:
 
   
  +
: In addition to the shell's variables, such as "<code>$(USERNAME)</code>" and "<code>$(PATH)</code>", a few additional pseudo variables are accessible:
 
:: $('''AttachmentsDirectory''') - the directory where all of project's attachment files are located. Notice that the attachment's filename can be different from the name in the expecco navigation tree, although usually the same name is used.
 
:: $('''AttachmentsDirectory''') - the directory where all of project's attachment files are located. Notice that the attachment's filename can be different from the name in the expecco navigation tree, although usually the same name is used.
   
Line 69: Line 72:
 
:: $('''DocumentsDirectory''') - your "documents" directory
 
:: $('''DocumentsDirectory''') - your "documents" directory
   
:: $('''TmpDirectory''') - the temporary directory (usually "/tmp", but this could have been redefined via the TMPDIR shell environment variable).
+
:: $('''TmpDirectory''') - the temporary directory (usually "<code>/tmp</code>", but this could have been redefined via the <code>TMPDIR</code> shell environment variable).
   
 
:: $('''LoginName''') - your login (user-) name
 
:: $('''LoginName''') - your login (user-) name
   
:: $('''UserName''') - your full name; only on Unix; on Windows, this is the same as $( LoginName)
+
:: $('''UserName''') - your full name. Only on Unix; on Windows, this will have the same value as <code>$( LoginName)</code>
   
 
==== Constant (Not Expanded) ====
 
==== Constant (Not Expanded) ====
Line 103: Line 106:
   
 
==== Smalltalk Expression ====
 
==== Smalltalk Expression ====
:the text is evaluated as a Smalltalk expression and the resulting value used to initialize the variable. For example, with "Filename desktopDirectory" as expression, the variable will be initialized to exactly that: the name of the user's desktop folder. Or with "Dialog request:'What?'", a dialog will pop up, asking for "what" (i.e. that is the behavior of the request from user initialization type).
+
:the text is evaluated as a Smalltalk expression and the resulting value used to initialize the variable.
  +
: For example, with "<code>Filename desktopDirectory</code>" as expression, the variable will be initialized to exactly that: the name of the user's desktop folder.
  +
: Or with "<code>Dialog request:'What?'</code>", a dialog will pop up, asking for "what" (i.e. that will give the same behavior as the ''request from user'' initialization type).
   
 
==== JavaScript Expression ====
 
==== JavaScript Expression ====

Latest revision as of 08:15, 12 August 2019

Environment Editor

The environment editor is used to create and edit variables in the environment of a test suite, a test plan or a compound block. In imported suites/libraries, a readonly version of this editor is shown. The view slightly differs when either the testsuite's or a compound block's environment is edited. It can be found on the "Environment" tab after selecting the test suite or a compound block in the navigation tree.

Environment variables can be used as input to a step by setting an input pin's input value (freeze value) to "Read from Environment" via the pin's menu. The standard library also contains a number of blocks to access environments and to dynamically add, remove, read or modify environment variables.

Environments are arranged as a hierarchy of local environments (lexical scope), so that an inner block's variable may hide an outer block's variable, up to the containing test plan and finally the test suite's environment. By default, an inner variable (i.e. from a deeper nesting) shadows a corresponding outer variable, but this behavior may be reversed for special situations, where an inner variable is used to provide a fallback value for testing.

Fields[edit]

Variable-Name
This field is a text input field that holds the variable's name, which must meet the rules for variable naming (alphanumeric or underline character, no embedded spaces). Click it once to give it the input focus.
ALM (i.e. visible in expecco ALM)
This checkbox determines whether the variable will be visible in expeccoALM or in exported parameter sets. Such externally visible variables can later be provided by whoever initiated the automatic test run (i.e. expecco ALM, another QM platform, an external parameter file or provided as command line argument). This option/column is only shown in the project's variable environment.
MON (monitor variable)
This option is only useful if expecco is used in conjunction with certain versions of expecco ALM. It defines the variable as a monitored variable. Such variables send out change notifications in realtime, which can be monitored by the QA platform (currently: only expecco ALM supports this). This column is hidden, if you've set a experience level below "expert".
OW (overwrite)
This check box determines whether the variable hides (overwrites) an inherited variable from an outer scope, or not. This option/column is only shown for compound blocks. The default mode is "overwrite", so variables behave as if defined in a lexical scoped fashion: variables can be redefined in a compound block to overwrite any outer definition of the same named variable.
The non-overwrite mode is useful if a block wants to define a variable's fall back or default value, to be used if no outer definition exists.
For example, an interface block which needs a port number (e.g.. an HTTP port number) might define the variable as a non-overwriting local variable, with a value of "80". If an outer block or the test suite needs a different parameter, it is free to define HTTP Port with another value (see also "Hints & Tricks" below).
R/W (read/write)
This field is a check box that determines whether the variable can be modified by the suite after initialization, or not. Parameter values should be set to read only, which is the default for new variables. If an action tries to modify a readonly variable, an exception will be raised.
Datatype
This drop down list provides a list of available data types. Choose the type of the variable here (typically, most likely a String- or Integer type is used). For externally visible variables (NET or MON variables), the datatype should be one of Boolean, String, Integer, Float or Number.
Initialisation ("init-type")
This drop down list provides a list of possible initialization methods for the variable's value. Most common are Constant and New Instance, but it is also possible to assign a value during initialization by evaluating a programmatic expression, or to request values from the user via a dialog. See the list below.
Initial Value/Expression
Use this text field to enter the initial value (if the initialization method is Constant) or the expression to evaluate the initial value (if the method is one of the compute methods).
Current Value
This field shows a variable's current value. This information is useful for writable variables (for example: counters or other measurements as gathered during execution) or for user-entered variables.
Notice, that the top-level test suite environment is initialized one, when the suite is loaded, so you may have to manually or programatically reset modified values in the global environment, if the suite is executed multiple times. A manual reset is done via the "Initialize the Environment" button in the toolbar. For programatic resetting, look for the environment action blocks in the standard library. It is good practice to reset any counters or statistic data in a test's (or test plan's) pre-action.

Initialization Types[edit]

Be reminded that variables are only initialized once, when the environment itself is instantiated. The instantiation happens at load time for the top-level project environment, or with every execution for the test suite, test case and action environments. At this time, each environment variable is initialised as specified in the description's init-type.

The concrete behavior, depending on the init-type, is:

Constant[edit]

a constant value, such as a port number, counter values, host name or other simple (unstructured) values.
This value is used "as is" except for variable expansion: you may refer to other variables by name or the operating system's shell environment variables.
Use "$(Name)" to refer to the value of the variable named "Name". The parenthesis are required. Thus if your environment contains a variable "Foo", with value "Hello", the value "$(Foo)World" will expand to "HelloWorld" whereas "$Foo World" remains as is.

Be careful when referring to variables from within the same environment, because they are initialized top-to-bottom, and the values of variables further down the list will not yet be setup.

If the variable is a compound action block's local variable (i.e. not a project-global or test plan / test item variable), then the action's pin values are accessible as "$(pinName)".
In addition to the shell's variables, such as "$(USERNAME)" and "$(PATH)", a few additional pseudo variables are accessible:
$(AttachmentsDirectory) - the directory where all of project's attachment files are located. Notice that the attachment's filename can be different from the name in the expecco navigation tree, although usually the same name is used.
$(ProjectDirectory) - the temporary directory where all of the project's files are located. This directory will be removed when another project is loaded, or expecco is terminated.
$(ProjectFilename) - the name of the suite's ".ets" file.
$(ExecutionDirectory) - the temporary directory where temporary files during a single test execution are located. This directory will be removed after the execution. Notice, that every execution creates a new temporary execution directory. This is done to allow for the same suite to be executed in parallel - either inside one expecco session or by two expecco applications (for example, when automated via jenkins, scripts or cron jobs).
$(ExpeccoInstallationDirectory) - the directory where expecco is installed.
$(ExpeccoPluginDirectory) - the directory where expecco plugins are installed.
$(CurrentDirectory) - the Unix "current" directory
$(HomeDirectory) - your home directory (login directory)
$(DesktopDirectory) - your desktop directory
$(DocumentsDirectory) - your "documents" directory
$(TmpDirectory) - the temporary directory (usually "/tmp", but this could have been redefined via the TMPDIR shell environment variable).
$(LoginName) - your login (user-) name
$(UserName) - your full name. Only on Unix; on Windows, this will have the same value as $( LoginName)

Constant (Not Expanded)[edit]

similar to the above, but the value is as-entered (i.e. no variables are expanded, and $(...) patters are left unchanged.

SecretConstant[edit]

like above, but its value is not shown in the variable-environment; useful for passwords. Notice: the algorithm for password storage is not secure. It will be easy for any experienced user/programmer to extract these values from either the saved test-suite or via an inspector from the running test-suite. The value may also be visible in the execution log, if it is passed as String through input/output pins (make sure that pins with secrets are excluded from the log).

New Instance[edit]

creates a new (empty) instance of the data type.

New Instance of Size[edit]

given its size in the environment, create a vector (collection-like) instance of the data type.

Ask for Size[edit]

will ask the user for the size of an instance of the data type to create.

Request from User[edit]

opens a dialog to ask for the variable's value, when the project is loaded. You can define the dialog box's question title in the comment (see below).

Request from User when first used[edit]

opens a dialog to ask for the variable's value, when the variable's value is needed for the very first time during the session. If the value is never needed (because the block(s) which need this value are not executed), no question dialog will be shown during execution. You can define the dialog box's question title in the comment (see below).

Secret from User[edit]

like "Request from User", but hides the password in the dialog, and only displays '*'s in the value fields.

Secret from User when first used[edit]

like "Request from User when first used", but hides the password in the dialog.

Smalltalk Expression[edit]

the text is evaluated as a Smalltalk expression and the resulting value used to initialize the variable.
For example, with "Filename desktopDirectory" as expression, the variable will be initialized to exactly that: the name of the user's desktop folder.
Or with "Dialog request:'What?'", a dialog will pop up, asking for "what" (i.e. that will give the same behavior as the request from user initialization type).

JavaScript Expression[edit]

as above, but uses JavaScript syntax for the evaluation.

Smalltalk Expression when first used (rel 18.1 aka 2.12)[edit]

as above, but the expression will be evaluated when the variable is first used, not when the environment is created. Use this, if the evaluation takes a long time and is possibly not needed during the test suite's execution.

JavaScript Expression when first used (rel 18.1 aka 2.12)[edit]

as above, but uses JavaScript syntax for the evaluation.

GUI-Value[edit]

a special type used with GUI blocks.

GUI-Action[edit]

a special type used with GUI blocks.

Notes on Variable Expansion[edit]

This feature has been added to the 19.2 release. In previous releases, only the above listed "special file names"and shell variables were expanded. In rare situations, this may lead to incompatibilities, please change the initialization mode to "Constant (unexpanded)" if required.

Notes on "From User" initialization[edit]

These will ask the user via an interactive UI-dialog. The boxes question text can be customized by adding lines starting with "#>" (hash-greater) to the variable's comment. These lines will be extracted from the comment and concatenated to form a multiline question string.

Notes on "When First Used" initialization[edit]

The "when first used" initialization-type is useful for passwords of dynamically alternative execution paths - for example, if your test suite needs either an FTP- or an HTTP password, depending on which communication path is chosen dynamically. Variables flagged as such will open a request-dialog, when the variable's value is needed for the very first time by an action's input pin or by elementary code. This is in contrast to the "request from user" initialisation (above), which will always ask for the value at the beginning of a test run.

Toolbar Buttons[edit]

  • Icon Add Variable.png Add
    Adds another variable to the environment

  • Icon Remove Variable.png Remove
    Removes the selected variable from the environment

  • Icon Initialize Environment.png Initialise
    Initialises the environment with the given values (only shown in the project's environment). This is done automatically, when a test suite is loaded initially, but may also be needed to reset read/write variables which have been modified by a test run back to their initial state, or to have computed values be recomputed or to ask again for user-answered variables. Lazy computed variables (i.e. those initialized when-first-used) will be reset to their initial, unknown state and will lead to reevaluation/reconfirmation when accessed again.

  • Icon Up.png Up
    Move the selected variable upwards in the list. As variables are initialized in the order of appearance, this may affect the order of execution of any computed values

  • Icon Down.png Down
    Move the selected variable downwards in the list

Filters[edit]

Two filters are available, which are useful if your environment contains many variables: by name and by comment. If toggled, only variables whose name matches the filter or which have a particular pattern in their comment are shown.

Popup Menu Functions (Right-Button-Menu)[edit]

The right mouse button shows the following popup menu items:

Undo
undo the last cut/paste operation (inside the variable editor)
Add Variable
add another variable
Cut Variable
remove the selected variable(s). These are put into the clipboard and can bbe pasted at another place.
Copy Variable
copy the selected variables to the clipboard
Paste Variable
paste the variables in the clipboard
Move Variable To..
to move the selected variables into another environment
Rename Variable...
rename a variable. Searches for uses of the selected variable and renames those references, too. However, if the name is constructed somewhere, it may not be found. So use with care.
Load Entries from File...
to load a file containing a set of values. The file can be in expecco's native XML format, in JSON or in CSV (i.e. imported from excel). For details, read the chapter on file formats below.
Save all Entries to File
to save all variables into a file. Specify XML (expecco's native format), JSON or CSV. For details, read the chapter on file formats below.
Save Selected Entries to File
like above, but only the selected variables are saved. For details, read the chapter on file formats below.
References
search for references to this variable in the suite
Inspect Current Value
opens an inspector window on the selected value.
More
additional, less frequently used menu items
Move Up / Down
to move the selected variable(s) up/down in the list

Variable Lookup Strategy, Order and Scoping[edit]

When an input pin is frozen from an environment variable, the lookup (search) behaviour is the same as of the "environmentAt" action block from the standard library. The search stops when the variable is found and the search order is:

  1. the activity chain is consulted, searching for a local definition of the variable, starting with the current activity, moving along the dynamic caller chain
  2. the current executor's environment is consulted (if running inside a test plan, this environment is initialised from those values, otherwise it is initially empty)
  3. a command line environment (if provided) is consulted
  4. a system environment (if provided) is consulted
  5. the top project's environment is consulted
  6. if the action-block comes from an imported library, that library's environment is consulted

Notice, that variables defined inside an imported library are invisible (i.e. private) in the importing main suite. If you want to expose values from such a library, you should provide an action inside the imported library, which contains an "environmentAt" action block, with its input frozen to the variable. Then place a step for this action into the main suite, to retrieve that variable's value. Be aware, that the variable may still be shadowed due to the above search order, and you may or may not retrieve the imported library's local value.

In very rare situations, it may be required to prevent shadowing; for this, place the "Get Local Project Variable by Name" action inside a compound action, inside the imported library. Then place that as a step into the importing library to access the library-value, independent of any shadowing elsewhere.

Another alternative is to automatically copy the variables from an imported library into the main suite's environment via the "Set Top Environment Variables from Imported" action. This also has to be placed into a compound inside the imported library (name this action eg. "Get my Values"), and place a step for this imported compound action into your initialisation action. This will copy all values from the imported library into the main suite's runtime environment (i.e. these will only be visible dynamically, via environment-variable accesses, but not statically in the main suites environment editor). Examples suites for these two use cases as found in the demos folder, as "d54_Importing_Variables.ets" and "d55_Variable_Imports.ets".

External File Formats[edit]

Parameters can be stored separately from a suite in a so called "parameter file". These can be generated by expecco itself (via the above described "Save As.." menu functions, or created manually in a text editor, or generated by another program.

Multiple formats are supported directly:

  • XML
    this is expecco's native and preferred format for parameter files.
  • JSON
    a JSON file should contain a single object which is an array of structure (dictionary) objects. Each should contain the name and value of a single parameter.
  • CSV
    two formats are supported: one describes each variable in a separate row as name-value pair (a vertical structure). The other contains exactly two rows, where the first contains all names, and the second contains all values (horizontal format). Both can easily be imported to or exported from excel and other table oriented calculators.

Other formats can be used by creating a reader-action block, which reads any custom format and sets the variables using environment-setter actions from the standard library.

To use one of the above file formats when executing standalone tests (i.e. without user interaction), specify its filename on the command line with the "--parameter" option. See "Command Line Options".

Special (Wellknown) Variables[edit]

The GUI-Browser subsystem can store xPath locators in the top level environment, to provide a mapping from widget-names to locators. All locator variables created by the GUI-Browser will have a "ui." prefix (i.e."ui.nameField", "ui.passwordField", etc.). These can be referred to in a locator-pin's freeze values as $(ui.xxx), also as prefix to a relative xPath (eg. "$(ui.panel)//button").

Hints and Tricks[edit]

Environment variables are your most powerful feature to ensure flexibility of your test suite and to minimize future maintenance efforts. Be reminded, that these variables can be preset individually via the command line or by providing a separate parameter file, which can also be specified via the command line. It is also possible to load a parameter set dynamically (eg. from a CSV, JSON or XML file, or via an SQL query from a database).

By placing configuration values (host names, port numbers, URLs, paths, version numbers, ui-locators etc.) into top-level environment variables, these make sure that:

  • magic constants are not spread all over your suite, which would make it hard to change such values later
  • the search functions can be used to find places in your suite, where a particular value is used
  • any test execution can later be customized, by providing either an individual variable's value or a complete set of values via the command line
  • if used with expecco ALM, these values can be defined in that central management tool, and will be passed down to the suite with every execution
  • if fetched dynamically from a database or file, tests can adjust themselves automatically to different versions or configurations
  • test sequences can be reused with different parameters by running individual actions inside another (compound-) block which redefines individual variable values

Overwritable Default Values[edit]

It is often useful to define a variable in an inner block (a compound block), so that the test-developer can run individual actions during test development. Typical scenarios involve host names or test-URLs, which may be different for the test-developer and the final (automated) test execution. In this case, the test-developer wants to see his own (private) values when he runs individual action blocks, whereas for automation, these values should be overwritten by providing values for the top-level environment. With normal scoping rules (i.e. inner definitions overwrite outer scopes), this would not be possible. However, the "OW" (Overwrite) flag allows for the scoping rules of an individual variable to be reversed, so that the inner scope value (on the block level) is only used iff no value is present on the outer (the suite) level.

Multiple Executions with Different Variable Values[edit]

Be reminded, that test plans can be nested, and offer a separate variable scope during their execution. You can wrap a whole testplan into another (master-) testplan, which simply executes the wrapped actions with different variable settings. For this, create one wrapper test plan for each run, providing different variable values as per run. Then place all those wrappers into a "master" testplan, which runs them in sequence. Of course, this can also be done with compound actions, by wrapping one action into another compound, which simple sets up a different variable scope for the inner action.

Dynamically Loading Variable Values[edit]

It is also possible to load complete sets of values from an external source dynamically, either at the start of your test, or in-between runs. Combining this with the above testplan wrapping, you can for example load parameter sets from a database or CSV file, change the variable values, and run the same sequence multiple times with different values.

Dynamically Changing Variable Values[edit]

The standard library provides blocks (named "environment-xxx") to dynamically add, modify or read environment variables.

Scope of Environments[edit]

Environments form a hierarchy, and when a variable is searched, it it searched (when a freeze-variable is accessed) along the following list (from first to last):

  • the environments along the calling chain of currently executing compound activities
  • the executor's environment
  • the command line environment (read only)
  • the system environment (read only)
  • the test suite's environment
  • the environment of the project in which the currently executing compound activity's action is defined

The standard library provides additional action blocks to manipulate some of those environments, others are readonly.

Hidden Environments[edit]

Two additional environments are provided, which can only be accessed via explicit environment-accessors from elementary Smalltalk and Javascript code:

  • the browser-window's environment (valid as long as there is a browser window)
  • the global browser environment (valid as long as expecco is running)

These should not be referred to by normal user action blocks - they are used internally (eg. by plugins) to remember open communication handles.

Predefined System Variables[edit]

The system environment contains configuration variables:

  • SYS$HEADLESS
    true, if expecco is running in headless (server) mode.
You can use this value as freeze value for web test (Selenium) connect blocks, to force headless mode of the webbrowser
  • OS$TYPE
    symbolic name of the operating system (one of "osx", "linux" or "windows")



Copyright © 2014-2018 eXept Software AG