Report Generation/en

Aus expecco Wiki (Version 2.x)
(Weitergeleitet von Report Generation)
Wechseln zu: Navigation, Suche

The results of a test execution can be visualized by creating a test report. These reports can be created as PDF or HTML documents. Its also possible to create an XML-, plain text or a CSV-file containing all relevant information on the test execution.

Report Templates

Report templates are used to specify the content and layout of created reports. By default the so called "Default Report Template" is used to create reports. Other report templates can be created in testsuites by adding a report template element to the tree. They will be saved together with the testsuite.

How to create a report

Report Settings

To create a report, a test has to be executed first. This can be either a single block test or a whole testplan. After the execution is finished, click on the "Print report" button (Report button) causing a dialog to open. Here you can select the output format (PDF, HTML, XML, CSV or TEXT), the report language, a paper format (either DIN A4 or Letter format) and a template for the report. This template can be the default Report template, a template from the current testsuite or a template which is stored separately in a file. You can also choose to edit the default report template.

How to customize a report template

To customize a report template you have to select it in the navigation tree. In the related editor the template can be modified. The same editor is also used to modify the default report template. You can access it either via the already mentioned dialog or via the general expecco settings.

Filter Settings

Filter Settings

These settings determine which items are part of the report. This way you can create either detailed or condensed reports.

Testsuite

  • Do not Print Testsuite Environment
    This option determines whether the testsuite environment, i.e. the variables defined in the current testsuite, are part of the test report. Please note that only variables which are defined as read-only will be shown in the report because the value of other variables can change during the test execution.

Testplan

  • Do not Print Testplan Environment
    This option determines whether the environment of each executed testplan, i.e. the variables defined in the respective testplan, are included in the report. Please note that only variables which are defined as read-only will be shown in the report.
  • Do not Print Successful Testcases
    This option removes all successful test cases from the report when checked, so that only failed tests are reported.
  • Only Print Testplan Summary
    If checked, this option will reduce every testplan to a summary of the results of the different testplan items, i.e. it removes all further information such as sub-activities or logs.
  • Print Unexecuted Test Cases
    This option also includes test cases to the report which have not been executed.
  • Print Pre/Post Condition
    This option determines whether the pre- and post executions are present in the report or not. There is also an option to include this information only for failed tests.

Activity

  • Nesting Limit
    This option determines to which level nested sub-activities are reported. This affects the length and also the level of detail of the generated report.
  • Activity Tags
    In order to include or exclude activities spawned by certain blocks from the report, you can add tags to each block. These tags can be set in the documentation tab of the block. Later you can filter the executed blocks by their tags. Please note that an inclusion to the report has priority over an exclusion.
    • Include Tag(s)
      All blocks which have one or more of those tags will be present in the report. Multiple tags must be separated by ";".
    • Exclude Tag(s)
      All blocks which do not have one or more of those tags will be present in the report. Multiple tags must be separated by ";".
    • Sub-activities Include Tag(s)
      All sub activities which have one or more of those tags will be present in the report. Multiple tags must be separated by ";".
    • Sub-activities Exclude Tag(s)
      All sub activities which do not have one or more of those tags will be present in the report. Multiple tags must be separated by ";".
    • Search Tags in Children
      This option determines whether the sub-activities of an activity are searched for existing tags, when filtering.
  • Print Input/Output Pins
    This option determines whether the data of input and output pins is present in the report. You can also add this information only in case the execution failed.
  • Do not Print Output Pins
    This option removes the data of the output pins from the report, so that only input values are printed.

Logs

These settings are used to specify how logs are reported.

  • Nesting Limit
    This option is used to determine down to which level logs of nested sub activities are reported.
  • Log Tags
    In order to include or exclude logs of certain blocks from the report you can add tags to each block. Please note that an inclusion to the report has priority over an exclusion.
    • Include Tag(s)
      All logs of blocks which have one or more of those tags will be present in the report. Multiple tags must be separated by ";".
    • Exclude Tag(s)
      All logs of blocks which do not have one or more of those tags will be present in the report. Multiple tags must be separated by ";".
    • Search Tags in Children
      This option determines whether the sub activities of an activity are searched for existing tags when filtering.
  • Log Types
    These checkboxes are used to include or exclude different log message types from the report. Please note that there is a difference between logs that only change the state of an activity, e.g. "logFail", and real exceptions that stop the execution, e.g. "fail". The "All" button is used to toggle all types.

Presentation Settings

Presentation Settings

These settings are used to determine how the report is presented.

Common

  • Print Full Activity Name
    This option determines whether the name of the corresponding block is printed when a step is reported. If the step-name is the same as the block name (which is the common case), then that option does not make a difference. But if a step has been renamed from the underlying block's name, then both names will be printed if they are different.
  • Print Activity Name in Logs as Well
    This option determines whether the name of the activity which logged a message will be printed in front of the message.
  • Indention Level Limit
    This option is used to determine to which nesting level reported sub activities are indented.
  • Highlight Top State in Every Row
    This option adds a colored area with the top state color in every row. This helps to keep track of the overall test case result in case that many sub activities are reported.
  • Page Break after Testplan Summary
    This option will add a page break after a testplan summary.
  • Page Break after every Testcase
    This option will add a page break after every test case.

Fonts

These settings are used to change the fonts used for the document, the column headers and the activity reports.

Colors

These settings are used to change the color of different report items. Its possible to use the default color as defined in the settings for error and failure logs. You can also print a "Only Black & White" report by toggling the corresponding check-box.

Test Overview Settings

Test Overview Settings

These settings can be used to customize the report summary.

  • Testplan Summary
    Here you can specify how the summary of a testplan is presented. The editor panel is described here: Editor Panel
  • Activity Summary
    Here you can specify how the summary of an activity is presented. The editor panel is described here: Editor Panel

Document Settings

Document Settings

These settings apply on the general design of the report. Note that the header and footer line will only be present in a PDF report.

  • Document Header
    Here you can create a header for the report. The editor panel is described here: Editor Panel
  • Columns Width
    Here you can resize the three columns of the report. Please note that each column has to fill at least 10 percent of the entire width.
  • Header
    These text fields are used to edit the header line of the report. The left fields are used to enter left aligned text, the center ones for center aligned text and the right ones for right aligned text. You can also create links by typing a target into the "link" fields under the text fields.
  • Footer
    These text fields are used to edit the footer line of the report. The usage is similar to the header line fields.

Logo Settings

These settings are used to specify images, e.g. company logos, which can be used in the report. By default the expecco logo is used. To replace the default images just click on the "replace" button behind the filenames and select the new image's file in the file dialog. Please note that the file type has to be ".jpg".

Description

Here you can add a description for the report template. This can be useful to inform other users for example when to use this template.

Editor Panel

This editor panel is used to create style information for report summaries and the report header.

The workspace is made up of a number of rows which are separated columns (limited to a maximum of 6 columns). A single field is called cell. It can contain either text or graphics, such as a pie chart or a logo.

To enter text, double click on the cell causing a text editor to open.

To insert graphics, right click on a cell and select "Insert Graphic". Now you can select the type of the graphic. Please note that pie charts are only available for testplan reports and logos only for the document header.

Inside text, the content of a placeholder variable, such as the name of the testsuite or the current date can be inserted: double click on a cell to open the editor and right click to open the context menu. Now select "Insert Variable" and choose one of the variables from the list. Currently, variables available in a testplan overview are:

  •  %testplan_name% - The name of the testplan
  •  %testcases_size% - The number of testcases in the testplan
  •  %testcases_executed% - The number of testcases in the report
  •  %hostname% - The hostname of the executing machine
  •  %username% - The username (login name)
  •  %start_date% - The date when the testplan was started
  •  %start_time% - The time when the testplan was started
  •  %run_time% - the overall execution time of the testplan (when finished)
  •  %elapsed_time% - the elapsed execution time of the testplan (only useful if a GenerateReport block is used from within an executing testplan. same as run_time when finished)

Variables available in an activity-overview are:

  •  %action_name% - The name of the activity
  •  %hostname% - The hostname of the executing machine
  •  %username% - The username (login name)
  •  %start_date% - The time when the activity was started
  •  %start_time% - The time when the activity was started
  •  %run_time% - the execution time of the activity

Buttons

  • Icon Insert Row.png Inserts a new row after the selected one
  • Icon Insert Column.png Inserts a new column right from the selected one. Please note that only the selected row is affected.
  • Icon Remove Row.png Removes the selected row
  • Icon Remove Column.png Removes the selected column
  • Icon Report Move Up.png Icon Report Move Down.png Moves the selected row up/down
  • Icon Report Move Left.png Icon Report Move Right.png Moves the selected column left/right
  • Icon Open Editor.png Opens an editor to modify the content of the selected column
  • Icon Save Close Editor.png Closes the editor with saving changes
  • Icon NoSave Close Editor.png Closes the editor without saving changes
  • Icon Left Alignment.png Icon Center Alignment.png Icon Right Alignment.png Aligns the content of the column left/center/right
  • Icon Show Hide Grid.png Shows/Hides a grid to resize the columns


Report vs. expecco Log-File

In contrast to the generated reports, which are meant to be either human readable archived documents (HTML, PDF, text) or machine readable, to be processed by other programs (CSV, XML), there is another, expecco-internal storage format, the so called "expecco log files", with a ".elf" extension. These files store all relevant tree elements, log data, in- and output in values which are required to reconstruct a full activity log/result tree. This also means, that they include a copy of the executed blocks and their definitions, independent of any changes done to the test suite in the meantime. As such, they can be also archived to proof exactly which version of a suite was executed.

Just like regular reports, these log files can be generated after the execution of a test (both single block- and full testplan tests), and saved to a file:

Save elf1.jpg

 

expecco ALM (formerly called "expeccoNET") also provides central storage of all log files and makes them available later to team members with appropriate access rights.

Save elf2.png

As you can see, there is a menu item named "Load Test-Results (from expeccoNET)“ which loads those ".elf" files from this central database. If you do not have expecco ALM installed, you have to save/load those from a regular file. After loading, the result tree is presented as it was right after the execution. You can also navigate through the tree, see pin values and other artefacts (however, of course, objects which where present on remote systems and only referred to via a bridge connection are no longer available. If required, these should be marshaled to XML or any other textual format, if that is required).


Another advantage of this format is that additional reports can be generated at any time later. For example, you can generate a new report with a different layout, look or detail. For example, if your company logo has changed in the meanwhile, or the product department needs another, less detailed report.


Print elf.png

Example: Executive Summary vs. Developer Report

Executive Summary
Developer Report.

The main advantage of report templates is the possibility to create standardized reports for different purposes. This way you can for example create a condensed report as a summary for an executive or a very detailed one for the development team to track down error causes or to document a test run.

In the shown example the executive summary solely shows the overall test results which are visualized by a pie chart. Other irrelevant information has been removed to keep the report clear and readable. A corresponding template can be found here: Datei:Executive Summary Template.ert

The shown developer report is more detailed and shows all relevant data to track down a possible error. In this case all irrelevant data from the developer's point of view is left out. For example the pie chart might not be useful in this case. A corresponding template can be found here: Datei:Developer Report Template.ert



Copyright © 2014-2016 eXept Software AG