Generating Testdata/en

Aus expecco Wiki (Version 2.x)
(Weitergeleitet von Generating Test Data)
Wechseln zu: Navigation, Suche

Introduction

One of the most frequently asked questions is "How can I generate testdata". Due to expecco's flexibility, there are multiple solutions to solving this task. Depending on the kind and amount of data to be generated, one of several patterns applies. The following gives you a rough overview on that theme.  

Existing (out-of-the-box) Solutions

Before writing your own generator, take a look at the existing blocks; often some of them already provides a useful function to generate what you need.

Enumerating Values from a Fixed Set

For a small number of fixed values, use a collection enumerator, with a frozen input-collection:

Test data collection1.png

As an alternative, you can also put the values into a variable, and read its value. Placing "likely-to-be-changed" values into a top-level environment variable makes them easier to find and customize later.  

Enumerating Values from a File Attachment

Alternatively, write them as lines into an attachment-file,

 

Test data attachment1.png

 

and read them line-wise (change the attachment-block's execution behavior to "Linewise", in the attachment-step's "Output-Data-Representation" popup-menu):  

Test data fromAttachment1.png

  The attachment can be an embedded file, as in the above example, or it can be accessed from a remote location via a URL-attachment.  

Using a Testdata Generator Block to Generate Values in a Timeline

This type of block uses a table consisting of rows (one row per generated value tuple) and columns (a timedelta and one column per value in the output tuple). The number of elements in the value tuples corresponds to the number of output pins, which have been specified in the testdata generator's schema definition. The datatypes of the values corresponds to the corresponding pin's datatype. To define a sequence of values, first define the names and types of the output pins in the schema, then switch to the data editor tab, and add one row for each tuple which is to be generated.

When executed, the generator will send data tuples controlled by a timeline to its output pin(s). With multiple output pins, tuples can be generated in a lock-step fashion. The default timedelay is 1 tuple per second, but this can be changed in the data table, down to a zero delay. In that case, values are generated immediately at the output pins, and probably buffered at the receiving input pins.

If the values have to be generated with non-zero delays in between (for example, to trigger some external machinery or program every other time), a test-data generator block is the easiest to use.
First, define a test-data generator block:

 

Test data generator1.png

 

and use its output values as in:  

Test data generator2.png

Notice that a test data generator with constant time deltas could also be easily written as the following simple elementary block:

   execute
      #(
           (1111  "string1"  1.6  true)    "/ tuple 1
           (2222  "string2"  2.6  false)   "/ tuple 2
           (3333  "string3"  3.6  false)   "/ tuple 3
           (4444  "string4"  4.6  true)    "/ tuple 4
           "/ ...
    ) do:[:eachTuple |
       outpin1 value: (eachTuple at:1).
       outpin2 value: (eachTuple at:2).
       outpin3 value: (eachTuple at:3).
       outpin4 value: (eachTuple at:4).
       Delay waitForSeconds: 1
   ]

 

Enumerating Values from an Arithmetic or Geometric Series

For numeric series (range checks), use a series-generator, as in:

 

Test data series1.png

or:

Test data series2.png  

Random Values

or even a random generator block:

Test data random1.png

 

Test-Adaptors and Data Generators

When a test is to be executed with a number of different testdata sets, you should split the action into two parts. One part, the data-generator part, which delivers tuples of test-values, and another part, which feeds the tuples to the SUT and checks for the expected result. Testdata tuples can be passed to the test-activity either via an input (with tuple-datatype to receive tuples of values), or by setting up a set of environment variables for each value combination and executing the test-activity within that environment.

Therefore, in the following, assume that the test-adaptor has been defined as:

Test adaptor.jpg

This adaptor is only used to receive value tuples and provide them to the actual test function; therefore, its internals look simply as:

Test adaptorInternal.jpg

Let us assume that a test-data generator is to be used to generate tuples. Its interface definition will look like:

Test generator.jpg

Notice, that the generator generates multiple tuples (i.e. a stream of individual data value tuples). For each such tuple, one run of the test adaptor is to be executed.

When combining the generator and the adaptor, we get the following picture:

Test harnish.jpg

Exceptions from the tested function are caught and reported as a failure-log. We have to catch exceptions explicitly here, in order to proceed with the next data tuple after a failure. Without the exception output handling, the whole testcase would stop with the first failing tuple value combination.

Testdata can be generated by multiple mechanisms, and are specified inside the generator block. Which mechanism to use, depends upon your concrete application. In many situations, existing blocks or customized compound blocks from existing ones can be used. Especially the Test Data Generator and CSV reader blocks will handle most cases. However, it may be required to create specialized testdata generators, which are not easily constructed from existing components. The following shall give you some info to start your own.


Programmatic Generation

Of course, by writing an elementary block, which generates data as output, almost any set of testdata can be generated. However, even for this, a number of already existing helper functions are useful. The examples below are shown in Smalltalk; the corresponding JS code would look similar. Some of the code fragments duplicate existing library blocks (as described above). These are included for your understanding and to give you a hint when exploring the underlying class library.

Combinatoric

To generate all possible combinations from a set of values, a useful method from the collection protocol can be used. For example, given the possible values #(10 20 30) for variable "a", (1 to: 10) for "b" and (-5 to:5) for "c", the elementary code to generate all permutations would be:

execute
    |valueRanges|

    valueRanges := OrderedCollection new.
    valueRanges add: #(10 20 30).
    valueRanges add: #(1 to: 10).
    valueRanges add: #(-5 to: 5).
    valueRanges combinationsDo:[:eachCombination |
        dataTuples value:eachCombination
    ].

When executed, tuples like (10 1 -5), (10 1 -4),..., (10 1 4), (10 1 5), (10 2 -5), (10 2 -4),..., (10 2 4), (10 2 5), ... (30 10 5).

Be aware of the combinatoric explosion - the above generates 3x10x11 = 330 output tuples.

Series

Fixed series can be generated in a similar way with:

execute
    |valueRanges|

    tuples := #(
        (10 1.0 'abc')
        (10 2.0 'def')
        (20 3.0 'xxx')
    ).
    tuples do:[:eachTuple |
        dataTuples value:eachTuple
    ].

Of course, this is very similar to what the data-generator block already. However, you will need a programmatic solution, if dynamic data (such as date, time or measurement values) affect the test data.

Random Data

Random tuple values:

execute
    |a b c|

    100 timesRepeat:[
        a := Random nextBetween:0 and:10.
        b := Random nextIntegerBetween:1 and:6.
        c := Random nextBoolean.

        dataTuples value:{a. b. c. }
    ].

Complex Random Values

By writing a small custom block, more complex random patterns can be generated. For example, the following elementary block generates random strings of the form: "xxx-##-$$$$$$$$$$" where "##" is a two-digit number and "$$$$$$$$$$" is a 10-digit random string consisting of lower case letters.

execute
    |randomInt randomString|

    randomInt := Random nextIntegerBetween:10 and:99.
    randomString := ((1 to:10) collect:[:i | ($a to: $z) atRandom ]) asString.
    out value:('xxx-%1-%2' bindWith:randomInt with:randomString).

Typical values generated are:

xxx-94-qbpigndyop
xxx-46-rhvrskmnka
xxx-26-adhnskbrbw
xxx-91-wfghiryjzj
xxx-63-cdgyumjwpg
xxx-94-unogbigssz
xxx-48-jrbtnmmuqc

Random UUID Values

The UUID class can generate new UUIDs, which by definition are globally unique (meaning that with a very very high probability, no two ids on any machine generated at any time are the same) and thus kind of random. For this, either use the existing "Generate UUID" elementary block, or create a new action block as:

execute
    numGeneratedPin value timesRepeat:[
        uuidOutPin value: (UUID genUUID)
    ]

Reading Data from an external Sources

Builtin Blocks to Read Files

For simple CSV (comma separated value) data, as provided by excel or other spreadsheet programs, use a CSV importer and use its output to feed the SUT.

Reading a DataBase

Using a Database-Query block, you can of course fetch test-data from any relational database which is accessible with an ODBC interface (which is in fact true for almost any database). For more details, please refer to the ODBC library documentation.

Getting the Data from another Program

You can execute any other program and read its standard output using either a standard shell command block, or by a direct call to the underlying class library. A call to the "tr" (transliterate) command under UNIX or cygwin (Windows) might look like:

execute
    |in out err|

    in := 'bla bla bla' readStream.
    out := WriteStream on:(String new).
    err := WriteStream on:(String new).
    retCode := OperatingSystem
        executeCommand: 'tr a b'
        inputFrom: in
        outputTo: out
        errorTo: err
        inDirectory: '~/someDirectory'.
    ...
    do something with "in contents"
    ...

Custom Blocks for Special Formats in Files

For special formats, use an elementary block. For example, the following reads measurement values from a binary file, where values come in multiples of 10 binary 32bit integer values:

execute
    |inFileName s|

    inFileName := in value. "/ filename via input pin
    s := inFileName asFilename readStream.
    [
        s binary.
        [s atEnd] whileFalse:[
            tuple := (1 to:10) collect:[:i | s nextUnsignedLongMSB:false].
            dataTuples value:tuple.
        ].
    ] ensure:[
        s close.
    ]

Concrete Example: Import of Equivalence-Class-Data

Assume that you have the equivalence class information about some variables at hand (this information could be extracted from the specification, from code analysis or any other non-expecco mechanism). Also assume, that the information is available in file or document, in a format which specifies boundary values; similar to:

a: 0
b: 0
c: 5,8

The entries specify boundary values for individual variables. For "a", we have one boundary (0) which defines 3 intervals: all values below 0 are within one class, values above 0 are in another, and 0 alone forms a third class.
The equivalence classes could be defined as:

x < 0,
x == 0,
x > 0.

Notice, that the above is only one possible way to define equivalence classes; another, more strict and test might try the values near the boundaries too, thus generating even more equivalence classes. As many errors occur around those boundaries (off by one errors, especially in loops), the equivalence classes below may be better for testing. The code-block described below generates this kind of values:

x < -1,
x == -1,
x == 0,
x == +1,
x > +1.


Notice, that this is a simplified format; a better way would be to specify a set of valid ranges for each variable, such as

a: (..<0),0,(0>..)
b: (..<0),0,(0>..)
c: (..5),(6..7),(8..)

Thus, a chain to read values from this spec, and provide the spec to a combinatoric generator looks like:

EqTest chain.jpg

and the complete setup for the test adaptor is:

EqTest harnish.jpg

In the strict sense, "any negative" and "-1" belong to the same equivalence class of "a"-values. However, in practice, many programmer errors occur at +/- 1 of a boundary. Therefore, the generator was programmed to generate tuples containing representant instances from each class PLUS all boundary values. For the value of "a", these are:

"any negative",
"-1" (boundary-1),
"0" (boundary),
"+1" (boundary +1)
and "any positive". 

Execution results in the following log:

EqTest result.jpg

Of course, with some more work, arbitrary complex description formats for the equivalence-description file are possible.

Sample Code

A sample project file demonstrating some of the above patterns ("DataGeneratorsTutorial.ets") is found in the examples directory of your expecco installation. It can also be downloaded here: http://wiki.expecco.de/images/e/e8/DataGeneratorsTutorial.ets -->



Back to Online Documentation



Copyright © 2014-2016 eXept Software AG