Mostly lazy data generators for property based testing using the Spock test framework

1. Intro

Providing test data, especially when attempting to test for a wide range of inputs is tedious if not impossible to do by hand. Generating inputs allows for more thorough testing without a dramatic increase in effort. In Spock data driven tests can have data provided by any Iterable. Spock Genesis provides a variety of classes that extend from Generator which meet that interface. Where possible the generators are lazy and infinite.

1.1. License

The Spock-Genesis project is open sourced under the MIT License.

1.2. Usage

``````repositories {
jcenter()
}

dependencies {
testCompile 'com.nagternal:spock-genesis:0.6.0'
}``````
 Change `0.6.0` with the latest available version (current version is 0.6.0)

The primary way of constructing generators is spock.genesis.Gen which provides static factory methods for data generators.

2. Values

Values could be of any simple type such as a String, Integer, Byte…​etc Before using any built-in generator remember to add the following import:

``//static import generator factory methods``

Then you should be able to generate a simple value from the available generators:

``````def 'using static factory methods'() {
expect:
string.iterator().next() instanceof String
bytes.iterator().next() instanceof byte[]
getDouble().iterator().next() instanceof Double
integer.iterator().next() instanceof Integer
getLong().iterator().next() instanceof Long
character.iterator().next() instanceof Character
date.iterator().next() instanceof Date
}``````

These examples are creating only the next available generated value from the corresponding generator. This way of using simple types generators doesn’t put any constraint to the generated value apart from generate a specific type of value. We’ll see later on how to add some boundaries to some of the value generators. For the time being, if for example, we would like a string we won’t care about the length of the string.

2.1. Strings

Like we saw in the previous section if we don’t care about the length or the content and we just wanted to generate a string then it’s enough to call `Gen.getString()` or in a more `groovier` way `Gen.string`

By length

In case we wanted to restrict the generated word length, we could use one of the following methods:

Restricting string length
``````def 'create a string by length'() {
when: 'establishing max string length'
def shortWord = string(5).iterator().next() (1)

then: 'word size should be less equal than max'
shortWord.size() <= 5

when: 'establishing min and max word size'
def largerWord = string(5,10).iterator().next() (2)

then: 'word should be larger equal min'
largerWord.size() >= 5

and: 'word should be less equal max'
largerWord.size() <= 10
}``````
 1 Establishing the maximum string length 2 Establishing both minimum and maximum length

By pattern

From a string pattern
``````def 'generate a string using a regular expression'() {
expect:
generatedString ==~ '(https?|ftp|file)://[-A-Z0-9+&@#/%?=~_|!:,.;]*[-A-Z0-9+&@#/%=~_|]\\d'
where:
generatedString << string(~'(https?|ftp|file)://[-A-Z0-9+&@#/%?=~_|!:,.;]*[-A-Z0-9+&@#/%=~_|]\\d').take(10)
}``````

2.2. Numbers

`spock.genesis.Gen` has access to several number generators. All basic number types have a direct method to generate a random value without establishing any restriction.

``````def 'generate numbers'() {
expect:
getDouble().iterator().next() instanceof Double
integer.iterator().next() instanceof Integer
getLong().iterator().next() instanceof Long
bytes.iterator().next() instanceof byte[]
}``````

For integers there are a couple more methods to set the boundaries of the generated values directly.

``````def 'create an integer with min and max'() {
when: 'establishing max possible number'
def firstNumber = integer(5..10).iterator().next() (1)

then: 'generated number will be less equals than max'
firstNumber >= 5
firstNumber <= 10

when: 'establishing min and max valid numbers'
def secondNumber = integer(5,10).iterator().next() (2)

then: 'generated number must be between both numbers'
secondNumber >= 5
secondNumber <= 10
}``````
 1 Using a Groovy range to establish min and max boundaries 2 Establishing min and max using two parameters

2.3. Date

In many applications could be handy to generate dates to validate some principles. For instance when booking a room to make sure the system doesn’t accept any check-out done the same date or before as the check-in right ?

``````def 'create a new date value range'() {
given: "yesterday's reference and tomorrow's"
def yesterday = new Date() - 1
def tomorrow  = new Date() + 1

when: 'getting a new date'
def newDate = date(yesterday, tomorrow).iterator().next()

then: 'new date should be between boundaries'
tomorrow.after(newDate)
newDate.after(yesterday)
}``````

2.4. From value

You’ve seen several ways of creating values from simple data types. But if you still wanted to create an infinite lazy generator for a given value you can use `Gen.value`

``````def 'create a value using the value() method'() {
expect: 'to get several copies of a value'
value(0).take(2).collect() == [0,0]

and: 'to get just one'
value(0).iterator().next() == 0
}``````

2.5. From enum

If you already have an enum type and you would like to generate random values from it, then you could use `these`:

``````enum Days {
SUNDAY, MONDAY, TUESDAY, WEDNESDAY, THURSDAY, FRIDAY, SATURDAY
}

def 'generate from an enum'() {
setup:
def gen = these Days
expect:
gen.collect() == Days.collect()
}``````
 `these` in general can generate values taken from a given source, in this case the source is an enum. To know more about `these` check the section `combine`.

3. Composites

Generate values of basic types is great but sometimes we may want to create instances of more complex types such as maps, lists, or pojos.

3.1. Tuple

A tuple is a finite ordered list of elements. As we’ll see afterwards you can create random sized lists with the `list` generator, but if you only wanted to create a fixed sized lists with fixed types, `tuple` could be your best option.

Tuple
``````def 'generate a tuple'() {
when: 'generating a tuple of numbers'
def tuple = tuple(integer, integer, string).iterator().next()

then: 'make sure we get a list of the expected size'
tuple.size() == 3

and: 'the type of the members are the expected'
tuple.first() instanceof Integer
tuple.get(1) instanceof Integer
tuple.last() instanceof String
}``````

In this example we’re creating a tuple (a fixed list) of three elments of types: Integer, Integer and String.

3.2. List

As we’ve just seen you can create fixed lists with `tuple` but if you want to vary the size of the list, or use random element types then you should be using `list`. A list needs a given value generator to take its elements from and could have some size constraints like the minimum or/and maximum number of elements. A basic list generator without any size boundaries:

Simple list
``````def 'generate a simple list'() {
when: 'generating a simple list'
def list = list(integer).iterator().next() (1)

then: 'we only can be sure about the type of the list'
list instanceof List

and: 'the type of elements due to the value generator used'
list.every { it instanceof Integer }
}``````
 1 `list(valueGenerator)` in this case we are using the `integer` generator to create values of type `Integer`

This example generates random sized lists with values taken from the value generator passed as parameter.

List length

On the other hand if you want to establish some size boundaries you could use `list(valueGenerator,min,max)` or `list(valueGenerator,max)`.

List with size boundaries
``````def 'generate a list with size boundaries'() {
when: 'establishing the list definition'
def list = list(integer, 1, 5).iterator().next() (1)

then: 'it should obey the following assertions'
list instanceof List                  (2)
list.size() >= 1                      (3)
list.size() <= 5                      (4)
list.every { it instanceof Integer }  (5)

}``````
 1 Creating a list with a minimum size of 1 and a maximum of 5 2 It should be an instance of list 3 It should have a minimum size of 1 4 It should have a maximum size of 5 5 All elements should be of type integer

3.3. Map

You can create instances of `java.util.Map` and also specify which type of values should be used per `key-value` entry.

Map
``````def 'generate a map'() {
when: 'defining a map with different fields'
def myMap = map(                            (1)
id: getLong(),                          (2)
name: string,                           (3)
age: integer(0, 120)).iterator().next() (4)

then: 'we should get instances of map'
myMap instanceof Map

and: 'the fields should follow the generators rules'
myMap.id instanceof Long
myMap.name instanceof String
myMap.age instanceof Integer
}``````
 1 Declaring a `map` generator 2 Declaring `id` will be a long 3 Declaring `name` will be a string 4 Declaring `age` will be an integer between 0 and 120. Then we get `next()` map generated value

3.4. Type

Given a class we may want to create a generator that can supply instances of that type. Here is an example of a given class:

Data
``````static class Data {
String s
Integer i
Date d
}``````

The following generator creates instances of the previous type and it has defined a different generator for each field:

Generator
``````def 'generate type with map'() {
setup:
def gen = type(Data, s: string, i: integer, d: date) (1)
when:
Data result = gen.iterator().next() (2)
then:
result.d
result.i
result.s
}``````
 1 Create generator 2 Take next instance

In the following example the type we would like to get instances from has a non default constructor:

TupleData
``````static class TupleData {
String s
Integer i
Date d

TupleData(String s, Integer i, Date d) {
this.s = s
this.i = i
this.d = d
}
}``````

But that is not an issue, as long as we respect the number of arguments after declaring the class.

Generator
``````def 'generate type with tuple'() {
expect:
result instanceof TupleData
result.d
result.i == 42
result.s

where:
result << type(TupleData, string, value(42), date).take(5)
}``````
 Notice here we are generating the same name for the `i` field over and over again (42)

3.5. Combinations with `permute`

Under some conditions you may want to test all combinations of inputs. Tuple, map with fixed keys, and type generators all implement the Permutable interface to accomplish that in a lazy fashion.

 Iterating through all combinations expands the number of iterations exponentially.

Outputs are produced using a depth first algorithm. Setting the max depth limits the number of values that are produced for each input.

using a map generated POJO
``````def 'permute is possible with map generator'() {
setup:
def generator = Gen.type(MapConstructorObj, string: ['a', 'b'], integer: [1,2,3]) (1)
when:
List<MapConstructorObj> results = generator.permute().collect() (2)
then: (3)
results == [
new MapConstructorObj(string: 'a', integer: 1),
new MapConstructorObj(string: 'b', integer: 1),
new MapConstructorObj(string: 'a', integer: 2),
new MapConstructorObj(string: 'b', integer: 2),
new MapConstructorObj(string: 'a', integer: 3),
new MapConstructorObj(string: 'b', integer: 3),
]
when: 'only to depth of 2'
List<MapConstructorObj> results2 = generator.permute(2).collect() (4)
then: (5)
results2 == [
new MapConstructorObj(string: 'a', integer: 1),
new MapConstructorObj(string: 'b', integer: 1),
new MapConstructorObj(string: 'a', integer: 2),
new MapConstructorObj(string: 'b', integer: 2),
]
}``````
 1 Declaring a generator 2 Call `permute` and get all of the values 3 The result contains all of the combinations 4 Call `permute` setting a max depth of 2 and get all the values 5 The result only contains the combinations of the first 2 values from each field

If no max depth is specified then the formula  |__root(n)(10000)__|  is used to determine the max depth

 input count depth iterations 1 10000 10000 2 100 10000 3 21 9261 4 10 10000 5 6 7776 6 4 4096 7 3 2187 8 3 6561

Groovy provides a combinations method that provides a similar capability but it is not lazy and does not control for the exponential issue.

4. Combine

When generating values, sometimes we may want to mix different types of generators, or introduce certain values in the generation, maybe special cases. In order to do that we need to combine generators, or values.

4.1. these

The `Gen.these` method creates a generator from a set of values. This values could be any of:

• java.util.Iterable

• java.util.Collection

• java.lang.Class

• Or a variable arguments parameter

When the generator produces new values it will be taking every element from the declared source in order until the source is exhausted

``````def 'generate from a specific set of values'() {
expect: 'to get numbers from a varargs'
these(1,2,3).take(3).collect() == [1,2,3]

and: 'to get values from an iterable object such as a list'
these([1,2,3]).take(2).collect() == [1,2]

and: 'to get values from a given class'
these(String).iterator().next() == String

and: 'to stop producing numbers if the source is exhausted'
these(1..3).take(10).collect() == [1,2,3]
}``````

4.2. then and `&`

Lets say you are confortable with the values produced by a given generator but once the generator is exhausted it would be nice to continue producing values from another generator, that’s exactly what the `then` method does.

``````def 'generate from multiple iterators in sequence'() {
setup:
def gen = these(1, 2, 3).then([4, 5])
expect:
gen.collect() == [1, 2, 3, 4, 5]
}``````

The `then` method is available in any generator and chains one generator with the next one.

Also you can use the `&` operator to combine to generators:

``````def 'create multi source generator with & operator'() {
setup:
def gen = string(100) & integer & date
expect:
gen instanceof MultiSourceGenerator
gen.any { it instanceof Integer }
gen.any { it instanceof String }
gen.any { it instanceof Date }
}``````

4.3. any

If `these` was producing elements from a source in order, `Gen.any` produces values from a given source but in random order.

``````def 'generate any value from a given source'() {
given: 'a source'
def source = [1,2,null,3]

expect: 'only that the generated value is any of the elements'
Gen.any(source).take(2).every { n -> n in source }
}``````

5. Cardinality

Once you chose a generator, you can tell the generator to produce a given number of values, lets see how.

5.1. @Iterations

If you want to limit the number of iterations that will be run you can use the `@Iterations` annotation. It is particularly useful when using infinite generators. It can be applied to the Specification or to individual features. If no value is provided it will default to 100.

Class
``````@Iterations(5)
class IterationsSpec extends Specification {

static List NUMBERS = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]

static List<Integer> VALUES = []

def 'method iterations is limited by class annotation'() {
expect:
value < 6
where:
value << NUMBERS
}``````
Feature
``````@Iterations(2)
def 'limiting iterations to 2 makes it so the first 2 iterations are all that run'() {
expect:
s instanceof String
i instanceof Integer
i < 3
where:
s << string(~/[A-Z][a-z]+( [A-Z][a-z]+)?/)
i << these(1,2,3,4,5,6)
}``````

5.2. Once

If you only want to produce a given value only once, then use `once`.

``````def 'generate a value once'() {
setup:
def gen = once value
expect:
gen.collect() == [value]
where:
value << [null, 1, 'b', [1,2]]
}``````

5.3. Using `multiply by`

You can tell how many items to generate from a given generator by using the `*` operator:

``````def 'multiply by int limits the quantity generated'() {
setup:
def gen = string * 3
when:
def results = gen.collect()
then:
results.size() == 3
}``````

5.4. Take

If you know in advanced how many items you will need then use `take`.

``````def 'generate a value repeatedly'() {
setup:
def gen = value(null).take(100)
when:
def result = gen.collect()
then:
result.size() == 100
result.every { it == null }
}``````

6. Output

Some times you need to control the output of a generator.

6.1. Seeding random generation

For random generators it can be useful to control the seed for random generation. This will cause a consistent sequence of values to be generated by an equivalent generator.

``````def 'setting seed returns the same values with 2 generators configured the same'() {
given:
def generatedA = string(10).seed(879).take(10).realized
def generatedB = string(10).seed(879).take(10).realized
expect:
generatedA == generatedB
}``````

Setting the seed to different values will vary the output but will always produce the same sequence.

``````def 'setting seed to different values produces different sequences'() {
given:
def generatedA = integer.seed(879).take(4).realized
def generatedB = integer.seed(3).take(4).realized
expect:
generatedA == [-1295148427, 2105117961, -922763979, 1733784787]
generatedB == [-1155099828, -1879439976, 304908421, -836442134]
}``````

6.2. Using `with`

You can use `with` if you would like to set some property of the generated value.

 `with` is different from the default groovy implementation! It always returns a new generator.
``````def 'call methods on generated value using with'() {
setup:
def gen = date.with { setTime(1400) }

expect:
gen.iterator().next().getTime() == 1400
}``````

6.3. Transforming with `map`

Sometimes output needs to be converted to another type. the `map` method works like groovy’s `collect` but will return a new generator that lazily performs the transformation. An example would be calling the `toString()` method.

``````@Iterations(10)
def 'transform the output of a generator'() {
expect:
result instanceof String
result.isInteger()
where:
result << integer.map { val -> val.toString() }
}``````

7. Development

7.1. Building `spock-genesis`

The only prerequisite is that you have JDK 7 or higher installed.

After cloning the project, type `./gradlew clean build` (Windows: `gradlew clean build`). All build dependencies, including Gradle itself, will be downloaded automatically (unless already present).

7.2. Groovydoc

`Groovydoc` can be found here.