Tomty

Tomty - Raku Test Framework.

Install

zef install Tomty

Build Status

Quick start

tomty --edit test-01

bash "echo Hello World"

tomty --edit test-02

bash "echo Upps && exit 1";

tomty --edit test-03

bash "echo Hello Again!";

tomty --all # Run all tests and make reports

[1/3] / [test-01] .......  2 sec. OK
[2/3] / [test-02] .......  3 sec. FAIL
[3/3] / [test-03] .......  3 sec. OK
=========================================
)=: (2) tests passed / (1) failed

# save tests to Git

echo ".tomty/.cache" >> .gitignore

git add .tomty

Guide

Writing tests

Tomty test is just a Raku scenario:

tomty --edit test-meta6-file-exist

#!raku

bash "test -f META6.json"

You can write more advanced tests, for example:

# Check if raku.org is accessible

tomty --edit test-raku-org-alive

#!raku

http-ok("https://raku.org");

# Check if META6.json file is a valid json

tomty --edit test-meta6-is-valid-json

#!raku

task-run "meta6 is a valid json", "json-lint";

Check out Sparrow6 DSL on what you can use writing your tests.

Running tests

To run all test just say tomty --all

It will find all the tests and run them in sequence.

To run single test just say tomty $test

For example:

tomty test-meta6-is-valid-json

Examining tests

To list all the tests just say tomty --list

This command will list all tests.

Managing tests

Removing test

To remove test use --remove option:

tomty --remove $test

Edit test source code

Use --edit to create test from the scratch or to edit existed test source code:

tomty --edit $test

Getting test source code

Use --cat command to print out test source code:

tomtit --cat $test

Use --lines flag to print out test source code with line numbers.

Environments

Tomty environments are configuration files, written on Raku and technically speaking are plain Raku Hashes
Environment configuration files should be placed at .tomty/conf directory:

.tomty/env/config.raku:

{
    dbname => "products",
    dbhost => "localhost"
}

When tomty runs it picks the .tomty/env/config.raku and read configuration from it variables will be accessible as config Hash, inside Tomty scenarios:

my $dbname = config<dbname>;
my $dbhost = config<dbhost>;

To define named configuration ( environment ), simply create .tomty/env/config{$env}.raku file and refer to it through --env=$env parameter:

nano .tomty/env/config.prod.raku

tomty --env=prod ... other parameters here # will run with production configuration

You can run editor for environment configuration by using --edit option:

tomty --env-edit test    # edit test environment configuration

tomty --env-edit default # edit default configuration

You can activate environment by using --env-set parameter:

tomty --env-set prod    # set prod environment as default
tomty --env-set         # to list active (current) environment
tomty --env-set default # to set current environment to default

To view environment configuration use --env-cat command:

tomty --env-cat $env

Use --lines flag to print out environment source code with line numbers.

You print out the list of all environments by using --env-list parameters:

tomty --env-list

Macros

Tomty macros allow to pre-process test scenarios. To embed macros use =begin tomty .. =end tomty syntax:

=begin tomty
%(
    tag => "slow"
)
=end tomty

Macros could be any Raku code, returning Hash. The example above set tag=slow for slow running tests, you can skip test execution by using --skip option:

tomty --skip=slow

Tags filtering

Tags filtering allows to run subsets of scenarios using tags as criteria.

Logical OR

By default logical OR is implied when using comma:

Examples:

tomty --skip=slow,windows # skip slow OR windows tests

tomty --only=frontend,backend # only frontend OR backend test

Logical AND

Use + to mimic logical AND:

tomty --only=database+mysql # execute only mysql database tests

--skip and --only could be combined to get more sophisticated scenarios:

tomty --only=database+mysql,skip=window # execute only mysql database tests BUT not for windows OS system

One can list available tags by:

tomty --list --tags

You can combine --tags with --only or --skip options to list tagged tests.

Examples:

tomty --tags --only=foo  # list tests tagged by `foo`

tomty --tags --only=foo+bar  # list tests tagged by `foo` AND `bar`

tomty --tags --only=foo,bar  # list tests tagged by `foo` OR `bar`

Filter untagged scenarios

Use special untagged keyword to filter untagged scenarios:

tomty --tags --only=untagged  # list all untagged tests
tomty --only=untagged         # run all untagged tests
tomty --skip=untagged         # run all but untagged tests

Profiles

Tomty profile sets command line arguments for a named profile:

cat .tomty/profile

%(
    default => %(
        skip => "broken"
    )
)

One can override following command line arguments through a profile:

skip
only
env
no-index-update

In the future more arguments will be supported.

A default profile sets default command line arguments when tomty cli run.

To add more profiles just add more Hash keys and define proper settings:

%(
    default => %(
        skip => "broken"
    ),
    development => %(
        only => "new-bugs"
    )
)

To chose profile use --profile option:

tomty --profile development

Bash completion

Tomty comes with nice Bash completion to easy cli usage, use --completion option to install completion:

tomty --completion

And then source ~/.tomty_completion.sh to activate one.

Tomty cli

Options

--all|-a

Run all tests

--show-failed

Show failed tests

--verbose

Runs tests in verbose mode, print more details about every test

--no-index-update

Don't update Sparrow repository index

--dump-task

Dump task code before execution, see SP6_DUMP_TASK_CODE Sparrow documentation

--color

Run in color mode

--list

List tests/tags

To list test only:

tomty --list

To lists tags only:

tomty --list --tags

--noheader

Omit header when list tests. Allow to edit tests one by one:

for i in $(tomty --noheader); do tomty --edit $i; done

--edit|--cat|--remove

Edit, dump, remove test

--env-edit|--env-list|--env-set

Edit, list, set environment

--completion

Install Tomty Bash completion

--log

Get log for given test run, useful when running in all tests mode:

tomty -all

tomty --log test-01

--skip | --only

Set tags filters.

Skip tests tagged as slow

tomty --skip=slow

Only run tests tagged as linux

tomty --only=linux

See also tags filtering for more details on tag filtering.

--tags

Show available tags:

tomty --list --tags # list all tags

You can combine --tags with --only or --skip options:

tomty --tags --only=foo+bar  # list tests tagged by `foo` AND `bar`

Environment variables

TOMTY_DEBUG

Use it when debugging Tomty itself:

TOMTY_DEBUG=1 tomty --all

Author

Alexey Melezhik

Thanks to

God, as my inspiration