Project Blin – Toasting Reinvented
Blin is a quality assurance tool for
Rakudo releases. Blin is based on
Whateverable.
Blin was inspired by
Toaster. Here
are some advantages:
- Fetches archives from
whateverable
instead of spending time to build rakudo
- Installs modules in proper order to avoid testing the same module more than once
- Deflaps modules that fail intermittently
- Automatically bisects regressed modules
- Avoids segfaults by producing no useful output instead of using DBIish
Installation
Install required packages:
sudo apt install zstd lrzip graphviz
Install dependencies:
zef install --deps-only .
Many modules require native dependencies. See
this page
for the list of packages to install.
Running
Currently it is supposed to run only on 64-bit linux systems.
If you want to test one or more modules:
RAKULIB=lib bin/blin.p6 SomeModuleHere AnotherModuleHere
Here is a more practical example:
time RAKULIB=lib bin/blin.p6 --old=2018.06 --new=2018.09 Foo::Regressed Foo::Regressed::Very Foo::Dependencies::B-on-A
You can also test arbitrary scripts. The code can depend on modules,
in which case they have to be listed on the command line (e.g. for a
script depending on WWW you should list WWW module, dependencies of
WWW will be resolved automatically).
Using this ticket as an example: https://github.com/rakudo/rakudo/issues/2779
Create file foo.p6 with this content:
use WWW;
my @stations;
@stations = | jpost "https://www.perl6.org", :limit(42);
Then run Blin:
./bin/blin.p6 --old=2018.12 --new=HEAD --custom-script=foo.p6 WWW
Then check out the output folder to see the results. Essentially, it
is a local Bisectable.
If you want to test the whole ecosystem:
time RAKULIB=lib bin/blin.p6
Estimated time to test the whole ecosystem with 24 cores is ≈60 minutes.
⚠☠ SECURITY NOTE: issues mentioned in Toaster still
apply.
Do not run this for the whole ecosystem on non-throwaway
installs. ☠⚠
Viewing
See output/overview file for a basic overview of results. More
details for specific modules can be found in installed/
directory. Betters ways to view the data should come soon (hopefully).
Docker
For info about the Docker image, have a look at the
Readme file in the docker directory.