Template::Jinja2

Complete Jinja2 template engine for Raku, targeting compatibility with the Python reference implementation (Jinja2 3.1). Designed for HuggingFace chat templates and general-purpose templating.

Synopsis

use Template::Jinja2;

my $env = Template::Jinja2.new;
my $tmpl = $env.from-string('Hello {{ name }}!');
say $tmpl.render(name => 'World');  # Hello World!

HuggingFace Chat Templates

use Template::Jinja2;
use JSON::Fast;

my $config = from-json('tokenizer_config.json'.IO.slurp);
my $chat-template = $config<chat_template>;

my $env = Template::Jinja2.new;
my $result = $env.from-string($chat-template).render(
    messages => [
        { role => 'system', content => 'You are helpful.' },
        { role => 'user', content => 'Hello!' },
    ],
    bos_token => '<s>',
    add_generation_prompt => True,
);

Produces byte-identical output to Python Jinja2 for ChatML, Llama 3, Mistral, Gemma 2, Zephyr, and Cohere Command A templates.

Features

Expressions

Arithmetic: +, -, *, /, //, %, **
Comparison: ==, !=, <, >, <=, >=>, chained
Logical: and, or, not
Membership: in, not in
Identity: is, is not
Ternary: expr if test else expr
String concat: ~
Filters: value | filter(args)
Subscript: items[0], items[-1], items[1:3], items[::-1]
Dot access: obj.attr, list.0
Literals: strings, integers, floats, hex/octal/binary, scientific notation, lists, dicts, tuples

Filters (52)

abs, attr, batch, capitalize, center, count, d/default, dictsort, e/escape, filesizeformat, first, float, forceescape, format, groupby, indent, int, items, join, last, length, list, lower, lstrip, map, max, min, pprint, random, reject, rejectattr, replace, reverse, round, rstrip, safe, select, selectattr, slice, sort, string, striptags, sum, title, tojson, trim, truncate, unique, upper, urlencode, urlize, wordcount, wordwrap, xmlattr

Tests (20+)

defined, undefined, none, boolean, integer, float, number, string, sequence, mapping, iterable, callable, sameas, eq/equalto, ne, lt/lessthan, le, gt/greaterthan, ge, even, odd, divisibleby, in, true, false, lower, upper

Template Inheritance

{# base.html #}
<html>{% block content %}default{% endblock %}</html>

{# child.html #}
{% extends "base.html" %}
{% block content %}{{ super() }} + override{% endblock %}

Supports multi-level inheritance with super() chaining.

Whitespace Control

{%- trim left -%}    {# trim both sides #}
{%+ no_lstrip +%}    {# disable lstrip/trim_blocks #}

my $env = Template::Jinja2.new(:trim-blocks, :lstrip-blocks);

Custom Delimiters

# ERB-style
my $env = Template::Jinja2.new(
    :block-start('<%'), :block-end('%>'),
    :variable-start('<%='), :variable-end('%>'),
    :comment-start('<%#'), :comment-end('%>'));

# PHP-style
my $env = Template::Jinja2.new(
    :block-start('<?'), :block-end('?>'),
    :variable-start('<?='), :variable-end('?>'),
    :comment-start('<!--'), :comment-end('-->'));

Loaders

use Template::Jinja2::Loader;

# File system
my $env = Template::Jinja2.new(
    loader => FileSystemLoader.new(searchpath => 'templates/'));

# In-memory
my $env = Template::Jinja2.new(
    loader => DictLoader.new(templates => {
        'base.html' => '...',
        'child.html' => '...',
    }));

Configuration

my $env = Template::Jinja2.new(
    :autoescape,                  # Auto-escape {{ }} output
    :trim-blocks,                 # Strip newline after block tags
    :lstrip-blocks,               # Strip leading whitespace before block tags
    :!keep-trailing-newline,      # Strip trailing newline from output
    :line-statement-prefix('#'),  # Line-based block syntax
    :line-comment-prefix('##'),   # Line-based comments
    globals => { site => 'My Site' },
    filters => { custom => sub ($v) { ... } },
    tests => { custom => sub ($v) { ... } },
);

Architecture

Template String -> [Lexer] -> Tokens -> [Parser] -> AST -> [Renderer] + Context -> Output

Lexer — Raku grammar tokenizer (default delimiters) or regex chunker (custom delimiters)
Parser — Pratt-style expression parser with block tag dispatch
AST — 30+ node types covering all Jinja2 constructs
Renderer — AST walker with expression evaluation, scope chain, filter/test dispatch
Context — Scope stack with Undefined sentinel and Namespace support

Testing

612 tests across 22 test files, including:

Ported tests from the Python Jinja2 reference test suite
Real-world HuggingFace chat template validation
Byte-identical output verification against Python Jinja2 3.1.6

# Run all tests
prove6 -I lib t/

# Run a single test
raku -I lib t/09-huggingface.rakutest

Author

Matt Doughty

License

Artistic-2.0