Rand Stats

Template::Jinja2

zef:apogee

Actions Status

Template::Jinja2

Complete Jinja2 template engine for Raku, targeting compatibility with the Python reference implementation (Jinja2 3.1). Designed for HuggingFace chat templates and general-purpose templating.

Synopsis

use Template::Jinja2;

my $env = Template::Jinja2.new;
my $tmpl = $env.from-string('Hello {{ name }}!');
say $tmpl.render(name => 'World');  # Hello World!

HuggingFace Chat Templates

use Template::Jinja2;
use JSON::Fast;

my $config = from-json('tokenizer_config.json'.IO.slurp);
my $chat-template = $config<chat_template>;

my $env = Template::Jinja2.new;
my $result = $env.from-string($chat-template).render(
    messages => [
        { role => 'system', content => 'You are helpful.' },
        { role => 'user', content => 'Hello!' },
    ],
    bos_token => '<s>',
    add_generation_prompt => True,
);

Produces byte-identical output to Python Jinja2 for ChatML, Llama 3, Mistral, Gemma 2, Zephyr, and Cohere Command A templates.

Features

Tags

All 15 Jinja2 tag types plus loop controls:

Expressions

Python Methods

Jinja2 has no method syntax of its own: value.method() resolves the name on the underlying Python object and calls it. Templates in the wild — HuggingFace chat templates especially — rely on that constantly, so the Python str and dict methods are implemented with Python semantics, not those of Raku's similarly-named methods.

{%- for m in messages -%}
    {#- strip() as both a guard and as output -#}
    {%- if m.content.strip() -%}
        <|{{ m.role }}|>{{ m.content.strip() }}
    {%- endif -%}

    {#- reasoning split, straight out of the GLM chat template -#}
    {%- if '</think>' in m.content -%}
        {%- set reasoning = m.content.split('</think>')[0].split('<think>')[-1] -%}
    {%- endif -%}

    {#- get() with a default, straight out of the Kimi chat template -#}
    {%- set name = m.get('name') or m['role'] -%}
    {%- set rc = m.get('reasoning', m.get('reasoning_content', '')) -%}
{%- endfor -%}

{#- dict iteration for tool schemas -#}
{%- for k, v in tool.items() -%}"{{ k }}": {{ v | tojson }}{%- endfor -%}

String methods:

Dict methods:

Method results chain, feed filters and tests, and bind through {% set %}:

{{ msg.content.strip().split('\n')[0] }}
{{ msg.content.strip() | tojson }}
{% set parts = ref.split('|') %}{{ parts[0] }}/{{ parts[-1] }}

Two deliberate divergences from CPython, both consequences of a Raku Hash having no insertion order to preserve:

Anything else raises a TemplateRuntimeError naming the method and the receiver's type — 'str' object has no method 'casefold' — rather than reaching a same-named Raku method or leaking an internal error. Calling a method on an undefined value raises UndefinedError identifying the expression that produced it.

Filters (52)

abs, attr, batch, capitalize, center, count, d/default, dictsort, e/escape, filesizeformat, first, float, forceescape, format, groupby, indent, int, items, join, last, length, list, lower, lstrip, map, max, min, pprint, random, reject, rejectattr, replace, reverse, round, rstrip, safe, select, selectattr, slice, sort, string, striptags, sum, title, tojson, trim, truncate, unique, upper, urlencode, urlize, wordcount, wordwrap, xmlattr

Tests (20+)

defined, undefined, none, boolean, integer, float, number, string, sequence, mapping, iterable, callable, sameas, eq/equalto, ne, lt/lessthan, le, gt/greaterthan, ge, even, odd, divisibleby, in, true, false, lower, upper

Template Inheritance

{# base.html #}
<html>{% block content %}default{% endblock %}</html>

{# child.html #}
{% extends "base.html" %}
{% block content %}{{ super() }} + override{% endblock %}

Supports multi-level inheritance with super() chaining.

Whitespace Control

{%- trim left -%}    {# trim both sides #}
{%+ no_lstrip +%}    {# disable lstrip/trim_blocks #}

my $env = Template::Jinja2.new(:trim-blocks, :lstrip-blocks);

Custom Delimiters

# ERB-style
my $env = Template::Jinja2.new(
    :block-start('<%'), :block-end('%>'),
    :variable-start('<%='), :variable-end('%>'),
    :comment-start('<%#'), :comment-end('%>'));

# PHP-style
my $env = Template::Jinja2.new(
    :block-start('<?'), :block-end('?>'),
    :variable-start('<?='), :variable-end('?>'),
    :comment-start('<!--'), :comment-end('-->'));

Loaders

use Template::Jinja2::Loader;

# File system
my $env = Template::Jinja2.new(
    loader => FileSystemLoader.new(searchpath => 'templates/'));

# In-memory
my $env = Template::Jinja2.new(
    loader => DictLoader.new(templates => {
        'base.html' => '...',
        'child.html' => '...',
    }));

Configuration

my $env = Template::Jinja2.new(
    :autoescape,                  # Auto-escape {{ }} output
    :trim-blocks,                 # Strip newline after block tags
    :lstrip-blocks,               # Strip leading whitespace before block tags
    :!keep-trailing-newline,      # Strip trailing newline from output
    :line-statement-prefix('#'),  # Line-based block syntax
    :line-comment-prefix('##'),   # Line-based comments
    globals => { site => 'My Site' },
    filters => { custom => sub ($v) { ... } },
    tests => { custom => sub ($v) { ... } },
);

Architecture

Template String -> [Lexer] -> Tokens -> [Parser] -> AST -> [Renderer] + Context -> Output

Testing

644 tests across 24 test files, including:

# Run all tests
prove6 -I lib t/

# Run a single test
raku -I lib t/09-huggingface.rakutest

Author

Matt Doughty

License

Artistic-2.0