NAME

SION - a serialization format a little more expressive than JSON

SYNOPSIS

use SION;    # exports to-sion() and from-sion()

my $data = from-sion($sion-text);    # Str or utf-8 Blob
my $sion = to-sion($data);

# options, a la JSON::Fast
my $sion = to-sion($data, :pretty, :sorted-keys);

DESCRIPTION

SION is a data serialization format whose origin is Swift literals, just like JSON's origin is JavaScript literals. It is a little more expressive than JSON:

[
    "nil":      nil,                // nil is allowed
    "bool":     true,
    "int":      -42,                // Int is distinguished from Double
    "double":   0x1.518f5c28f5c29p+5,   // C99 hexadecimal notation
    "string":   "漢字、カタカナ、ひらがなの入ったstring😇",
    "array":    [nil, true, 1, 1.0, "one", [1], ["one":1.0]],
    "dictionary": ["nil":nil, "array":[], "object":[:]],
    "data":     .Data("R0lGODlh"),  // binary data in Base64
    "date":     .Date(0x0p+0),      // date in seconds since epoch
    "ext":      .Ext("1NTU"),       // msgpack-style extension
    1:          "non-string keys are also allowed", // comments, too!
]

This module implements a SION encoder/decoder for Raku. It is standalone — no dependency on other modules.

MAPPING

Raku is expressive enough that almost every SION type maps to a native Raku type:

SION	Raku
`nil`	`Any` (undefined)
`true/false`	`Bool`
`Int`	`Int`
`Double`	`Num`
`String`	`Str`
`Data`	`Blob`
`Date`	`DateTime` (fractional epoch preserved)
`Ext`	`SION::Ext`
`Array`	`Array`
`Dictionary`	`Hash` (object hash for non-`Str` keys)

Int vs Double

Int and Num are distinct types in Raku, so the distinction round-trips with no scalar-flag tricks: 1 encodes to 1 while 1e0 encodes to 0x1p+0. Nums are always encoded in C99 hexadecimal floating point notation, which is lossless; nan, inf and -inf are also supported. Rat and other Reals encode as Double.

String vs Data

A Str is text and a Blob is binary, so no heuristics are needed: Str encodes as a SION String while any Blob (including buf8) encodes as .Data("…") in Base64, and vice versa on decoding.

dates

.Date(n) — n being seconds since epoch, possibly fractional — decodes to a DateTime, and DateTime, Instant and Date all encode as .Date(…). The fractional part survives the round trip: .Date(0x1p-1) is DateTime.new(Instant.from-posix(0.5)).

dictionaries

SION allows any value as a Dictionary key. When every key is a Str, from-sion returns an ordinary Hash; otherwise it returns an object hash, so from-sion('[1:"one"]'){1} works and the Int-ness of the key is preserved — no stringification.

FUNCTIONS

from-sion

my $data = from-sion($sion);                 # Str
my $data = from-sion($octets);               # utf-8 encoded Blob
my $data = from-sion($sion, :max-depth(64));

Decodes SION text to a Raku data structure. Dies on malformed input with the character offset of the error. :max-depth (default 512) bounds the nesting level.

to-sion

my $sion = to-sion($data);
my $sion = to-sion($data, :pretty);       # multi-line, indented by 4
my $sion = to-sion($data, :sorted-keys);  # deterministic dictionaries
my $sion = to-sion($data, :ascii);        # non-ASCII as \uXXXX escapes
my $sion = to-sion($data, :max-depth(64));

Encodes a Raku data structure to SION text. Compact by default; :sorted-keys sorts dictionary entries by their encoded keys so the output is deterministic (and a fixed point of to-sion(from-sion($_), :sorted-keys)).

HELPER CLASSES

SION::Ext

my $ext = SION::Ext.new($blob);   # encodes to .Ext("base64")
$ext.data;                        # the Blob payload

msgpack-style extension. It is a value type: two SION::Ext objects with the same payload are eqv.

HEXADECIMAL FLOATING POINT

SION serializes Double in C99 hexfloat notation (0x1.518f5c28f5c29p+5) because it is exact. Rakudo is gaining native support for it — sprintf '%a' landed for v6.e in rakudo/rakudo#6525 — but this module does not require it: it probes at load time and falls back to a pure-Raku encoder/decoder (exact Int/FatRat arithmetic, so the result is correctly rounded). Subnormals are emitted normalized (0x1p-1074), byte-compatible with js-sion and BSD printf %a.

AUTHOR

Dan Kogai

LICENSE AND COPYRIGHT

This is free software, licensed under the Artistic License 2.0.