Rand Stats

Inline::J

zef:elcaro

NAME

Inline::J - Use the J programming language inside Raku.

SYNOPSIS

use Inline::J;

my \j = Inline::J.new(:load-profile);

say j.eval('i. 6');
# 0 1 2 3 4 5

say my $n = j.noun('>: 2 4 $ i. 8');
# 1 2 3 4
# 5 6 7 8

say j.eval("{(1,2) »×» 10} + $n");
# 11 12 13 14
# 25 26 27 28

say my $a = $n.getm;
# [[1 2 3 4]
#  [5 6 7 8]]

say $a.^name => $a.shape;
# Array[Int] => [2 4]

say j.gets(‘2 5 $ 'HelloWorld'’);
# [[H e l l o]
#  [W o r l d]]

WARNING

This module is under development. As such, the API is subject to change at any time, and there might be bugs. The use of this module in Production is not recommended.

If you insist on using this module on important code during version 0, you should pin the exact point release as I reserve the right to break/change the API at any time before version 1.

INSTALLATION

To install this module you need to have J already installed. See the J Software Wiki for more info.

During installation of this module, it will attempt to locate the bin path where J shared lib is located. It will search the most common install locations for POSIX or Windows depending on your system. Alternatively, you can export the bin location the environment variable JBINPATH before installation.

GETTING STARTED

The start using Inline::J, first create a new Inline::J object. To load the default J profile, pass a truthy load-profile named argument. You should load-profile if you wish to use names defined in the Standard Library.

my \j = Inline::J.new(:load-profile);
say j.eval(‘toupper 'hello, world!'’);
# HELLO, WORLD!

Alternatively you can use the load-profile method, which return self.

my \y = Inline::J.new.load-profile;

Note that each object is it's own separate interpreter.

j.eval('n =: 1');
y.eval('echo n');  # fail: value error

USAGE

Inline::J

do

The do method accepts a single J expression and passes it to the J interpreter. Any non-Str values will be coerced to a Str.

If there are any errors, it will return a Failure with the error message.

This method returns self, so that multiple expressions can be chained.

j.do('a =. i. 3').do('a =. 5 + a');  # ok
j.do('notaverb 1');                  # fail: value error

getr

The getr method returns the output of the last expression passed to do, with the trailing newline chompd.

j.do('a').getr;
# 5 6 7

eval

This is a convenience function that simply chains a do and getr call into one function

say j.eval('(+/ % #) 3 1 4 1 5 9');
# 3.83333

Since this wraps do, the eval method can also return a Failure.

free

This calls the JFree C function to free the J interpreters memory. Ideally, this should not need to be run, as Inline::J objects call this method on DESTROY.

END { j.free }

noun

This method creates an Inline::J::Noun object that references a noun in J. The gist of this object is the J representation of this noun.

If a noun already exists in J, you can create an Inline::J::Noun by providing it's name.

my $a = j.noun(name => 'a');
say $a;
# 5 6 7

NOTE: When an Inline::J::Noun object is DESTROY'd, the J noun will be erased and it's storage freed by calling (4!:55) <'name' in J.

Alternatively, you can declare a new variable in J by also providing an expression

my $b = j.noun('10 20 30', name => 'b');
say $b;
# 10 20 30

This is essentially the same as doing j.eval(b =. 10 20 30), only now you have a Raku object that references the noun.

If you do not provide a name, a random name will be generated for you.

my $v = j.noun('42');
say $v.name;
# ijn_d63a244e

Inline::J::Noun

An Inline::J::Noun (IJN) object references a noun in J. The gist of this object is the J representation of this noun.

EXPERIMENTAL: Currently, an IJN stringifies to it's name. This allows them to be interpolated into J expressions.

say j.eval("a + $b");
# 15 26 37

I'm not sure if this is a good idea yet, but it's fun. The alternative would be to explicitly interpolate it's name into expressions, eg.

say j.eval("a + {$b.name}");

Which is not as much fun.

Methods

The following examples will use the following value

my $m = j.noun('2 3 4 $ i. 24');

gist

Returns the J representation of the noun An objects .gist method is called automatically by say.

say $m;
#  0  1  2  3
#  4  5  6  7
#  8  9 10 11
#
# 12 13 14 15
# 16 17 18 19
# 20 21 22 23

name (Str)

Returns the name of the noun in J, which may be different from the variable name in Raku, and is randomly generated when not specified

This method is also called when coercing the object to a string (eg. $m.Str, "$m", ~$m, put $m, etc.)

say $m.name;
# ijn_f39324d1

datatype

Returns an Inline::J::Datatype enum.

The numeric value of the enum will match J's type identifier

say $m.datatype;
# integer
say +$m.datatype;
# 4

shape

Returns a Seq of Ints that count the length (number of items) in each axis.

say $m.shape;
# (2 3 4)

tally

Returns an Int of the count of items in the array.

As per the J documentation

> The items of an array are the cells whose rank is 1 lower than the rank of the array.
> Thus, the items of a list are atoms, and the items of a table are lists.

The tally can also be derived from the shape by taking the first element of the shape.

say $m.tally;
# 2

rank

Returns an Int of the count of axis in the array.

The rank can be derived from the shape by taking the count of element of the shape.

say $m.rank;
# 3

elems

Returns an Int of the total count of scalar items in the array

The elems can be derived from the shape by taking the product of the shape.

say $m.elems;
# 24

AT-POS

EXPERIMENTAL

I've added an .AT-POS method (which is called when you index into an object).

Currently this just returns a string from J.

say $m[1;0];
# 12 13 14 15

Inline::J::Verb

Similar to IJN's, an Inline::J::Verb (IJV) object references a verb defined in J. The object does Callable so that it acts like a Raku function. IJV callables accept 0, 1 or 2 arguments, and return an IJN.

When calling with 0 arguments, the J function is actually called with the empty string value.

The arguments can either be an existing IJN, or a Raku value that can be converted to a J value: Bool, Int, Num, Str, or a homogeneous array of one of those types). Raku types will be coerced to an Array and attempted to be converted to an IJN before being passed to J.

my &v = j.verb('>:');

my $y = j.noun('i. 5');
my $x = j.noun(2);

say v($y);
# 1 2 3 4 5

say v($y, $x);
# 0 0 1 1 1

say v($y, $x).getm;
# [False False True True True]

LIMITATION: The underlying method that converts Raku types to J only supports Arrays, so Scalars (J: nouns with no shape) are unu coerce to Array, are currently unsupported.

NOTE: Since version 0.4.0, the IJVs y argument is always the first argument. This means you will need to explicitly swap the arguments if you are creating infix operators in Raku

sub infix:<minus>($x, $y) { j.verb('-')($y, $x) }

Or use ~ (reflex, aka: commute) on the J verb

my &infix:<minus> := j.verb('-~');

Example of passing a Raku Array to a J verb

# Raku shaped Array
say my Int @a[4;4] = ([1..4] xx 4);
# [[1 2 3 4]
#  [1 2 3 4]
#  [1 2 3 4]
#  [1 2 3 4]]

# Passing it to a IJV, returns an IJN
# Convert back to Raku by calling .getm
say j.verb('|."_1~ i.@#')(@a).getm;
# [[1 2 3 4]
#  [2 3 4 1]
#  [3 4 1 2]
#  [4 1 2 3]]

Like IJNs, IJV's also get a random name (if none is provided)

my &t = j.verb('|:');
say &t.name;
# ijv_c79ce86a

They can tell you their rank, and have a hard-coded arity/count of 1/2.

say &t.rank;
# _ 1 _

say &t.arity;
# 1

say &t.count;
# 2

LIMITATION: Currently only monadic and dyadic verbs are supported, (4!:0) will be checked to ensure a value of 3 (verb) type, otherwise an Exception is thrown

my &c =. j.verb(';.');  # dies: Not a verb

EXPERIMENTAL: Since IJV's are Raku objects, Raku operators could provide multi's specific for them. Once experiment I've added is a multi for &infix:<∘>, which will compose 2 IJVs in J using @: (At)

my $x = j.noun(<1 2 3>);
my $y = j.noun(<2 2 2>);

my &f = j.verb('+/');
my &g = j.verb('*:');
my &h = j.verb('-');

#| sum of squared differences
my &sum-sq-diff = &f ∘ &g ∘ &h;
say sum-sq-diff($x, $y);
# 2

NOTE: Doing the compose (∘) in J avoids unnecessary IJN creation, which happens when IJV's return data to Raku.

DATA CONVERSION

WARNING: The code for handling data conversion is in a messy state and largely untested.

J to Raku

J scalars, lists, and matrices can be converted to Raku scalars and arrays. Currently this module strongly prefers creating shaped & typed Raku arrays.

Currently the only J datatypes supported are:

unicode values are encoded as UTF-16 in J, but normalised to UTF-8 when converted to Raku. unicode4 values are encoded as UTF-32 in J. Raku (specifically, it's VM) does not have a UTF-32 decoder, so I have written a naïve utf32-to-utf8 function in pure Raku, which may be buggy, and is likely to be slow on large inputs.

Current notable omissions are 'extended', 'rational', and 'complex' numbers, as well as 'boxed' arrays.

getm

The Inline::J object provides the following method to get a J noun and convert it to a Raku

my $arr = j.getm($m.name);
say $arr;
# [[[0 1 2 3]
#   [4 5 6 7]
#   [8 9 10 11]]
#  [[12 13 14 15]
#   [16 17 18 19]
#   [20 21 22 23]]]

Note however, that Inline::J::Noun objects also have a getm method available, so the above could also be expressed as:

my $arr = $m.getm;

As stated, the Raku array returned is shaped and typed.

say  $arr.^name => $arr.shape;
# Array[Int] => [2 3 4]

The only minor issue with getm is that it relies on the noun existing in J. This is probably fine on a small scale, but on a larger scale could cause an explosion of declared nouns in J if not handled properly.

gets

Sometimes, you might want to create a Raku data structure from a J expression without declaring a noun. J provides a foreign function (3!:3) which displays an ASCII (hex) representation of the data, which Inline::J can parse.

say j.gets('2 5 $ i. 10');
# [[0 1 2 3 4]
#  [5 6 7 8 9]]

For (currently) no obvious reason, Inline::J::Noun objects also have a gets method, which bypasses the underlying JGetM C function, and parses the ASCII representation instead

say $a.gets;
# [5 6 7]

WARNING: This relies on string output from J. By default, J truncates to 256 columns and 222 rows. I have increased this not truncate, but have still run into issues converting arrays of several thousand values.

Raku to J

WARNING: Barely developed.

Currently, only Raku shaped and typed Arrays can be converted to J arrays and matrices.

The only Raku types currently supported are:

* When converting an Array of Str to J, Inline::J will check if any codepoints are > 255, and if so, the Str's are encoded as UTF-16 (J's unicode) before passing to J.

setm

The Inline::J object provides the following method to set a J noun from a Raku data structure

my Bool @b[2;2] = [(True, False), (False, True)];
j.setm('b', @b);
say j.eval('b');
# 1 0
# 0 1

In addition to creating a noun in the J instance, the setm method also returns an IJN.

my $b = j.setm('b', @b);  # Eqv to `my $b = j.noun('b');`
say $b.^name;
# Inline::J::Noun

Existing IJN objects can also have their value over-written with the setm method

$m.setm(Array[Str].new(:shape(2;3), [<A B Ć>, <D E F>]));

say $m;
# ABĆ
# DEF

say $m.^name => $m.datatype;
# Inline::J::Noun => unicode

CAVEATS AND LIMITATION

This module was developed on a x86-64 (little endian) system, and byte-data returned from J is assumed to be 64-bit little endian. Expect things to fail when running on 32-bit builds/systems, or systems that use big endian.

It should be possible to support 32-bit (and/or big endian) systems with a bit of effort, although I do not have a need for it, and will not be developing this support. Pull requests welcome.

TODO

The following items are pending further development

Pull request welcome.

FINAL THOUGHTS

I'm not a C programmer, and I've barely used NativeCall in the past, so it's entirely possibly I'm doing things in a less-than-optimum way. If anyone is a C/NativeCall guru, I would appreciate and feedback on whether there are better ways I could be handling the data conversions. The header file for libj is here.

So far it's just been me playing around with this module, trying to work out what useful methods and functionality would be useful. I haven't had a good look at it, but a potential source to mine ideas from is Py'n'APL library.

In general, I'm open to feedback from fellow J+Raku users on settling on the API of this module.