Rand Stats

ONNX::Runtime

zef:slavenskoj

ONNX Runtime Bindings for Raku

Native Raku bindings for ONNX Runtime, allowing you to run ONNX models directly in Raku using the C API.

This is an initial implementation focusing on core functionality and is in development and testing.

Features

Prerequisites

  1. Install ONNX Runtime library:

    • macOS: Download from ONNX Runtime releases
    • Linux: apt install libonnxruntime or download from releases
    • Windows: Download from releases

  2. Ensure the library is in your system's library path:

    • macOS: Copy to /usr/local/lib/ or set DYLD_LIBRARY_PATH
    • Linux: Copy to /usr/lib/ or set LD_LIBRARY_PATH
    • Windows: Copy to system directory or set PATH

Installation

From zef ecosystem (when published)

zef install ONNX::Runtime

From source

# Clone the repository
git clone https://github.com/slavenskoj/raku-onnx-raku-bindings.git
cd raku-onnx-raku-bindings

# Install with zef
zef install .

Setting library path

If ONNX Runtime is not in your system's default library path, set the environment variable:

export ONNX_RUNTIME_LIB=/path/to/libonnxruntime.so  # Linux
export ONNX_RUNTIME_LIB=/path/to/libonnxruntime.dylib  # macOS
set ONNX_RUNTIME_LIB=C:\path\to\onnxruntime.dll  # Windows

Usage

Basic Example

use ONNX::Runtime;

# Load a model
my $onnx = ONNX::Runtime.new(
    model-path => "path/to/model.onnx",
    log-level => ORT_LOGGING_LEVEL_WARNING
);

# Query model information
say "Inputs: ", $onnx.input-names;
say "Outputs: ", $onnx.output-names;

# Prepare input data
my %inputs = (
    input_name => [[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]],  # 2x3 tensor
);

# Run inference
my %outputs = $onnx.run(%inputs);

# Process results
for %outputs.kv -> $name, $data {
    say "$name: ", $data;
}

Russian Accent Detection Example

use ONNX::Runtime;

# Load the Russian accent detection model
my $onnx = ONNX::Runtime.new(model-path => "russian_accent.onnx");

# Load audio samples (16kHz)
my @audio-samples = load-audio-file("speech.wav");

# Run inference
my %outputs = $onnx.run(input => @audio-samples);

# Get probability
my $probability = %outputs<output>[0];
say "Russian accent probability: {$probability.fmt('%.2%')}";

Using Different Tensor Types

use ONNX::Runtime;
use ONNX::Runtime::Types;

my $onnx = ONNX::Runtime.new(model-path => "model.onnx");

# Integer tensor input
my @int-data = [1, 2, 3, 4, 5];
my %inputs = (
    int_input => @int-data,
    float_input => [1.5, 2.5, 3.5]
);

# Explicitly specify types if needed
my %types = (
    int_input => ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32,
    float_input => ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT
);

my %outputs = $onnx.run(%inputs, :%types);

Image Processing with UInt8

# Images are typically uint8 (0-255)
my @image-data = load-image-as-array("photo.jpg");  # Returns uint8 values

my %inputs = (
    image => @image-data  # Shape: [1, 3, 224, 224] for RGB image
);

my %outputs = $onnx.run(%inputs);

API Reference

ONNX::Runtime

Main class for loading and running ONNX models.

Constructor

my $onnx = ONNX::Runtime.new(
    model-path => Str,    # Path to ONNX model file (required)
    log-level => Int      # Logging level (optional, default: WARNING)
);

Methods

Logging Levels

Current Limitations

  1. String tensors not yet supported
  2. Basic tensor reshaping (full N-dimensional support coming)
  3. CPU execution only (GPU support planned)
  4. No custom operators yet

Development Status

This is an initial implementation focusing on core functionality:

Project Structure

onnx-raku-bindings/
├── lib/
│   ├── ONNX/
│   │   ├── Runtime.rakumod          # High-level API
│   │   └── Runtime/
│   │       ├── API.rakumod          # Low-level C bindings
│   │       └── Types.rakumod        # Type definitions
├── t/
│   └── 01-basic.t                   # Basic tests
├── examples/
│   └── basic-usage.raku             # Usage examples
├── onnxruntime-osx-universal2-1.16.3/  # ONNX Runtime library
└── README.md

Contributing

https://github.com/slavenskoj/raku-onnx-raku-bindings

Author

Danslav Slavenskoj

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments