ONNX Runtime Bindings for Raku
Native Raku bindings for Open Neural Network Exchange (ONNX) Runtime, allowing you to run ONNX models directly in Raku using the C API.
Access to Transformer Models - Run BERT, GPT-2, RoBERTa, and other modern NLP models directly in Raku without Python dependencies
Hugging Face Integration - Any model from Hugging Face can be exported to ONNX and used in Raku
Production-Ready Deep Learning - ONNX models are optimized for inference, offering better performance than Python for deployment
This is an initial implementation focusing on core functionality and is in development and testing.
Status
⚠️ Important: Due to a bug in Raku's module precompilation system when using NativeCall with function pointers, the bindings cannot be packaged as a traditional Raku module. Instead, use the standalone scripts provided.
Working Examples
The following scripts are fully functional and tested:
onnx-working.raku
- Basic ONNX Runtime functionality demonstrationmnist-final.raku
- Complete MNIST digit recognition example
Usage
# Make scripts executable
chmod +x onnx-working.raku mnist-final.raku
# Run basic example
./onnx-working.raku
# Run MNIST example
./mnist-final.raku
Features
- Load ONNX models
- Run inference
- Support for float32 tensors
- Automatic memory management
- Model introspection (input/output names and shapes)
Requirements
- Raku (tested with Rakudo Star 2025.05)
- ONNX Runtime library (not included)
- An ONNX model file (MNIST model tested)
Technical Details
ONNX Runtime uses a C API based on function pointers accessed through OrtGetApiBase()
. The implementation:
- Gets the API base structure
- Retrieves the API version
- Obtains function pointers by index
- Uses these function pointers for all operations
Known Issues
Module Precompilation: Raku's precompilation system hangs when modules use NativeCall to call function pointers during initialization. This is why the code is provided as standalone scripts.
Limited Type Support: Currently only supports float32 tensors. Other data types can be added following the same pattern.
Example Output
$ ./mnist-final.raku
Getting API base...
API base retrieved successfully
Getting API version 16...
API retrieved successfully
Getting function pointers...
All function pointers retrieved successfully
Creating environment...
Environment created successfully
Creating session options...
Session options created successfully
Setting optimization level...
Optimization level set successfully
Creating session...
Session created successfully
Creating memory info...
Memory info created successfully
Getting allocator...
Allocator retrieved successfully
Model loaded successfully!
...
Predicted digit: 3 (confidence: 54.82%)
Future Work
- Report the precompilation bug to Raku developers
- Add support for more tensor types
- Create a workaround for the module issue
- Add more examples and documentation
Contributing
https://github.com/slavenskoj/raku-onnx-bindings
Author
Danslav Slavenskoj
License
This project is licensed under the MIT License - see the LICENSE file for details.
Acknowledgments
- ONNX Runtime team for the C API
- Raku community for NativeCall support