WWW::OpenAI Raku package
In brief
This Raku package provides access to the machine learning service OpenAI, [OAI1].
For more details of the OpenAI's API usage see the documentation, [OAI2].
Remark: To use the OpenAI API one has to register and obtain authorization key.
Remark: This Raku package is much "less ambitious" than the official Python package, [OAIp1], developed by OpenAI's team.
Gradually, over time, I expect to add features to the Raku package that correspond to features of [OAIp1].
The original design and implementation of "WWW::OpenAI" were very similar to those of
"Lingua::Translation::DeepL", [AAp1].
Major refactoring of the original code was done -- now each OpenAI functionality targeted by "WWW::OpenAI"
has its code placed in a separate file.
Installation
Package installations from both sources use zef installer
(which should be bundled with the "standard" Rakudo installation file.)
To install the package from Zef ecosystem use the shell command:
zef install WWW::OpenAI
To install the package from the GitHub repository use the shell command:
zef install https://github.com/antononcube/Raku-WWW-OpenAI.git
Usage examples
Remark: When the authorization key, auth-key, is specified to be Whatever
then the functions openai-* attempt to use the env variable OPENAI_API_KEY.
Universal "front-end"
The package has an universal "front-end" function openai-playground for the
different functionalities provided by OpenAI.
Here is a simple text-generation call. The default model is gpt-6.1-sol and
the default path is the Responses API:
use WWW::OpenAI;
openai-playground('Where is Roger Rabbit?', max-tokens => 128, format => 'values');
# Roger Rabbit lives in **Toontown**, the cartoon neighborhood of Los
Another one using Bulgarian:
openai-playground('Колко групи могат да се намерят в този облак от точки.', max-tokens => 128, format => 'values');
# Моля, качете изображението на облака от точки, за да преценя колко групи (клъстери) се виждат.
The function openai-completion can also be used for text generation.
Use openai-chat-completion with path => 'chat/completions' when explicitly
calling the Chat Completions endpoint with a compatible model.
Models
The current OpenAI models can be found with the function openai-models:
openai-models
# (chat-latest chatgpt-image-latest ... whisper-1)
The returned list reflects the models available to your API account. The
package's built-in model list includes current and legacy IDs. Deprecated IDs
remain selectable while OpenAI still serves them; retirement dates are listed
in OpenAI's deprecations page.
For example, gpt-5 and whisper-1 can be selected explicitly during their
deprecation periods. An ID remaining in the package cannot restore access after
OpenAI shuts it down.
Code generation
For new text-generation code, use the Responses path:
openai-completion(
'generate Raku code for making a loop over a list',
model => 'gpt-6.1-sol',
max-tokens => 120,
format => 'values');
# Use a `for` loop to iterate over a list in Raku:
#
# ```raku
# my @fruits = <apple banana cherry>;
#
# for @fruits -> $fruit {
# say $fruit;
# }
# ```
#
# Output:
#
# ```text
# apple
# banana
# cherry
# ```
You can preserve separate system and user messages with openai-chat-completion:
openai-chat-completion(
[system => 'Answer with Raku code only.',
user => 'Write a loop over a list.'],
model => 'gpt-6.1-sol',
format => 'values');
# my @items = <apple banana cherry>;
#
# for @items -> $item {
# say $item;
# }
Legacy text-completion models require type => 'text' and an active model ID.
Image generation
Remark: See the files "Image-generation*" for more details.
Images can be generated with the function openai-create-image -- see the section
"Images" of [OAI2].
Here is an example:
my $imgB64 = openai-create-image(
"racoon with a sliced onion in the style of Raphael",
model => 'gpt-image-2.6-flare',
n => 1,
size => '1024x1024',
format => 'values',
method => 'tiny');
Here are the options descriptions:
- GPT Image models return Base64 image data;
response-format applies to older DALL·E models n takes a positive integer, for the number of images to be generatedsize accepts auto, square, landscape, or portrait for the current GPT Image models
Here we embed a generated image using the returned Base64 data:
my @imgRes = |openai-create-image(
"racoon and onion in the style of Roy Lichtenstein",
n => 1,
size => 'square',
format => 'values',
method => 'tiny');
'';
The default image model is gpt-image-2.6-flare. Older DALL·E model names
remain in the package for historical compatibility; OpenAI has shut down
dall-e-2 and dall-e-3.
Image variation
The image variations endpoint used by the examples below relied on dall-e-2,
which OpenAI has shut down. For current image-to-image work, use
openai-edit-image with a GPT Image model.
Remark: See the files "Image-variation*" for more details.
Images variations over image files can be generated with the function openai-variate-image
-- see the section
"Images" of [OAI2].
Here is an example:
my $imgB64 = openai-variate-image(
$*CWD ~ '/resources/RandomMandala.png',
response-format => 'b64_json',
n => 1,
size => 'small',
format => 'values',
method => 'tiny');
Here are the options descriptions:
response-format takes the values "url" and "b64_json"n takes a positive integer, for the number of images to be generatedsize takes the values '1024x1024', '512x512', '256x256', 'large', 'medium', 'small'.
Remark: Same arguments are used by openai-generate-image. See the previous sub-section.
Here we generate an image, get its URL, and place (embed) a link to it via the output of the code cell:
my @imgRes = |openai-variate-image(
$*CWD ~ '/resources/RandomMandala.png',
response-format => 'url',
n => 1,
size => 'small',
method => 'tiny');
'';
Image edition
Remark: See the files "Image-variation*" for more details.
Editions of images can be generated with the function openai-edit-image -- see the section
"Images" of [OAI2].
Here are the descriptions of positional arguments:
file is a file name string (a PNG image with RGBA color space)prompt is a prompt tha describes the image edition
Here are the descriptions of the named arguments (options):
mask-file a file name of a mask image (can be an empty string or Whatever)n takes a positive integer, for the number of images to be generatedsize takes the values '1024x1024', '512x512', '256x256', 'large', 'medium', 'small'.response-format takes the values "url" and "b64_json"method takes the values "tiny" and "curl"
Here is a random mandala color (RGBA) image:

Here we generate a few editions of the colored mandala image above, get their URLs,
and place (embed) the image links using a table:
my @imgRes = |openai-edit-image(
$*CWD ~ '/../resources/RandomMandala2.png',
'add cosmic background',
response-format => 'url',
n => 2,
size => 'small',
format => 'values',
method => 'tiny');
@imgRes.map({ '' }).join("\n\n")
Vision
In the fall of 2023 OpenAI introduced image vision model
"gpt-4-vision-preview", [OAIb1].
If the function openai-completion is given a list of images, textual results corresponding to those images is returned.
The argument "images" is a list of image URLs, image file names, or image Base64 representations. (Any combination of those element types.)
Here is an example with three images:
my $url1 = 'https://i.imgur.com/LEGfCeq.jpg';
my $url2 = 'https://i.imgur.com/UcRYl9Y.jpg';
my $fname3 = $*CWD ~ '/resources/ThreeHunters.jpg';
my @images = [$url1, $url2, $fname3];
say openai-completion("Give concise descriptions of the images.", :@images, max-tokens => 900, format => 'values');
# 1. A wide-eyed raccoon perches on a mossy branch, surrounded by colorful butterflies and lush foliage.
# 2. Two raccoons run toward a flowering tree adorned with a “2023.11” sign, amid butterflies and a whimsical autumn landscape.
# 3. Three raccoons gather on a moss-covered branch in a sunlit forest filled with orange leaves and colorful butterflies.
The function encode-image from the namespace WWW::OpenAI::ChatCompletions can be used
to get Base64 image strings corresponding to image files. For example:
my $img3 = WWW::OpenAI::ChatCompletions::encode-image($fname3);
say ""
When a file name is given to the argument "images" of openai-completion then the function encode-image is applied to it.
Moderation
Here is an example of using
OpenAI's moderation:
my @modRes = |openai-moderation(
"I want to kill them!",
format => "values",
method => 'tiny');
for @modRes -> $m { .say for $m.pairs.sort(*.value).reverse; }
# violence => 0.9527683472498025
# harassment/threatening => 0.3972887313593611
# harassment => 0.34057871905923465
# illicit => 0.11078220244277934
# illicit/violent => 0.06772771747410274
# hate/threatening => 0.02235435107822504
# hate => 0.013671582768712567
# self-harm => 0.0005152672191311762
# self-harm/intent => 0.000287036046633266432
# sexual => 0.000091105119440064544
# violence/graphic => 0.0000503335575452811456
# sexual/minors => 4.98595821272702e-06
# self-harm/instructions => 2.586743108135441e-06
Audio transcription and translation
Here is an example of using
OpenAI's audio transcription:
my $fileName = $*CWD ~ '/resources/HelloRaccoonsEN.mp3';
say openai-audio(
$fileName,
format => 'json',
method => 'tiny');
# {
# "languages": [
# {
# "code": "en"
# }
# ],
# "usage": {
# "type": "duration",
# "seconds": 6
# },
# "text": "Raku practitioners around the world, eat more onions."
# }
Transcription defaults to gpt-transcribe. You can still select
model => 'whisper-1' while that model remains available. Translation uses
whisper-1; specify type => 'translations':
my $fileName = $*CWD ~ '/resources/HowAreYouRU.mp3';
say openai-audio(
$fileName,
type => 'translations',
format => 'json',
method => 'tiny');
# {
# "text": "How are you, bandits, hooligans? I have long gone mad from you. I have been working as a guard all my life."
# }
Audio speech generation
Here is an example of text-to-speech generation - type, prompt, have to be specified:
my $fileName = $*CWD ~ '/resources/EveryDay.mp3';
my $res = openai-audio(
$fileName,
prompt => 'Every day is a summer day!',
type => 'speech',
format => 'mp3',
voice => 'alloy',
speed => 1,
method => 'tiny');
# Buf[uint8]:0x<FF F3 C4 C4 00 5B E4 39 AC 01 5A C0 00 34 45 02 C2 4D 6C 73 B3 84 F0 D7 38 AB 41 C9 D1 00 B4 05 B7 2C B9 68 D3 AE 79 C0 50 05 34 96 B3 88 65 04 E5 93 41 77 F9 40 19 22 D3 31 18 D0 83 52 8D 8C 36 30 D4 83 31 00 C0 5E 92 B0 B8 4D B7 3D F4 F3 30 73 DD 24 54 64 94 13 95 1C 04 78 2D 81 64 0C 06 37 38 FF ...>
Embeddings
Embeddings
can be obtained with the function openai-embeddings. Here is an example of finding the embedding vectors
for each of the elements of an array of strings:
my @queries = [
'make a classifier with the method RandomForeset over the data dfTitanic',
'show precision and accuracy',
'plot True Positive Rate vs Positive Predictive Value',
'what is a good meat and potatoes recipe'
];
my $embs = openai-embeddings(@queries, format => 'values', method => 'tiny');
$embs.elems;
# 4
Here we show:
- That the result is an array of four vectors each with length 1536
- The distributions of the values of each vector
use Data::Reshapers;
use Data::Summarizers;
say "\$embs.elems : { $embs.elems }";
say "\$embs>>.elems : { $embs>>.elems }";
records-summary($embs.kv.Hash.&transpose);
# $embs.elems : 4
# $embs>>.elems : 1536 1536 1536 1536
# +----------------------------------+----------------------------------+-----------------------------------+----------------------------------+
# | 1 | 3 | 0 | 2 |
# +----------------------------------+----------------------------------+-----------------------------------+----------------------------------+
# | Min => -0.088196 | Min => -0.111389 | Min => -0.10614 | Min => -0.098328 |
# | 1st-Qu => -0.0169449 | 1st-Qu => -0.017197 | 1st-Qu => -0.017029 | 1st-Qu => -0.0169449 |
# | Mean => -0.0004749156457061569 | Mean => 0.00014255968077729145 | Mean => -0.00031491291398803395 | Mean => -0.0003526161114374797 |
# | Median => -0.00058 | Median => 0.0002962350845336914 | Median => 0.0007591 | Median => -0.0003255605697631836 |
# | 3rd-Qu => 0.0173874 | 3rd-Qu => 0.0165939 | 3rd-Qu => 0.016388 | 3rd-Qu => 0.0161057 |
# | Max => 0.108398 | Max => 0.098877 | Max => 0.111084 | Max => 0.081177 |
# +----------------------------------+----------------------------------+-----------------------------------+----------------------------------+
Here we find the corresponding dot products and (cross-)tabulate them:
use Data::Reshapers;
use Data::Summarizers;
my @ct = (^$embs.elems X ^$embs.elems).map({ %( i => $_[0], j => $_[1], dot => sum($embs[$_[0]] >>*<< $embs[$_[1]])) }).Array;
say to-pretty-table(cross-tabulate(@ct, 'i', 'j', 'dot'), field-names => (^$embs.elems)>>.Str);
# +---+----------+----------+----------+----------+
# | | 0 | 1 | 2 | 3 |
# +---+----------+----------+----------+----------+
# | 0 | 1.000369 | 0.207500 | 0.197296 | 0.085141 |
# | 1 | 0.207500 | 1.000164 | 0.346319 | 0.103709 |
# | 2 | 0.197296 | 0.346319 | 1.000013 | 0.077962 |
# | 3 | 0.085141 | 0.103709 | 0.077962 | 1.000569 |
# +---+----------+----------+----------+----------+
Remark: Note that the fourth element (the cooking recipe request) is an outlier.
(Judging by the table with dot products.)
Chat completions with engineered prompts
Here is a prompt for "emojification" (see the
Wolfram Prompt Repository
entry
"Emojify"):
my $preEmojify = q:to/END/;
Rewrite the following text and convert some of it into emojis.
The emojis are all related to whatever is in the text.
Keep a lot of the text, but convert key words into emojis.
Do not modify the text except to add emoji.
Respond only with the modified text, do not include any summary or explanation.
Do not respond with only emoji, most of the text should remain as normal words.
END
Here is an example of chat completion with emojification:
openai-chat-completion([ system => $preEmojify, user => 'Python sucks, Raku rocks, and Perl is annoying'], max-tokens => 200, format => 'values')
# Python 🐍 sucks, Raku 🦋 rocks 🤘, and Perl 🐪 is annoying 😒
For more examples see the document "Chat-completion-examples".
Finding textual answers
The models of OpenAI can be used to find sub-strings in texts that appear to be
answers to given questions. This is done via the package
"ML::FindTextualAnswer", [AAp3],
using the parameter specs llm => 'chatgpt' or llm => 'openai'.
Here is an example of finding textual answers:
use ML::FindTextualAnswer;
my $text = "Lake Titicaca is a large, deep lake in the Andes
on the border of Bolivia and Peru. By volume of water and by surface
area, it is the largest lake in South America";
find-textual-answer($text, "Where is Titicaca?", llm => 'openai')
# On the border of Bolivia and Peru.
By default find-textual-answer tries to give short answers.
If the option "request" is Whatever then depending on the number of questions
the request is one those phrases:
- "give the shortest answer of the question:"
- "list the shortest answers of the questions:"
In the example above the full query given to OpenAI's models is:
Given the text "Lake Titicaca is a large, deep lake in the Andes
on the border of Bolivia and Peru. By volume of water and by surface
area, it is the largest lake in South America"
give the shortest answer of the question:
Where is Titicaca?
Here we get a longer answer by changing the value of "request":
find-textual-answer($text, "Where is Titicaca?", llm => 'chatgpt', request => "answer the question:")
# Lake Titicaca is in the Andes, on the border of Bolivia and Peru.
Remark: The function find-textual-answer is inspired by the Mathematica function
FindTextualAnswer;
see [JL1].
Multiple questions
If several questions are given to the function find-textual-answer
then all questions are spliced with the given text into one query (that is sent to OpenAI.)
For example, consider the following text and questions:
my $query = 'Make a classifier with the method RandomForest over the data dfTitanic; show precision and accuracy.';
my @questions =
['What is the dataset?',
'What is the method?',
'Which metrics to show?'
];
# [What is the dataset? What is the method? Which metrics to show?]
Then the query send to OpenAI is:
Given the text: "Make a classifier with the method RandomForest over the data dfTitanic; show precision and accuracy."
list the shortest answers of the questions:
- What is the dataset?
- What is the method?
- Which metrics to show?
The answers are assumed to be given in the same order as the questions, each answer in a separated line.
Hence, by splitting the OpenAI result into lines we get the answers corresponding to the questions.
If the questions are missing question marks, it is likely that the result may have a completion as
a first line followed by the answers. In that situation the answers are not parsed and a warning message is given.
Default models
Default models can be specified with the operating system environmental variables:
OPENAI_DEFAULT_MODELOPENAI_DEFAULT_IMAGE_GENERATION_MODELOPENAI_DEFAULT_EMBEDDING_MODEL
The built-in defaults are gpt-6.1-sol for text, gpt-image-2.6-flare for
image generation and editing, and text-embedding-3-small for embeddings.
Transcription defaults to gpt-transcribe.
Command Line Interface
Playground access
The package provides a Command Line Interface (CLI) script:
openai-playground --help
# Usage:
# openai-playground [<words> ...] [--path=<Str>] [-n[=UInt]] [--mt|--max-tokens[=UInt]] [-m|--model=<Str>] [-r|--role=<Str>] [-t|--temperature[=Real]] [-i|--images=<Str>] [-l|--language=<Str>] [--response-format=<Str>] [-a|--auth-key=<Str>] [--timeout[=UInt]] [-f|--format=<Str>] [--method=<Str>] [--base-url=<Str>] -- Command given as a sequence of words.
#
# --path=<Str> Path, one of 'responses', 'chat/completions', 'images/generations', 'images/edits', 'images/variations', 'moderations', 'audio/transcriptions', 'audio/translations', 'embeddings', or 'models'. [default: 'responses']
# -n[=UInt] Number of completions or generations. [default: 1]
# --mt|--max-tokens[=UInt] The maximum number of tokens to generate in the completion. [default: 4096]
# -m|--model=<Str> Model. [default: 'Whatever']
# -r|--role=<Str> Role. [default: 'user']
# -t|--temperature[=Real] Temperature. [default: 0.7]
# -i|--images=<Str> Image URLs or file names separated with comma (','). [default: '']
# -l|--language=<Str> Language. [default: '']
# --response-format=<Str> The format in which the generated images are returned; one of 'url' or 'b64_json'. [default: 'url']
# -a|--auth-key=<Str> Authorization key (to use OpenAI API.) [default: 'Whatever']
# --timeout[=UInt] Timeout. [default: 10]
# -f|--format=<Str> Format of the result; one of "json", "hash", "values", or "Whatever". [default: 'Whatever']
# --method=<Str> Method for the HTTP POST query; one of "tiny" or "curl". [default: 'tiny']
# --base-url=<Str> URL of the Web API service. [default: 'Whatever']
Remark: When the authorization key argument "auth-key" is specified set to "Whatever"
then openai-playground attempts to use the env variable OPENAI_API_KEY.
Mermaid diagram
The following flowchart corresponds to the steps in the package function openai-playground:
graph TD
UI[/Some natural language text/]
TO[/"OpenAI<br/>Processed output"/]
WR[[Web request]]
OpenAI{{https://platform.openai.com}}
PJ[Parse JSON]
Q{Return<br>hash?}
MSTC[Compose query]
MURL[[Make URL]]
TTC[Process]
QAK{Auth key<br>supplied?}
EAK[["Try to find<br>OPENAI_API_KEY<br>in %*ENV"]]
QEAF{Auth key<br>found?}
NAK[/Cannot find auth key/]
UI --> QAK
QAK --> |yes|MSTC
QAK --> |no|EAK
EAK --> QEAF
MSTC --> TTC
QEAF --> |no|NAK
QEAF --> |yes|TTC
TTC -.-> MURL -.-> WR -.-> TTC
WR -.-> |URL|OpenAI
OpenAI -.-> |JSON|WR
TTC --> Q
Q --> |yes|PJ
Q --> |no|TO
PJ --> TO
Potential problems
Tested on macOS only
Currently this package is tested on macOS only.
Not all models work
Not all models listed and proclaimed by OpenAI's documents
work with the corresponding endpoints. Certain models are not available for text- or chat completions,
although the documentation says they are.
See and run the file "Models-run-verification.raku"
to test the available models per endpoint.
Related is a (current) deficiency of the package "WWW::OpenAI" -- the known models are hardcoded.
(Although, there the function openai-models uses an endpoint provided by OpenAI.)
SSL certificate problems (original package version)
(This subsection is for the original version of the package, not for the most recent one.)
On macOS I get the errors:
Cannot locate symbol 'SSL_get1_peer_certificate' in native library
See longer discussions about this problem
here
and
here
Interestingly:
- I did not get these messages while implementing the changes of ver<1.1> of this package
- I do not get these messages when using Raku in Markdown or Mathematica notebooks, [AA1],
via the package "Text::CodeProcessing"
Because of those SSL problems I implemented the method option that takes the values 'cro' and 'curl'.
The method "curl":
- Requires
curl to be installed - Invokes the procedure
shell - Again, this is tested on macOS only.
After "discovering" "HTTP::Tiny" and given the problems with "Cro::HTTP::Client", I removed the 'cro' method.
(I.e. the methods are 'tiny' and 'curl' in ver<0.2.0+>.)
TODO
DONE Comprehensive unit tests
- Note that this requires OpenAI auth token and costs money. (Ideally, not much.)
- DONE Basic usage
- DONE Completions - chat
- DONE Completions - text
- DONE Moderation
- DONE Audio transcription
- DONE Audio translation
- DONE Image generation
- DONE Image variation
- DONE Image edition
- DONE Embeddings
- DONE Finding of textual answers
- DONE Audio speech generation
DONE HTTP(S) retrieval methods
- DONE
curl - DONE "Cro"
- DONE "HTTP::Tiny"
DONE Models implementation
DONE Embeddings implementation
DONE Refactor the code, so each functionality (audio, completion, moderation, etc)
has a separate file.
DONE Refactor HTTP(S) retrieval functions to be simpler and more "uniform."
DONE De-Cro the request code.
- Given the problems of using "Cro::HTTP::Client" and the implementations with
curl and
"HTTP::Tiny",
it seems it is better to make the implementation of "WWW::OpenAI" more lightweight.
DONE Implement finding of textual answers
DONE Factor out finding of textual answers into a separate package
DONE Implement vision (over images)
DONE Implement handling of tools
References
Articles
[AA1] Anton Antonov,
"Connecting Mathematica and Raku",
(2021),
RakuForPrediction at WordPress.
[JL1] Jérôme Louradour,
"New in the Wolfram Language: FindTextualAnswer",
(2018),
blog.wolfram.com.
[OAIb1] OpenAI team,
"New models and developer products announced at DevDay",
(2023),
OpenAI/blog.
Packages
[AAp1] Anton Antonov,
Lingua::Translation::DeepL Raku package,
(2022),
GitHub/antononcube.
[AAp2] Anton Antonov,
Text::CodeProcessing,
(2021),
GitHub/antononcube.
[AAp3] Anton Antonov,
ML::FindTextualAnswer,
(2023),
GitHub/antononcube.
[OAI1] OpenAI Platform, OpenAI platform.
[OAI2] OpenAI Platform, OpenAI documentation.
[OAIp1] OpenAI,
OpenAI Python Library,
(2020),
GitHub/openai.
Videos
[AAv1] Anton Antonov,
"Jupyter Chatbook LLM cells demo (Raku)",
(2023),
YouTube/@AAA4prediction.
[AAv2] Anton Antonov,
"Integrating Large Language Models with Raku",
(2023),
The Raku Conference at YouTube.