Rand Stats

WWW::OpenAI

zef:antononcube

WWW::OpenAI Raku package

In brief

This Raku package provides access to the machine learning service OpenAI, [OAI1]. For more details of the OpenAI's API usage see the documentation, [OAI2].

Remark: To use the OpenAI API one has to register and obtain authorization key.

Remark: This Raku package is much "less ambitious" than the official Python package, [OAIp1], developed by OpenAI's team. Gradually, over time, I expect to add features to the Raku package that correspond to features of [OAIp1].

The original design and implementation of "WWW::OpenAI" were very similar to those of "Lingua::Translation::DeepL", [AAp1]. Major refactoring of the original code was done -- now each OpenAI functionality targeted by "WWW::OpenAI" has its code placed in a separate file.

Installation

Package installations from both sources use zef installer (which should be bundled with the "standard" Rakudo installation file.)

To install the package from Zef ecosystem use the shell command:

zef install WWW::OpenAI

To install the package from the GitHub repository use the shell command:

zef install https://github.com/antononcube/Raku-WWW-OpenAI.git

Usage examples

Remark: When the authorization key, auth-key, is specified to be Whatever then the functions openai-* attempt to use the env variable OPENAI_API_KEY.

Universal "front-end"

The package has an universal "front-end" function openai-playground for the different functionalities provided by OpenAI.

Here is a simple call for a "chat completion":

use WWW::OpenAI;
openai-playground('Where is Roger Rabbit?', max-tokens => 64);

# [{finish_reason => stop, index => 0, logprobs => (Any), text => 
# 
# Roger Rabbit is a fictional character from the movie "Who Framed Roger Rabbit" and he does not exist in real life. He can only be found in the movie and its related merchandise.}]

Another one using Bulgarian:

openai-playground('Колко групи могат да се намерят в този облак от точки.', max-tokens => 64);

# [{finish_reason => length, index => 0, logprobs => (Any), text => 
# 
# Не може да се даде точен отговор на този въпрос без да се предостави допълнителна информация за облака от точки. Броят на групите може да зав}]

Remark: The function openai-completion can be used instead in the examples above. See the section "Create chat completion" of [OAI2] for more details.

Models

The current OpenAI models can be found with the function openai-models:

openai-models

# (babbage-002 dall-e-2 dall-e-3 davinci-002 gpt-3.5-turbo gpt-3.5-turbo-0125 gpt-3.5-turbo-0301 gpt-3.5-turbo-0613 gpt-3.5-turbo-1106 gpt-3.5-turbo-16k gpt-3.5-turbo-16k-0613 gpt-3.5-turbo-instruct gpt-3.5-turbo-instruct-0914 gpt-4 gpt-4-0125-preview gpt-4-0613 gpt-4-1106-preview gpt-4-turbo-preview gpt-4-vision-preview text-embedding-3-large text-embedding-3-small text-embedding-ada-002 tts-1 tts-1-1106 tts-1-hd tts-1-hd-1106 whisper-1)

Code generation

There are two types of completions : text and chat. Let us illustrate the differences of their usage by Raku code generation. Here is a text completion:

openai-completion(
        'generate Raku code for making a loop over a list',
        type => 'text',
        max-tokens => 120,
        format => 'values');

# my @list = (1, 2, 3, 4, 5, 6); # example list
# 
# for @list -> $item { # loop over the list using the for loop
#     say $item; # print each item in the list
# }
# 
# # Output:
# # 1
# # 2
# # 3
# # 4
# # 5
# # 6

Here is a chat completion:

openai-completion(
        'generate Raku code for making a loop over a list',
        type => 'chat',
        max-tokens => 120,
        format => 'values');

# Here is an example of Raku code that loops over a list and prints each element:
# 
# ```raku
# my @list = 1, 2, 3, 4, 5;
# 
# for @list -> $element {
#     say $element;
# }
# ```
# 
# This code will output:
# ```
# 1
# 2
# 3
# 4
# 5
# ```

Remark: The argument "type" and the argument "model" have to "agree." (I.e. be found agreeable by OpenAI.) For example:

Image generation

Remark: See the files "Image-generation*" for more details.

Images can be generated with the function openai-create-image -- see the section "Images" of [OAI2].

Here is an example:

my $imgB64 = openai-create-image(
        "racoon with a sliced onion in the style of Raphael",
        response-format => 'b64_json',
        model = 'dalle-e-3',
        n => 1,
        size => '1024x1024',
        format => 'values',
        method => 'tiny');

Here are the options descriptions:

Here we generate an image, get its URL, and place (embed) a link to it via the output of the code cell:

my @imgRes = |openai-create-image(
        "racoon and onion in the style of Roy Lichtenstein",
        response-format => 'url',
        n => 1,
        size => 'small',
        method => 'tiny');

'![](' ~ @imgRes.head<url> ~ ')';

Remark: The argument "model" can be Whatever of one of "dall-e-2" or "dall-e-3". Not all parameters that are valid for one of the models are valid or respected by the other -- see the subsection "Create image" of OpenAI's documentation.

Image variation

Remark: See the files "Image-variation*" for more details.

Images variations over image files can be generated with the function openai-variate-image -- see the section "Images" of [OAI2].

Here is an example:

my $imgB64 = openai-variate-image(
        $*CWD ~ '/resources/RandomMandala.png',
        response-format => 'b64_json',
        n => 1,
        size => 'small',
        format => 'values',
        method => 'tiny');

Here are the options descriptions:

Remark: Same arguments are used by openai-generate-image. See the previous sub-section.

Here we generate an image, get its URL, and place (embed) a link to it via the output of the code cell:

my @imgRes = |openai-variate-image(
        $*CWD ~ '/resources/RandomMandala.png',
        response-format => 'url',
        n => 1,
        size => 'small',
        method => 'tiny');

'![](' ~ @imgRes.head<url> ~ ')';

Image edition

Remark: See the files "Image-variation*" for more details.

Editions of images can be generated with the function openai-edit-image -- see the section "Images" of [OAI2].

Here are the descriptions of positional arguments:

Here are the descriptions of the named arguments (options):

Here is a random mandala color (RGBA) image:

Here we generate a few editions of the colored mandala image above, get their URLs, and place (embed) the image links using a table:

my @imgRes = |openai-edit-image(
        $*CWD ~ '/../resources/RandomMandala2.png',
        'add cosmic background',
        response-format => 'url',
        n => 2,
        size => 'small',
        format => 'values',
        method => 'tiny');

@imgRes.map({ '![](' ~ $_ ~ ')' }).join("\n\n")

Vision

In the fall of 2023 OpenAI introduced image vision model "gpt-4-vision-preview", [OAIb1].

If the function openai-completion is given a list of images, textual results corresponding to those images is returned. The argument "images" is a list of image URLs, image file names, or image Base64 representations. (Any combination of those element types.)

Here is an example with three images:

my $url1 = 'https://i.imgur.com/LEGfCeq.jpg';
my $url2 = 'https://i.imgur.com/UcRYl9Y.jpg';
my $fname3 = $*CWD ~ '/resources/ThreeHunters.jpg';
my @images = [$url1, $url2, $fname3];
say openai-completion("Give concise descriptions of the images.", :@images, max-tokens => 900, format => 'values');

# 1. The first image is a vibrant illustration of a raccoon perched on a tree branch surrounded by a flurry of colorful butterflies.
# 
# 2. The second image depicts two raccoons running along a path next to a large tree with a sign, surrounded by butterflies, in a colorful autumn landscape.
# 
# 3. The third image shows a trio of raccoons sitting in a tree hollow, with a warm, luminous backdrop and butterflies fluttering around, set in an autumnal forest scene.

The function encode-image from the namespace WWW::OpenAI::ChatCompletions can be used to get Base64 image strings corresponding to image files. For example:

my $img3 = WWW::OpenAI::ChatCompletions::encode-image($fname3);
say "![]($img3)"

When a file name is given to the argument "images" of openai-completion then the function encode-image is applied to it.

Moderation

Here is an example of using OpenAI's moderation:

my @modRes = |openai-moderation(
"I want to kill them!",
format => "values",
method => 'tiny');

for @modRes -> $m { .say for $m.pairs.sort(*.value).reverse; }

# violence => 0.998648464679718
# harassment/threatening => 0.5217957496643066
# harassment => 0.49981600046157837
# hate => 0.1619441658258438
# hate/threatening => 0.04833448305726051
# violence/graphic => 0.000130274318507872528
# sexual => 0.0000252172667387640096
# self-harm => 4.838872882828582e-06
# self-harm/intent => 2.9584793992398772e-06
# sexual/minors => 1.4673852888336114e-07
# self-harm/instructions => 1.7596822887711028e-09

Audio transcription and translation

Here is an example of using OpenAI's audio transcription:

my $fileName = $*CWD ~ '/resources/HelloRaccoonsEN.mp3';
say openai-audio(
        $fileName,
        format => 'json',
        method => 'tiny');

# {
#   "text": "Raku practitioners around the world, eat more onions!"
# }

To do translations use the named argument type:

my $fileName = $*CWD ~ '/resources/HowAreYouRU.mp3';
say openai-audio(
        $fileName,
        type => 'translations',
        format => 'json',
        method => 'tiny');

# {
#   "text": "How are you, bandits, hooligans? I have long gone mad from you. I have been working as a guard all my life."
# }

Audio speech generation

Here is an example of text-to-speech generation - type, prompt, have to be specified:

my $fileName = $*CWD ~ '/resources/EveryDay.mp3';
my $res = openai-audio(
        $fileName, 
        prompt => 'Every day is a summer day!',
        type => 'speech',
        format => 'mp3',
        voice => 'alloy',
        speed => 1,
        method => 'tiny');

# Buf[uint8]:0x<FF F3 E4 C4 00 5F 94 39 D8 17 5A C0 00 06 53 52 1C 77 E6 BC B9 73 8C C2 A3 68 E0 DB 30 35 48 0C D1 43 20 20 C8 89 30 E0 C0 40 CB 36 5A 72 CB 98 10 26 0C 29 85 02 62 03 18 60 C6 18 21 84 08 61 02 18 20 46 08 00 08 01 6B 0B 46 5B 32 CB 96 4C B2 65 A7 2E 59 6C CB 36 66 11 88 05 98 2E 42 0E 24 42 62 28 ...>

Embeddings

Embeddings can be obtained with the function openai-embeddings. Here is an example of finding the embedding vectors for each of the elements of an array of strings:

my @queries = [
    'make a classifier with the method RandomForeset over the data dfTitanic',
    'show precision and accuracy',
    'plot True Positive Rate vs Positive Predictive Value',
    'what is a good meat and potatoes recipe'
];

my $embs = openai-embeddings(@queries, format => 'values', method => 'tiny');
$embs.elems;

# 4

Here we show:

use Data::Reshapers;
use Data::Summarizers;

say "\$embs.elems : { $embs.elems }";
say "\$embs>>.elems : { $embs>>.elems }";
records-summary($embs.kv.Hash.&transpose);

# $embs.elems : 4
# $embs>>.elems : 1536 1536 1536 1536
# +--------------------------------+-------------------------------+--------------------------------+-------------------------------+
# | 3                              | 1                             | 2                              | 0                             |
# +--------------------------------+-------------------------------+--------------------------------+-------------------------------+
# | Min    => -0.6047187           | Min    => -0.6678709          | Min    => -0.6308942           | Min    => -0.590719           |
# | 1st-Qu => -0.012928714         | 1st-Qu => -0.012335346        | 1st-Qu => -0.012563366         | 1st-Qu => -0.01316472075      |
# | Mean   => -0.00075381957348203 | Mean   => -0.0007606080686471 | Mean   => -0.00072762704473066 | Mean   => -0.0007624376152279 |
# | Median => -0.00081807276       | Median => -0.000239203835     | Median => -0.00061062546       | Median => -0.00098278925      |
# | 3rd-Qu => 0.01215142175        | 3rd-Qu => 0.0114232735        | 3rd-Qu => 0.011824543          | 3rd-Qu => 0.012312613         |
# | Max    => 0.22180624           | Max    => 0.2273228           | Max    => 0.21278104           | Max    => 0.21206765          |
# +--------------------------------+-------------------------------+--------------------------------+-------------------------------+

Here we find the corresponding dot products and (cross-)tabulate them:

use Data::Reshapers;
use Data::Summarizers;
my @ct = (^$embs.elems X ^$embs.elems).map({ %( i => $_[0], j => $_[1], dot => sum($embs[$_[0]] >>*<< $embs[$_[1]])) }).Array;

say to-pretty-table(cross-tabulate(@ct, 'i', 'j', 'dot'), field-names => (^$embs.elems)>>.Str);

# +---+----------+----------+----------+----------+
# |   |    0     |    1     |    2     |    3     |
# +---+----------+----------+----------+----------+
# | 0 | 1.000000 | 0.724379 | 0.756525 | 0.665386 |
# | 1 | 0.724379 | 1.000000 | 0.808996 | 0.716001 |
# | 2 | 0.756525 | 0.808996 | 1.000000 | 0.698533 |
# | 3 | 0.665386 | 0.716001 | 0.698533 | 1.000000 |
# +---+----------+----------+----------+----------+

Remark: Note that the fourth element (the cooking recipe request) is an outlier. (Judging by the table with dot products.)

Chat completions with engineered prompts

Here is a prompt for "emojification" (see the Wolfram Prompt Repository entry "Emojify"):

my $preEmojify = q:to/END/;
Rewrite the following text and convert some of it into emojis.
The emojis are all related to whatever is in the text.
Keep a lot of the text, but convert key words into emojis.
Do not modify the text except to add emoji.
Respond only with the modified text, do not include any summary or explanation.
Do not respond with only emoji, most of the text should remain as normal words.
END

# Rewrite the following text and convert some of it into emojis.
# The emojis are all related to whatever is in the text.
# Keep a lot of the text, but convert key words into emojis.
# Do not modify the text except to add emoji.
# Respond only with the modified text, do not include any summary or explanation.
# Do not respond with only emoji, most of the text should remain as normal words.

Here is an example of chat completion with emojification:

openai-chat-completion([ system => $preEmojify, user => 'Python sucks, Raku rocks, and Perl is annoying'], max-tokens => 200, format => 'values')

# 🐍 Python sucks, 🦝 Raku rocks, and Perl is 😒 annoying

For more examples see the document "Chat-completion-examples".

Finding textual answers

The models of OpenAI can be used to find sub-strings in texts that appear to be answers to given questions. This is done via the package "ML::FindTextualAnswer", [AAp3], using the parameter specs llm => 'chatgpt' or llm => 'openai'.

Here is an example of finding textual answers:

use ML::FindTextualAnswer;
my $text = "Lake Titicaca is a large, deep lake in the Andes 
on the border of Bolivia and Peru. By volume of water and by surface 
area, it is the largest lake in South America";

find-textual-answer($text, "Where is Titicaca?", llm => 'openai')

# Andes

By default find-textual-answer tries to give short answers. If the option "request" is Whatever then depending on the number of questions the request is one those phrases:

In the example above the full query given to OpenAI's models is:

Given the text "Lake Titicaca is a large, deep lake in the Andes on the border of Bolivia and Peru. By volume of water and by surface area, it is the largest lake in South America" give the shortest answer of the question:
Where is Titicaca?

Here we get a longer answer by changing the value of "request":

find-textual-answer($text, "Where is Titicaca?", llm => 'chatgpt', request => "answer the question:")

# Lake Titicaca is located in the Andes on the border of Bolivia and Peru in South America.

Remark: The function find-textual-answer is inspired by the Mathematica function FindTextualAnswer; see [JL1].

Multiple questions

If several questions are given to the function find-textual-answer then all questions are spliced with the given text into one query (that is sent to OpenAI.)

For example, consider the following text and questions:

my $query = 'Make a classifier with the method RandomForest over the data dfTitanic; show precision and accuracy.';

my @questions =
        ['What is the dataset?',
         'What is the method?',
         'Which metrics to show?'
        ];

# [What is the dataset? What is the method? Which metrics to show?]

Then the query send to OpenAI is:

Given the text: "Make a classifier with the method RandomForest over the data dfTitanic; show precision and accuracy." list the shortest answers of the questions:

  1. What is the dataset?
  2. What is the method?
  3. Which metrics to show?

The answers are assumed to be given in the same order as the questions, each answer in a separated line. Hence, by splitting the OpenAI result into lines we get the answers corresponding to the questions.

If the questions are missing question marks, it is likely that the result may have a completion as a first line followed by the answers. In that situation the answers are not parsed and a warning message is given.

Command Line Interface

Playground access

The package provides a Command Line Interface (CLI) script:

openai-playground --help

# Usage:
#   openai-playground [<words> ...] [--path=<Str>] [-n[=UInt]] [--mt|--max-tokens[=UInt]] [-m|--model=<Str>] [-r|--role=<Str>] [-t|--temperature[=Real]] [-i|--images=<Str>] [-l|--language=<Str>] [--response-format=<Str>] [-a|--auth-key=<Str>] [--timeout[=UInt]] [-f|--format=<Str>] [--method=<Str>] [--base-url=<Str>] -- Command given as a sequence of words.
#   
#     --path=<Str>                Path, one of 'chat/completions', 'images/generations', 'images/edits', 'images/variations', 'moderations', 'audio/transcriptions', 'audio/translations', 'embeddings', or 'models'. [default: 'chat/completions']
#     -n[=UInt]                   Number of completions or generations. [default: 1]
#     --mt|--max-tokens[=UInt]    The maximum number of tokens to generate in the completion. [default: 100]
#     -m|--model=<Str>            Model. [default: 'Whatever']
#     -r|--role=<Str>             Role. [default: 'user']
#     -t|--temperature[=Real]     Temperature. [default: 0.7]
#     -i|--images=<Str>           Image URLs or file names separated with comma (','). [default: '']
#     -l|--language=<Str>         Language. [default: '']
#     --response-format=<Str>     The format in which the generated images are returned; one of 'url' or 'b64_json'. [default: 'url']
#     -a|--auth-key=<Str>         Authorization key (to use OpenAI API.) [default: 'Whatever']
#     --timeout[=UInt]            Timeout. [default: 10]
#     -f|--format=<Str>           Format of the result; one of "json", "hash", "values", or "Whatever". [default: 'Whatever']
#     --method=<Str>              Method for the HTTP POST query; one of "tiny" or "curl". [default: 'tiny']
#     --base-url=<Str>            URL of the Web API service. [default: 'Whatever']

Remark: When the authorization key argument "auth-key" is specified set to "Whatever" then openai-playground attempts to use the env variable OPENAI_API_KEY.

Mermaid diagram

The following flowchart corresponds to the steps in the package function openai-playground:

graph TD
	UI[/Some natural language text/]
	TO[/"OpenAI<br/>Processed output"/]
	WR[[Web request]]
	OpenAI{{https://platform.openai.com}}
	PJ[Parse JSON]
	Q{Return<br>hash?}
	MSTC[Compose query]
	MURL[[Make URL]]
	TTC[Process]
	QAK{Auth key<br>supplied?}
	EAK[["Try to find<br>OPENAI_API_KEY<br>in %*ENV"]]
	QEAF{Auth key<br>found?}
	NAK[/Cannot find auth key/]
	UI --> QAK
	QAK --> |yes|MSTC
	QAK --> |no|EAK
	EAK --> QEAF
	MSTC --> TTC
	QEAF --> |no|NAK
	QEAF --> |yes|TTC
	TTC -.-> MURL -.-> WR -.-> TTC
	WR -.-> |URL|OpenAI 
	OpenAI -.-> |JSON|WR
	TTC --> Q 
	Q --> |yes|PJ
	Q --> |no|TO
	PJ --> TO

Potential problems

Tested on macOS only

Currently this package is tested on macOS only.

Not all models work

Not all models listed and proclaimed by OpenAI's documents work with the corresponding endpoints. Certain models are not available for text- or chat completions, although the documentation says they are.

See and run the file "Models-run-verification.raku" to test the available models per endpoint.

Related is a (current) deficiency of the package "WWW::OpenAI" -- the known models are hardcoded. (Although, there the function openai-models uses an endpoint provided by OpenAI.)

SSL certificate problems (original package version)

(This subsection is for the original version of the package, not for the most recent one.)

TODO

References

Articles

[AA1] Anton Antonov, "Connecting Mathematica and Raku", (2021), RakuForPrediction at WordPress.

[JL1] Jérôme Louradour, "New in the Wolfram Language: FindTextualAnswer", (2018), blog.wolfram.com.

[OAIb1] OpenAI team, "New models and developer products announced at DevDay", (2023), OpenAI/blog.

Packages

[AAp1] Anton Antonov, Lingua::Translation::DeepL Raku package, (2022), GitHub/antononcube.

[AAp2] Anton Antonov, Text::CodeProcessing, (2021), GitHub/antononcube.

[AAp3] Anton Antonov, ML::FindTextualAnswer, (2023), GitHub/antononcube.

[OAI1] OpenAI Platform, OpenAI platform.

[OAI2] OpenAI Platform, OpenAI documentation.

[OAIp1] OpenAI, OpenAI Python Library, (2020), GitHub/openai.

Videos