AIGC: Label: "1" ContentProducer: 001191440300708461136T1XGW3 ProduceID: aed9bd2c2f3c6f83a54981e0ddb92ca2_2ff46853d75a11f18947525400de85a5 ReservedCode1: aCy5yttkD9A2gcPJBug8DKlMg8u9pcZZNTrT80qVrEaBhbmAj0FBt7ImjkXwsgd8v5ISlUYZ5Dmx1KDIG1ZNnVMN69icYZ5WsE72c+D+GEZv15zi1TJxqdJvR02xEFSZNRrgKTgjz3mKdPEnfDOYnYDX128/TMLCvPZL42SukyScjkoWULPJ+OhJweQ= ContentPropagator: 001191440300708461136T1XGW3 PropagateID: aed9bd2c2f3c6f83a54981e0ddb92ca2_2ff46853d75a11f18947525400de85a5 ReservedCode2: aCy5yttkD9A2gcPJBug8DKlMg8u9pcZZNTrT80qVrEaBhbmAj0FBt7ImjkXwsgd8v5ISlUYZ5Dmx1KDIG1ZNnVMN69icYZ5WsE72c+D+GEZv15zi1TJxqdJvR02xEFSZNRrgKTgjz3mKdPEnfDOYnYDX128/TMLCvPZL42SukyScjkoWULPJ+OhJweQ=

HTTP::Core

A minimal HTTP/1.x request-head parser for Raku: a hand-written C kernel bound through NativeCall with no shim layer.

This is a probe, not a framework. It exists to answer one question — how far does a raw NativeCall binding get before you need an intermediate layer? — and then to be the parsing primitive a real server can build on.

On top of that kernel sit a codec, a router, a server and a client, so the answer is measured rather than asserted: the whole stack, from parse-request to a keep-alive connection pool, is built out of the same nine exported C functions. Suite status: 17 test files, 101 tests, all passing on Rakudo 2026.09 / MoarVM.

Contents

Install

Rakudo 2026.09 (MoarVM) or newer. Two dependencies, Compress::Zlib and JSON::Fast; the test suite additionally uses IO::Socket::Async::SSL.

git clone https://gitee.com/skyter10086/raku-HTTP-Core.git
cd raku-HTTP-Core

raku -I. Build.rakumod                # build the kernel; CC=... overrides the compiler
raku -Ilib -e 'use HTTP::Core; say core-version(); say library-path()'

Build.rakumod writes http_wire.dll, libhttp_wire.so or libhttp_wire.dylib into resources/libraries/. That directory is ignored by git on purpose — a compiled kernel belongs to whoever compiled it — so a fresh clone has no library until you run the build.

The binding searches, in order: %*ENV<HTTP_CORE_LIB>, then resources/libraries/, then the platform loader path. If you keep the library somewhere else, name it explicitly:

HTTP_CORE_LIB=/opt/http-core/libhttp_wire.so raku -Ilib -e 'use HTTP::Core'

Using it from this tree needs nothing else: -Ilib is the whole install. zef install . additionally ships the provides and resources entries from META6.json, which is convenient once the kernel is built but no substitute for it — build first, install second.

Windows

No Visual Studio, no WSL, no MSYS shell: Rakudo's Windows build runs the script directly and zig cc emits a plain .dll that NativeCall loads as-is.

git clone https://gitee.com/skyter10086/raku-HTTP-Core.git
cd raku-HTTP-Core

raku -I. Build.rakumod                                          # picks zig cc / cc / gcc / clang
raku -Ilib -e "use HTTP::Core; say core-version(); say library-path()"
zef install .                                                   # builds, tests, installs
raku -e "use HTTP::Core; say core-version(); say library-path()" # now from the site repo

Four differences from the POSIX recipe above:

• CC is an environment variable, not a command prefix. CC="gcc" raku … is POSIX shell syntax and does not exist in PowerShell. Set it first, then run the same command: $env:CC = 'gcc' (set CC=gcc in cmd.exe, Remove-Item Env:\CC to get the default back).
• The compiler is picked, not assumed. With CC unset, Build tries zig cc, cc, gcc, clang and keeps the first one that actually produces a library, so a machine with only llvm-mingw and one with only zig both build without being told anything. It compiles rather than probing with --version: WSL inherits the Windows PATH, so a zig.exe sitting there answers zig version cheerfully and then cannot build an ELF target — the build falls through to cc instead of stopping on the first candidate. zig cc needs nothing else installed: a single zig.exe, no Visual Studio and no runtime redistributable; the DLL imports nothing beyond KERNEL32.dll and the UCRT that ships with Windows 10/11. gcc and clang from llvm-mingw work too and produce a smaller DLL with the same imports. MSVC cl is untested and unnecessary — it wants a developer prompt and its DLL would add a vcruntime dependency.
• The kernel is http_wire.dll, no lib prefix. That is the PE convention and _find-lib follows the platform (stem and extension), so a libhttp_wire.so left in resources/libraries/ by a WSL session is never picked. zig cc also drops http_wire.lib and http_wire.pdb next to it; both are ignored by git and neither is packaged.
• A Windows checkout is byte-identical to a WSL one. .gitattributes pins * text=auto eol=lf, so core.autocrlf does not matter and the kernel never sees a stray \r: the 14 \r\n sequences in http_wire.c are spelled as escapes, and the file itself has no CR bytes.

If zef install . finishes with Testing [OK] and the last command prints an absolute path under C:\rakudo\share\perl6\site\resources\, the install is complete. Anything else — see Troubleshooting.

Quick start

Serve something:

raku -Ilib examples/serve.raku --port=8080
curl -s http://127.0.0.1:8080/health

Request something:

raku -Ilib examples/client.raku --url=http://127.0.0.1:8080/health --verbose
raku -Ilib examples/client.raku --self-test     # no network, no second terminal

Both scripts parse their own arguments; --help prints the generated usage. Option values may be written either way: --port 8080 and --port=8080 are equivalent, and so are --timeout 5 and --timeout=5. Raku's MAIN binds a space-separated value only to a Str option, so the numeric options are declared Str and converted by a small int-option / num-option helper; a value that is not a number is reported with its own name instead of the usage block.

What it does

• Parses a request line and header block out of a flat byte buffer.
• Allocates nothing: every returned string is a copy taken from your buffer.
• Reports the head length so the body can be sliced from the same buffer.
• Distinguishes Incomplete (feed more bytes) from ParseError (give up).
• Runs on MoarVM safely because it does no callbacks and owns no event loop.

It does not do chunked transfer decoding, HTTP/2, URL decoding, or header coalescing. Those belong above this layer.

Why no shim

The one hazard that did bite: NativeCall refuses to pass a native size_t @array to a CArray parameter, so every out-parameter is declared as a bare Pointer and fed with nativecast(Pointer, @native-array). See lib/HTTP/Core/Native.rakumod.

The second hazard, learned the hard way: do not hoist those buffers into state. Doing so made the single-threaded benchmark 17% faster and silently corrupted 47% of concurrent parses, because state is not thread-local in Rakudo 2026.09. t/03-threads.rakutest exists to catch a regression. The full post-mortem is in bench/RESULTS.md §8.

Byte offsets, not graphemes

Str.index counts grapheme clusters, and "\r\n" is a single cluster under UAX #29. So $text.index("\r\n\r\n") on a wire image of 152 bytes returns 141, not 146 — seven CRLFs, seven missing positions. Every offset in this module is a byte offset, so any search for a terminator must run on the Buf, not on a decoded Str. t/06-codec.rakutest has the byte-level helper.

Server

use HTTP::Core::Server;
use HTTP::Core::Response;

my $app = HTTP::Core::Server.new(port => 8080);

$app.router.get:  '/',          -> $req, %p { HTTP::Core::Response.html('<h1>hi</h1>') };
$app.router.get:  '/users/:id', -> $req, %p { 'user ' ~ %p<id> };
$app.router.post: '/echo',      -> $req, %p { $req.body };

$app.start;

A handler may return a Response, a Str (sent as text/plain), or a Blob (sent as octet-stream). Anything else is stringified. A handler that dies turns into a 500 and the connection stays consistent.

What the loop gives you for free:

start only arms the listener and returns immediately. A connect issued in the same tick can still be refused, so scripts either retry the first exchange or sleep one short tick after start — every example here does one of the two.

$app.connections reports live entries in the connection registry, and $app.drop-connections closes everything outstanding. The registry is keyed on a monotonic Connection.serial, not on a socket address, because an address is recycled the moment the old connection is collected — the bug, the fix and the 192/192/0 measurement are in bench/RESULTS-CRO.md.

Deliberately missing: chunked request bodies, TLS, Expect: 100-continue, HTTP/2. For TLS, put this behind a reverse proxy or wrap the accepted socket — do not bind OpenSSL by hand.

Client

The same kernel read backwards, so a client needs one use and two calls:

use HTTP::Core;
use HTTP::Core::Client;

my $bytes = encode-request(
    method  => 'POST',
    target  => '/items?page=2',
    headers => ('Host' => 'example.com', 'Content-Type' => 'application/json').List,
    body    => '{"a":1}',
);

my $res = parse-response($incoming);      # or decode-response, same thing
say $res.status;
say $res.body;
say $res.framing;                          # no-body / content-length / chunked / until-close
say response-extent($res);                 # bytes on the wire, or -1 if unknown from the head

And the pooled form, which is what the client is for:

my $client = HTTP::Core::Client.new(:timeout(5));
my $res = await $client.get('http://127.0.0.1:8080/health');
say $res.status, ' ', $res.body.decode;
$client.close;

Headers are an ordered List of pairs, not a hash: a response may repeat Set-Cookie, and the order of those repeats is meaningful (RFC 6265 §3). header-pairs returns the wire verbatim; header-values('set-cookie') returns every value, while header('X') returns the first.

Content-Length is the one header the codec derives. It is arithmetic on the bytes you handed over, not policy, so: a body with no framing header gets the length added; a caller-supplied Transfer-Encoding is left alone; a caller-supplied Content-Length must match the body or encode-request dies rather than put an unparseable request on the wire. Host and Date remain yours.

Response parsing tells you more than a request parse needs:

parse-response($blob, :head) is the HEAD-request form: the head is parsed but the body is never expected.

Failures are values, not die messages: await gives you an HTTP::Core::X exception (Timeout, Network, Truncated, Redirect, Protocol, InvalidUrl …), all descending from X::Base, so one CATCH can classify the lot.

Examples

registry-watch.raku is the executable form of the leak fix: it opens one keep-alive connection and prints the expected and observed count at four points, each line ending in ok or FAIL. With DEBUG=1 each accept line and its drop line carry the same conn#N; the id= field differs between them for the same connection, which is exactly why the address could not be the registry key.

Tests

prove -e 'raku -Ilib' t/*.rakutest   # the whole suite: 17 files, 101 tests
raku -Ilib t/01-parse.rakutest            # 37 assertions, parser only
raku -Ilib t/02-server.rakutest           # 13 subtests, every request issued by Client
raku -Ilib t/06-codec.rakutest            # 11 subtests: both directions, all four framings
raku -Ilib t/07-live-client.rakutest      # 3 subtests: client codec against a real socket
raku -Ilib t/17-registry-release.rakutest # 4 subtests: the registry gives its entries back

Test files use the .rakutest extension so a harness (prove -e 'raku -Ilib' t/*.rakutest, or anything that globs t/**/*.rakutest) discovers them on its own; .rakumod is reserved for modules. The helpers under t/fixtures/ (CurlCaptures, CurlResponses) stay .rakumod — they are libraries, not tests. prove6 works too if it is installed; plain prove needs no extra distribution.

Layout

Build.rakumod              compiles src/http_wire.c via %*ENV<CC>
src/http_wire.c            the kernel: pure function, zero allocation
src/http_wire.h            C ABI + phr_header layout
lib/HTTP/Core.rakumod      facade: parse-request + self-check
lib/HTTP/Core/Native.rakumod   the only file that knows about pointers
lib/HTTP/Core/Request.rakumod  userspace object, lazily materialised strings
lib/HTTP/Core/Response.rakumod serialisation, status phrases, constructors
lib/HTTP/Core/Codec.rakumod    the four directions: encode/decode, both sides
lib/HTTP/Core/Codec/Content.rakumod  content-encoding negotiation (gzip/deflate)
lib/HTTP/Core/Codec/Raku.rakumod     the string-building oracle the C path is tested against
lib/HTTP/Core/Bytes.rakumod    Str -> UTF-8 without the per-call transcoder
lib/HTTP/Core/Router.rakumod   :params and * wildcards
lib/HTTP/Core/Framing.rakumod     message boundaries: buf in, Outcome out
lib/HTTP/Core/Connection.rakumod  buffer + parked promises, event-driven
lib/HTTP/Core/Transport.rakumod   role: incoming / write / close / id
lib/HTTP/Core/Transport/TCP.rakumod  IO::Socket::Async implementation
lib/HTTP/Core/Transport/TLS.rakumod  the same, behind a handshake
lib/HTTP/Core/Transport/InMemory.rakumod  a socket-less transport for tests
lib/HTTP/Core/Server.rakumod   routing, dispatch, keep-alive policy only
lib/HTTP/Core/Client.rakumod   URL, redirects, deadlines, pooling
lib/HTTP/Core/Pool.rakumod     leases, idle/active accounting, reuse metrics
lib/HTTP/Core/X.rakumod    Incomplete / ParseError / Network / Timeout / ...
tools/                     scratch helpers used while developing the parser
t/                         .rakutest files, one concern each
t/fixtures/                CurlCaptures / CurlResponses: capture fixtures (modules, not tests)
examples/                  runnable scripts, each with --help
docs/API.md                every public method, one table per module
bench/                     RESULTS.md, RESULTS-CRO.md: numbers and post-mortems

API reference

The full surface — every public method, argument and default, one table per module — lives in docs/API.md. The short version:

Troubleshooting

Cannot locate native library 'http_wire' — the kernel was never built, or it sits outside the search path. Build it, or point at it: HTTP_CORE_LIB=$PWD/resources/libraries/libhttp_wire.so raku -Ilib -e 'use HTTP::Core'.

invalid ELF header from inside a precomp — a Windows http_wire.dll and a Linux libhttp_wire.so are sitting in the same directory and the loader grabbed the wrong one. The binding orders candidates by platform (stem and extension both), but an explicit HTTP_CORE_LIB removes all doubt.

'zig' is not recognized / The term 'CC=zig cc' is not recognized — two different Windows mistakes. The first is harmless if any other compiler is present: the probe falls through to cc, gcc, clang and the build carries on. With no compiler at all, install zig (winget install zig.zig, or unzip a release and put zig.exe on PATH), or point CC at one you have. The second is POSIX shell syntax pasted into PowerShell: CC="zig cc" raku … assigns nothing and tries to run a command literally named CC=zig cc; set the variable on its own line.

Could not find Compress::Zlib while installing — HTTP::Core::Codec::Content loads it at the top of the file, and zef compiles every module in provides before it calls the install done, so the module has to be resolvable at that moment. META6.json declares it, which means a modern zef will fetch it; if your zef is old enough to ignore depends, or the dependency lives in the user repository (~/.raku) while zef is staging into site — its throwaway repository chain is staging + site + vendor + core and does not include ~/.raku — install into the user repository as well: zef install --install-to=home ., or zef install --install-to=site Compress::Zlib first.

Cannot locate native library right after zef install . — the installed copy resolves its DLL through the distribution resource table, which zef fills in while staging. A binding that froze the path at BEGIN (a constant LIB) recorded a path inside zef's staging directory, which is deleted the moment the install finishes, so it works during Testing and fails afterwards. Current code resolves on first call (is native(&_find-lib)); if you still see this, a stale lib/HTTP/Core/Native.rakumod is installed — zef uninstall HTTP::Core and install again.

Saw 1 occurrence of deprecated code on every run — Rakudo 2026.09 deprecates every path accessor on Distribution::Resource (.IO, .Str, .absolute, .resolve) and suggests "using %?RESOURCES<key> directly", but that value is a resource object, not a path, and NativeCall needs a string — so the installed-distribution lookup still calls .absolute. The alternatives were tried and do not work: $?DISTRIBUTION.content('resources/libraries/…') returns Any for an installed distribution, and .open just reaches the same deprecated Str from the inside. The notice is noise once per process, it comes from lib/HTTP/Core/Native.rakumod, and it goes away when the replacement accessor lands. To silence it now, tell Rakudo not to report deprecations — RAKUDO_NO_DEPRECATIONS=1 in the environment:

RAKUDO_NO_DEPRECATIONS=1 raku -Ilib -e 'use HTTP::Core; say library-path()'

$env:RAKUDO_NO_DEPRECATIONS = '1'; raku -e "use HTTP::Core; say library-path()"

Nothing is hidden by this beyond the notice: the deprecated call is one line whose behaviour is unchanged, and an empty setting restores the message.

Which library actually loaded? raku -Ilib -e 'use HTTP::Core; say library-path(); say assert-abi()'. assert-abi compares the kernel's ABI constant against the binding's and dies instead of limping along after a partial rebuild.

EADDRINUSE on start — read $app.bind-error for the reason, or pass a port you did not choose: my $port = free-port().

The first request after start is refused — start arms the listener and returns; a connect in the same tick can race the bind. Retry the first exchange (every example does) rather than assuming the server is down.

Offsets look seven bytes short — you searched a decoded Str for "\r\n\r\n". Use a Buf; see Byte offsets, not graphemes.

Concurrent parses return the wrong values — a scratch buffer was hoisted into state, which is not thread-local. Keep buffers per call; t/03-threads.rakutest is the guard.

connections never goes down — that was the registry leak fixed in 55c8bae / c8d9af9: !drop used a Str key against an Int id, threw inside the frame callback, and the entry leaked forever. Run raku -Ilib examples/registry-watch.raku; every line should read ok.

Stale precompilation — rm -rf lib/.precomp and run again. A rebuild of the kernel changes nothing Raku-side, but a partially written precomp after an interrupted run will keep failing until it is removed.

Benchmarks

bench/RESULTS.md holds the parse microbenchmarks — including §8, the state post-mortem, and §12, the cost of encode versus rebuilding a head by hand. bench/RESULTS-CRO.md holds the connection-registry investigation: root cause, the 192/192/0 measurement, and why the key had to change.

Limits

The transport is IO::Socket::Async, i.e. libuv, which MoarVM already owns, so there is exactly one event loop and no thread juggling. Anything that needs C to call back into Raku (a server framework that owns its own loop) still hits the MoarVM limitation and needs a sidecar process instead.

Also out of scope, on purpose: chunked request bodies, Expect: 100-continue, HTTP/2, WebSocket upgrades, and an HTTP client that hides the response bytes. This library is the layer those would be built on, not the layer that hides it.

License

MIT, as declared in META6.json. The full text is in LICENSE.