Revision history for Vips::Native
0.3.0 2026-04-16T02:48:04+01:00
- Root-caused the macOS image-load failures to a C varargs
ABI bug that had been latent across all platforms. libvips's
public API is entirely variadic (`vips_image_new_from_file
(filename, ...)` etc.), and Raku's NativeCall marshals every
declared argument according to the *non*-variadic ABI since
it has no way to mark a binding as variadic. On Apple arm64
specifically, the variadic ABI diverges: named args go in
registers (x0–x7), unnamed args go on the stack. When we
put our NULL terminator in a register, libvips read the
stack looking for the first vararg and found whatever
garbage happened to be there — error messages like
"pngload: no property named `\xa0\x13Yk\x01'" from bytes
that were stack memory, not a property name. Linux x86_64
and Windows x64 happened to work because their variadic
ABIs reuse the same registers as non-variadic for the
first several args.
- Fix: `src/vips_native_shim.c` — a tiny non-variadic shim
that wraps each variadic libvips entry point in C (where
the compiler knows the signature is variadic) and exposes
honest non-variadic symbols Raku's NativeCall can bind
directly. Shipped on all platforms where the ABI matters:
* macOS arm64: `libvips_shim.dylib` compiled in
build-binaries.yml, shipped in the prebuilt tarball.
* Linux x86_64 + aarch64: `libvips_shim.so` compiled
in build-binaries.yml, shipped in the prebuilt tarball.
aarch64 has the same AAPCS64 variadic divergence as
macOS arm64; x86_64 is defence-in-depth (eliminates
latent UB even though the current ABI happens to work).
* Windows: shim not yet compiled in prebuilt (needs
MinGW setup; x64 ABI overlap means tests pass without
it; arm64 Windows Raku doesn't exist yet). TODO
documented in build-binaries.yml.
* System-libvips fallback (all POSIX): Build.rakumod
compiles the shim at install time via `cc` with
`-undefined dynamic_lookup` (macOS) or plain `-shared`
(Linux). Symbols resolve lazily against the system
libvips already loaded by NativeCall. Non-fatal if no
C compiler present — falls back to direct variadic
bindings with a warning about arm64 risk.
FFI.rakumod detects the shim at module load and routes
the variadic entry points through it when present; falls
back to direct libvips bindings otherwise. Raku-facing
signatures unchanged — tests didn't need to move.
- test.yml: run prebuilt-path install + tests *before*
`brew install vips` / `apt install libvips-dev` so the
self-contained bundle is exercised with no system libvips
present. Also saves several minutes on every failed run
since the slow system-libvips install is only reached
after the fast path passes.
- BINARY_TAG bumped to binaries-vips-8.18.2-r7 (r6 shipped
without the shim on Linux/Windows; r7 includes it on all
platforms where we compile it).
- macOS build-binaries.yml: recursive transitive-dep bundler
for vips-modules. dylibbundler only follows link-time deps,
so the vips-* modules (heif/jxl/magick/openslide/poppler) —
which libvips dlopens at runtime — left their own deps
(libheif, libjxl, libMagickCore, libopenslide, libpoppler
and everything those transitively pulled, incl. libgio via
libpoppler-glib) pointing at Homebrew absolute paths. At
runtime on macOS, that meant dyld loaded Homebrew copies of
libgio alongside ours, lighting up the duplicate-ObjC-class
and duplicate-GObject-type registration paths and breaking
image loads on runners whose Homebrew glib happens to match
our bundled version. The new recursive pass walks every
module's otool -L deps, copies missing transitive deps into
bundle/ at top level with @loader_path-relative install_names,
and rewrites every ref so the whole bundle is self-contained.
Follows symlinks to copy the real file (keeps the ref-site
basename). Best-effort on @rpath refs via a brew-prefix find.
Post-bundle audit fails the build if any Mach-O still
references /opt/homebrew or /usr/local/Cellar — ship-time
guarantee that no surprise Homebrew lib gets loaded at
runtime. User-visible effect: full HEIF / JXL / Magick /
OpenSlide / Poppler support in the bundled install,
previously only available on the system-libvips fallback
path.
- BINARY_TAG bumped to binaries-vips-8.18.2-r5.
- FFI.rakumod: env var setup (VIPSHOME, GIO_MODULE_DIR) now
goes through a libc setenv(3) NativeCall, not Raku's %*ENV.
Raku's %*ENV = Y on macOS writes to a Raku-internal env
table that *doesn't* propagate to Darwin's __environ array
— verified empirically: `getenv("VIPSHOME")` returns NULL
inside the same process where `%*ENV` shows a
value. Same class of issue as Windows' CRT-vs-kernel32 split
we hit earlier for DLL search. libvips / GLib / everything
linked against libc reads via getenv(3), so we go through
libc directly. %*ENV is still updated too, so subprocess
spawns inherit a correct environ. This was the root cause
of the r3/r4 macOS test failures — VIPSHOME + GIO_MODULE_DIR
were set from Raku's perspective but completely invisible
to libvips, so libvips kept falling back to its compile-time
Homebrew prefix and loading Homebrew's modules (→ Homebrew's
libgio via transitive loads → ObjC dup-class crash).
- macOS: third pass on the r3 bundle fixes. r3 extracted
cleanly and libvips loaded, but libvips then searched
Homebrew's vips-modules- dir at init time and dlopened
Homebrew's modules — dragging in Homebrew's libgio
(duplicate ObjC class registration → GIO module scan bypass
didn't help, because the modules were vips's not GIO's).
Root-caused via VIPS-INFO diagnostics:
* I wrongly assumed libvips honours a VIPS_MODULEDIR env
var. It doesn't — libvips's `vips_guess_prefix` checks
VIPSHOME → probes argv[0] on PATH → falls back to
compile-time prefix. When argv[0] is `raku`, probe
fails; with no VIPSHOME set, libvips landed at Homebrew's
compile-time prefix. FFI.rakumod now sets VIPSHOME to
the staged parent dir; respects pre-set VIPSHOME.
* Our bundled libvips.42.dylib had LC_ID_DYLIB still set
to `/opt/homebrew/opt/vips/lib/libvips.42.dylib` —
dylibbundler only rewrites deps on its -x target, not
the target's own install_name. Added an explicit
install_name_tool -id '@loader_path/libvips.42.dylib'
post-dylibbundler so dyld identifies the bundled copy
as authoritative and doesn't double-load Homebrew's.
* Bundled modules dir was named `vips-modules` — libvips
computes the path as
`$VIPSHOME/lib/vips-modules-./` with the
version suffix. Preserve the exact dir name from the
source (`$(basename $MODULES_SRC)` → `vips-modules-8.18`
for Homebrew 8.18, `vips-modules-8.18` from build-win64-
mxe) so the runtime lookup resolves.
- BINARY_TAG bumped to binaries-vips-8.18.2-r4.
- macOS: point GIO_MODULE_DIR at a guaranteed-nonexistent path
(`/dev/null/vips-native-no-gio-modules`) at module load. Our
bundled libgio was compiled with GIO_MODULE_DIR baked to the
Homebrew cellar used to produce the bundle; at g_type_init
time, GIO scanned that dir and loaded extension modules built
against the *builder's* libgio, ending with two
libgio-2.0.0.dylibs in-process and duplicate ObjC class
registration (GNotificationCenterDelegate etc.) — which macOS
treats as a fatal load error. libvips only needs GIO's core
type system (statically linked into libgio itself); the
extension modules aren't touched by our tests, so disabling
module loading is safe. Applies on Linux too (same mechanism,
just doesn't crash without ObjC) for symmetry. Respects a
pre-set GIO_MODULE_DIR if the user wants to override.
- Second pass on the r2 macOS + Windows bundle fixes. r2
shipped broken:
* macOS: the modules-pass `dylibbundler -d bundle` call
re-entered bundle/ and rewrote libvips.42.dylib's own
install_names with the modules' `@loader_path/../`
prefix, breaking sibling-dep resolution (every lookup
landed in the parent dir where nothing was staged).
Replaced with direct install_name_tool -change walks
per module — bundle/ is now read-only with respect to
the modules pass.
* Windows: %*ENV prepend didn't propagate to the
Win32 loader's DLL search. Added a direct
SetDllDirectoryW call via kernel32 (always pre-loaded)
alongside the PATH prepend so LoadLibrary finds bundled
sibling DLLs regardless of CRT/env-propagation state.
- BINARY_TAG bumped to binaries-vips-8.18.2-r3. The r2 release
artefacts remain published but are known-broken; do not
use. checksums.txt will refresh on the r3 release.
0.2.2 2026-04-16T00:29:10+01:00
- Fix macOS + Windows test failures against the
binaries-vips-8.18.2-r1 archives by reworking the bundle
layout and adding runtime env setup.
* macOS: libvips 8.15+ splits format loaders (pngload,
jpegload, heifload, …) into dlopen'd modules under
$VIPS_MODULEDIR. dylibbundler only follows link-time
deps so these never made it into the r1 bundle; libvips
loaded fine, but vips_image_new_from_file() silently
returned NULL on every format. build-binaries.yml now
stages $VIPS_CELLAR/lib/vips-modules-*/ into
bundle/vips-modules/, rebundles each module's refs via
`dylibbundler -p '@loader_path/../'`, and codesigns them.
* Windows: NativeCall loads libvips-42.dll by absolute
path, which makes Windows resolve libvips's own DLL
deps starting from raku.exe's directory rather than
libvips-42.dll's — so every sibling DLL in the staged
lib dir was invisible and libvips-42.dll failed to
load with ERROR_MOD_NOT_FOUND (0x7e). FFI.rakumod now
prepends the staged lib dir to %*ENV at module
load time (before any `is native(...)` binding fires).
build-binaries.yml also now includes bin/vips-modules/
from build-win64-mxe, which the r1 copy-glob missed.
* All platforms: FFI.rakumod sets %*ENV
to /vips-modules when the dir exists. No-op on
the conda-forge Linux build (loaders baked in), load-
bearing on macOS/Windows, forward-compatible if any
future Linux bundle source switches to split modules.
Respects VIPS_NATIVE_LIB_DIR / VIPS_MODULEDIR overrides.
- BINARY_TAG bumped to binaries-vips-8.18.2-r2 to force a
re-download; r1 archives remain broken and should not be
used. resources/checksums.txt will refresh on the next
release.
0.2.1 2026-04-16T00:01:51+01:00
- Build.rakumod: detect system glibc via `ldd --version` and
skip the prebuilt download when it's older than the prebuilt
target (currently v2.35, matching the ubuntu-22.04 CI runner).
Previously, users on Ubuntu 20.04 / Debian 11 / RHEL 8
downloaded a prebuilt libvips that loaded but failed at first
symbol use with "GLIBC_2.xx not found". The guard fires before
the download and routes to this module's existing system-
libvips fallback (Native.rakumod resolves via the OS dynamic
loader) — users just need `libvips` from their distro.
VIPS_NATIVE_BINARY_ONLY=1 now hard-fails with a clear message
on old-glibc systems rather than producing a broken install.
- New CI workflow .github/workflows/glibc-fallback.yml: runs
`zef install .` inside an ubuntu:20.04 container (glibc 2.31,
with apt-installed libvips42) and asserts both that the
fallback message appears in the build log and that vips_init()
against the system libvips succeeds.
0.2.0 2026-04-15T04:11:52+01:00
- Prebuilt-binary-first install path. Build.rakumod now attempts
to download a per-platform archive from the repo's GitHub
Releases (libvips + libgobject + libglib + the format-handling
libs for jpeg / png / webp / tiff / gif / heif / avif, all
relocated to load via @loader_path / $ORIGIN / sibling-DLL),
before falling back to the original behaviour of resolving
'vips' and 'gobject-2.0' via the OS dynamic loader from a
system-installed libvips.
- Sources we repackage from (the trust chain): Homebrew bottle
for macOS arm64, conda-forge for Linux x86_64 / aarch64,
libvips/build-win64-mxe for Windows x86_64. We don't compile
vips ourselves — the upstream sources are reputable and well-
tested, and rebuilding vips's dep tree from source in CI
would be a maintenance nightmare.
- SHA256 verification against bundled resources/checksums.txt;
refuses any prebuilt whose hash isn't recorded. Empty /
unverifiable hashes fail safe to system-libvips fallback.
- Stage native libs to $XDG_DATA_HOME/Vips-Native//
lib/ (or $LOCALAPPDATA on Windows, ~/.local/share fallback)
under their real filenames — NOT into META6 resources, because
zef hashes resource filenames and that breaks vips's inter-lib
refs (libvips needs libgobject-2.0.0.so next to it on disk by
that exact name). Same fix Notcurses-Native uses.
- Refactored lib/Vips/Native/FFI.rakumod resolver: env override
(VIPS_NATIVE_LIB_DIR) → XDG-staged dir (versioned-aware
filename matching) → bare short name for OS dynamic loader.
Drops the hard MacOS::NativeLib dependency since the resolver
now handles macOS lib lookup itself; macOS users still get
Homebrew vips just as cleanly via the fallback path.
- New BINARY_TAG file at repo root + resources/BINARY_TAG (the
tiny text file that survives zef's resource-hashing intact)
as single source of truth for the pinned binary release tag.
- New .github/workflows/build-binaries.yml: dispatches per
libvips version, downloads from each upstream source, repack-
ages with @loader_path / $ORIGIN / sibling-DLL relocation,
publishes to GitHub Releases.
- Test workflow now exercises both paths: prebuilt-via-zef-
install AND a second pass with VIPS_NATIVE_PREFER_SYSTEM=1.
- New env knobs: VIPS_NATIVE_PREFER_SYSTEM=1 to skip prebuilt
download; VIPS_NATIVE_BINARY_ONLY=1 to refuse the system
fallback; VIPS_NATIVE_BINARY_URL to override release base URL;
VIPS_NATIVE_CACHE_DIR for download cache; VIPS_NATIVE_DATA_
DIR for staged-libs base; VIPS_NATIVE_LIB_DIR for runtime
lib-dir override.
- All five platforms in the prebuilt matrix: macOS arm64, Linux
x86_64 + aarch64 glibc, Windows x86_64 + arm64.
0.1.3 2025-07-10T16:06:03+01:00
- Document script usage
- Make script die with a useful message when params not provided
0.1.2 2025-07-10T15:48:10+01:00
- Add vips install info to readme
0.1.1 2025-07-10T15:45:21+01:00
- Remove windows from test runner, installing vips on github actions is too awkward
0.1.0 2025-07-10T15:43:17+01:00
- Initial version