|
libmizu 0.0.1
Lock-free shared-memory IPC
|
libmizu binding-author API (the unstable tier). More...

Go to the source code of this file.
Data Structures | |
| struct | mizu_read_ctx_s |
| struct | mizu_binding_s |
| struct | mizu_bytes_s |
| A byte buffer. More... | |
| struct | mizu_shm_s |
| The public opaque type in mizu.h; defined here for bindings (the core's control-region mappings embed one in the handle structs). More... | |
| struct | mizu_parker_s |
| One parker per waiting entity: a 32-bit monotonic epoch word in the shared region plus, on Windows only, one named auto-reset event (the epoch compare is not atomic with the sleep there). More... | |
| struct | mizu_pool_sig_s |
| The opaque pool-signal trio a map runner loads relaxed once per batch transition: the help_wanted doorbell, the pool's shared shutdown word, and the handle's process-local listener-written owner_dead flag. More... | |
| struct | mizu_morsel_hdr_s |
| The 128-byte region header. More... | |
| struct | mizu_morsel_sizer_s |
| Batch-sizing state: process-private, never wire state, reset at each run's first-call CLAIM CAS. More... | |
| struct | mizu_morsel_span_s |
| Worker-death lost set: issued = [0, bound), lost = issued minus the union of the collected batch histories — a batch in no history was issued but never completed (its claimant died, or the map fn errored mid-batch); a dead runner's whole history lands here too — it publishes only at exhaustion. More... | |
| struct | mizu_result_sink_s |
| The result-sink struct: the core fills it from a claimed entry and hands it to exec_fn; the binding passes it back to the publish verbs. More... | |
| struct | mizu_mizs_geom_s |
| The small pure helpers over mizu.h's wire-format conventions, dual form like the zc accessors above (see the banner): the timeout unit conversion, the NA_real_ store (for a binding filling converted output without R headers), the aux packing of the pool RAWSPILL and SHM_VEC kinds and its decode pair, the flat MIZH header write/check, the MIZS / MIZL layout checks, the validity-section setter, and the NA sentinel <-> bitmap primitives (one NA-test implementation behind both bindings' validity paths, so two bindings' bitmaps cannot drift apart from the sentinels they describe). More... | |
| struct | mizu_mizl_entry_s |
| One MIZL directory entry plus its validity-table pair (mizu.h documents the 32-byte entry): mizu_mizl_elem's out-param. More... | |
| struct | mizu_ix_item_s |
| struct | mizu_ix_s |
| The pull cursor. More... | |
| struct | ArrowSchema |
| struct | ArrowArray |
| struct | ArrowArrayStream |
| struct | mizu_cvt_row_s |
| struct | mizu_arrow_warn_s |
| The conversion warning counts, accumulated by mizu_cvt_convert and the frame writer; the binding raises them after the stage returns, never mid-stage. More... | |
| struct | mizu_arrow_col_s |
Macros | |
| #define | MIZU_EXT_INLINE static inline |
| Some static inlines below go unused in some TUs; the amalgamation folds this header into the single mizu.c TU, where they would trip -Wunused-function (header inlines are exempt only in a real header). | |
| #define | MIZU_EXT_LOAD_ACQ(p) atomic_load_explicit((p), memory_order_acquire) |
| The dual-form inlines touch atomics; spell the acquire load per language (mizu.h's MIZU_ATOMIC gives C++ consumers std::atomic). | |
| #define | MIZU_PREFIX_LITERAL "/mizu_" |
| #define | MIZU_ALIGN64(x) (((x) + 63) & ~(size_t) 63) |
| #define | MIZU_ZC_FLOOR ((size_t) 32768) |
| Staging-policy floors for a binding's stage_fn (not used by the core): SHM_VEC escalates only past max(inline budget, MIZU_ZC_FLOOR); the channel's raw floor is higher (the arena copy has no region machinery to amortize) and lifts entirely under the churn signal. | |
| #define | MIZU_ZC_FLOOR_RAW ((size_t) (256 << 10)) |
| #define | MIZU_OPEN_CACHE_MAX 16 |
| The core's consumer SHM_RAW mapping cache size; a binding sizes its own view cache to match. | |
| #define | MIZU_CODEC_MAGIC 0x52u |
| The self-describing stream dispatch byte: payload byte 0 of an INLINE frame, where an R native stream carries 'B'/'X'. | |
| #define | MIZU_PYMIZU_CODEC_MAGIC 0x50u |
| 'P' | |
| #define | MIZU_INTEROP_MAGIC 0x49u |
| 'I': the interchange stream (DESIGN.md's Interchange codec section) | |
| #define | MIZU_LANG_NONE 0u |
| The language registry: the identity word's byte 0, the pool's worker identity word, and each binding's handle-level peer_lang. | |
| #define | MIZU_LANG_BYTES 1u |
| the core's bytes binding and test bindings | |
| #define | MIZU_LANG_R 2u |
| #define | MIZU_LANG_PYTHON 3u |
| #define | MIZU_CAP_MIZS (1u << 0) |
| Reader capabilities: a 32-bit mask, one bit per layout or format extension a reader implements beyond the baseline (MIZH atomic and INT64 views, 'I' format 0x01 as specified in DESIGN.md's Interchange codec section). | |
| #define | MIZU_CAP_ATTRS (1u << 1) |
| reads an 'I' attribute blob on a layout root or MIZL leaf | |
| #define | MIZU_CAP_MIZL (1u << 2) |
| wraps a generic MIZL tree as views | |
| #define | MIZU_CAP_TASKREF (1u << 3) |
| reads 0x13 ref leaves: task arguments by reference (task streams and the map descriptor, which shares the value grammar) | |
| #define | MIZU_CAP_MIZL_REF (1u << 4) |
| reads MIZL remote leaves — directory tag 33 | |
| #define | MIZU_CAP_MIZL_TSN (1u << 5) |
| reads MIZL INT64 leaves carrying a tscale attribute — verbatim nanosecond timestamps and durations | |
| #define | MIZU_CAP_MIZL_U64V (1u << 6) |
| reads MIZL INT64 leaves carrying a u64 attribute — verbatim uint64 frame columns | |
| #define | MIZU_IDENT(lang, caps) ((uint64_t) (uint8_t) (lang) | ((uint64_t) (uint32_t) (caps) << 32)) |
| The identity word: the language in bits 0-7, the 32-bit capability mask in bits 32-63, bits 8-31 reserved — written zero and ignored by readers (a negotiation word, so a later field needs no reader taught to skip it). | |
| #define | MIZU_READ_CONSUME 1u |
| read_fn flags (ctx->flags, zero at each read's start): CONSUME makes a NULL return consume the slot exactly as on success while the verb still returns MIZU_ERR — the foreign/corrupt-payload contract; see the seam comment above. | |
| #define | MIZU_MORSEL_CANCEL_OFF ((uint64_t) 0) |
| A binding's parallel map rides one fresh region per map call: a 128-byte header, the language-specific descriptor stream, an optional bare-bytes x section, the morsel state, and an optional template output area. | |
| #define | MIZU_MORSEL_GEN_OFF ((uint64_t) 4) |
| #define | MIZU_MORSEL_CURSOR_OFF ((uint64_t) 64) |
| #define | MIZU_MORSEL_CLAIM_OFF ((uint64_t) 128) |
| #define | MIZU_MORSEL_GEN_MASK ((uint32_t) 0x3FFFFFFF) |
| Generation comparisons mask to the CLAIM word's 30 bits (wrap takes 2^30 resets of one region: harmless). | |
| #define | MIZU_MORSEL_T_TARGET 200e-6 |
| Batch sizing policy (mizu_morsel_next): k targets a batch duration, growing at most 2x per step and shrinking immediately on overshoot, clamped to the cap — which bounds lost-set coarseness and the ramp worst case. | |
| #define | MIZU_MORSEL_BATCH_CAP 64 |
| #define | MIZU_MORSEL_MAGIC 0x4D495A4Du |
| The one map-region magic: "MIZM" read as hex, the core's magic packing (MIZU_MAGIC, the pool magic, MIZH/MIZS/MIZL). | |
| #define | MIZU_IX_VERSION 0x01u |
| The byte-level half of DESIGN.md's Interchange codec section: a validating pull cursor (mizu_ix_open / mizu_ix_next / mizu_ix_end, in src/interop.c) and the dual-form emit helpers below, shared by every binding so the wire grammar has exactly one implementation. | |
| #define | MIZU_IX_DEPTH_MAX 64 |
| #define | MIZU_IX_ERR_OVERHEAD 25u |
| The err framer's budget constants (mizu_ix_write_err below) — the wire authority for the bounded err item: magic + version + tag + flags + index + three counted lengths of headroom, and the type span's share cap. | |
| #define | MIZU_IX_ERR_TYPE_SHARE 128u |
| #define | MIZU_IX_ATTR_NAMES "names" |
| The attribute vocabulary (DESIGN.md's byte-shape helper registry): the shape whitelist's wire strings — attr keys, class strings and difftime units. | |
| #define | MIZU_IX_ATTR_LEVELS "levels" |
| #define | MIZU_IX_ATTR_DIM "dim" |
| #define | MIZU_IX_ATTR_CLASS "class" |
| #define | MIZU_IX_ATTR_UNITS "units" |
| #define | MIZU_IX_ATTR_ROWNAMES "row.names" |
| #define | MIZU_IX_ATTR_TZONE "tzone" |
| #define | MIZU_IX_ATTR_TSCALE "tscale" |
| #define | MIZU_IX_ATTR_U64 "u64" |
| #define | MIZU_IX_CLASS_FACTOR "factor" |
| #define | MIZU_IX_CLASS_DATE "Date" |
| #define | MIZU_IX_CLASS_POSIXCT "POSIXct" |
| #define | MIZU_IX_CLASS_POSIXT "POSIXt" |
| #define | MIZU_IX_CLASS_DIFFTIME "difftime" |
| #define | MIZU_IX_CLASS_DATAFRAME "data.frame" |
| #define | MIZU_IX_UNIT_SECS "secs" |
| #define | MIZU_IX_UNIT_MINS "mins" |
| #define | MIZU_IX_UNIT_HOURS "hours" |
| #define | MIZU_IX_UNIT_DAYS "days" |
| #define | MIZU_IX_UNIT_WEEKS "weeks" |
| #define | ARROW_C_DATA_INTERFACE |
| The Arrow C Data Interface structs, defined per the spec under its two include guards (the layout is frozen ABI; no headers, no dependency): ArrowSchema / ArrowArray under ARROW_C_DATA_INTERFACE, ArrowArrayStream under ARROW_C_STREAM_INTERFACE (nanoarrow guards the stream struct separately; one shared guard would leave it redefinable by a later nanoarrow or arrow include). | |
| #define | ARROW_C_STREAM_INTERFACE |
| #define | MIZU_ARROW_FLAG_DICTIONARY_ORDERED 1 |
| The ArrowSchema flags word bits the writer reads (the spec's values). | |
| #define | MIZU_ARROW_FLAG_NULLABLE 2 |
Typedefs | |
| typedef struct mizu_handle_s | mizu_handle |
| Generic handle view for the stager services; the core passes the handle a binding's callbacks were registered with. | |
| typedef struct mizu_result_sink_s | mizu_result_sink |
| Valid only during an exec callback. | |
| typedef int(* | mizu_stage_fn) (void *obj, mizu_slot_hdr *hdr, uint8_t *payload, uint32_t inline_max, mizu_handle *, void *ctx) |
| The core never sees a language object. | |
| typedef struct mizu_read_ctx_s | mizu_read_ctx |
| typedef void *(* | mizu_read_fn) (const mizu_slot_hdr *, const uint8_t *payload, size_t limit, mizu_read_ctx *) |
| typedef int(* | mizu_exec_fn) (const mizu_slot_hdr *hdr, const uint8_t *payload, size_t limit, mizu_result_sink *, int catching, mizu_read_ctx *ctx) |
| typedef int(* | mizu_check_fn) (void *ctx) |
| typedef void(* | mizu_park_fn) (void *ctx, int entering) |
| typedef void(* | mizu_sweep_fn) (void *ctx) |
| typedef void(* | mizu_drop_fn) (void *ctx, void *pin) |
| typedef struct mizu_binding_s | mizu_binding |
| typedef struct mizu_bytes_s | mizu_bytes |
| A byte buffer. | |
| typedef struct mizu_parker_s | mizu_parker |
| One parker per waiting entity: a 32-bit monotonic epoch word in the shared region plus, on Windows only, one named auto-reset event (the epoch compare is not atomic with the sleep there). | |
| typedef enum mizu_park_result_e | mizu_park_result |
| typedef struct mizu_death_watch_s | mizu_death_watch |
| Translates a watched pid's exit into *flag = 1 plus a directed unpark of pk (optional, copied). | |
| typedef enum mizu_live_probe_e | mizu_live_probe |
| Exclusive flock (POSIX) / LockFileEx (Windows) held for a process's entire lifetime and released by the kernel on any exit path. | |
| typedef struct mizu_pool_sig_s | mizu_pool_sig |
| The opaque pool-signal trio a map runner loads relaxed once per batch transition: the help_wanted doorbell, the pool's shared shutdown word, and the handle's process-local listener-written owner_dead flag. | |
| typedef struct mizu_morsel_hdr_s | mizu_morsel_hdr |
| The 128-byte region header. | |
| typedef struct mizu_morsel_sizer_s | mizu_morsel_sizer |
| Batch-sizing state: process-private, never wire state, reset at each run's first-call CLAIM CAS. | |
| typedef struct mizu_morsel_span_s | mizu_morsel_span |
| Worker-death lost set: issued = [0, bound), lost = issued minus the union of the collected batch histories — a batch in no history was issued but never completed (its claimant died, or the map fn errored mid-batch); a dead runner's whole history lands here too — it publishes only at exhaustion. | |
| typedef struct mizu_mizs_geom_s | mizu_mizs_geom |
| The small pure helpers over mizu.h's wire-format conventions, dual form like the zc accessors above (see the banner): the timeout unit conversion, the NA_real_ store (for a binding filling converted output without R headers), the aux packing of the pool RAWSPILL and SHM_VEC kinds and its decode pair, the flat MIZH header write/check, the MIZS / MIZL layout checks, the validity-section setter, and the NA sentinel <-> bitmap primitives (one NA-test implementation behind both bindings' validity paths, so two bindings' bitmaps cannot drift apart from the sentinels they describe). | |
| typedef struct mizu_mizl_entry_s | mizu_mizl_entry |
| One MIZL directory entry plus its validity-table pair (mizu.h documents the 32-byte entry): mizu_mizl_elem's out-param. | |
| typedef struct mizu_ix_item_s | mizu_ix_item |
| typedef struct mizu_ix_s | mizu_ix |
| The pull cursor. | |
| typedef struct ArrowSchema | ArrowSchema |
| typedef struct ArrowArray | ArrowArray |
| typedef struct ArrowArrayStream | ArrowArrayStream |
| typedef struct mizu_cvt_row_s | mizu_cvt_row |
| typedef struct mizu_arrow_warn_s | mizu_arrow_warn |
| The conversion warning counts, accumulated by mizu_cvt_convert and the frame writer; the binding raises them after the stage returns, never mid-stage. | |
| typedef struct mizu_arrow | mizu_arrow |
| The Arrow -> MIZL frame writer: a classified, batch-vetted frame plus its size-pass results, staged through mizu_arrow_stage. | |
| typedef struct mizu_arrow_col_s | mizu_arrow_col |
Enumerations | |
| enum | { MIZU_HTYPE_CHANNEL = 1 , MIZU_HTYPE_POOL = 2 } |
| Handle-kind discriminator. More... | |
| enum | mizu_park_result_e { MIZU_PARK_WOKEN = 0 , MIZU_PARK_TIMEOUT , MIZU_PARK_INTR } |
| enum | mizu_live_probe_e { MIZU_LIVE_ACQUIRED = 0 , MIZU_LIVE_HELD = 1 } |
| Exclusive flock (POSIX) / LockFileEx (Windows) held for a process's entire lifetime and released by the kernel on any exit path. More... | |
| enum | { MIZU_MORSEL_IDLE = 0 , MIZU_MORSEL_RUNNING , MIZU_MORSEL_ABANDONED } |
| enum | { MIZU_MORSEL_X_DESC = 0 , MIZU_MORSEL_X_RAW } |
| enum | { MIZU_IX_TAG_NIL = 0x00 , MIZU_IX_TAG_LGL1 = 0x01 , MIZU_IX_TAG_INT = 0x02 , MIZU_IX_TAG_REAL = 0x03 , MIZU_IX_TAG_STR = 0x04 , MIZU_IX_TAG_BYTES = 0x05 , MIZU_IX_TAG_LGLV = 0x06 , MIZU_IX_TAG_INTV = 0x07 , MIZU_IX_TAG_REALV = 0x08 , MIZU_IX_TAG_CPLXV = 0x09 , MIZU_IX_TAG_RAWV = 0x0a , MIZU_IX_TAG_STRV = 0x0b , MIZU_IX_TAG_LIST = 0x0c , MIZU_IX_TAG_DICT = 0x0d , MIZU_IX_TAG_I64V = 0x0e , MIZU_IX_TAG_ATTR = 0x0f , MIZU_IX_TAG_CPLX = 0x10 , MIZU_IX_TAG_ERR = 0x11 , MIZU_IX_TAG_TASK = 0x12 , MIZU_IX_TAG_REF = 0x13 } |
| The wire tags (DESIGN.md's tag table). | |
| enum | { MIZU_IX_NIL = 0 , MIZU_IX_LGL , MIZU_IX_INT , MIZU_IX_REAL , MIZU_IX_CPLX , MIZU_IX_STR1 , MIZU_IX_STR , MIZU_IX_BYTES , MIZU_IX_VEC , MIZU_IX_STRV , MIZU_IX_LIST , MIZU_IX_DICT , MIZU_IX_ATTR , MIZU_IX_ERR , MIZU_IX_TASK , MIZU_IX_REF } |
| What mizu_ix_next yields. More... | |
| enum | { MIZU_CVT_COPY = 0 , MIZU_CVT_I8_INT , MIZU_CVT_I16_INT , MIZU_CVT_U16_INT , MIZU_CVT_U32_REAL , MIZU_CVT_U64_REAL , MIZU_CVT_F16_REAL , MIZU_CVT_F32_REAL , MIZU_CVT_BOOL8_LGL , MIZU_CVT_BIT_LGL , MIZU_CVT_C64_CPLX } |
| The fixed-width conversion rows: an element width pair plus the kernel, producing a MIZU_TYPE_* leaf. More... | |
| enum | { MIZU_ARROW_CVT = 0 , MIZU_ARROW_STR , MIZU_ARROW_DICT , MIZU_ARROW_DATE , MIZU_ARROW_TS , MIZU_ARROW_TD } |
| The per-column classification, mizu_arrow_col_info's out-param: the kind, the conversion row, and the string-form particulars a binding's own emitters key on. More... | |
Functions | |
| int | mizu_handle_kind (const mizu_handle *) |
| Handle queries standing in for field access into the private handle struct. | |
| int | mizu_handle_churn (const mizu_handle *) |
| int64_t | mizu_handle_keep_out (const mizu_handle *) |
| The tx-keeper count: outstanding staging retains on a channel handle (the reap gate's own count, so side-effect free). | |
| void | mizu_handle_spill_info (const mizu_handle *, uint32_t *fl_entries, uint32_t *ledger_entries) |
| void | mizu_binding_init (mizu_binding *) |
| Zero and size-stamp a binding struct. | |
| uint64_t | mizu_channel_peer_ident (const mizu_channel *) |
| The identity words (the language registry above). | |
| uint64_t | mizu_pool_worker_ident (const mizu_pool *) |
| void * | mizu_stage_arena_alloc (mizu_handle *, size_t n, uint64_t *off) |
| Core services for a stager (per handle), valid only during a stage_fn call — staging is single-threaded per handle role, so at most one checkout is in flight per handle: | |
| mizu_status | mizu_stage_spill_get (mizu_handle *, size_t n, mizu_shm **out) |
| void | mizu_stage_retain (mizu_handle *, mizu_shm *region) |
| void | mizu_stage_retain_zc (mizu_handle *, mizu_shm *region) |
| void | mizu_stage_pin (mizu_handle *, void *pin) |
| void | mizu_stage_reap (mizu_handle *) |
| void | mizu_stage_rollback (mizu_handle *) |
| void * | mizu_stage_raw_spill (mizu_handle *, uint64_t n, int wire_type, mizu_slot_hdr *, uint8_t *payload, uint32_t inline_max) |
| Raw-tier reservation (stage_raw.c), valid only during stage_fn: choose the tier for n bare bytes of wire_type and return the destination with hdr stamped — the caller memcpys n bytes into it. | |
| static void * | mizu_stage_raw (mizu_handle *h, uint64_t n, int wire_type, mizu_slot_hdr *hdr, uint8_t *payload, uint32_t inline_max) |
| mizu_shm * | mizu_read_region (mizu_read_ctx *, const uint8_t *name, uint32_t len) |
| Read-side service, invoked through the read_ctx handed to read_fn: a borrowed consumer mapping for a SHM_RAW-class payload name, from the handle's open cache (open/fstat/mmap on a miss, LRU-evicted). | |
| int | mizu_result_publish (mizu_result_sink *, void *value) |
| Result publish, valid during exec only. | |
| int | mizu_result_publish_err (mizu_result_sink *, void *flattened, uint32_t inline_n) |
| void | mizu_result_publish_lost (mizu_result_sink *) |
| void | mizu_binding_bytes (mizu_binding *) |
Fill binding with the bytes binding: stage/read only — exec, check, park, sweep, drop are NULL (a bytes handle is a channel peer or a pool submitter, never a worker). | |
| void | mizu_bytes_free (mizu_bytes *) |
| int | mizu_parker_attach (mizu_parker *pk, uint32_t *epoch, const char *region_name, int entity, int create) |
| region_name/entity name the Windows event ("<region>.pk.<entity>"), created by the region's host (create = 1) and opened by name by attachers; unused on POSIX. | |
| void | mizu_parker_detach (mizu_parker *pk) |
| int | mizu_park (mizu_parker *pk, uint32_t snapshot, long timeout_ms) |
| Sleeps while the epoch still equals snapshot, up to timeout_ms (0 = poll: never sleeps; < 0 = indefinite — on POSIX an untimed wait would silently restart under SA_RESTART and swallow an interrupt until the next genuine wake, so indefinite parks are internally bounded and rely on directed unparks). | |
| void | mizu_unpark (mizu_parker *pk) |
| static uint32_t | mizu_parker_snapshot (const mizu_parker *pk) |
| The park handshake's first step (dual form — see the banner). | |
| mizu_death_watch * | mizu_death_watch_start (long pid, int *flag, const mizu_parker *pk) |
| void | mizu_death_watch_stop (mizu_death_watch *w) |
| void | mizu_death_listener_teardown (void) |
| Library-unload teardown; joins the Linux epoll thread (no-op elsewhere: macOS dispatch sources and Windows thread-pool waits are per-watch). | |
| const char * | mizu_live_dir (void) |
| Directory for liveness lock files: the MIZU_LIVENESS_DIR override (read-through, checked every call) else a per-platform default resolved once — /dev/shm on Linux, the per-user temp dir on macOS and Windows. | |
| int | mizu_live_open (const char *path, intptr_t *out) |
| int | mizu_live_try (intptr_t h) |
| void | mizu_live_close (intptr_t h) |
| void | mizu_preamble_write (void *region, const mizu_preamble *p) |
| Host writes at create, immutable thereafter; the peer validates before any shared atomic is read or written. | |
| const char * | mizu_preamble_validate (const void *region, size_t region_size, mizu_preamble *out) |
| mizu_pool_sig * | mizu_pool_signals (mizu_pool *) |
| A malloc'd copy of the signal trio (the caller frees). | |
| int | mizu_pool_help_once (mizu_pool *) |
| One doorbell help beat (claims a map runner and re-homes it onto the helper's own deque) and the test-harness injection pull. | |
| int | mizu_pool_deque_pull (mizu_pool *, uint32_t n) |
| int | mizu_pool_map_caps (mizu_pool *, uint32_t *free_rs, uint32_t *inj_cap, uint32_t *inline_entry) |
| A map's batch-sizing inputs: claims the worker's nested-submitter slot when unclaimed (as a first nested submit does), then reports the caller's FREE result slots, the injection cap, and the entry inline budget. | |
| mizu_status | mizu_pool_submit_flags (mizu_pool *, void *task_obj, uint16_t flags, mizu_task *out, double timeout_ms) |
| The map runner's submit: mizu_pool_submit plus the entry flags (MIZU_ENTRY_RUNNER marks a map's join ticket — a doorbell help beat re-homes it onto the helper's own deque instead of executing it). | |
| uint64_t | mizu_morsel_layout (mizu_morsel_hdr *h, uint64_t n, uint64_t morsel_size, uint64_t desc_len, uint32_t x_type, uint64_t x_len, uint32_t out_type, uint64_t out_m, uint32_t claim_n) |
| Fill *h with the section layout for a map of n elements (morsel_size the geometry input, desc_len the descriptor stream size, x_type/x_len the optional raw x section — x_len must be exactly n elements — out_type/out_m the optional template output area, claim_n the CLAIM word count). | |
| const char * | mizu_morsel_hdr_check (const void *base, size_t size, mizu_morsel_hdr *out) |
| Validate a region's header against the mapping size, including section placement. | |
| void | mizu_morsel_sizer_init (mizu_morsel_sizer *) |
| int | mizu_morsel_next (void *base, const mizu_morsel_hdr *h, mizu_morsel_sizer *sz, uint32_t r, uint32_t gen, const mizu_pool_sig *sig, uint64_t pin_k, double now, uint64_t *m_out, uint64_t *k_out, int *help_out) |
| One whole batch transition — generation-fenced lane claim, cancel and pool-signal checks, sized cursor issue — over the mapped region. | |
| uint32_t | mizu_morsel_reset (void *base, const mizu_morsel_hdr *h) |
| Prepared-run re-arm, O(1) in n (no per-morsel state exists to clear): bump the generation, stamp (new_gen << 2) | IDLE over the CLAIM array, zero the cursor, clear the cancel word. | |
| int | mizu_morsel_abandon (void *base, const mizu_morsel_hdr *h, uint32_t r, uint32_t gen) |
| The exhausted-runner trim's CAS, folding its own trigger: a no-op unless the cursor is exhausted or the cancel word is set. | |
| void | mizu_morsel_cancel_set (void *base, const mizu_morsel_hdr *h) |
| The cancel word: set by the submitter on timeout / interrupt / death, and by an erroring runner itself before its ERR publish — the fail-fast store that stops every peer within ~a batch. | |
| int | mizu_morsel_cancel_get (const void *base, const mizu_morsel_hdr *h) |
| uint32_t | mizu_morsel_generation (const void *base, const mizu_morsel_hdr *h) |
| Single reads of the mutable words: the generation (masked), the cursor clamped to n_morsels, and one CLAIM word (the protocol tests' view). | |
| uint64_t | mizu_morsel_cursor (const void *base, const mizu_morsel_hdr *h) |
| uint32_t | mizu_morsel_claim (const void *base, const mizu_morsel_hdr *h, uint32_t r) |
| void | mizu_morsel_span_of (const mizu_morsel_hdr *h, uint64_t m, uint64_t k, uint64_t *lo, uint64_t *hi) |
| A batch's element range, 0-based half-open: [m * morsel_size, min((m + k) morsel_size, n)). | |
| size_t | mizu_morsel_lost (mizu_morsel_span *spans, size_t n, uint64_t bound, mizu_morsel_span *out) |
| void | mizu_pool_eval_mark (mizu_pool *, int in_flight) |
| The eval-in-flight marker a binding sets around its catching = 0 task eval only (R: around Rf_eval, never the decode), so its unwind path can tell a task error from infrastructure failure. | |
| int | mizu_pool_unwind_sink (mizu_pool *, mizu_result_sink *out) |
| The unwind path for a binding whose catching = 0 exec abandons (R's longjmp): when the eval marker says a task eval was in flight, clears it and mints that task's sink (from the identity the core saved at execute) for the err publish. | |
| static uint32_t * | mizu_zc_rc (void *base) |
| The zc refcount / flags words of a MIZU* region header (bytes [24-31] of the reserved band — the wire-format offsets MIZU_ZC_REFCOUNT_OFF / MIZU_ZC_FLAGS_OFF in mizu.h). | |
| static uint32_t * | mizu_zc_flags_ (void *base) |
| static mizu_mizs_geom | mizu_mizs_geometry (int64_t n) |
| The string block's section offsets for n strings (each section 64-byte aligned from the block start). | |
| static int | mizu_ext_flags_known (const void *base) |
| The [32-35] format flags word admits only the assigned S4 bit. | |
| static int | mizu_ext_valid_ok (int64_t off, int64_t count, uint64_t n, size_t size) |
| The three-state validity pair of an MIZH header or one MIZL leaf-table entry: {0, 0} absent, {0, -1} known-NA-free, or a 64-byte-aligned offset whose ceil(n / 8)-byte bitmap fits the region, 0 <= count <= n. | |
| static int | mizu_ext_mizl_tag_ok (int32_t sexptype) |
| A directory entry's sexptype: MIZU_MIZL_S4 masked off, the remainder a listed tag — 0 (a serialized leaf), the atomic tags, STR, VEC, INT64, and the remote leaf (33). | |
| static int | mizu_ext_na_real_bits (uint64_t bits) |
| One per-element NA test per NA-capable wire type (anything else is never null): INT32_MIN for LGL and INT, INT64_MIN, and the NA_real_ payload discriminated from other NaNs — any NaN whose low word is 1954 (0x7A2), R's own ISNA test, so a writer's quiet-bit-clear NA_real_ (R's verbatim form) and the wire's quiet-bit-set twin both read as NA while a genuine NaN stays a value. | |
| static int | mizu_ext_na_at (int type, const void *src, uint64_t i) |
| static void | mizu_ext_na_store (int type, void *dst, uint64_t i) |
| Write the wire type's missing sentinel over element i (the CPLX pair both parts). | |
| static int | mizu_ext_mizl_ent (const void *base, size_t size, int64_t i, mizu_mizl_entry *e) |
| One MIZL directory entry's checks, shared by mizu_mizl_check's pass and mizu_mizl_elem: alignment and extent, the attrs tail, the listed tag, and the length against the leaf kind — the string block's fixed sections for STR, the element extent for an atomic leaf. | |
| static int | mizu_ext_mizh_check_impl (const void *base, size_t size, int *wire_type, int64_t *n_elems, int64_t valid[2]) |
| The bodies behind the larger dual-form functions below: both the header inline and the exported symbol in src/ext.c are one-line delegations to these, so the two forms cannot drift apart. | |
| static int | mizu_ext_mizs_check_impl (const void *base, size_t size, int64_t *n, int64_t *str_size, int64_t *attrs_size) |
| static int | mizu_ext_mizl_check_impl (const void *base, size_t size, int64_t *n, int64_t *attrs_off, int64_t *attrs_size, int64_t valid[2]) |
| static int | mizu_ext_mizl_elem_impl (const void *base, size_t size, int64_t i, mizu_mizl_entry *elem) |
| static uint64_t | mizu_ext_na_build_impl (int type, uint8_t *bitmap, const void *src, uint64_t n, uint64_t bit_off) |
| static uint64_t | mizu_ext_na_apply_impl (int type, void *dst, const void *src, const uint8_t *bitmap, uint64_t n) |
| static double | mizu_timeout_ms (double seconds) |
| seconds (a binding's convention; <= 0 polls, non-finite waits indefinitely) to the core's timeout_ms (0 polls, < 0 indefinite). | |
| static void | mizu_store_na_real (void *dst) |
| static uint64_t | mizu_aux_rawspill_pool (int type, uint32_t name_len) |
| aux = wire type tag | region name length << 8. | |
| static uint64_t | mizu_aux_shm_vec (int type, uint64_t total) |
| aux = layout type tag | exact used bytes << 8. | |
| static int | mizu_aux_type (uint64_t aux) |
| The decode half of the aux split (low byte the type tag, the kind's field above it): the tag, and the packed field (the pool RAWSPILL region name length; the SHM_VEC exact used bytes). | |
| static uint64_t | mizu_aux_hi (uint64_t aux) |
| static void | mizu_mizh_write (void *base, int wire_type, int64_t n_elems) |
| The flat MIZH header: magic, type, element count, zero attrs, and the reserved band [24, 64) zeroed. | |
| static int | mizu_mizh_check (const void *base, size_t size, int *wire_type, int64_t *n_elems, int64_t valid[2]) |
| Validate the MIZH header at base: region size, magic, a known atomic wire type, the element/attribute extents against the region size, the format flags word, and the validity section (handed back through valid) — 0 on success, -1 on any rejection. | |
| static void | mizu_mizh_validity_set (void *base, int64_t off, int64_t count) |
| Stamp the validity pair of an MIZH or MIZL header (the one write site): {0, 0} absent, {0, -1} known-NA-free, or the section offset and null count. | |
| static int | mizu_mizs_check (const void *base, size_t size, int64_t *n, int64_t *str_size, int64_t *attrs_size) |
| Validate the MIZS header at base against the region: magic, the format flags word, and the string block's fixed sections plus the two end offsets (the block's and the attrs blob's) — 0 on success, -1 on any rejection. | |
| static int | mizu_mizl_check (const void *base, size_t size, int64_t *n, int64_t *attrs_off, int64_t *attrs_size, int64_t valid[2]) |
| Validate the MIZL header at base and its directory's extent against the region: magic, the format flags word, every entry (mizu_ext_mizl_ent), and the header's validity pair — whose count totals the nulls across the leaves, so the directory entry count does not bound it; the sum of the atomic leaves' lengths does. | |
| static int | mizu_mizl_elem (const void *base, size_t size, int64_t i, mizu_mizl_entry *elem) |
| One bounds-checked MIZL directory entry (index i), with its validity-table pair: a {0, 0}/{0, -1} header state covers every leaf; otherwise the table's entry i is validated by the MIZH rule against the leaf's length. | |
| static uint64_t | mizu_na_build (int type, uint8_t *bitmap, const void *src, uint64_t n, uint64_t bit_off) |
| The sentinels-to-bitmap scan: bit (bit_off + i) of bitmap records element i of src, 1 = present (the MIZS string block's convention); a type with no missing sentinel is all-present. | |
| static uint64_t | mizu_na_apply (int type, void *dst, const void *src, const uint8_t *bitmap, uint64_t n) |
| The bitmap-to-sentinels copy: n elements land at dst, the clear bits' elements written as the wire type's missing sentinel. | |
| mizu_status | mizu_ix_open (mizu_ix *cur, const void *buf, size_t len) |
| Open a cursor over a whole stream: the magic and version checks, the unknown version taking the informative "the peer uses a newer format" decline. | |
| mizu_status | mizu_ix_next (mizu_ix *cur, mizu_ix_item *item) |
| The next item: bounds, depth, UTF-8 and grammar checks per the spec. | |
| mizu_status | mizu_ix_end (mizu_ix *cur) |
| The finishing check: exactly one value per stream — fails on a truncated root value or on bytes past it. | |
| mizu_status | mizu_ixt_open (mizu_ix *cur, const void *buf, size_t len, mizu_ix_item *item, int max_kind) |
| The task-stream decode shim (DESIGN.md's byte-shape helper registry): the per-field shape checks every task-stream decode shares, once. | |
| mizu_status | mizu_ixt_want_code (mizu_ix *cur, mizu_ix_item *item) |
| The next item must be a non-NA STR1 (the 0x04 code string). | |
| mizu_status | mizu_ixt_want_list (mizu_ix *cur, mizu_ix_item *item) |
| The next item must be a LIST begin (the positional field). | |
| mizu_status | mizu_ixt_want_dict (mizu_ix *cur, mizu_ix_item *item) |
| The next item must be a DICT begin (the named field). | |
| static int | mizu_ext_ix_utf8_valid_impl (const void *s, size_t n) |
| The byte-shape helpers' bodies (DESIGN.md's byte-shape helper registry): pure byte math over the wire constants, dual-form below. | |
| static int | mizu_ext_ix_tag_of_impl (int wire_type) |
| static size_t | mizu_ext_ix_put_header_impl (unsigned char *dst) |
| The interop emit helpers' bodies (the interchange stream section below): every count and value follows its tag byte directly at unaligned offsets, little-endian, so writes are memcpy, never casts. | |
| static size_t | mizu_ext_ix_put_tag8_impl (unsigned char *dst, uint32_t tag, uint64_t v) |
| static size_t | mizu_ext_ix_put_nil_impl (unsigned char *dst) |
| static size_t | mizu_ext_ix_put_lgl_impl (unsigned char *dst, int value) |
| static size_t | mizu_ext_ix_put_int_impl (unsigned char *dst, int64_t value) |
| static size_t | mizu_ext_ix_put_real_impl (unsigned char *dst, double value) |
| static size_t | mizu_ext_ix_put_cplx_impl (unsigned char *dst, double re, double im) |
| static size_t | mizu_ext_ix_put_str_impl (unsigned char *dst, const void *s, int32_t len) |
| static size_t | mizu_ext_ix_put_bytes_impl (unsigned char *dst, const void *data, uint64_t count) |
| static size_t | mizu_ext_ix_put_vec_impl (unsigned char *dst, int wire_type, const void *data, uint64_t count) |
| static size_t | mizu_ext_ix_put_strv_begin_impl (unsigned char *dst, uint64_t count) |
| static size_t | mizu_ext_ix_put_strelt_impl (unsigned char *dst, const void *s, int32_t len) |
| static size_t | mizu_ext_ix_put_list_begin_impl (unsigned char *dst, uint64_t count) |
| static size_t | mizu_ext_ix_put_dict_begin_impl (unsigned char *dst, uint64_t count) |
| static size_t | mizu_ext_ix_put_key_impl (unsigned char *dst, const void *s, uint32_t len) |
| static size_t | mizu_ext_ix_put_attr_impl (unsigned char *dst) |
| static size_t | mizu_ext_ix_put_err_impl (unsigned char *dst, int has_index, uint64_t index, const void *type, uint32_t type_len, const void *message, uint32_t message_len, const void *detail, uint32_t detail_len) |
| static size_t | mizu_ext_ix_put_task_impl (unsigned char *dst, int target, int kind, uint64_t ident) |
| static size_t | mizu_ext_ix_put_ref_impl (unsigned char *dst, const void *name, uint32_t name_len) |
| static size_t | mizu_ext_ix_err_floor_impl (const void *s, size_t n, size_t share) |
| static size_t | mizu_ext_ix_write_err_impl (unsigned char *dst, uint32_t inline_max, const void *type, size_t type_n, const void *msg, size_t msg_n, const void *detail, size_t detail_n, int has_index, uint64_t index) |
| static int | mizu_ix_utf8_valid (const void *s, size_t n) |
| The emit helpers: each returns its byte count, writing only when dst is not NULL, so a binding's two-pass walk sizes (dst NULL) and writes through the same byte-level code. | |
| static int | mizu_ix_tag_of (int wire_type) |
| The wire-type -> vector-tag mapping: MIZU_TYPE_LGL / INT / REAL / CPLX / RAW / INT64 -> 0x06 / 0x07 / 0x08 / 0x09 / 0x0a / 0x0e; any other input returns 0 (MIZU_IX_TAG_NIL, never a vector tag, so the failure is distinguishable from every mapping). | |
| static size_t | mizu_ix_put_header (unsigned char *dst) |
| static size_t | mizu_ix_put_nil (unsigned char *dst) |
| static size_t | mizu_ix_put_lgl (unsigned char *dst, int value) |
| value on the wire: 0, 1, or 2 (NA). | |
| static size_t | mizu_ix_put_int (unsigned char *dst, int64_t value) |
| static size_t | mizu_ix_put_real (unsigned char *dst, double value) |
| real and cplx take doubles and memcpy them: NaN payloads are bitwise-preserved. | |
| static size_t | mizu_ix_put_cplx (unsigned char *dst, double re, double im) |
| static size_t | mizu_ix_put_str (unsigned char *dst, const void *s, int32_t len) |
| The 0x04 string scalar; len -1 is NA (s may then be NULL). | |
| static size_t | mizu_ix_put_bytes (unsigned char *dst, const void *data, uint64_t count) |
| static size_t | mizu_ix_put_vec (unsigned char *dst, int wire_type, const void *data, uint64_t count) |
| The fixed-width vector tags: wire_type one of MIZU_TYPE_LGL / INT / REAL / CPLX / RAW / INT64 (any other returns 0 — the bindings pass only the six). | |
| static size_t | mizu_ix_put_strv_begin (unsigned char *dst, uint64_t count) |
| static size_t | mizu_ix_put_strelt (unsigned char *dst, const void *s, int32_t len) |
| A bare strv element; len -1 is NA. | |
| static size_t | mizu_ix_put_list_begin (unsigned char *dst, uint64_t count) |
| static size_t | mizu_ix_put_dict_begin (unsigned char *dst, uint64_t count) |
| static size_t | mizu_ix_put_key (unsigned char *dst, const void *s, uint32_t len) |
| A bare dict key: never NA, so the length is unsigned. | |
| static size_t | mizu_ix_put_attr (unsigned char *dst) |
| static size_t | mizu_ix_put_err (unsigned char *dst, int has_index, uint64_t index, const void *type, uint32_t type_len, const void *message, uint32_t message_len, const void *detail, uint32_t detail_len) |
| static size_t | mizu_ix_put_task (unsigned char *dst, int target, int kind, uint64_t ident) |
| The task header: target language byte, kind byte, the reserved u16 flags written zero, then the submitter identity. | |
| static size_t | mizu_ix_put_ref (unsigned char *dst, const void *name, uint32_t name_len) |
| The ref leaf: the region identifier span (1-255 bytes; 0 returns 0). | |
| static size_t | mizu_ix_write_err (unsigned char *dst, uint32_t inline_max, const void *type, size_t type_n, const void *msg, size_t msg_n, const void *detail, size_t detail_n, int has_index, uint64_t index) |
| A complete 'I' err stream in one call: the two-byte header, then the 0x11 err item, under the bounded budget (DESIGN.md's err framer) — type truncated at MIZU_IX_ERR_TYPE_SHARE, message at half the inline budget, detail at what remains, each cut at a UTF-8 boundary. | |
| static int | mizu_ext_arrow_valid_impl (const ArrowArray *arr, int64_t k) |
| static int | mizu_ext_arrow_str_at_impl (const ArrowArray *arr, int64_t k, int form, const uint8_t **ptr, int32_t *len) |
| static int | mizu_arrow_valid (const ArrowArray *arr, int64_t k) |
| The validity bit for element k of a batch (Arrow LSB-first). | |
| static int | mizu_arrow_str_at (const ArrowArray *arr, int64_t k, int form, const uint8_t **ptr, int32_t *len) |
| The string forms' per-batch element access: utf8 (i32 offsets), large_utf8 (i64), string_view (16-byte views). | |
| const mizu_cvt_row * | mizu_cvt_for_arrow (const char *fmt) |
| The Arrow-format keying of the conversion table (every supported row a single character); NULL when the format has no row. | |
| void | mizu_cvt_convert (uint8_t *dst, const uint8_t *src, const uint8_t *valid, uint64_t off, size_t n, const mizu_cvt_row *row, mizu_arrow_warn *warn) |
| One run's convert into a reserved destination: the BIT_LGL / masked / plain dispatch. | |
| mizu_status | mizu_arrow_open (mizu_arrow **out, const ArrowSchema *schema, const ArrowArray *const *batches, size_t n_batches, uint32_t peer_caps) |
| Classify every column off the schema (a struct schema; the type map of DESIGN.md's frame section), vet every batch (buffer counts, offsets, null_count range, column lengths, a uint8 column's nulls, unnamed / duplicate names, a zero-column frame) and validate the dictionaries (batch 0's levels canonical, a null level or a level set that differs across batches declines, every index range-checked). | |
| void | mizu_arrow_col_info (const mizu_arrow *, int64_t col, mizu_arrow_col *out) |
| The per-column classification, for a binding's own emitters over the same batches (pymizu's inline frame forms). | |
| size_t | mizu_arrow_col_body (const mizu_arrow *, int64_t col, uint8_t *dst, mizu_arrow_warn *warn) |
| One column's by-value body bytes over every batch — the cvt lanes, a dictionary's 1-based codes, date / timestamp / duration doubles; the same loops the MIZL writer's leaves use, for a binding's own 'I' emitters. | |
| size_t | mizu_arrow_col_blob (const mizu_arrow *, int64_t col, uint8_t *dst) |
| One column's attribute dict as a bare 0x0d dict value (the §3.5 blob without its stream header — a binding splices it into its own 'I' attr form). | |
| size_t | mizu_arrow_write_ix_col (const mizu_arrow *, int64_t col, uint8_t *dst, mizu_arrow_warn *warn) |
| One column's 'I' value form (the inline small-table path): the plain vectors bare (the cvt tag off the row's wire type; the strv form over mizu_arrow_str_at, a malformed record put as NA like a null), the attr-marked forms splicing mizu_arrow_col_blob's bare dict. | |
| size_t | mizu_arrow_write_ix (const mizu_arrow *, uint8_t *dst, mizu_arrow_warn *warn) |
| One complete 'I' stream of the whole frame (the small-table path): the header, the attr mark, the list of the columns' value forms (mizu_arrow_write_ix_col), then the root dict {names, class = "data.frame", row.names = c(NA, -n)} — one C write straight off the Arrow buffers where a below-gate frame previously converted back to a language value first. | |
| int | mizu_arrow_remote (mizu_arrow *, int64_t col, const char *ident, size_t ident_len, void *ref_base, size_t ref_size, int64_t ref_leaf) |
| A remote-leaf override (pymizu's provenance match): column col crosses as directory tag 33 — the identifier span, the referenced leaf's attrs size and validity claim read off the referenced region (mizu_mizl_elem), and one REFHELD OR per distinct referenced region at write. | |
| void | mizu_arrow_root_blob (mizu_arrow *, const void *blob, size_t len) |
| The root-blob override: a complete 'I' stream for the frame attributes the caller built (pymizu's custom row_names). | |
| void | mizu_arrow_tz (mizu_arrow *, int64_t col, const char *tz) |
| The tzone override: the POSIXct leaf blob writes tz verbatim (a same-language sender's naivety-preserving form; the classified default writes "UTC" for a naive timestamp). | |
| size_t | mizu_arrow_size (mizu_arrow *) |
| The region span in bytes: runs the size pass on first call (the result memoizes; an override registered after it re-runs the pass on the next call). | |
| mizu_status | mizu_arrow_stage (mizu_handle *, mizu_arrow *, mizu_slot_hdr *hdr, unsigned char *payload, mizu_arrow_warn *warn) |
| The one-call stage over the handle seam, valid only during a stage_fn call. | |
| void | mizu_arrow_close (mizu_arrow *) |
| double | mizu_now (void) |
| Monotonic seconds (the spin/deadline clock) and the current pid (cached, with an atfork reset). | |
| long | mizu_self_pid (void) |
| void | mizu_err_describe (mizu_errcat, const char **summary, const char **hint) |
| Category + remediation text for an MIZU_ERRCAT (what the handle / thread-local error slot records). | |
| void | mizu_rng_jump (int *seed) |
| One 2^127-step L'Ecuyer-CMRG stream jump in place over a 6-word state (rng_jump.c; a binding's map derives per-element streams with it). | |
| void | mizu_tune (void) |
| malloc tuning (tune.c), called by the binding at load. | |
libmizu binding-author API (the unstable tier).
The sanctioned surface for language bindings and the built-in bytes binding: the callback seam, the stager/read/publish services, and the promoted internals the first-party bindings (mizu for R, pymizu for Python) compile against.
Tier policy: this header is version-pinned per minor release. Any declaration, struct layout, or macro here may change on a minor version bump, without deprecation — including after mizu.h's stable promise starts at 1.0. Consumers rebuild against each libmizu minor release (the first-party bindings vendor the sources at a pinned commit). mizu.h carries the stable-ABI promise; src/internal.h stays private. There is no third state: a symbol not here or in mizu.h is private.
Conventions differ from mizu.h where the consumers differ: this tier is C-only binding surface, so macros are fine, and the hot accessors plus the small wire-format helpers ship in dual form (below). Every other operation is a real exported function (MIZU_API), so FFI consumers see the full tier.
Dual-form fast paths (the CPython Py_INCREF pattern): mizu_parker_snapshot, mizu_zc_rc, mizu_zc_flags_, the wire helpers (mizu_timeout_ms, mizu_store_na_real, mizu_aux_rawspill_pool, mizu_aux_shm_vec, mizu_mizh_write, mizu_mizh_check, mizu_mizh_validity_set, mizu_mizs_geometry, mizu_mizs_check, mizu_mizl_check, mizu_mizl_elem, mizu_na_build, mizu_na_apply), and the interchange emit helpers (mizu_ix_put_*) ship as static inline here AND as same-named exported functions (src/ext.c). A C TU inlines its own copy — zero cost — while an FFI binds the exported symbol. This is legal C: internal and external linkage of the same name in different TUs do not clash; taking the address in an ordinary TU yields the local inline's address, which is harmless (the accessors are pure offset math over the wire-format offsets). A TU defining the exported forms defines MIZU_EXT_NO_INLINES before including this header: a static inline and an extern definition of one name cannot coexist in a TU. Single-sourcing: both forms are thin wrappers over the same offset macros / struct definitions below — the macros, not the bodies, are the one definition.
| #define MIZU_ZC_FLOOR ((size_t) 32768) |
Staging-policy floors for a binding's stage_fn (not used by the core): SHM_VEC escalates only past max(inline budget, MIZU_ZC_FLOOR); the channel's raw floor is higher (the arena copy has no region machinery to amortize) and lifts entirely under the churn signal.
Bindings pick their own.
| #define MIZU_CODEC_MAGIC 0x52u |
The self-describing stream dispatch byte: payload byte 0 of an INLINE frame, where an R native stream carries 'B'/'X'.
The core is codec-agnostic; DESIGN.md's codec registry allocates the magic bytes ('R' = mizu, 'P' = pymizu). 'R' here and MIZU_DROP_R share the letter deliberately — both denote an R-binding payload, in disjoint contexts. 'R'
| #define MIZU_LANG_NONE 0u |
The language registry: the identity word's byte 0, the pool's worker identity word, and each binding's handle-level peer_lang.
Append-only: a value, once shipped, is never reassigned (the byte is equality-compared, so a reassignment fails behaviorally, as false join rejections across mixed builds). 0 is "none" — no peer attached yet, a pool no worker has joined — never a stored binding identity: a binding whose identity word has a zero language byte is rejected at create, attach and join.
| #define MIZU_CAP_MIZS (1u << 0) |
Reader capabilities: a 32-bit mask, one bit per layout or format extension a reader implements beyond the baseline (MIZH atomic and INT64 views, 'I' format 0x01 as specified in DESIGN.md's Interchange codec section).
Readers ignore bits they do not know — they never reject them; absent means unsupported, and a writer stages a layout or tag only for a peer that sets its bit. A bit names a byte layout, not a feature: an incompatible change to a gated layout, tag or shape allocates a new bit and retires the old, which is never reassigned. Features that always land together in every binding share a bit. reads MIZS string layouts
| #define MIZU_IDENT | ( | lang, | |
| caps | |||
| ) | ((uint64_t) (uint8_t) (lang) | ((uint64_t) (uint32_t) (caps) << 32)) |
The identity word: the language in bits 0-7, the 32-bit capability mask in bits 32-63, bits 8-31 reserved — written zero and ignored by readers (a negotiation word, so a later field needs no reader taught to skip it).
The pool word's exact-match join compares the whole word.
| #define MIZU_READ_CONSUME 1u |
read_fn flags (ctx->flags, zero at each read's start): CONSUME makes a NULL return consume the slot exactly as on success while the verb still returns MIZU_ERR — the foreign/corrupt-payload contract; see the seam comment above.
The flag is honoured on a single receive and on a batch's first message only; past that the core defers the consume decision to the next receive.
| #define MIZU_MORSEL_CANCEL_OFF ((uint64_t) 0) |
A binding's parallel map rides one fresh region per map call: a 128-byte header, the language-specific descriptor stream, an optional bare-bytes x section, the morsel state, and an optional template output area.
Map regions are private to a binding install (workers spawn from the same package), so the layout is unified here and each binding keeps only its magic tag. The protocol:
| #define MIZU_MORSEL_T_TARGET 200e-6 |
Batch sizing policy (mizu_morsel_next): k targets a batch duration, growing at most 2x per step and shrinking immediately on overshoot, clamped to the cap — which bounds lost-set coarseness and the ramp worst case.
Frozen by mizu's 2026-08-04 gate sweep (M4 Pro, W = 4).
| #define MIZU_MORSEL_MAGIC 0x4D495A4Du |
The one map-region magic: "MIZM" read as hex, the core's magic packing (MIZU_MAGIC, the pool magic, MIZH/MIZS/MIZL).
The morsel module stamps and checks it itself — the descriptor's codec identity rides the descriptor stream's own first byte, so the magic carried nothing else, and no path can hand a runner another binding's region.
| #define MIZU_IX_VERSION 0x01u |
The byte-level half of DESIGN.md's Interchange codec section: a validating pull cursor (mizu_ix_open / mizu_ix_next / mizu_ix_end, in src/interop.c) and the dual-form emit helpers below, shared by every binding so the wire grammar has exactly one implementation.
The cursor owns bounds, the depth cap, UTF-8 validity, the container arity accounting (a task is one element of kind-determined arity), and the informative unknown-tag / unknown-version / unknown-kind declines, recording them in the thread-local error slot (mizu_last_error_message). A binding keeps only a builder, allocating a native object per item, and a value walk emitting through the put helpers; the golden corpus (tests/interop/) certifies the grammar once here.
| #define MIZU_IX_ATTR_NAMES "names" |
The attribute vocabulary (DESIGN.md's byte-shape helper registry): the shape whitelist's wire strings — attr keys, class strings and difftime units.
The set is append-only: a rename is a wire change; a later whitelist shape adds rows. Lengths ride sizeof - 1 at the call sites that pass one.
| #define ARROW_C_DATA_INTERFACE |
The Arrow C Data Interface structs, defined per the spec under its two include guards (the layout is frozen ABI; no headers, no dependency): ArrowSchema / ArrowArray under ARROW_C_DATA_INTERFACE, ArrowArrayStream under ARROW_C_STREAM_INTERFACE (nanoarrow guards the stream struct separately; one shared guard would leave it redefinable by a later nanoarrow or arrow include).
The frame writer reads caller-owned structs; a binding's import front-ends produce them.
| typedef struct mizu_handle_s mizu_handle |
Generic handle view for the stager services; the core passes the handle a binding's callbacks were registered with.
Never constructed by bindings; the struct definition stays private (src/internal.h).
| typedef struct mizu_result_sink_s mizu_result_sink |
Valid only during an exec callback.
The struct definition is below (the ERR-envelope frame buffer is binding surface).
| typedef int(* mizu_stage_fn) (void *obj, mizu_slot_hdr *hdr, uint8_t *payload, uint32_t inline_max, mizu_handle *, void *ctx) |
The core never sees a language object.
A binding registers these callbacks at create/attach/join; the core copies them into the handle, so a hot-path call is one load + a predicted indirect branch.
stage: frame obj as (hdr, payload) — payload capacity inline_max. Spill/arena/retain via the mizu_stage_* services on the handle. ctx is the handle's binding.ctx. Returns 0 on success, nonzero on staging failure (the verb fails as MIZU_ERR / MIZU_ERRCAT_STAGE). May also not return (a binding's longjmp): staging is transactional — the core mutates no shared state before stage returns, a mid-stage arena chunk is FIFO-reclaimed like any other, and an uncommitted region checkout or pin rolls back at the next verb entry or at destroy. An abandoned stage leaves the handle consistent. read: produce the binding's object for a received frame; the return value goes to the verb's caller, opaque to the core. May also not return (a binding's longjmp): invocation points leave the handle consistent and nothing consumed. payload is always a dereferenceable byte range: the transport resolves its arena-referencing kinds (ARENA, channel RAWSPILL) to their byte range before the call, bounds-checked against the arena, passing the validated capacity as limit (inline_max for slot-resident frames). The kinds are NOT rewritten. For a pool collect, ctx->outcome carries the result's terminal state (MIZU_RS_OK/ERR with a payload frame; MIZU_RS_CANCEL/DIED without one — read_fn builds the binding's error object for all three). A vanished out-of-line region: mizu_read_region sets ctx->gone and read_fn propagates by returning NULL. A NULL return consumes nothing — unless the read first set ctx->flags |= MIZU_READ_CONSUME: then the transport consumes exactly as on success (channel: the head advance and its batched publication; pool OK/ERR: the FREE transition, the task-keeper drop, and the producer-keeper wake) while the verb still returns MIZU_ERR. Use for foreign or corrupt payloads that must not wedge the ring behind an unreadable slot: the binding carries its specific message in its own state (the core records no generic error for a consumed read), and a retried read sees the next slot. The consume decision sits with the binding on a single receive and on a batch's first message; past that the core defers it — a mid-batch read failure ends the batch with MIZU_OK and the consumed prefix, and the next receive reproduces the failure and decides. exec: pool workers only — run one claimed task frame and publish through the sink. ctx is a read ctx on the handle (outcome MIZU_RS_OK): decode reads ride it (mizu_read_region, ctx->gone), and the binding ctx remains reachable as ctx->binding_ctx. Must not abandon: the binding catches every task condition into the sink; an escape degrades to worker death plus the reaper verdict (the hard-crash semantics, never the path for an ordinary task error). catching marks a reentrant invocation (nested-collect help, nested submit's inline execute): contain task conditions there instead of letting them unwind through the worker loop. Returns 0 on success; nonzero is infrastructure failure and takes the worker down (MIZU_EXIT_ERROR). check: interrupt/cancel poll, invoked from core wait/work loops at abandon-safe points only (no shared-state mutation in progress, no cleanup pending), once per spin/park iteration. Return 0 to continue, nonzero to abandon: the verb unwinds as MIZU_ERR with MIZU_ERRCAT_INTERRUPTED, consuming nothing (a recv interrupted mid-wait has claimed no slot; a collect interrupted while parked has consumed no result). The hook may also not return (a binding's longjmp) — invocation points are chosen so an abandoned wait leaves the handle consistent. Verb-calling thread only, never death-watch or reaper threads. NULL for plain-C consumers: the poll skips at one load + a predicted branch. park: around-park hook, bracketing each bounded park — after the parker's announce, around the sleep only, never around shared-state mutation: entering is nonzero before the sleep, zero after the wake. For runtimes with a global lock: a Python worker holds the GIL through collect (nested-collect help reenters exec_fn); this hook drops it for each bounded sleep so other threads in the worker process run while the worker parks. Verb-calling thread only. NULL for runtimes without a global lock, plain-C consumers, and submitter handles (which release around the whole verb instead). sweep: idle hook, invoked when a pool worker goes idle or departs (the core's keeper-table sweep points): drop binding-side caches. NULL for bindings without per-handle caches. drop: pin-release hook, invoked when a retained staging entry is released — at the consumer-done points (collect, slot reuse, the worker keeper sweep), on a cancelled publish, on a staging rollback, and at handle teardown. pin is the opaque token the stager registered with mizu_stage_pin. Fires only on the handle-owning thread. NULL when the binding never pins. ctx: opaque to the core.
| typedef struct mizu_bytes_s mizu_bytes |
A byte buffer.
Send: the consumer fills {data, len} and passes its address as the verb's obj. Recv: the binding returns a malloc'd mizu_bytes (data rides the same allocation); release with mizu_bytes_free — an export, not documented free(), so a shared-library build never crosses allocator domains. Fixed layout, never extended.
| typedef struct mizu_parker_s mizu_parker |
One parker per waiting entity: a 32-bit monotonic epoch word in the shared region plus, on Windows only, one named auto-reset event (the epoch compare is not atomic with the sleep there).
Park sites follow snapshot -> announce -> re-check -> sleep-bounded; any unpark that observes the announcement bumps the epoch after the snapshot, so the sleep returns immediately. Spurious wakes are absorbed by the caller's re-check. This handshake is the sole guarantee against lost wakeups — no watchdog sits behind it.
| typedef struct mizu_death_watch_s mizu_death_watch |
Translates a watched pid's exit into *flag = 1 plus a directed unpark of pk (optional, copied).
A pid already dead fires immediately. The flag target and the parker's epoch word / event must stay valid until mizu_death_watch_stop returns: stop synchronizes with any in-flight callback, so after it returns nothing touches them. Detection is a wake trigger only — the liveness lock is the verdict; pid-reuse races are absorbed there. Callback threading: the core's own reap watch (internal start2) fires on OS listener threads; start's flag store + unpark is the whole contract here.
| typedef enum mizu_live_probe_e mizu_live_probe |
Exclusive flock (POSIX) / LockFileEx (Windows) held for a process's entire lifetime and released by the kernel on any exit path.
fd-scoped, not PID-scoped: pid reuse cannot fake "alive". A probe is a non-blocking acquire on the fd kept from open — ACQUIRED means the previous holder is dead (and the caller now holds the lock, serializing survivor cleanup); HELD means alive.
| typedef struct mizu_pool_sig_s mizu_pool_sig |
The opaque pool-signal trio a map runner loads relaxed once per batch transition: the help_wanted doorbell, the pool's shared shutdown word, and the handle's process-local listener-written owner_dead flag.
Borrowed; the binding's map dereferences the words and stays pool-layout-free.
| typedef struct mizu_morsel_hdr_s mizu_morsel_hdr |
The 128-byte region header.
Not pool wire format — it rides its own region, keyed by the same ABI version — but the same rules apply: the struct is the layout, 64-byte-aligned sections follow it.
| typedef struct mizu_morsel_sizer_s mizu_morsel_sizer |
Batch-sizing state: process-private, never wire state, reset at each run's first-call CLAIM CAS.
The binding embeds one per map context; a doorbell help that claims a queued runner of the same map aliases it (the cost is a mis-sized batch or a re-ramp on resume — harmless).
| typedef struct mizu_morsel_span_s mizu_morsel_span |
Worker-death lost set: issued = [0, bound), lost = issued minus the union of the collected batch histories — a batch in no history was issued but never completed (its claimant died, or the map fn errored mid-batch); a dead runner's whole history lands here too — it publishes only at exhaustion.
spans (caller-assembled from the histories, element space, lo <= hi) is sorted in place; the gaps (at most n + 1) land in out as 0-based half-open ranges; returns the gap count.
| typedef struct mizu_mizs_geom_s mizu_mizs_geom |
The small pure helpers over mizu.h's wire-format conventions, dual form like the zc accessors above (see the banner): the timeout unit conversion, the NA_real_ store (for a binding filling converted output without R headers), the aux packing of the pool RAWSPILL and SHM_VEC kinds and its decode pair, the flat MIZH header write/check, the MIZS / MIZL layout checks, the validity-section setter, and the NA sentinel <-> bitmap primitives (one NA-test implementation behind both bindings' validity paths, so two bindings' bitmaps cannot drift apart from the sentinels they describe).
The layout layouts themselves are mizu.h's MIZH / MIZS / MIZL documentation. The MIZS string block's four section offsets from a string count (mizu.h documents the block). data doubles as the size of everything before the string bytes.
| typedef struct mizu_mizl_entry_s mizu_mizl_entry |
One MIZL directory entry plus its validity-table pair (mizu.h documents the 32-byte entry): mizu_mizl_elem's out-param.
sexptype is the wire value — MIZU_MIZL_S4 is the S4 bit, the remainder a listed tag.
The pull cursor.
Stack-allocated by the caller; the frame stack is the depth cap's accounting (LIST / DICT / STRV / ATTR / TASK frames, the dict's remaining counting keys and values alike).
| typedef struct mizu_arrow mizu_arrow |
The Arrow -> MIZL frame writer: a classified, batch-vetted frame plus its size-pass results, staged through mizu_arrow_stage.
Opaque; the caller owns the schema and batch structs it handed over (they must outlive the handle — the writer reads them only inside mizu_arrow_write).
| anonymous enum |
Handle-kind discriminator.
The staging services are kind-specific: mizu_stage_arena_alloc / mizu_stage_reap are channel-only.
| enum mizu_live_probe_e |
Exclusive flock (POSIX) / LockFileEx (Windows) held for a process's entire lifetime and released by the kernel on any exit path.
fd-scoped, not PID-scoped: pid reuse cannot fake "alive". A probe is a non-blocking acquire on the fd kept from open — ACQUIRED means the previous holder is dead (and the caller now holds the lock, serializing survivor cleanup); HELD means alive.
| anonymous enum |
What mizu_ix_next yields.
A scalar carries its value; STR1 (the 0x04 scalar) and STR (a bare string: an strv element or a dict key — key is 1 for the latter) carry a UTF-8-validated span, na set on the -1 form (keys are never na); BYTES and VEC carry a (ptr, count) span already bounds-checked; STRV / LIST / DICT / ATTR are counted begins whose elements arrive as following items (a dict's keys as STR items); ERR carries its fields (top level only); TASK is a header item whose kind-determined arity (count) of fields arrive as ordinary items; REF carries the region identifier span (ptr, len), bounds-checked with the content opaque to the cursor (the resolver owns the identifier rules).
| anonymous enum |
The fixed-width conversion rows: an element width pair plus the kernel, producing a MIZU_TYPE_* leaf.
| anonymous enum |
The per-column classification, mizu_arrow_col_info's out-param: the kind, the conversion row, and the string-form particulars a binding's own emitters key on.
The column bodies and attribute dicts are the per-column writers below.
| int mizu_handle_kind | ( | const mizu_handle * | ) |
Handle queries standing in for field access into the private handle struct.
kind returns MIZU_HTYPE_*. churn reads the spill-churn signal (raised on Linux only: a spill pop missed with lent regions outstanding; cleared when a ledger sweep or force-reclaim returns one) — while set, a stager falls back to the copy tiers. The churn read is an extern call: gate it behind the payload's size check so it runs only for payloads already proven large. spill_info fills the producer free-list entry count and the lent-region ledger count (debug/dump surface; the per-kind public snapshots are mizu_channel_info_get / mizu_pool_dump_get).
| int64_t mizu_handle_keep_out | ( | const mizu_handle * | ) |
The tx-keeper count: outstanding staging retains on a channel handle (the reap gate's own count, so side-effect free).
-1 for a pool handle (its keepers live on both sides of every queue; no one count).
| void mizu_binding_init | ( | mizu_binding * | ) |
Zero and size-stamp a binding struct.
Call before filling the fn pointers.
| uint64_t mizu_channel_peer_ident | ( | const mizu_channel * | ) |
The identity words (the language registry above).
Each channel side publishes its binding's ident in its entity block: the host at create, the peer at attach before ready_set. mizu_channel_peer_ident reads the other side's word through the mapping (0 until the peer attaches: a host knows its peer's identity once ready_wait returns). A pool's first worker join CASes its word into the pool header — an exact match joins, a differing word fails — and mizu_pool_worker_ident reads that word through the mapping (0 until the first join; never reset for the pool's lifetime).
| void * mizu_stage_arena_alloc | ( | mizu_handle * | , |
| size_t | n, | ||
| uint64_t * | off | ||
| ) |
Core services for a stager (per handle), valid only during a stage_fn call — staging is single-threaded per handle role, so at most one checkout is in flight per handle:
region must be the current checkout.| void * mizu_stage_raw_spill | ( | mizu_handle * | , |
| uint64_t | n, | ||
| int | wire_type, | ||
| mizu_slot_hdr * | , | ||
| uint8_t * | payload, | ||
| uint32_t | inline_max | ||
| ) |
Raw-tier reservation (stage_raw.c), valid only during stage_fn: choose the tier for n bare bytes of wire_type and return the destination with hdr stamped — the caller memcpys n bytes into it.
NULL hands the object to the binding's serialized tiers (a reservation failure degrades, never errors). Policy: n <= inline_max → RAWVEC inline (aux = wire_type). channel: arena RAWSPILL (aux = wire_type, payload = chunk offset) when n <= MIZU_ZC_FLOOR_RAW or under churn; else flat SHM_VEC (MIZH header + retain_zc, aux = wire_type | exact used bytes << 8, payload = region name) when n >= max(inline_max, MIZU_ZC_FLOOR) and no churn, reaping before the checkout; an arena retry on region failure; else NULL. pool: flat SHM_VEC on the same zc gate; else a RAWSPILL region (aux = wire_type | name_len << 8, payload = name; n <= UINT32_MAX); else NULL. Bare bytes carry no identifier, so nothing is pinned. Eligibility probes (which objects are raw), STR1/NIL/REF, and the string/list-tree SHM_VEC layouts stay binding-side. Dual form (see the banner): the inline stamps the RAWVEC fast path and falls through to the exported slow path.
| mizu_shm * mizu_read_region | ( | mizu_read_ctx * | , |
| const uint8_t * | name, | ||
| uint32_t | len | ||
| ) |
Read-side service, invoked through the read_ctx handed to read_fn: a borrowed consumer mapping for a SHM_RAW-class payload name, from the handle's open cache (open/fstat/mmap on a miss, LRU-evicted).
The mapping is cache-owned: copy the payload out before consumer-done. On a vanished region, sets ctx->gone and returns NULL; read_fn propagates by returning NULL. View tiers do not use it — their mappings are binding-owned (mizu_shm_open_view), pinned by the view.
| int mizu_result_publish | ( | mizu_result_sink * | , |
| void * | value | ||
| ) |
Result publish, valid during exec only.
publish stages value via the worker handle's stage_fn and publishes MIZU_RS_OK — a result past the inline budget spills to a region whose consumer is the submitter, so a large result never has to fit the slot. publish_err publishes MIZU_RS_ERR: when inline_n != 0 the binding has already framed the flattened envelope INLINE in the sink's frame buffer (a self-contained stream pins nothing); otherwise the flattened object rides the tiered stage (the tiny-slot fallback). The ERR envelope is INLINE-framed by construction wherever a classed condition fits the slot, so the publish cannot fail: a task error fails the task, never the worker. publish_lost is the status-only terminal (MIZU_RS_LOST) for a task whose out-of-line payload vanished with its enqueuer: it carries no death verdict on the claiming worker, so the collect side neither force-reclaims the claimant's lent regions nor builds a claimant record — the task simply never runs anywhere, and the drain continues in this thief. The publish pair return 1 when the publish CAS won, 0 when a cancel beat it, and -1 on infrastructure failure (recorded on the handle — the worker cannot continue).
| void mizu_binding_bytes | ( | mizu_binding * | ) |
Fill binding with the bytes binding: stage/read only — exec, check, park, sweep, drop are NULL (a bytes handle is a channel peer or a pool submitter, never a worker).
Stage rides the INLINE/ARENA/SHM_RAW tiers exactly as a serialize stream does; read copies out before consumer-done. The copy tiers of a foreign peer (RAWVEC/RAWSPILL/STR1) read as their bare bytes; the view tiers (SHM_VEC/REF) fail the read — they are binding-owned by design. Pool collects surface non-OK outcomes only as the verb's mizu_status — the binding builds no error object. This is the FFI zero-callback path, the reference stager for binding authors, and the C test tiers' stager.
| int mizu_parker_attach | ( | mizu_parker * | pk, |
| uint32_t * | epoch, | ||
| const char * | region_name, | ||
| int | entity, | ||
| int | create | ||
| ) |
region_name/entity name the Windows event ("<region>.pk.<entity>"), created by the region's host (create = 1) and opened by name by attachers; unused on POSIX.
Returns 0 on success.
| int mizu_park | ( | mizu_parker * | pk, |
| uint32_t | snapshot, | ||
| long | timeout_ms | ||
| ) |
Sleeps while the epoch still equals snapshot, up to timeout_ms (0 = poll: never sleeps; < 0 = indefinite — on POSIX an untimed wait would silently restart under SA_RESTART and swallow an interrupt until the next genuine wake, so indefinite parks are internally bounded and rely on directed unparks).
Returns MIZU_PARK_*.
| const char * mizu_live_dir | ( | void | ) |
Directory for liveness lock files: the MIZU_LIVENESS_DIR override (read-through, checked every call) else a per-platform default resolved once — /dev/shm on Linux, the per-user temp dir on macOS and Windows.
NULL if unresolvable. Only region creators call this; participants read the embedded copy.
| void mizu_preamble_write | ( | void * | region, |
| const mizu_preamble * | p | ||
| ) |
Host writes at create, immutable thereafter; the peer validates before any shared atomic is read or written.
validate returns NULL and fills out on success, else a static error message. The pool header check (magic + version) lives in the pool attach path, not here.
| mizu_pool_sig * mizu_pool_signals | ( | mizu_pool * | ) |
A malloc'd copy of the signal trio (the caller frees).
NULL on failure.
| int mizu_pool_help_once | ( | mizu_pool * | ) |
One doorbell help beat (claims a map runner and re-homes it onto the helper's own deque) and the test-harness injection pull.
help_once returns 1 when it claimed, 0 when not, -1 on an exec_fn infrastructure failure (recorded on the handle).
| int mizu_pool_map_caps | ( | mizu_pool * | , |
| uint32_t * | free_rs, | ||
| uint32_t * | inj_cap, | ||
| uint32_t * | inline_entry | ||
| ) |
A map's batch-sizing inputs: claims the worker's nested-submitter slot when unclaimed (as a first nested submit does), then reports the caller's FREE result slots, the injection cap, and the entry inline budget.
Only this process allocates from its own subrange, so the FREE count can only grow under it. Returns -1 on a claim failure (the handle error slot holds it).
| uint64_t mizu_morsel_layout | ( | mizu_morsel_hdr * | h, |
| uint64_t | n, | ||
| uint64_t | morsel_size, | ||
| uint64_t | desc_len, | ||
| uint32_t | x_type, | ||
| uint64_t | x_len, | ||
| uint32_t | out_type, | ||
| uint64_t | out_m, | ||
| uint32_t | claim_n | ||
| ) |
Fill *h with the section layout for a map of n elements (morsel_size the geometry input, desc_len the descriptor stream size, x_type/x_len the optional raw x section — x_len must be exactly n elements — out_type/out_m the optional template output area, claim_n the CLAIM word count).
Sections land 64-aligned in order: descriptor, x, morsel state, output. Returns the region size, 0 on invalid geometry or overflow (the binding raises its own error). The caller memcpys the header into the fresh region.
| const char * mizu_morsel_hdr_check | ( | const void * | base, |
| size_t | size, | ||
| mizu_morsel_hdr * | out | ||
| ) |
Validate a region's header against the mapping size, including section placement.
Returns NULL and fills *out (when non-NULL) on success, else a static message. Runs once per mapping.
| int mizu_morsel_next | ( | void * | base, |
| const mizu_morsel_hdr * | h, | ||
| mizu_morsel_sizer * | sz, | ||
| uint32_t | r, | ||
| uint32_t | gen, | ||
| const mizu_pool_sig * | sig, | ||
| uint64_t | pin_k, | ||
| double | now, | ||
| uint64_t * | m_out, | ||
| uint64_t * | k_out, | ||
| int * | help_out | ||
| ) |
One whole batch transition — generation-fenced lane claim, cancel and pool-signal checks, sized cursor issue — over the mapped region.
Returns 1 and fills m (the 0-based first morsel), k (morsel count after the final partial grant) and help (the doorbell flag); 0 means stop: the lane was lost to the trim or a reset re-armed it, the cancel word fired, the pool is stopping, the owner died, or the cursor is exhausted. sig (may be NULL) is the worker-local pool-signal trio; pin_k != 0 fixes k (a test entry, bypassing the sizing policy); now is the caller's mizu_now() read. Atomics only, no park: safe to call under a global lock.
| uint32_t mizu_morsel_reset | ( | void * | base, |
| const mizu_morsel_hdr * | h | ||
| ) |
Prepared-run re-arm, O(1) in n (no per-morsel state exists to clear): bump the generation, stamp (new_gen << 2) | IDLE over the CLAIM array, zero the cursor, clear the cancel word.
The stamped generation is the fence against a stale trimmed runner from the prior run: its first-call CAS expects the old generation and fails against the re-armed word however the reset interleaves. Returns the new generation — the value the next run's runner payloads must carry.
| int mizu_morsel_abandon | ( | void * | base, |
| const mizu_morsel_hdr * | h, | ||
| uint32_t | r, | ||
| uint32_t | gen | ||
| ) |
The exhausted-runner trim's CAS, folding its own trigger: a no-op unless the cursor is exhausted or the cancel word is set.
Returns the morsel-state verdict: MIZU_MORSEL_ABANDONED (won, or already trimmed), MIZU_MORSEL_RUNNING (executing or published — collect it), MIZU_MORSEL_IDLE (trigger unarmed: defer rather than park).
| void mizu_morsel_cancel_set | ( | void * | base, |
| const mizu_morsel_hdr * | h | ||
| ) |
The cancel word: set by the submitter on timeout / interrupt / death, and by an erroring runner itself before its ERR publish — the fail-fast store that stops every peer within ~a batch.
Idempotent.
| void mizu_morsel_span_of | ( | const mizu_morsel_hdr * | h, |
| uint64_t | m, | ||
| uint64_t | k, | ||
| uint64_t * | lo, | ||
| uint64_t * | hi | ||
| ) |
A batch's element range, 0-based half-open: [m * morsel_size, min((m + k) morsel_size, n)).
Bindings with 1-based inclusive ranges add 1 to lo.
| void mizu_pool_eval_mark | ( | mizu_pool * | , |
| int | in_flight | ||
| ) |
The eval-in-flight marker a binding sets around its catching = 0 task eval only (R: around Rf_eval, never the decode), so its unwind path can tell a task error from infrastructure failure.
The core heals it at worker step/run entry.
| int mizu_pool_unwind_sink | ( | mizu_pool * | , |
| mizu_result_sink * | out | ||
| ) |
The unwind path for a binding whose catching = 0 exec abandons (R's longjmp): when the eval marker says a task eval was in flight, clears it and mints that task's sink (from the identity the core saved at execute) for the err publish.
Returns 1 then; 0 when the abandonment came from outside any task eval — infrastructure failure, the worker goes down.
|
inlinestatic |
The zc refcount / flags words of a MIZU* region header (bytes [24-31] of the reserved band — the wire-format offsets MIZU_ZC_REFCOUNT_OFF / MIZU_ZC_FLAGS_OFF in mizu.h).
Shared by the core release machinery and the bindings' view wrap/resolve paths. Dual form (see the banner): the exported symbols in src/ext.c are these same bodies.
|
inlinestatic |
The string block's section offsets for n strings (each section 64-byte aligned from the block start).
Dual form, like the helpers below.
|
inlinestatic |
A directory entry's sexptype: MIZU_MIZL_S4 masked off, the remainder a listed tag — 0 (a serialized leaf), the atomic tags, STR, VEC, INT64, and the remote leaf (33).
Anything else unlisted rejects.
|
inlinestatic |
One per-element NA test per NA-capable wire type (anything else is never null): INT32_MIN for LGL and INT, INT64_MIN, and the NA_real_ payload discriminated from other NaNs — any NaN whose low word is 1954 (0x7A2), R's own ISNA test, so a writer's quiet-bit-clear NA_real_ (R's verbatim form) and the wire's quiet-bit-set twin both read as NA while a genuine NaN stays a value.
The CPLX pair either part.
|
inlinestatic |
One MIZL directory entry's checks, shared by mizu_mizl_check's pass and mizu_mizl_elem: alignment and extent, the attrs tail, the listed tag, and the length against the leaf kind — the string block's fixed sections for STR, the element extent for an atomic leaf.
A remote leaf (tag 33) skips the attrs-tail and body checks: the span is the identifier (1–255 bytes), and length / attrs_size describe the referenced column as resolved; the S4 bit rejects. Fills *e except the valid pair.
|
inlinestatic |
seconds (a binding's convention; <= 0 polls, non-finite waits indefinitely) to the core's timeout_ms (0 polls, < 0 indefinite).
A finite seconds whose product would overflow reports indefinite: the wait is unbounded either way, and downstream deadline arithmetic stays in double range.
|
inlinestatic |
The flat MIZH header: magic, type, element count, zero attrs, and the reserved band [24, 64) zeroed.
Write BEFORE mizu_stage_retain_zc: the zeroing covers the zc refcount word (a recycled region carries a stale count), and the retain stores the producer loan after.
|
inlinestatic |
Validate the MIZH header at base: region size, magic, a known atomic wire type, the element/attribute extents against the region size, the format flags word, and the validity section (handed back through valid) — 0 on success, -1 on any rejection.
Attribute policy (whether the word at [16, 24) may be nonzero, and what it means) stays binding-side.
|
inlinestatic |
Validate the MIZS header at base against the region: magic, the format flags word, and the string block's fixed sections plus the two end offsets (the block's and the attrs blob's) — 0 on success, -1 on any rejection.
The block's own validity bitmap serves where MIZH / MIZL carry the header section, so there is no valid out-param.
|
inlinestatic |
Validate the MIZL header at base and its directory's extent against the region: magic, the format flags word, every entry (mizu_ext_mizl_ent), and the header's validity pair — whose count totals the nulls across the leaves, so the directory entry count does not bound it; the sum of the atomic leaves' lengths does.
0 on success, -1 on any rejection.
|
inlinestatic |
One bounds-checked MIZL directory entry (index i), with its validity-table pair: a {0, 0}/{0, -1} header state covers every leaf; otherwise the table's entry i is validated by the MIZH rule against the leaf's length.
0 on success, -1 on any rejection.
|
inlinestatic |
The sentinels-to-bitmap scan: bit (bit_off + i) of bitmap records element i of src, 1 = present (the MIZS string block's convention); a type with no missing sentinel is all-present.
Returns the null count.
|
inlinestatic |
The bitmap-to-sentinels copy: n elements land at dst, the clear bits' elements written as the wire type's missing sentinel.
Returns the null count.
| mizu_status mizu_ix_open | ( | mizu_ix * | cur, |
| const void * | buf, | ||
| size_t | len | ||
| ) |
Open a cursor over a whole stream: the magic and version checks, the unknown version taking the informative "the peer uses a newer format" decline.
The stream is borrowed, never copied: buf must outlive the cursor, and item spans (mizu_ix_item.ptr) point into it. MIZU_OK, or MIZU_ERR with the error in the TLS slot.
| mizu_status mizu_ix_next | ( | mizu_ix * | cur, |
| mizu_ix_item * | item | ||
| ) |
The next item: bounds, depth, UTF-8 and grammar checks per the spec.
MIZU_OK and *item filled, or MIZU_ERR with the informative text in the TLS slot (an unknown tag or task kind declines as "the peer uses a newer format", never a bare corrupt-stream error).
| mizu_status mizu_ixt_open | ( | mizu_ix * | cur, |
| const void * | buf, | ||
| size_t | len, | ||
| mizu_ix_item * | item, | ||
| int | max_kind | ||
| ) |
The task-stream decode shim (DESIGN.md's byte-shape helper registry): the per-field shape checks every task-stream decode shares, once.
Exported only — it wraps the cursor and records through the TLS slot, the cursor's own form. mizu_ixt_open opens the cursor, pulls the first item and requires the TASK header with task_kind <= max_kind — the decode site's ceiling within the cursor's registry, so a runner stream at the exec decode fails informatively instead of misparsing. On success *item is the header item, the submitter identity riding item->u64[0]; the binding stashes it by its own discipline. Every failure is MIZU_ERR with the text in the thread-local slot.
|
inlinestatic |
The byte-shape helpers' bodies (DESIGN.md's byte-shape helper registry): pure byte math over the wire constants, dual-form below.
Strict RFC 3629: rejects overlong forms, surrogates, and code points past U+10FFFF. The empty span is valid.
|
inlinestatic |
The wire-type -> vector-tag mapping; 0 (MIZU_IX_TAG_NIL — never a vector tag) for any other input.
|
inlinestatic |
tag + one trailing u64: the vector/container/bytes wire shape
|
inlinestatic |
tag + u8 length + identifier bytes; 0 is malformed on the wire, so the helper refuses it the put_vec way (0 returned, nothing written)
|
inlinestatic |
The UTF-8-boundary floor: n itself within share, else the largest len <= share that does not split a multi-byte sequence.
|
inlinestatic |
The bounded budget (DESIGN.md's err framer): type capped at its share, message at half the inline budget, detail at what remains — each floored at a UTF-8 boundary, so the stream fits the slot by construction (inline_max >= MIZU_IX_ERR_OVERHEAD) and the writer cannot fail. The spans are valid UTF-8 by the caller's guarantee.
|
inlinestatic |
The emit helpers: each returns its byte count, writing only when dst is not NULL, so a binding's two-pass walk sizes (dst NULL) and writes through the same byte-level code.
The write pass relies on the size pass's count, so no limit is carried. Bodies are impl delegations, the dual-form single-sourcing discipline. Strict RFC 3629 UTF-8 validation: 1 when the n bytes at s are a well-formed byte sequence (shortest forms only, no surrogates, no code point past U+10FFFF), 0 otherwise. The empty span is valid; s shall be NULL only when n is 0. A pure predicate: O(n), no allocation, no error record.
|
inlinestatic |
A complete 'I' err stream in one call: the two-byte header, then the 0x11 err item, under the bounded budget (DESIGN.md's err framer) — type truncated at MIZU_IX_ERR_TYPE_SHARE, message at half the inline budget, detail at what remains, each cut at a UTF-8 boundary.
inline_max shall be at least MIZU_IX_ERR_OVERHEAD; the return is then at most inline_max, so the caller stamps INLINE with the keeperless claim and the writer cannot fail. The three spans shall be valid UTF-8; the framer never validates. A NULL dst is the size query.
|
inlinestatic |
The string forms' per-batch element access: utf8 (i32 offsets), large_utf8 (i64), string_view (16-byte views).
0 on a null (ptr/len unset), -1 on a malformed record. The Arrow C Data Interface carries no buffer lengths, so beyond the view form the byte extent stays the producer's word; what is checked here are the catastrophic cases — a view's buffer index past the buffer array, negative lengths or offsets, non-monotonic offset spans (which would size one way and gather another), and, for a producer that wrote the spec's sizes array, a view span past its declared size (a view naming the trailing slot proves a no-sizes export, and its span crosses unbounded).
| void mizu_cvt_convert | ( | uint8_t * | dst, |
| const uint8_t * | src, | ||
| const uint8_t * | valid, | ||
| uint64_t | off, | ||
| size_t | n, | ||
| const mizu_cvt_row * | row, | ||
| mizu_arrow_warn * | warn | ||
| ) |
One run's convert into a reserved destination: the BIT_LGL / masked / plain dispatch.
src points at the first source element (the caller applies the element offset); off is the bit offset into data and validity for the bit-packed forms, and into validity for the masked forms. valid NULL converts every lane. The warning counts accumulate into warn.
| mizu_status mizu_arrow_open | ( | mizu_arrow ** | out, |
| const ArrowSchema * | schema, | ||
| const ArrowArray *const * | batches, | ||
| size_t | n_batches, | ||
| uint32_t | peer_caps | ||
| ) |
Classify every column off the schema (a struct schema; the type map of DESIGN.md's frame section), vet every batch (buffer counts, offsets, null_count range, column lengths, a uint8 column's nulls, unnamed / duplicate names, a zero-column frame) and validate the dictionaries (batch 0's levels canonical, a null level or a level set that differs across batches declines, every index range-checked).
batches are caller-owned and must outlive the handle; the writer reads them only inside mizu_arrow_write. peer_caps is the peer's capability mask: the stage's capability conjunction and the per-column leaf choices key off it. MIZU_OK, or MIZU_ERR with the decline in the thread-local slot (mizu_last_error_message) naming the column, its format and the remedy.
| void mizu_arrow_col_info | ( | const mizu_arrow * | , |
| int64_t | col, | ||
| mizu_arrow_col * | out | ||
| ) |
The per-column classification, for a binding's own emitters over the same batches (pymizu's inline frame forms).
col shall be in range.
| size_t mizu_arrow_col_body | ( | const mizu_arrow * | , |
| int64_t | col, | ||
| uint8_t * | dst, | ||
| mizu_arrow_warn * | warn | ||
| ) |
One column's by-value body bytes over every batch — the cvt lanes, a dictionary's 1-based codes, date / timestamp / duration doubles; the same loops the MIZL writer's leaves use, for a binding's own 'I' emitters.
Returns the byte count; dst NULL sizes. 0 for a string column: the 'I' string form is the binding's per-element emitter (mizu_arrow_str_at), the MIZS block region-only. The converting form always — the MIZL-only verbatim leaves (the ns INT64 counts, the u64 lanes) never appear here. The conversion counts accumulate into warn like the MIZL write's.
| size_t mizu_arrow_col_blob | ( | const mizu_arrow * | , |
| int64_t | col, | ||
| uint8_t * | dst | ||
| ) |
One column's attribute dict as a bare 0x0d dict value (the §3.5 blob without its stream header — a binding splices it into its own 'I' attr form).
Returns the byte count; dst NULL sizes. 0 for a plain column (CVT, STR): the by-value tiers carry no attribute dict on them, and the MIZL-only marker pairs (tscale, u64) never appear here.
| size_t mizu_arrow_write_ix_col | ( | const mizu_arrow * | , |
| int64_t | col, | ||
| uint8_t * | dst, | ||
| mizu_arrow_warn * | warn | ||
| ) |
One column's 'I' value form (the inline small-table path): the plain vectors bare (the cvt tag off the row's wire type; the strv form over mizu_arrow_str_at, a malformed record put as NA like a null), the attr-marked forms splicing mizu_arrow_col_blob's bare dict.
Returns the byte count; dst NULL sizes. col shall be in range (0 returned otherwise). The conversion counts accumulate into warn on the writing pass only.
| size_t mizu_arrow_write_ix | ( | const mizu_arrow * | , |
| uint8_t * | dst, | ||
| mizu_arrow_warn * | warn | ||
| ) |
One complete 'I' stream of the whole frame (the small-table path): the header, the attr mark, the list of the columns' value forms (mizu_arrow_write_ix_col), then the root dict {names, class = "data.frame", row.names = c(NA, -n)} — one C write straight off the Arrow buffers where a below-gate frame previously converted back to a language value first.
The root dict is always the automatic form (the root-blob override is the region writer's); the MIZL-only leaves (the ns INT64 counts, the u64 lanes) never appear. Returns the byte count; dst NULL sizes. The conversion counts accumulate into warn on the writing pass only.
| int mizu_arrow_remote | ( | mizu_arrow * | , |
| int64_t | col, | ||
| const char * | ident, | ||
| size_t | ident_len, | ||
| void * | ref_base, | ||
| size_t | ref_size, | ||
| int64_t | ref_leaf | ||
| ) |
A remote-leaf override (pymizu's provenance match): column col crosses as directory tag 33 — the identifier span, the referenced leaf's attrs size and validity claim read off the referenced region (mizu_mizl_elem), and one REFHELD OR per distinct referenced region at write.
ident is the referenced region's name, ref_base / ref_size its mapping, ref_leaf the referenced 0-based directory leaf; register only for a peer with MIZU_CAP_MIZL_REF. 0, or -1 on a torn referenced tree (the caller leaves the column local).
| void mizu_arrow_root_blob | ( | mizu_arrow * | , |
| const void * | blob, | ||
| size_t | len | ||
| ) |
The root-blob override: a complete 'I' stream for the frame attributes the caller built (pymizu's custom row_names).
Absent, the writer emits names + class data.frame + the automatic c(NA, -n) row.names.
| mizu_status mizu_arrow_stage | ( | mizu_handle * | , |
| mizu_arrow * | , | ||
| mizu_slot_hdr * | hdr, | ||
| unsigned char * | payload, | ||
| mizu_arrow_warn * | warn | ||
| ) |
The one-call stage over the handle seam, valid only during a stage_fn call.
First the gate, touching no shared state: the capability conjunction (MIZU_CAP_MIZL | MIZU_CAP_ATTRS, plus MIZU_CAP_MIZS when any column is a string) off the classification and the peer_caps given at open, then the churn check. Then mizu_stage_reap, the spill checkout, the write, the SHM_VEC stamp (name payload, aux type VEC + used bytes) and mizu_stage_retain_zc. MIZU_OK staged; otherwise a non-OK status — the gate's decline or the spill status when no region — with nothing claimed: the caller's copy tiers take every non-OK alike. The caller pins nothing: the region is self-contained unless it registered remote leaves, whose pins are its own. warn (zeroed here) returns the conversion counts; the binding raises them after the stage returns.
| void mizu_tune | ( | void | ) |
malloc tuning (tune.c), called by the binding at load.
MIZU_EXT_H