binary compression · typed apis · rust core
hyperfly
Binary compression for typed APIs at the edge of entropy.
Your schema already fixes every field name, type, and bound. Your traffic already reveals what the values usually look like. Hyperfly compiles both into a binary protocol for that exact route — and speaks JSON to anything that hasn't been told.
What never leaves the machine
A typed API is a contract. Everything the contract already settles is not data — it is repetition, sent again on every response, to a peer that could have derived it.
- field namesposition is the name
- type descriptorsthe codec is already typed
- object structurecompiled into the reader
- enum membersan index into a set both sides hold
- value boundsan offset from the declared minimum
- optionalityone bit in a shared bitmap
What remains is the part your API actually had to say.
The size of a response
A general-purpose compressor rediscovers your structure on every response, from scratch, with no idea what the next byte is allowed to be. A compiled codec never has to.
500 messages · 20–50 audit records each · recurring actors and user agents
An audit log repeats itself across requests, not within one — the same actors, the same user agents, request after request — and its ids are machine-made: evt_, a sequence number, eight hex digits. The profile learns the recurring values as a dictionary, the id shape as a grammar whose lanes ship the sequence as deltas and the hex at four bits a digit, and actorEmail as a function of actorId, which then costs nothing at all. The profile takes 1 439 bytes off this route.
Bytes per message, averaged over 500 independent responses per route, measured with the TypeScript reference implementation — reproduce with `bun run bench`. Each Hyperfly row adds exactly one thing: schema-compiled layout, then a dictionary trained on the route's own traffic. That dictionary is an out-of-band artifact; the repo reports its size and how many requests it takes to pay for itself — ten for events, thirty-five for orders. The Brotli rows are q4, the level edges actually run on dynamic responses. Protobuf gets proper enums and int64. Corpora are synthetic but shaped like real routes: a fixed device fleet, a fixed product catalogue, a recurring cast of authors.
Schema in, protocol out
Four stages. The first is enough to encode. The second is what makes it fast.
- 01
Schema
Zod or Pydantic in, canonical IR out. Field order, types, bounds, and enum sets become one stable description that both sides can agree on.
CandleResponse → ir:8f3c91 - 02
Profile
Optionally, sampled traffic for a single route: value distributions, repetition, cardinality, monotonic runs — what the data usually is, not what it could be.
/v1/candles ← 1.2M samples - 03
Planner
The planner picks a codec per column: dictionary, delta, frame-of-reference bit packing, grammar lanes for machine-made ids, one column derived from another — or raw, when nothing beats raw.
plan · 12 columns · 5 codecs - 04
Wire
The result is an immutable codec profile — one artifact, identical behaviour on every runtime, negotiated per request.
hf · v1 · fp:8cf38e4e
The envelope
Nineteen bytes of header, then the payload. The fingerprint names the entire codec — same schema under a different plan is a different artifact, so bytes are never misread, only refused.
- A peer that does not hold the profile says so in the request, and gets JSON. There is no failure mode where the response is unreadable.
- Profiles are content-addressed and immutable. Retraining produces a new profile; it never mutates one that is already in flight.
- The same bytes come out of Rust, Node, Python, and the browser — or it is a bug, not a dialect.
Two lines at the boundary
The schema is the configuration. Compile it once, then encode and decode where you already serialise.
import { compile } from "hyperfly/zod";
import { CandleResponse } from "./schema";
const codec = compile(CandleResponse);
const bytes = codec.encode(response);
const value = codec.decode(bytes);Planned API. Nothing is published yet.
Properties
Schema-aware
Compiled from the types you already ship. No second schema language to maintain.
Production-trained
Profiles built from observed traffic on one route, not from a guess about the average payload.
Rust-native
One core, bound outward. The planner and the codecs are the same code everywhere.
Browser-ready
A decode path that runs in the tab, without a proxy or a round trip to translate.
Transparent fallback
Version negotiation up front. An unknown profile gets JSON instead of an error.
Reproducible
Benchmarks you can run against your own payloads, not screenshots of ours.
Architecture
One core, one intermediate representation, one artifact. Everything above it is a binding.