binary compression · typed apis · rust core
hyperfly
Binary compression for typed APIs at the edge of entropy.
Your schema already fixes every field name, type, and bound. Your traffic already reveals what the values usually look like. Hyperfly compiles both into a binary protocol for that exact route — and speaks JSON to anything that hasn't been told.
What never leaves the machine
A typed API is a contract. Everything the contract already settles is not data — it is repetition, sent again on every response, to a peer that could have derived it.
- field namesposition is the name
- type descriptorsthe codec is already typed
- object structurecompiled into the reader
- enum membersan index, log₂(n) bits wide
- value boundsthe bits the range requires
- optionalityone bit, packed with the rest
What remains is the part your API actually had to say.
The size of a response
A general-purpose compressor rediscovers your structure on every response, from scratch, with no idea what the next byte is allowed to be. A compiled codec never has to.
1 000 OHLCV rows · 7 numeric columns · monotonic timestamps
Columnar layout separates the series; timestamps and prices become deltas against their own neighbours instead of independent decimal strings.
The figures above model the design. They are not measurements, and results depend entirely on the payload — structure compresses, prose does not. Reproducible benchmarks against JSON, gzip, Brotli, Protobuf, MessagePack, and CBOR ship with the first release.
Schema in, protocol out
Four stages. The first is enough to encode. The second is what makes it fast.
- 01
Schema
Zod or Pydantic in, canonical IR out. Field order, types, bounds, and enum sets become one stable description that both sides can agree on.
CandleResponse → ir:8f3c91 - 02
Profile
Optionally, sampled traffic for a single route: value distributions, repetition, cardinality, monotonic runs — what the data usually is, not what it could be.
/v1/candles ← 1.2M samples - 03
Planner
A Rust planner picks a codec per field: entropy coding, dictionary, delta, columnar layout, or raw bytes when nothing beats raw bytes.
plan · 7 columns · 4 codecs - 04
Wire
The result is an immutable codec profile — one artifact, identical behaviour on every runtime, negotiated per request.
hf1 · 0x04 · profile:3f9c
The envelope
Sixteen bytes of header, then the payload. Small enough to read in one glance, versioned so that nothing breaks when the profile moves.
- A peer that does not hold the profile says so in the request, and gets JSON. There is no failure mode where the response is unreadable.
- Profiles are content-addressed and immutable. Retraining produces a new profile; it never mutates one that is already in flight.
- The same bytes come out of Rust, Node, Python, and the browser — or it is a bug, not a dialect.
Two lines at the boundary
The schema is the configuration. Compile it once, then encode and decode where you already serialise.
import { compile } from "hyperfly/zod";
import { CandleResponse } from "./schema";
const codec = compile(CandleResponse);
const bytes = codec.encode(response);
const value = codec.decode(bytes);Planned API. Nothing is published yet.
Properties
Schema-aware
Compiled from the types you already ship. No second schema language to maintain.
Production-trained
Profiles built from observed traffic on one route, not from a guess about the average payload.
Rust-native
One core, bound outward. The planner and the codecs are the same code everywhere.
Browser-ready
A decode path that runs in the tab, without a proxy or a round trip to translate.
Transparent fallback
Version negotiation up front. An unknown profile gets JSON instead of an error.
Reproducible
Benchmarks you can run against your own payloads, not screenshots of ours.
Architecture
One core, one intermediate representation, one artifact. Everything above it is a binding.