Open-source market infrastructure · Gate 1 in progress
Astral makes it possible to determine whether a trading idea actually survives fees, slippage, latency, liquidity and realistic fills — and to reproduce that determination later.
$ astra-record capture --venue binance --symbol BTC/USDT --channel book_diff
frames 601
checked 601
conn_gaps 0
seq_gaps 0
stop_reason duration_elapsedStart here
No keys, no network except the first capture. Every step prints what it did and refuses bad inputs loudly.
$ astra-record capture --venue binance --symbol BTC/USDT --channel book_diff
$ astra-record check --input ./demo # verdict healthy
$ astra-normalize --input ./demo --output ./norm # twice: identical bytes
$ astra-replay --input ./demo --seed 7 --strategy book-top # one hash, any seed
$ astra-exec --input ./demo --side buy --price 100000 --quantity 0.001 --fee-bps 5
$ astra-harness run --input ./demo --config ./bench.json --record ./experiments
verdict reproducedNo venue access? Skip the capture entirely: the seed corpus verifies offline — run astra-harness verify --registry benchmark-corpus/registry --hash cb050f8f7f0384baa45bd434237da76f0e4b1a76d1fc144e95e9abcea34f78f7 --input benchmark-corpus/capture and it must print verdict reproduced. The loop runs before anything is recorded.
What is Astral
They ask would this idea have made money — on data nobody can inspect, with fills nobody can verify, under assumptions nobody wrote down. A quoted spread is treated as an executable opportunity. A paper result is presented as performance.
Astral asks a smaller, harder question instead: can we measure whether an apparent opportunity is actually executable — after fees, slippage, latency, liquidity, market impact, execution constraints, realistic fills and capital constraints?
It answers with a continuously-collected, provenance-tracked market-data corpus, a deterministic replay engine, and an honest execution-cost harness. Every result carries its dataset hash, its assumptions, and a reproduction command. A rigorous “this has no edge” is a successful result here.
Architecture
Raw venue bytes in. Reproducible claims out. Nothing in between is a black box.
Raw frames written to compressed, content-addressed chunks with capture timestamps and a SHA-256 integrity index. Binance depth diffs, partial books, trades, book ticker, funding and liquidations — plus Bybit order books on spot and perp. Read-to-stored p99 around a millisecond on live feeds (per-venue table and methodology in the README verification record — live-measured, not reference hardware). Drops and gaps are recorded as data, never absorbed silently.
L2 books rebuilt from diffs on a snapshot bootstrap, with venue update-ID continuity enforced. A broken book rejects events until a new snapshot — it never silently resumes. The live book updates as frames arrive at single-digit-microsecond p50 (Binance p99 68µs, Bybit p99 32µs, live-measured).
Same capture replayed twice: identical signal hashes, verified live — over book diffs, trade prints (bundles expanded per print), and top-of-book. Cross-machine proof awaits a second machine — the procedure is written (docs/cross-machine-repro.md), stated, not assumed.
Open source
Apache-2.0. The full research loop runs locally on your own machine — record, verify, reconstruct, report. Everything essential is in the open.
crate · astra-types
Fixed-point decimals, nanosecond timestamps, venue and instrument identifiers, capture records with quality flags.
crate · astra-record
init · capture · check · reconstruct · verify — the lossless recorder and its audit tooling.
crate · astra-book
Venue-neutral level updates, snapshot bootstrap, continuity enforcement, top-of-book comparison.
crate · astra-normalize
Exact-decimal datasets with whole-batch validation, Hive-partitioned, byte-deterministic across runs — and chunk-streamed, so a 200k-frame capture normalizes in 28 MB instead of 256.
crate · astra-replay
Event core (book, trade, top-of-book), seeded strategies, and a CLI — same capture in, identical signal hashes out. Streams chunk-by-chunk with sequence enforcement, and replays Parquet trees with identical signals to raw. Spec and repro procedure in docs/, soak operator in ops/.
crate · astra-exec
Limit-maker simulation over trade prints — through-print fills, maker fees, gaps void — plus post-fill markouts per horizon (adverse always negative, gaps report unknown) and a probe CLI that runs resting orders over real captures. Calibration and shadow mode open.
crate · astra-harness
One replay pass plus N probes into a canonical hashed JSON report — plus a local registry with verify reproduction verdicts and a sweep subcommand running whole config directories into comparison tables. Same inputs, byte-identical outputs.
crate · astra-python
Exact decimals, conservative fills, and clock reads — plus the full record → normalize → replay loop from a notebook with built-in strategies. Embedded-tested, shipped as a maturin wheel, CI-verified on install.
The contracts behind the crates — read these before citing a result.
doc · schema
The Parquet format the normalizer targets: book_diff, trade, top_of_book tables, partitioning, validation rules.
doc · replay
The contract the engine satisfies: event model, ordering, virtual time, gap semantics, determinism rules.
doc · repro
How a second machine reproduces a result: three hash equalities, exact commands, known limits.
doc · fills
When a resting limit order counts as filled: trade-print-through only, maker fees, gaps void. Spec written, code open.
doc · audit
145 tests, zero lints, 3 bugs fixed same-day, 13 open findings ranked. The throttle report.
doc · soak
Why the soak runs on a cheap VPS instead of a laptop: sizing, shakedown, supervision, judging.
doc · bench
One capture plus one config into a hashed report: canonical JSON, fail-closed parsing, determinism rules.
doc · status
Numbers, flowcharts, gates, and what ends the OSS — every figure sourced from the repo itself. Supersedes 2026-10-10.
doc · policy
What each subcommand does with the same corruption: abort, finding, skip, or null-row — read off the code.
doc · stream
Chunk-by-chunk replay and normalization that keeps every determinism guarantee — designed first, then built and measured (256 MB → 28 MB).
git clone https://github.com/VarunGore36/AstralProject.git
Research
Every note ships with its dataset, assumptions and reproduction command — including the ones that find nothing.
Verified · live venue data
601 frames captured at 10.1/s against the live book-diff feed. All 601 continuity-checked, zero gaps, chunk hash independently cross-checked.
Verified · live venue snapshot
400-frame capture against a mid-stream depth snapshot: 133 pre-snapshot events skipped — matching an independent count exactly — 267 applied, zero gaps.
Verified · venue-published depth
Best bid/ask and levels 1–2 match the venue's own published book on all 300 checks. Deeper mismatches trace to the venue's servers disagreeing with each other — stated, not hidden.
Verified · live venue data
Second venue connected with its own subscribe protocol: 1 snapshot + 316 deltas on spot, 1 snapshot + 374 deltas on perp, zero gaps on both — versions continuity-checked on every market frame, and in-band snapshots bootstrap the book mid-stream.
Probed · auth wall
The venue refuses unauthenticated level2 subscriptions outright. Ticker and trades are public but carry no checkable sequence — so no rule is claimed. Evidence over assumptions.
Verified · live venue data
Third venue connected: 33 genuine matches and 35 tickers captured live. No order book exists publicly, and the shared sequence admits no strict rule — so both report unchecked, honestly.
Verified · live replay
151 live frames replayed ten times under one seed: ten identical signal hashes. Repetition is the test — passed on real data.
Verified · same-connection books
One connection, two channels, a 1000-level bootstrap: 149 of 150 depth-10 checks exact. Partial bootstraps diverge by rank drift; two connections by server disagreement. Both mechanisms proven, not assumed.
Verified · live replay
96 live frames replayed under two seeds: identical signal hashes. Seed-independence was expected — now it is measured.
Measured · synthetic scale
300k synthetic frames replay chunk-by-chunk with identical signal hashes to load-all (141 MB → 12 MB on the quiet strategy); 200k normalize at 28 MB (was 256) with identical file layout. Synthetic inputs, real guarantees.
Verified · fixture equivalence
A mixed 7-frame capture — diffs, bundle trades, topbook, gap, one unparseable frame — emits byte-identical signal hashes through raw and normalized replay. The null row replays as a counted skip in both.
Seeded · bronze corpus
benchmark-corpus/ holds a 10-frame capture, a config, and a recorded registry run that verifies reproduced immediately — honestly labeled bronze, with tier graduation rules written down.
Enforced · CI gate
ops/determinism.sh check runs on every push: normalize-twice byte equality plus pinned replay and tree hashes over an 8-frame fixture. Verified green — and verified red on a doctored index.
Procedure defined
Same capture bytes plus same commit plus same seed must yield equal verdicts, equal Parquet tree hashes, and equal signal hashes. The steps are written down — awaiting a second machine.
Published · research note 003
14,979 prints over 30 minutes: 529 in the busiest second, silence in a fifth of them, 73% sharing a millisecond. Bars average what never existed.
Published · research note 002
14,979 prints over 30 minutes: median $8, top 1% carries 46.4% of volume. Equal-weighted signals let dust outvote size.
Published · research note 001
15,655 live prints over 10 minutes: per-minute buy share swings 12.8%–95.3% by count but sums to 50.6% by volume. The naive signal dies at fees — a first-class negative result.
Published · research note 004
BTC long-spot/short-perp over Q1 2024: $556.74 funding against $20 fees and $12.88 of adverse basis drift — net +$523.85 on $20k. One window, hourly bars, every assumption on the table.
Published · research note 005
Identical hold over Q4 2022: $78.08 funding against the same $20 fees — net +$55.51, inside every unmodeled cost at once. Carry has weather; fees are constant.
Published · research note 006
Enter rich premiums, exit weak ones: drift flips −$53 to +$255 — but 163 round trips cost $3,260 against $455 harvested. Selectivity is real edge before turnover.
Published · research note 007
Five hysteresis widths, one monotonic gradient: turnover dominates at every width. The widest band scrapes +$81 and still trails doing nothing by $300.
Published · research note 008
All of 2023, no chosen windows: unconditional +$746.55 against timed +$241.48. Eleven of twelve months paid; turnover is the only variable that ever decides the sign.
Published · research note 009
One week, ~15M prints, spot vs perp: the US third carries 42–44% on both; spot taker-buy slides 44%→38% while perp sits at 50%; both batch half their prints into shared milliseconds.
Published · research note 010
OI flushed −7.4% the day before the March 2023 spot halt, then the long/short ratio flipped 0.70→0.94 while spot was dark. Halting one venue relocates positioning; it does not freeze it.
Published · research note 011
Hourly taker-buy share predicts the next hour at r=+0.36 — identically in Asia, EU, and US. Direction varies by session; memory does not.
Published · research note 012
Loud hours carry the full +0.32 while quiet ones read +0.03; perp manages +0.12 and forgets. Persistence belongs on spot, sized by conviction.
Published · research note 013
SVB week at 7× flow: r=+0.49, gradient replicates, session split unresolved within error bars. Suggestive strengthening, established persistence.
Published · research note 014
Luna week: r=+0.62 overall, +0.75 in loud hours, sessions uniform throughout. Calm, SVB, Luna order monotonically — memory follows stress.
Published · research note 015
Batched vs streaming fills agree on everything across 4,096 adversarial streams. Intra-millisecond order never changes a v1 conclusion.
Published · research note 016
Short-spot/long-perp in the most negative month: +$17.37 funding vs $20 fees nets −$2.23 at zero borrow. Carry is one-directional at retail size.
In progress
Cross-exchange spreads measured as fill probability plus cost-adjusted edge — never as P&L. The platform’s calibration benchmark.
Planned
Can a simple delta-neutral funding strategy survive realistic fees, slippage, basis movement and execution constraints? Asked honestly, answered with data.
Audit · 2026-10-06
Full suite, lints, panic-surface scan, and a line-by-line review of every production path in the measurement pipeline. Three bugs fixed same-day. Thirteen findings ranked; twelve closed since, one documented open. Full detail in the audit doc.
17 book · 16 normalize · 86 record · 10 replay · 15 types, plus one overflow guard each at three sites. Zero ignored, zero failed.
cargo clippy --workspace --all-targets -- -D warnings passes; cargo fmt --check clean. The lint caught a dead assignment in the auditor's own fix mid-audit.
Every production unwrap inspected. All live in test modules except one guard-proven framing read — flagged as hygiene, not a bug.
Fixed same-day
Fixed · overflow
Live tracker, offline audit, and reconstructor computed previous + 1 on venue-supplied IDs — exit 101 in debug on a hostile payload. Now checked_add with regression tests at all three sites.
Fixed · soak behavior
Waits doubled without reset across a long soak, while a failed attempt already ends the capture. Now a fixed 250ms wait; dead constant removed.
Fixed · audit gap
check trusted the manifest it auditedReplay and normalize enforced the frame count; the soak's judge did not, and never printed stop_reason. Now reports manifest_mismatch as unhealthy.
Closed since the audit
Closed · overflow
Three previous + 1 sites panicked in debug on hostile IDs — proven exit 101. Now checked_add with a regression test each.
Closed · backoff
Waits doubled toward 30s across long soaks for no benefit. Now a fixed 250ms — a failed attempt already ends the capture.
Closed · manifest trust
check verifies frame countsReplay and normalize enforced the manifest; the soak's judge did not. Now reports manifest_mismatch as unhealthy, plus stop_reason.
Closed · schema gate
check, normalize, replay, and reconstruct refuse a newer schema_version — one gate test per reader.
Closed · book errors
Bad levels no longer vanish: every capture prints updates and errors side by side, covered by a hostile-diff test.
Closed · output contracts
Every report renders through a golden-tested library function — including the frames and verdict lines the soak scripts parse.
Closed · decimal range
Hostile 13-digit values parsed fine but never fit Decimal128(20,8). All three validators reject out-of-range batches whole.
Closed · sides & prints
Side labels fail closed to Buy/Sell-or-unknown; non-positive prints can never fill. Fills need real counterparties.
Closed · gap accounting
reconstruct counts connection gapsMarkers classified on the walk — connection-type counted, sequence-type left to the book's own break — and undecodable markers abort like everywhere else.
Closed · streaming
Replay, normalize, reconstruct and compare walk chunks with O(chunk) inputs: 300k frames replay at 12 MB (was 141), 200k normalize at 28 MB (was 256). Same-capture bytes unchanged; cross-chunking equality at read-back level.
Closed · unification
replay_normalized feeds Parquet trees to any strategy with byte-identical signal hashes to raw replay. The policy table's last structural divergence is resolved, not documented.
Closed · snapshots
Both --snapshot loaders share one venue-aware refusal: Binance format spelled out, Bybit payloads told no file is needed. No format parsed from memory.
Still open, ranked
Closed · 2
Starting where chunks exist errors loudly instead of colliding sequences. Resume is future work; every live path uses fresh dirs.
Closed · 4
reconstruct shares the strictnessManifest-gated, connection gaps counted, sequence breaks counted — the full check prelude, minus only the verdict itself.
Open · 9
The final disconnect ends the capture with no trailing gap marker — correct, since nothing follows, but the soak judge must read the manifest's stop_reason, never the stream. Documented, one item left.
Next: evidence
Twelve of thirteen findings closed plus the streaming, unification, and snapshot follow-ups. What remains needs wall-clock time (soak, shadow) or other humans (reproductions) — not more auditing. The VPS run is the next gate.
Principles
Same inputs, same code, same result — on any machine. Every claim carries a reproduction command.
No fill without a trade print through your price. No quoted spread presented as an opportunity.
Every byte traced to its venue, its capture, its schema version. Quality tiers, not blind trust.
Backtests are labelled backtests. Paper is not live. Optimistic models are marked unrealistic in the output itself.
Fees, slippage, latency, capital, execution model — on the chart, not in a footnote.
“No edge” is a first-class, publishable finding. Refutation is the mechanism that makes positive claims worth trusting.
Roadmap
Each stage must earn the next one with evidence. If a gate fails, the project stops or pivots — that outcome is designed in.
Lossless recording, verified book reconstruction, offline audit. Judged by a 72-hour soak: ops/soak.sh check reports a per-stream verdict and exits non-zero on any issue — zero unexplained loss required. Latency methodology published; cross-machine procedure defined.
Shadow mode measures predicted fills against what was actually tradable. Dies if model error exceeds ~10 bps on liquid pairs.
Three honest early notes plus a first funding-carry window (+$523.85, every assumption stated). Dies if three months of real use produces nothing substantive — a live-captured carry study is the next candidate.
A seed corpus reproduces out of the box and the determinism gate runs in CI — the machinery is ready. Dies quietly if nobody comes: 5 users, 2 external reproductions.
Further stages are defined from what Gate 4 teaches, not before. The public roadmap ends where the evidence ends.