Chill Bits

Egis Technical Journal

—

Budget Drift, Exchange Sync, and Transaction Safety

Context

Egis is a mobile-first trading system built with:

  • Laravel 12 / PHP 8.2
  • Inertia.js + React
  • shadcn/ui + TailwindCSS
  • Dark theme only
  • Logged-in system, Admin + User roles separated at UI level

Exchanges:

  • Paper Exchange (simulated, local DB)
  • Binance Futures (TESTNET only in dev)

Key architectural rule:

Internal positions table is the single source of truth.
Exchange tables are mirrors / execution logs only.

1. The Core Problem: Budget Drift Keeps Coming Back

Symptom

Repeated errors like:

  • Not enough budget. Available: $-0.01
  • Available: $0.00, Requested: $9.00
  • Even when no positions are open

This happened multiple times under different forms.

Root Causes (Confirmed)

  1. Multiple sources of truth
    • Budget read from internal positions
    • Execution state lived in paper_positions / exchange_positions
  2. Incorrect filtering
    • Closed positions counted as reserved
    • Wrong bot_id or status used
  3. Rounding drift
    • Requested size vs executed size mismatch
    • SUM() returning decimals slightly below zero
  4. Post-execution sync missing
    • Requested $10
    • Exchange actually executed $8.828
    • Egis still reserving $10

2. Hard Architectural Decisions (Non-Negotiable)

2.1 Single Source of Truth

  • positions table is authoritative for:
    • Budget
    • UI
    • Risk
  • Exchange tables are derived and repairable

If all exchange tables are deleted, Egis must still be logically correct.

2.2 Budget Formula (Frozen Contract)

available_budget =
  bot.initial_budget_usd
- SUM(positions.size_usd WHERE status = open AND bot_id = X)

Rules:

  • ONLY internal positions
  • ONLY status = open
  • NO exchange tables
  • NULL sums must become 0
  • Budget service must be centralized (no inline math)

3. Executed Size Must Replace Requested Size

Real-World Reality

On Binance:

  • User requests $10
  • Exchange executes $8.828 due to:
    • lot size
    • step size
    • rounding
    • partial fills

Fix

After opening a position:

  • Store both:
    • requested_size_usd (audit)
    • size_usd (authoritative, executed)
  • size_usd = executed_qty * avg_entry_price
  • Budget and UI use executed size only

UI improvement:

  • Show $8.83
  • Optionally show (requested $10) for transparency

4. Critical Binance Bug: Order Placed, DB Failed

The Incident

Log:

Binance order placed but DB transaction failed
No query results for model [ExchangePosition]

Meaning:

  • Binance order succeeded
  • DB transaction crashed afterward
  • Egis lost atomicity → drift risk

Root Cause

  • firstOrFail() called after external side-effect
  • DB expected ExchangePosition to exist but it didn’t

Correct Fix Pattern

Never allow this sequence:

External order succeeds
↓
DB fails
↓
System crashes

Required Guarantees

  1. Preflight DB state before external calls
    • If required DB rows are missing → DO NOT place order
  2. OR Safe recovery path
    • If order placed but DB fails:
      • Persist a pending_reconciliation record
      • Log intent + exchange_order_id
      • Return controlled error to UI
  3. Never allow firstOrFail() after an exchange order

5. Intent-Based Order System (Already Saved Egis Once)

Bug That Happened

  • Closing a LONG on Binance uses MARKET SELL
  • Egis interpreted it as OPEN SHORT
  • Phantom short positions created

Final Rule

Every order MUST carry explicit intent:

  • open
  • close

Only:

  • open intent can create positions
  • close intent must reference existing position
  • Close can NEVER create a new one

Regression tests now mandatory.

6. Paper Exchange Reconciliation (DB-Only)

Philosophy

  • Exchange state is priority
  • Paper DB state must follow exchange data

Command

php artisan egis:reconcile-paper-positions

Rules:

  • Dry-run by default
  • --confirm applies fixes
  • --debug shows diffs
  • NO external API calls
  • Never touches internal positions

Fix logic:

  • Exchange OPEN + paper missing → create paper row
  • Exchange CLOSED + paper OPEN → close paper row
  • Mismatched size/price → update paper row

7. SQLite-Only Tests (Hard Guardrail)

Non-Negotiable Safety Rules

  • ALL tests use SQLite
  • If DB_CONNECTION != sqlite → tests FAIL immediately
  • Binance API NEVER called in tests (always mocked)

Must-Have Tests

  • Budget with no open positions = full budget
  • Closed positions do not reserve budget
  • Executed size replaces requested size
  • Close intent never creates positions
  • Exchange order success + DB failure does not crash system
  • Reconciliation idempotency

8. Debugging Strategy That Finally Works

When budget is wrong, log:

  • bot_id
  • initial_budget_usd
  • list of OPEN internal positions
  • summed reserved size
  • rounding steps
  • final available value

Debug must be:

  • gated (debug flag)
  • structured
  • never shown to normal users

9. UI Improvements (Bot Detail Page)

Positions Section

  • Delete button moved to sane location
  • Show authoritative size
  • Optional “requested size” hint
  • Manual “Sync Paper State” button
    • dry-run first
    • confirm to apply

10. Final Mental Model (Very Important)

  • Egis is not a reflection of the exchange
  • Egis is a ledger
  • Exchange is just one actor that can fail, lag, or lie
  • Internal consistency > external accuracy
  • Drift must be:
    • detectable
    • debuggable
    • repairable

Status at End of This Journal

  • Budget bugs understood at root level
  • Exchange atomicity issues identified
  • Correct sync direction enforced
  • Tests defined to prevent regression
  • System now trending toward financial correctness

If Egis ever breaks again, it will be loud, traceable, and fixable — not silent.