Back to Blog
Engineering

How We Built a Sub-2-Second Approval System

Deep dive into the architecture that powers RCCP's low-latency approval workflow.

January 5, 2025
10 min read

The Challenge

When Claude Code needs approval, every second counts. A slow approval system means wasted development time. We set an ambitious target: P95 latency under 2 seconds.

Architecture Overview

WebSocket-First Design

We chose WebSockets over HTTP polling for real-time communication:

  • Persistent connections between agents and server
  • Bi-directional message flow
  • Sub-100ms message delivery

    • The Outbox Pattern

    To ensure exactly-once delivery, we implemented the outbox pattern:

  • Agent persists message to local SQLite
  • Sends message to server
  • Waits for ACK
  • Deletes from outbox on ACK
  • Replays on reconnection

    • Event-Driven Backend

    Our Fastify backend processes events asynchronously:

  • Message received → validated → persisted
  • Push notifications dispatched in parallel
  • WebSocket broadcast to connected clients

    • Performance Optimizations

    Connection Pooling

  • Reuse database connections
  • Keep WebSocket connections alive
  • Heartbeat every 30 seconds

    • Edge Caching

  • Static assets on CDN
  • API responses cached at edge
  • Device tokens cached in Redis

    • Efficient Serialization

  • Protocol buffers for internal messages
  • JSON for client-facing APIs
  • Compressed WebSocket frames

    • Results

    After optimization:

  • P50 latency: 800ms
  • P95 latency: 1.8s
  • P99 latency: 2.5s

    • Lessons Learned

  • WebSockets beat polling for real-time apps
  • Local persistence is essential for reliability
  • Push notifications add ~500ms latency
  • Database queries are often the bottleneck

    • What's Next

    We're working on:

  • Regional edge nodes for lower latency
  • Offline-first mobile app
  • Predictive pre-caching

    • Stay tuned for more engineering deep dives!

    Ready to try RCCP?

    Get Started Free