Backend/2025
Relay / Event-driven API
A resilient order workflow built around explicit state, safe retries and traceable events.
Illustrative case study, not a claim of client work or measured production outcomes.
Concept case study. Demo and repository links are intentionally omitted; replace this content with your own verifiable project and public links.
Context
Overview
Relay is a concept for an order-processing API that has to coordinate slow or unreliable external services. Rather than presenting another CRUD service, the case study examines what happens when requests time out, callbacks arrive twice, or a downstream system is temporarily unavailable.
The design prioritises explainable behaviour. An operator should be able to find an order, see its state transitions and understand whether a retry is safe.
The problem
A network timeout does not tell the caller whether an operation was committed. Retrying a payment-adjacent request without an idempotency policy can create duplicate work. Treating all failures as the same generic server error makes both the user experience and incident diagnosis worse.
The system must also avoid writing an order successfully and then losing the event that should trigger its next step.
The solution
The proposed solution combines request idempotency, an explicit order state machine and a transactional outbox. The business write and its outgoing event are committed together. A worker publishes pending events and tracks delivery attempts.
Consumers must tolerate repeat delivery. Failed work moves through bounded retries and a reviewable dead-letter path instead of an infinite retry loop. Correlation identifiers tie request logs to worker activity.
Ownership, made explicit
My contribution
- API contracts: define validation, idempotency-key scope, conflict responses and versioned event envelopes.
- Persistence design: model order transitions, outbox records and unique keys inside explicit transactions.
- Workers: design retry policy, duplicate handling, backoff and an operator-initiated replay path.
- Testing: specify crash, timeout and duplicate-delivery scenarios instead of testing only successful HTTP responses.
- Observability: connect correlation identifiers, structured logs and queue-age indicators to a troubleshooting runbook.
Architecture
The API writes orders and outbox entries to PostgreSQL within the same transaction. A publisher worker reads the outbox and passes events to a broker boundary. Consumers perform idempotent work and report status back through the application.
The broker is a replaceable component in this conceptual design. The architecture does not promise exactly-once delivery: correctness comes from idempotent state transitions and durable records.
Key features
Idempotent order creation
Repeated requests with the same scope and key return a consistent result without creating a second order.
Explicit order state
Documented transitions prevent callbacks from moving an order into an impossible state.
Operator-friendly replay
A bounded retry policy and replay tooling make failure visible and recoverable.
Challenges & trade-offs
The hardest trade-off is not picking a queue: it is deciding what must be consistent immediately and what may converge later. The UI needs to distinguish pending, completed and attention-required states.
Outbox cleanup, poison messages and replay permissions need operational policies. A successful local prototype would not establish production resilience without failure testing under representative load.
Results & impact
The deliverable is a conceptual API contract, an event-flow diagram and a failure-testing checklist. The proposed boundaries make duplicate handling and operational ownership visible.
Production error rates, processing latency and incident recovery times require real measurement. They are intentionally not fabricated for this portfolio demo.
Screenshots
Technology stack
Reproducible application builds and runtime environments.
DocumentationApplication services, integrations and asynchronous workers.
DocumentationRelational modelling, transactions and query planning.
DocumentationCaching, coordination and short-lived state.
Documentation