An API gateway simplifies entry into the system right up to the point where it becomes an overloaded center for auth, routing, policy, and observability.
This case helps show how routing, rate limiting, authn/authz, request shaping, and backpressure converge on a single ingress layer, creating both convenience and a potential new bottleneck.
For interviews and architecture discussions, it is useful because it tests whether you can separate platform capability from business logic and reason about the blast radius of a centralized control plane.
Control Plane
Focus on policy, limits, routing, and stable edge behavior under variable load.
Data Path
Keep latency and throughput predictable while traffic and burst pressure increase.
Failure Modes
Cover fail-open/fail-close behavior, graceful degradation, and safe fallback paths.
Ops Ready
Show monitoring for saturation, retry storms, and practical operational guardrails.
Related book
Building Microservices
Sam Newman details the role of API Gateway in a microservices architecture.
API Gateway — a single entry point for all client requests in a microservice architecture. This is a critical component that provides routing, security, request transformation, and many cross-cutting concerns.
Why do you need an API Gateway?
Functional Requirements
Routing
- Path-based routing (/users → User Service)
- Header-based routing (A/B testing)
- Query parameter routing
- Weighted routing (canary deployments)
Security
- JWT/OAuth2 validation
- API key management
- IP whitelisting/blacklisting
- mTLS for service-to-service
Rate Limiting
- Per-user/per-IP limits
- Per-endpoint throttling
- Burst handling
- Quota management
Transformation
- Request/Response modification
- Protocol translation (REST → gRPC)
- Response aggregation
- Schema validation
Related case
Rate Limiter
Detailed analysis of rate limiting algorithms: Token Bucket, Sliding Window.
Non-functional requirements
< 10ms added
Latency
P99
100K+ RPS
Throughput
per instance
99.99%
Availability
SLA
Horizontal
Scalability
stateless
High-level architecture
Mobile App
Web App
Partner API
IoT Devices
Load Balancer
L4/L7
Gateway 1
Gateway 2
Gateway N
User Service
Order Service
Payment Service
External Services
Click a client or start the animation to simulate the request flow
Key patterns
Backend for Frontend (BFF)
A separate API Gateway for each type of client, optimized for its needs.
Backend for Frontend Pattern
A separate BFF for each client type
CLIENTS
Mobile App
Optimized for mobile
Web App
Full-featured
Admin Panel
Extended permissions
BFF LAYER
MICROSERVICES
API Composition
Gateway aggregates data from multiple services into a single response.
Advantages:
- Reducing the number of round trips
- Hiding the internal structure
Risks:
- Increased latency
- Difficulty handling errors
Related book
Release It!
Michael Nygard is the author of the Circuit Breaker pattern and other stability patterns.
Circuit Breaker in Gateway
Protection against cascading failures when downstream services are unavailable.
Circuit Breaker Simulation
Protection pattern against cascading failures
CLOSED
Normal operation. Requests pass through to the service.
State Transitions
Configuration
0
Total Requests
0
Successful
0
Rejected (Fast)
Fail Fast
Immediate rejection instead of waiting for timeout
Cascading Prevention
Protection from failure propagation
Self-Healing
Automatic recovery after timeout
Documentary
Inside Envoy: The Proxy for the Future
The history of the creation of Envoy Proxy and its role in the service mesh ecosystem.
Popular solutions
Kong
- Plugins ecosystem
- Lua/Go extensibility
- Declarative config
AWS API Gateway
- Lambda integration
- WebSocket support
- Pay-per-request
Envoy
- L7 proxy
- Service mesh ready
- xDS API
NGINX Plus
- High performance
- Native modules
- Active health checks
Traefik
- Auto-discovery
- Let's Encrypt
- Kubernetes native
Apigee (Google)
- Analytics
- Developer portal
- Monetization
Interview tips
1. Start with "why"
Explain why API Gateway is needed: single entry point, security, cross-cutting concerns. Not just “this is a standard in microservices.”
2. Discuss stateless vs stateful
Gateway must be stateless for horizontal scaling. Status (rate limits, sessions) - in Redis/external store.
3. Consider single point of failure
Gateway is a critical component. Discuss: multiple instances, health checks, graceful degradation, fallback responses.
4. Don't make Gateway too smart
Avoid business logic in Gateway. It should only deal with cross-cutting concerns: auth, routing, rate limiting.
Key Findings
- API Gateway - single entry point providing routing, security and observability
- Must be stateless to scale horizontally (state in Redis)
- Backend for Frontend (BFF) - separate gateway for each type of client
- The Composition API reduces round trips, but adds latency and complexity
- Circuit Breaker protects against cascading failures of downstream services
- Avoid business logic in Gateway - only cross-cutting concerns
For additional practice with the API Gateway pattern, review microservices.io: API Gateway pattern and AWS API Gateway Developer Guide.
Related chapters
- System Design Primer (short summary) - provides the baseline design principles for decomposition, API boundaries, and reliability decisions in gateway systems.
- Continuous API Management (short summary) - extends API lifecycle, governance, and contract-driven management around gateway-enabled platforms.
- Customer-friendly API: convenient API for clients - goes deeper into BFF and client-facing facade design on top of internal microservice APIs.
- Rate Limiter - details traffic-shaping algorithms and distributed quota control that are typically enforced at the gateway edge.
- API Security Patterns - complements gateway security responsibilities: authN/authZ, token policies, and boundary hardening.
- Inter-Service Communication Patterns - explains where gateway responsibilities stop and how downstream sync/async service communication takes over.
- Service Mesh Architecture - clarifies edge gateway vs service-mesh roles across ingress traffic, data plane behavior, and control policies.
- Inside Envoy: The Proxy for the Future - adds practical context on Envoy as a common L7 foundation for modern API gateway and ingress stacks.
- Resilience Patterns - strengthens timeout, retry, circuit-breaker, and graceful-degradation strategies at the gateway layer.
- Zero Trust - connects gateway design with continuous verification, policy-based access control, and trust minimization.
- Kubernetes Fundamentals - covers operational deployment context for gateways in Kubernetes: ingress, autoscaling, and rollout patterns.
- Notification System - shows how gateways integrate with multi-channel delivery systems while governing external API traffic.