Scaling and Architecture Trade-offs
Choose complexity in response to an observed constraint. · Technical Fluency · Lesson 30 · 4 min
Scaling and Architecture Trade-offs · 4 min
Situation
An architecture for 100 users is not a plan for every scale.
A report product handles 100 users comfortably. The team expects rapid growth and proposes splitting it into many services before acquiring more customers.
Growth can create bottlenecks, but it does not tell you which ones will matter first. An architecture that handles ten million users may impose unnecessary cost and coordination on a small team.
Mental model
Scale the workload, not just the user count.
Latency is how long a request takes. Throughput is how much work the system completes per unit of time. A bottleneck limits the flow of work.
Ten million registered users making one request a month create a different workload from ten thousand users uploading large videos simultaneously. Ask about concurrency, request patterns, data size, and service expectations.
Example
Find the constrained step.
Suppose a report request spends most of its time scanning a large database table. Adding more web servers may not help because they all wait on the same query.
An index, a narrower query, or precomputed summaries might address that bottleneck. Each has costs: extra storage, write overhead, delayed freshness, or implementation work. Ask engineering to measure before choosing.
Compare
Several options buy different things.
Caching can reduce repeated work but creates freshness decisions. Queues can absorb bursts but add waiting. More capacity may improve throughput but increase cost. Splitting services can isolate workloads but adds network boundaries and operational complexity.
The choice should follow the failing workload and required behavior. No technique is a universal upgrade.
Failure case
Designing for imagined flexibility.
A general platform can consume months while the product's actual direction remains uncertain. More components also mean more deployments, ownership boundaries, alerts, and failure modes.
Conversely, ignoring a measured capacity limit can create a predictable incident. Restraint means matching investment to evidence, not postponing every architectural change.
PM decision
Express the requirement as a service target.
“For the next quarter, support the expected peak report volume with most reports ready within the agreed wait time, at an affordable operating cost.” Establish actual numbers with the team and usage evidence.
Ask for a forecast range, load-test results, the first expected limit, and the cost of the next increment. Include a monitoring trigger for revisiting the design.
Think it through
What would make the recommendation change?
A slow interactive action may need immediate results. A weekly report can tolerate background processing. The same computational work can deserve a different architecture because the user contract differs.
Explain which constraint is firm and which can move. Product scope, freshness, and timing can be architecture tools too.
Remember this
Buy complexity only when it solves a real constraint.
Make performance, scale, cost, and operational ownership explicit. A defensible architecture decision is appropriate to a workload and a team, not impressive in isolation.