This chapter matters because it is not really about a pair of devices, but about dramatic jumps in latency, capacity, and durability across the hierarchy.
In real engineering work, it helps you design hot and cold paths deliberately: what belongs in memory, what moves to SSD or HDD, and how those choices reshape APIs, caches, and user experience.
In interviews and design discussions, it turns storage trade-offs into something concrete through tail latency, durability, and infrastructure budget.
Practical value of this chapter
Memory hierarchy
Guides hot and cold path design using realistic latency and cost differences.
Data placement
Clarifies what should live in RAM versus SSD or HDD and how that affects API behavior.
Load stability
Shows how to avoid degradation as the working set grows and cache pressure increases.
Interview trade-offs
Strengthens discussion of speed, durability, and infrastructure-budget compromises.
Source
Random-access memory
RAM structure and key properties of volatile memory.
RAM and storage are not competing options but different layers of one hierarchy: RAM gives the lowest latency for hot data, while SSD and HDD provide capacity, durability, and better cost efficiency. Good system design is mostly about deciding what stays in memory, what moves to faster storage, and what can safely live on colder, cheaper tiers.
RAM and storage: what's the difference
How RAM works
- DRAM is organized into cell matrices addressed by the memory controller.
- Access is random, but reads within one row are usually faster because of the row buffer.
- Memory requires periodic refresh, so part of the time is spent on background work.
- CPU goes through L1/L2/L3 caches, which hides part of RAM access latency.
How storage works
- Storage reads and writes data in blocks and pages.
- HDD adds mechanical seek time, while SSD works with NAND pages and erase blocks.
- Performance depends on I/O queues, concurrency, and block size.
- File systems and DBMSs build journals, indexes, and compaction strategies on top of that base.
RAM
- Volatile: data disappears when power is removed.
- Minimal access latency and extremely high read/write speed.
- Best suited for hot data, indexes, and caches.
- Much more expensive per GB than storage.
Storage
- Data survives power loss.
- Access latency is orders of magnitude higher than in RAM.
- Available capacity is much larger.
- Storage cost per GB is much lower.
Dynamic visualization: capacity and storage-tier calculator
This simplified model shows how data is split between RAM, SSD, and HDD after replication and compression.
Daily data volume
600 GB/day
Retention period
30 days
Replication
x3
Compression
2.0x
Hot data share
20%
SSD share of the warm tier
60%
Index overhead
35%
Capacity calculator output
Total data volume
26.37 TB
RAM for hot set
3780 GB
SSD volume
12.66 TB
HDD volume
8.44 TB
Data split across storage tiers
RAM coverage for the hot tier: 70%
Estimated monthly cost: $13781 (~$165372/year).
Source
Solid-state drive
SSD fundamentals and why SSD latency is much lower than HDD latency.
Dynamic visualization: latency calculator
Adjust the workload profile and watch how cache hit ratio, random I/O, and queue depth change p95 latency for each storage layer.
Cache hit ratio
85%
Random I/O share
70%
Queue depth
16
Target IOPS
50,000
P95 latency SLO
15 ms
RAM
p95: 0.10 us
Device only: 0.10 us
Load pressure: 0.20x
Within the SLO
NVMe SSD
p95: 46 us
Device only: 306 us
Load pressure: 0.20x
Within the SLO
SATA SSD
p95: 313 us
Device only: 2.08 ms
Load pressure: 0.71x
Within the SLO
HDD
p95: 1084.1 ms
Device only: 7227.2 ms
Load pressure: 142.86x
Outside the SLO
Recommended primary tier: NVMe SSD
These parameters still fit the target SLO for the selected cache-hit profile.
Where RAM, SSD, and HDD fit best
Choose a workload profile and the balance between performance and cost. The simulator suggests how much of the system should live in RAM, SSD, and HDD.
Cost priority
45%
0 = performance first, 100 = cost first
Yearly growth
+35%
Target p99
6 ms
Target IOPS
120,000
Recommended split by storage tier
Projected ingest: 540 GB/day -> projected storage envelope: 23.73 TB
Risk: validate traffic growth and queue depth so hidden peak-time degradation does not creep in.
How to shape the storage layout
- Use aggressive RAM caching for hot keys and indexes.
- Use SSD as the primary working layer for read and write paths.
- The cold tier stays small here: this profile values latency more than storage cost.
Source
Hard disk drive
Mechanical disk characteristics and random/sequential access behavior.
HDD and SSD: practical comparison
Branch-heavy APIs and transactional systems usually require SSD for random I/O. HDD remains valuable for archives, historical reads, and long-term retention where minimum latency is not critical.
HDD
- Mechanical components with rotating platters.
- High random I/O latency because of head movement and seek time.
- Large capacity and low price per TB.
- Good for cold data, backups, and archives.
SSD
- Flash memory with no moving parts.
- Low latency and high IOPS relative to HDD.
- Much better for random access workloads.
- Write endurance is finite, so lifecycle policy and monitoring still matter.
Common mistakes
Keeping the whole data set on SSD
Without storage tiers, cost grows quickly while cold data still occupies the expensive fast layer.
Ignoring the working set
If the hot working set does not fit in RAM, p95/p99 latency can degrade sharply.
Looking only at average latency
For storage, tail behavior matters most: p95, p99, and queue-driven degradation under load.
No lifecycle policy
Without automatic RAM -> SSD -> HDD transitions, storage cost and latency become unpredictable.
Practical recommendations
Storage tiers by access pattern
Keep hot keys and indexes in RAM, operational data on SSD, and long-tail history on HDD or object storage.
Capacity planning with headroom
Plan for replication, compression, index overhead, and yearly traffic growth in one model.
Set the SLO before choosing the medium
Set p99, p95, and IOPS targets first, then choose RAM, SSD, and HDD, and only after that optimize cost.
Storage observability
Monitor cache hit ratio, queue depth, fsync latency, and saturation for each storage tier.
Practical conclusion
Choosing between RAM, SSD, and HDD is not a binary decision. It is the design of a multi-tier storage layout. Start with the target SLO and workload profile, then size the system, and only after that optimize cost through storage tiers and a clear data-lifecycle policy.
Related chapters
- Why foundational knowledge matters - explains how memory and disk constraints shape architecture decisions.
- Structured Computer Organization (short summary) - adds the hardware context: memory hierarchy, buses, and the way CPU interacts with storage I/O.
- Operating system: overview - extends the topic with page cache behavior, I/O scheduling, and kernel-driven latency effects.
- Why understand storage systems - broadens the topic to replication, indexing, and long-term storage strategy.
- Database Selection Framework - helps choose the right DB and storage pattern for a specific workload.
- Performance Engineering - adds practical methods for profiling latency and throughput and for finding bottlenecks in the I/O path.
- CPU and GPU: overview and differences - connects storage choices with memory bandwidth and the cost of moving data.