Random IOPS vs. Sequential IOPS, and How to Read Benchmark Results

Applicability of This Document

Ready-to-Run Commands

# 対象: Linux + fio 3.x
# 権限: 対象ファイルへの読み取り権限（デバイス直接指定なら root）
# 変更作業: なし（読み取りのみ）だが実行中は他処理の I/O 性能に影響する
# Production 実行: 非推奨。検証環境で実行すること
fio --name=rand_read_4k \
    --filename=/mnt/testdir/fio_testfile \
    --size=64G \
    --rw=randread \
    --bs=4k \
    --ioengine=libaio \
    --direct=1 \
    --iodepth=32 \
    --numjobs=4 \
    --runtime=300 \
    --time_based \
    --ramp_time=30 \
    --group_reporting

# 対象: Linux + fio 3.x
# 権限: 対象ファイルへの読み取り権限
# 変更作業: なし（読み取りのみ）だが I/O 帯域を占有する
# Production 実行: 非推奨。検証環境で実行すること
fio --name=seq_read_1m \
    --filename=/mnt/testdir/fio_testfile \
    --size=64G \
    --rw=read \
    --bs=1M \
    --ioengine=libaio \
    --direct=1 \
    --iodepth=8 \
    --numjobs=1 \
    --runtime=300 \
    --time_based \
    --group_reporting

# 対象: Linux + fio 3.x
# 権限: 対象ファイルへの読み書き権限
# 変更作業: あり（テストファイルに書き込む。既存データを指定しないこと）
# Production 実行: 不可。専用の検証領域でのみ実行すること
fio --name=rand_rw_70_30 \
    --filename=/mnt/testdir/fio_testfile \
    --size=64G \
    --rw=randrw \
    --rwmixread=70 \
    --bs=8k \
    --ioengine=libaio \
    --direct=1 \
    --iodepth=32 \
    --numjobs=4 \
    --runtime=600 \
    --time_based \
    --ramp_time=60 \
    --group_reporting

# 対象: Linux（sysstat / nvme-cli）
# 権限: 一般ユーザー（nvme list は root が必要な場合がある）
# 変更作業: なし（参照のみ）
# Production 実行: 可能
lsblk -o NAME,SIZE,TYPE,ROTA,MOUNTPOINT,MODEL
sudo nvme list
iostat -x 1 10

How to Read the Results

When to Use This

• A product catalog's IOPS value differs significantly from what you measured yourself
• The same storage shows numbers that vary by multiples depending on who measures it
• A benchmark shows fast numbers, but database processing doesn't get as fast as expected
• Switching from SATA SSD to NVMe SSD made no perceptible difference
• Adding many NVMe drives didn't scale performance proportionally
• It's unclear whether to look at the sequential or random figure

Possible Causes (Most Likely First)

1. 01

   An IOPS figure is not defined on its own

   IOPS is just "I/O operations per second" — it only becomes meaningful once the size of a single I/O, the number of I/Os issued concurrently (queue depth), the read/write ratio, and whether the cache is bypassed are all fixed. Numbers missing any of these four cannot be compared.

2. 02

   Physical mechanics dominate random access on HDD

   On rotating media, every random I/O involves moving the head (seek) and waiting for the sector to come around (rotational latency). Sequential access involves almost none of this, so the same device shows an order-of-magnitude gap between random and sequential.

3. 03

   The gap is not zero even on SSD/NVMe

   With no moving parts, the random-vs-sequential gap shrinks dramatically compared to HDD, but it remains due to flash's page/block structure, write amplification, garbage collection, and mapping-table lookup cost. For writes in particular, the figure differs between the steady state (free space depleted) and the initial state.

4. 04

   SATA (AHCI) and NVMe have different queue structures

   AHCI is fundamentally designed around a single command queue, limiting how many commands can be outstanding at once. NVMe is designed to have many deep queues per CPU core. This difference matters at high parallelism and shrinks at a queue depth of 1 with sequential access. If "switched to NVMe but it's not faster," the measurement conditions may be low-parallelism.

5. 05

   The benchmark is measuring the cache

   Without `direct=1`, with a test size smaller than RAM, or with a run so short it only covers warm-up, what's being measured is the page cache or the controller's write-back cache.

6. 06

   The database's access pattern doesn't match the measurement conditions

   Access to a database's data files is generally random-leaning, while writes to the transaction log are sequential and synchronous. Log writes are governed by write latency, not IOPS or throughput. A good sequential-read number does not guarantee either of these.

Diagnostic Steps

1. 1

   Measure the real workload's I/O characteristics first

   Read-only

   Use `iostat -x` to get `rareq-sz`/`wareq-sz` (average I/O size), `r/s`/`w/s`, and `aqu-sz` (queue length). Match the benchmark conditions to these.

2. 2

   Check the device configuration

   Read-only

   Use `lsblk -o ...,ROTA,MODEL` and `nvme list` to determine what's actually being measured (HDD or SSD, local or network-attached).

3. 3

   Run fio with the measurement conditions explicitly stated

   Low

   Specify `--bs`, `--iodepth`, `--numjobs`, `--rw`, and `--direct=1`, and record the conditions as-is. A benchmark result without recorded conditions cannot be interpreted later.

4. 4

   Measure at several points with different queue depths

   Low

   Vary iodepth while recording both IOPS and latency. The point where IOPS plateaus but latency keeps climbing is the practical ceiling.

5. 5

   Run write tests until the steady state

   Medium

   On SSD, a short measurement only shows the initial-state value. Set a long `--runtime` and exclude the ramp-up with `--ramp_time`.

6. 6

   Isolate layer by layer for a multi-device configuration

   Low

   Measure in order — a single device, the RAID/volume layer, the filesystem layer, and the application layer — to identify where the ceiling is.

Solutions

Differences by Version and Environment

If This Doesn't Resolve It

• Check whether the CPU is saturated

  A high-IOPS measurement can consume all available CPU on interrupt handling and I/O issuing. Check CPU usage with `mpstat` alongside `iostat`.

• Check the I/O scheduler setting

  `none` is often appropriate for NVMe; leaving a rotating-media scheduler in place introduces unnecessary reordering.

• Check filesystem alignment

  A misaligned partition or stripe boundary splits a single logical I/O into multiple physical I/Os.

• Check the application's concurrency

  If things are slow without storage hitting a ceiling, the limiting factor is application-side parallelism or lock contention.

• Obtain the measurement conditions for anything being compared

  When comparing against a number from elsewhere, check whether that measurement's bs, iodepth, numjobs, rw, and direct flag are published. A figure without that is not comparable.

Basis and Limitations of This Document

FAQ

Questions This Document Covers

• Difference in Random Read IOPS between SATA SSD and NVMe SSD
• Where the bottleneck is in a 24-drive NVMe SSD configuration
• Why a benchmark's IOPS value differs from the product catalog
• Does a database care more about random I/O or sequential

What the Risk Labels Mean

• Read-onlyDoes not change data or settings.
• LowImpact is limited, but permissions and load should be checked.
• MediumMay affect performance, locking, or cost.
• HighMay cause an outage, data loss, or require recovery work.
• Expert Review RequiredRequires a separate review before applying to production.

GIIP's Scope of Support

Everything up to this point is reproducible by anyone once the measurement conditions are aligned. What's hard in real operations is not measuring correctly — it's recording the conditions used and maintaining a state where the next measurement can be compared against it. At GIIP, an AI agent continuously collects I/O size, queue length, and wait time under identical conditions across several databases on AWS and Azure and roughly 30 web services, and a human specialist investigates the cause once a value deviates from the historical distribution. This mechanism is built to use a time series collected under consistent conditions as the basis for judgment, rather than a one-off benchmark figure.

Related Services

• データベース性能チューニング
• 構成と測定条件について相談する