wiki:db/HardwareModel
Last modified 10 years ago Last modified on 06/09/2009 09:13:47 AM

Database Hardware Model

The goal is to maximize the sustained disk bandwidth of the entire system for the minimum cost. The disk bandwidth can be limited by many things:

  • The sustained bandwidth from memory to core.
  • The sustained bandwidth of the disk controller.
  • The sustained bandwidth available to memory (half used to transfer data from disk and half used to transfer data to the cores). Note that this may be much less than the number of cores times the maximum bandwidth per core. For this reason, decreasing the number of cores per box may be advantageous.
  • The maximum bandwidth available across all attached spindles, which is in turn limited by the number of disks that can be attached to a box, the maximum sustained disk bandwidth from a single spindle, and the maximum sustained bandwidth of the disk interconnect.

A secondary goal is to maximize the number of random I/O operations per second, again for minimum cost.

These numbers are needed at (at least) three timeframes: DC4 (around 2011), when large-scale database serving will be required; DR1, the first data release of the survey; and DR11, the last data release of the survey, ten years later.

The "Today (GW)" numbers in the table below are based on the GrayWulf configuration. The "Today (eee)" numbers are based on an Asus eee PC with SSD, which may be able to provide similar performance at lower cost and lower power consumption. Both use numbers from Alex Szalay.

Parameter Unit Today (GW) Today (eee) DC4 (2011) DR1 (2016) DR11 (2026)
Sustained bandwidth from disk to 1 core MB/s 180 100 ? 500 1800
Sustained disk bandwidth from 1 spindle MB/s 50 200 ? 500 1800
Sustained bandwidth from disks to 1 controller MB/s 720 200 ? 1200 2600
Sustained bandwidth to/from memory on 1 box GB/s 2.88 > 0.4 ? 5 24
Disks that can be attached to 1 box disks 30 1 ? 120 1000
Disk size for 1 (cheap) spindle GB 750 256 ? 2000 64000
Maximum random I/O rate for 1 spindle IO/s 30 180 ? 256 2048
Memory per box GB > 8 > 1 ? 64 2048
Number of cores per box cores 8 2 ? 16 512