This article is part of the Technology Insight series, made possible with funding from Intel.
________________________________________________________________________________________
In case you hadn’t noticed, solid-state drives keep getting bigger and faster. Back in 2008, a state-of-the-art enterprise SSD offered 32GB of capacity and moved files at up to 250 MB/s. Today, a 32TB version can read data sequentially at 3,200 MB/s. That’s a 1000x size increase and more than 10x speed-up.
Those incredible gains are made possible by storing more bits of data in every memory cell, and then fitting more memory cells in each NAND flash chip. For example, the X25-E’s single-level cell flash held one bit in each cell; the new SSD D5-P4326 packs four bits into the same space.
The industry is moving toward higher-capacity SSDs in its effort to keep data close to processing resources. But simply buying the largest SSD out there isn’t the best way for IT decision-makers to construct complex storage systems. Before picking the drives for your next application, make sure you understand how NAND flash affects performance, endurance, and density.
Are you ready for the zettabyte age?
- ~32ZB of data were created in 2018, according to IDC
- Current forecasts suggest ~103ZB will be created in 2023
- Scaling solid-state storage to help satisfy this demand requires denser NAND (x/y axis), more layers of NAND per die (z axis), and more bits per memory cell
- Quad-level cell (QLC) NAND offers a 33% scaling advantage compared to existing triple-level cell (TLC) memory, but presents write performance and endurance challenges
- As a result, TLC remains an important memory technology in write-intensive workloads. Expect the two technologies to complement each other.
3D QLC NAND: Where we’re going
The NAND flash in Intel’s SSD D5-P4326 is referred to as 3D QLC. When we talk about QLC, or quad-level cell technology, we’re referring to each memory cell’s ability to save four bits of data across 15 different threshold voltages. 3D is a reference to the way memory cells are built.
It used to be that those cells were arranged side by side on a silicon substrate. Their density increased as new lithography processes made it possible to fit more of them on a planar surface. But as it became increasingly difficult to scale along the x- and y-axis, manufacturers started organizing cells vertically, three-dimensionally along the z-axis.
The benefits of 3D NAND over 2D planar NAND naturally include much higher density. 3D NAND can also be written to and erased more times than planar NAND thanks to its larger memory cells. The technology offers lower power consumption, better performance, and less cost per bit of storage.
In a flash device built up 64 layers-tall, 3D NAND enables 64 times the cell density of planar memory. From there, cramming more data into every cell serves as a multiplier. So, QLC technology takes that 64x and turns it into 256x. Specific to Intel’s 64-layer 3D NAND, which it uses in the SSD D5-P4326, the company can fit 1Tb density per die. And more flash memory per die translates to higher-capacity SSDs in the same familiar form factors.
3D TLC NAND: Still cutting-edge memory technology
Whereas QLC NAND stores four bits per cell by sensing one of 16 possible charge states, triple-level cell (TLC) NAND only tracks eight. Of course, that’s still a formidable task. But because fewer bits are written to TLC NAND compared to QLC memory, TLC can withstand a higher number of program/erase cycles before its cells start wearing out.
Above: Storing four bits in a QLC memory cell requires differentiating between 16 different charge states. Three bits in TLC NAND can be achieved with eight charge states. Both are far more complex than older MLC or SLC technologies.
TLC flash is faster than QLC, too. It turns out that differentiating between twice as many charge states makes QLC more prone to mistakes than TLC flash. And although both technologies employ error-correcting code algorithms to maintain the integrity of your data, this process consumes a greater number of processing cycles on QLC-based drives, hitting write performance especially hard.
Picking the right performance profile for your application
According to a presentation given at the 2019 Flash Memory Summit, Micron’s Kent Smith made it clear that the latest QLC-based SSDs are designed to augment existing TLC SSDs, not replace them. He pointed out that QLC pricing puts the technology in striking distance of the 55 million 7,200 RPM (or higher) hard drives expected to ship in 2019.
Knowing that 3D TLC and 3D QLC are out there, side by side on the shelf, how (and perhaps more important, why), how do you choose between them? It’s all about understanding your storage application.
Because QLC NAND can be read sequentially just as fast as TLC NAND, it’s great for read-heavy workloads. Conversely, TLC NAND has the upper hand in write performance. When you apply those strengths to the spectrum of read and write ratios, it’s easy to visualize where each technology fits best. Smith went a step further, adding block sizes to his breakdown. He showed QLC SSDs for mixed workloads handling large blocks of data.
Above: TLC and QLC SSDs complement each other. The former excels in write-heavy workloads, while the latter offers excellent read performance at a lower cost-per-bit than TLC flash.
Better still, Smith’s presentation offered up a number of performance-sensitive workloads historically run on hard drives that read data at least 90% of the time, or rely heavily on random reads and sequential writes. AI data lakes, edge analytics (including 5G), big data (Hadoop), object stores, SQL databases, content delivery networks, cloud services, vSAN capacity tiers, and financial regulatory and compliance storage are all prime candidates to make the move to QLC-based SSDs.
Whereas a traditional datacenter I/O pattern might involve four reads for every write, the deep learning algorithms that feed AI are estimated at 5,000 reads for every write, according to data presented by Micron. A larger, cheaper QLC-based SSD is ideal in an application like that.
“Netflix is another good example of where QLC NAND works well,” said Michael Scriber, senior director of server solution management at Supermicro. “They’re going to write a movie to their system once. Then, customers are going to read that movie out a zillion times at the same performance and lower cost compared to TLC.”
Endurance matters, too
Beyond performance, your application’s ratio of reads to writes also affects endurance. Since QLC NAND is rated for fewer program/erase cycles than TLC, write-heavy workloads wear its memory cells faster. Those tasks seem to be the exception, though. According to Micron, four out of five enterprise SSDs shipped in 2018 were rated for less than one drive-write per day (DWPD). That metric tells you what percentage of an SSD’s capacity you can write to the drive each day over its warranty period.
Above: According to Micron, 4/5 of all enterprise drives shipped in 2018 were rated for less than 1 DWPD, illustrating a decreasing need for high-endurance SSDs.
Back in the day of Intel’s X25-E, one drive write—a mere 32GB—would have been grossly insufficient. But when you factor in the capacity of today’s SSDs, a lower endurance rating is easier to stomach.
“If I have an 8TB (TLC) drive good for 1 DWPD, I can write 8TB every day for five years and my warranty is still good,” said Supermicro’s Scriber. “On the other hand, if I have a 16TB (QLC) drive that’s only good for 0.5 DWPD, I can still write 8TB per day for the next five years and it’ll still be fine.”
When you think about 32 SSD D5-P4326s across the front of a 1U server, and the 500TB of pooled capacity they represent, ask if your application will be writing 250TB or 300TB on a daily basis before sounding an alarm over endurance.
TLC and QLC NAND complement each other
By 2025, Western Digital predicts that 50% of the NAND flash bits shipped will be 3D QLC, with 3D TLC making up most of what’s left. QLC NAND will slowly displace some TLC NAND volume between now and then. However, both technologies remain important moving forward.
Above: By 2025, half of the bits shipped will be based on 3D QLC NAND. The other half will be 3D TLC. Planar NAND will have all but disappeared.
As a case in point, Intel’s SSD DC P4510 lives alongside its SSD D5-P4326. Both are available in E1.L form factors at capacities as large as 15.36TB, and both are covered by five-year warranties. But the SSD DC P4510 is composed of 3D TLC NAND stacked 64 layers-high and capable of 3.1 GB/s sequential reads and writes. The SSD D5-P4326 employs 64-layer 3D QLC NAND that pushes sequential reads up to 3.2 GB/s, but drops to 1.6 GB/s when you write. Although some of their specifications overlap, these drives are designed for different applications.
Bottom line
Decision-makers have more flexibility than ever to tap the best storage option for their workloads, balancing performance, endurance, density, and cost. QLC NAND’s strengths finally make a case for replacing mechanical disks with much faster and more reliable solid-state drives. Meanwhile, TLC-based SSDs remain the better choice in write-heavy applications. Address each of your workloads with the right storage technology and you’ll hammer out bottlenecks without overspending.
