Why is making enough RAM to go around so hard now? I know the cause of it - AI cunts - but what is the actual bottleneck in the production of RAM that means it can’t be pumped out fast enough to meet demand?
AI’s demand for memory is pretty difficult to really get across because there’s a lot of complex factors, but whatever you can imagine is the demand, it’s higher than that.
You can look at pre and post AI to get a slightly better picture, but then the numbers don’t look terrible and so the demand isn’t as clear.
2020-2023 primary customers were smartphones, laptops, PC. Data centers were eating about 32% of the global market for RAM. Monolithic DDR4/DDR5 was the main product and High Bandwidth Memory (HBM) was about 8%. Total memory set being sold was like 16GB kits to 64GB kits, obviously server kits were going out, just the majority was those mostly for PCs.
2025 hits and the primary customer is AI Data Centers. To put it at scale, you have literally everything that uses memory (and I mean literally every fucking thing on this planet) and AI Data Centers. And the break between those two bins are 30% and 70%. AI data centers are consuming more than twice the memory of literally everything combined that uses RAM that isn’t an AI data center.
The primary RAM being made now is HBM, which is way more complex. 23% of all the wafers that will be used to make integrated circuits will be HBM RAM. And by wafers, I mean all the chips that will be made this year, lock, stock, and barrel. If you randomly picked up a wafer out of a fab you have a almost 1 in 4 chance to pick up RAM. And finally the average kit going out is 1TB to 2TB kits, which is a lot more than the old 16GB to 64GB kits.
Now I mention HBM because it eats more wafer, that’s because unlike DDR4/5 RAM, HBM RAM is a three-dimensional circuit. 12 to 16 layers of silicon is stacked on top of each other. So HBM consumes about 300% more silicon than other memory (not every layer is one-to-one in size). So you don’t just have one fab making chips, you have several fabs making the layers.
The next thing is that building fabs is complex. I hate trying to explain the complexity, but you can’t do it overnight. Usually you have to build these things over the course of five years. Just to give you some idea of how technical the construction is. If you had a road within 500 feet of a chip fabricator sitting on a regular concrete floor, the car driving on the road would create enough shakiness in the Earth to cause the chip fabricator to bounce around too much. So when they build the place that have to literally isolate the small earth quakes humans walking around inside the place cause. This requires very complex floor building. And this is just the floor, not to mention how clean the place has to be kept, isolated as much as possible atmosphere, literally specific sections are under vacuum. It’s massively complex to build ONE of these.
The complexity comes with a price tag. Average cost to build one memory making factory is around $15B to $20B. It’s serious cash, but even if you have 5 years and $20B, there’s a specific bottleneck. ASML. ASML is the only company on the entire face of the Earth that makes the chip making machines. They’ve indicated that if you ordered a machine today, you can expect it roughly 1½ to 2 years from now. That’s how many people have put in an order for the machines to make memory.
So all that aside, there’s one more bottleneck. HBM has to be stacked in layers, there are very few people on this planet that can do that, and they have years long backlog. And even then, most times the stacking fails. About 30% to 50% of all HBM is trashed because the layers fell apart. And the people who stack are entirely different people than the layer makers. But they’re the same people that take that DDR4/5 wafer and cap it into that little black rectangle you see on your sticks of memory. So they have pretty much ~100% of their employees doing nothing but stacking layers of memory together.
Another thing is economic prioritization, HBM is about 500% more than DDR4/5’s price tag per GB. A fab producing wafers of DDR4/5 is making about $x.xx. A fab producing a couple of the layers for HBM is making about 500% × $x.xx on average (it’s complicated because of the layers), even with the stacking issues. And the profit margin on HBM is 70% versus DDR4/5 before AI which was fingernail thin. SK Hynix was actually taking a loss on production of DDR5 at about -1.6%. So going from -1.6% to 70% profit has created a crowding out effect. Not to mention that since there was a bit of a bleeding out period after COVID, some literally stopped making RAM. Which has made the issue even worse.
The last thing before I run out of characters is the AI growth. AI needs about 300% more memory every ten months. That’s how fast these models are growing. That’s caused a panic buying and also caused a rushing to fulfill. The industry is losing it’s collective mind because the money to be made is big and so lots think it can’t last and trying to get their cut before the gravy train derails.
There are very few places on earth that are capable of producing the silicon wafers used in RAM. These factories are still producing at the same rate as before but buyers who pay more (large companies with data centers) are buying them so there are fewer left over for normal consumers (hence the high prices). So why not scale up by making the factories bigger or faster? They are, it will take decades to do that because the process is so advanced. Why not just scale out by building more factories for producing the parts? They are, but that too will take decades.
I’m not going to look this up, because I’m expecting that there’s a number of reasons, but you can pretty much point the finger at supply and demand of components, as well as inability for companies to scale production, and the fact this is likely a bubble.
Think about it from the standing of a ram manufacturer. You’re already pumping out as much product as physically possible, or you need components or materials from other vendors. You aren’t just sitting on idle machines, if there’s a market what what you’re doing, you’re going to push as much of it as you can. All of a sudden, you can charge more because there’s a ram shortage, but you can’t scale up your business because with this rise there’s likely to be an equal fall in the not too distant future, and scaling up production is a slow process. Even if you did, there’s a strong possibility that you’d be restricted by other bottlenecks.
It’s a huge business risk for not much reward. Better to take the win, do what you can try maximise it without exposing yourself to potential losses.
Actually this is incorrect. The RAM manufacturers operate as a cartel and purposefully restrict the supply of RAM so they can sell less at much higher margins. They have been caught doing this 20 years ago and they are doing it again but now everyone is buying their excuse.
c/NoStupidQuestions style question:
Why is making enough RAM to go around so hard now? I know the cause of it - AI cunts - but what is the actual bottleneck in the production of RAM that means it can’t be pumped out fast enough to meet demand?
AI’s demand for memory is pretty difficult to really get across because there’s a lot of complex factors, but whatever you can imagine is the demand, it’s higher than that.
You can look at pre and post AI to get a slightly better picture, but then the numbers don’t look terrible and so the demand isn’t as clear.
2020-2023 primary customers were smartphones, laptops, PC. Data centers were eating about 32% of the global market for RAM. Monolithic DDR4/DDR5 was the main product and High Bandwidth Memory (HBM) was about 8%. Total memory set being sold was like 16GB kits to 64GB kits, obviously server kits were going out, just the majority was those mostly for PCs.
2025 hits and the primary customer is AI Data Centers. To put it at scale, you have literally everything that uses memory (and I mean literally every fucking thing on this planet) and AI Data Centers. And the break between those two bins are 30% and 70%. AI data centers are consuming more than twice the memory of literally everything combined that uses RAM that isn’t an AI data center.
The primary RAM being made now is HBM, which is way more complex. 23% of all the wafers that will be used to make integrated circuits will be HBM RAM. And by wafers, I mean all the chips that will be made this year, lock, stock, and barrel. If you randomly picked up a wafer out of a fab you have a almost 1 in 4 chance to pick up RAM. And finally the average kit going out is 1TB to 2TB kits, which is a lot more than the old 16GB to 64GB kits.
Now I mention HBM because it eats more wafer, that’s because unlike DDR4/5 RAM, HBM RAM is a three-dimensional circuit. 12 to 16 layers of silicon is stacked on top of each other. So HBM consumes about 300% more silicon than other memory (not every layer is one-to-one in size). So you don’t just have one fab making chips, you have several fabs making the layers.
The next thing is that building fabs is complex. I hate trying to explain the complexity, but you can’t do it overnight. Usually you have to build these things over the course of five years. Just to give you some idea of how technical the construction is. If you had a road within 500 feet of a chip fabricator sitting on a regular concrete floor, the car driving on the road would create enough shakiness in the Earth to cause the chip fabricator to bounce around too much. So when they build the place that have to literally isolate the small earth quakes humans walking around inside the place cause. This requires very complex floor building. And this is just the floor, not to mention how clean the place has to be kept, isolated as much as possible atmosphere, literally specific sections are under vacuum. It’s massively complex to build ONE of these.
The complexity comes with a price tag. Average cost to build one memory making factory is around $15B to $20B. It’s serious cash, but even if you have 5 years and $20B, there’s a specific bottleneck. ASML. ASML is the only company on the entire face of the Earth that makes the chip making machines. They’ve indicated that if you ordered a machine today, you can expect it roughly 1½ to 2 years from now. That’s how many people have put in an order for the machines to make memory.
So all that aside, there’s one more bottleneck. HBM has to be stacked in layers, there are very few people on this planet that can do that, and they have years long backlog. And even then, most times the stacking fails. About 30% to 50% of all HBM is trashed because the layers fell apart. And the people who stack are entirely different people than the layer makers. But they’re the same people that take that DDR4/5 wafer and cap it into that little black rectangle you see on your sticks of memory. So they have pretty much ~100% of their employees doing nothing but stacking layers of memory together.
Another thing is economic prioritization, HBM is about 500% more than DDR4/5’s price tag per GB. A fab producing wafers of DDR4/5 is making about $x.xx. A fab producing a couple of the layers for HBM is making about 500% × $x.xx on average (it’s complicated because of the layers), even with the stacking issues. And the profit margin on HBM is 70% versus DDR4/5 before AI which was fingernail thin. SK Hynix was actually taking a loss on production of DDR5 at about -1.6%. So going from -1.6% to 70% profit has created a crowding out effect. Not to mention that since there was a bit of a bleeding out period after COVID, some literally stopped making RAM. Which has made the issue even worse.
The last thing before I run out of characters is the AI growth. AI needs about 300% more memory every ten months. That’s how fast these models are growing. That’s caused a panic buying and also caused a rushing to fulfill. The industry is losing it’s collective mind because the money to be made is big and so lots think it can’t last and trying to get their cut before the gravy train derails.
There are very few places on earth that are capable of producing the silicon wafers used in RAM. These factories are still producing at the same rate as before but buyers who pay more (large companies with data centers) are buying them so there are fewer left over for normal consumers (hence the high prices). So why not scale up by making the factories bigger or faster? They are, it will take decades to do that because the process is so advanced. Why not just scale out by building more factories for producing the parts? They are, but that too will take decades.
I’m not going to look this up, because I’m expecting that there’s a number of reasons, but you can pretty much point the finger at supply and demand of components, as well as inability for companies to scale production, and the fact this is likely a bubble.
Think about it from the standing of a ram manufacturer. You’re already pumping out as much product as physically possible, or you need components or materials from other vendors. You aren’t just sitting on idle machines, if there’s a market what what you’re doing, you’re going to push as much of it as you can. All of a sudden, you can charge more because there’s a ram shortage, but you can’t scale up your business because with this rise there’s likely to be an equal fall in the not too distant future, and scaling up production is a slow process. Even if you did, there’s a strong possibility that you’d be restricted by other bottlenecks.
It’s a huge business risk for not much reward. Better to take the win, do what you can try maximise it without exposing yourself to potential losses.
Actually this is incorrect. The RAM manufacturers operate as a cartel and purposefully restrict the supply of RAM so they can sell less at much higher margins. They have been caught doing this 20 years ago and they are doing it again but now everyone is buying their excuse.
Correct. They’re also manufacturing slightly less than they did the year prior iirc.