A quick personal note first. I went independent on October 1. Chipstrat and Semi Doped are now my full-time work, and I’m taking on analyst, consulting, and public speaking engagements alongside this newsletter and the Semi Doped podcast.
Yesterday I moderated the Semiconductor Industry Outlook panel at the CASPA CEO Summit with Stacy Rasgon of Bernstein and Dylan Patel of SemiAnalysis. We covered the cost of capital, bottlenecks, pricing power up and down the stack, Nvidia, GPUs vs XPUs, and enterprise adoption.
I had foundry teed up next, but we ran out of time unfortunately! Also thanks to Ian Cuttress for recommending me as moderator for this panel.
CASPA posted the full video. The transcript follows, lightly edited for readability.
Financing the AI Buildout
Austin Lyons: All right. Hello, everyone. Y’all ready for this? This should be good. This should be fun. We are going to talk about the industry. We’re going to look ahead to 2027, 2028 and hear these two gentlemen’s thoughts about where the industry is headed. We’ll try to cover it all from financing to GPU, CPU down to foundry. So, let’s get into it. I actually want to start with the finance side.
Lately a lot of the news has been about the cost of capital. We know hyperscalers are investing a ton and free cash flow is going toward negative. And now we’ve got 10-year Treasury over 5%. Even if you zoom out to CoreWeave, there’s talk about their bonds at 15% yields. My question is, should we be concerned? Is there going to be an impact as companies are investing more and more that it’s becoming more and more expensive to finance this? I’ll start with Stacy.
Stacy Rasgon: Look, there are a lot of constraints in general that are moderating the pace at which this buildout can happen. These are power and supply and semiconductor tools and now maybe financing. I’m a Wall Street analyst. If you look at the stocks, they’re still up a bunch on this, but they’re clearly way down from the peaks. The market is telling you something. The market is telling me that they’re worried that there’s a lot of uncertainty in terms of the pace and sustainability of the buildouts and maybe the peak is approaching. Clearly financing is one of those concerns. It’s probably up near the top.
You mentioned a number of different reasons. The hyperscalers, these were the most cash generative businesses in the history of humankind. And they’re burning cash now and probably going to be raising equity. Meta just had a new share class structure that among other things will enable them to do that. We saw Google raised and others. You mentioned CoreWeave’s bonds. That’s their unsecured debt. It’s not what they’re paying right now. That’s where they’re trading in the market. But that would suggest if they go out to the market today to get new unsecured debt, that’s in the ballpark of where they’ll be spending. And it’s a lot of money.
So I think these are all broad concerns right now. Though I think at least for the next year or two, which is as far out, frankly, as I can look, we’re probably okay. But I do think it’s pretty clear that we need to have revenues grow to the point where this becomes less of a concern, and maybe that happens. I don’t know. SpaceX reported earnings this week, last week, I can’t remember. He’s running out, clearly he’s running out in the spot market, and so things are very, very constrained. The only thing we’re hearing is nobody has enough compute.
But he’s renting capacity out on short-term contracts, at something like $50 billion a gigawatt at those kinds of rates. I don’t think it’s actually that hard to imagine a scenario where this stuff gets covered and then some. But again, it gets to the question of how sustainable are those, and that has broader questions. It really depends on what demand actually looks like relative to supply and how long does a supply demand mismatch last. These are questions that maybe Dylan would probably have a better idea on than I do. But absolutely it is a concern. It is one of many constraints that are out there that I think are moderating.
I’ll say one more thing though. The idea that these constraints are moderating growth, nobody likes that, but at the same time it may be a good thing in some sense too because it stops things from going nuts. We saw this during the tech bubble. There’s dark fiber in the ground today that was overbuilt and never used.
Some of these things may actually give us a stronger for longer kind of behavior because it almost forces the demand that is there to push out supply even more and allow things to be better matched, rather than completely going nuts today and then risking a crash down the line. So, I don’t actually think constraints in general are necessarily a bad thing, but I do know how they’re perceived and clearly they are impacting the trajectory of the stocks in the near term.
AL: Yeah, totally.
Dylan Patel: Echoing Stacy’s point, in any buildout, everyone’s going to want to build.
When we look at any buildout over the last few decades, we’ve always had the industry double order, triple order, and too much capacity get built out. Supply chains are really bad at forecasting demand. As Stacy mentioned, there is a bit of self-moderation, especially on the finance side. If we go back a year or two ago, any cloud that wanted capital could get it. They would pay a pretty high interest rate, but they could get it.
Now if you don’t have really high-grade offtake, you’re not able to build if you’re some random cloud. Ultimately the hyperscalers, sure they can take their free cash flow down to zero, they can raise some debt, and their earnings are obviously growing. So their capacity that they can build out grows, but they can’t do north of two trillion capex next year. They can’t do two and a half trillion capex next year. So that moderates how much they can spend. And when you look at the Neocloud industry, which accounts for non-hyperscaler demand, nearly half of the stuff that’s being built out, if they don’t have offtake from a hyperscaler, how are they going to get it financed?
The market’s already saying, hey, if you’ve got offtake from Anthropic or OpenAI, we’re going to price it at 13, 14, 15% debt. We saw that recently with Volta. They’re making a cluster for Anthropic and their debt is basically a 12% coupon, but they sold the bonds for 95 cents. So it ends up being like a 15% interest rate. If we look out at everyone who’s not getting offtake from these companies, hyperscalers, the major AI labs, how do you build that capacity? How do you build capacity to service smaller enterprises?
How do you build capacity to service new startups? You just literally cannot unless it’s coming from a hyperscaler. And so that’s Elon’s whole gambit is he’s making all this capacity that he can sell at very high pricing on short-term rental contracts to startups, to the AI labs. But ultimately it is quite a dampener. We’ve seen Nvidia recognized this as a very big problem for them because, hey, they have five customers who can buy as much as they want or as much as they have capital for, but what about the other part of the market?
And how do they enable that? So they’ve gone and done really creative structures where with Sharon AI and Fermi and a few others, they’ve taken revenue shares on the cluster in order to guarantee that it’s financed. So they’ll say, “Hey, we’ll rent the cluster from you at a guaranteed price of $3.50, which is the cloud making zero money, and then if you rent it to anyone else above that price, we get a revenue share.” And we’re starting to see more and more of these structures. In fact, even to build a cluster for Anthropic, Lambda had to do a revenue share with Nvidia.
In this case, it wasn’t a 50% like Sharon and Fermi had, but it was still a 10% revenue share. So how these clouds can get the capital to build and get to that point where they build everything they want to build and get to the over supply scenario is really tough just because of the financing of it all. We’re seeing more and more turn from these long-term loans and long-term debt to something that’s more shorter term. So Broadcom’s done the same thing with Anthropic where they’re basically leasing them the chips in a sense. It’s a rent to own. They’re stretching out the payments over a few years.
The whole name of the game here is if anyone wants to get capacity if you’re not the big companies, you’ve got to come with a deal, a structure that has Nvidia or Broadcom or AMD backstopping these clusters. Ultimately, the financing of this all is a big dampener. The hyperscalers are also very, very large off-balance sheet and on-balance sheet commitments. Meta has $1.4 trillion dollars of commitments for compute and data centers and so on and so forth.
That’s then able to get financed by the whole market where they go loan buy a data center, rent a data center from someone or rent chips and then those guys can go turn around and get capital for it. But ultimately, there’s a big struggle for anyone to do more than that. A lot of people have worried about even Meta’s financial structure. I mean Meta is completely fine, but even the capital markets are getting spooked by that. So this is the financing is part of the holy trinity. It’s land, power, shell, it’s chips, and it’s financing.
Supply Chain Bottlenecks
AL: That’s good. That’s interesting. So it’s getting more difficult to finance. Although the hyperscalers are fine, maybe a little bit harder for neoclouds. We’re seeing these new creative structures come around to help that. If we set finance aside and go back to the traditional bottlenecks that are ultimately limiting the pace of the growth of the industry, talk to me about where you see that 2027, 2028. There’s obviously ASML says they’re sold out in 2028.
TSMC always has more demand than they have supply even though they’re doubling capacity for two nanometers. And then people talk about power as a bottleneck. Dylan, I know that your team recently this week maybe put out a post about up to 50 gigawatts coming online next year. So maybe I’ll start with you. How about you stack rank some of these different parts of the semiconductor supply chain and where should we be looking at the constraints?
DP: Yeah. So, on the semiconductor side, the biggest constraint, of course, besides memory, is substrates. Substrates, PCB materials, substrate materials, all of these components were severely underinvested in. Much of the supply chains that create the materials in Japan, those firms are a lot more conservative and unwilling to invest heavily in capacity. So we’ve seen tremendous bottlenecks there in terms of what can be made over the next two years, for accelerators as well as CPUs and every other component in the supply chain.
So substrates is I think the biggest bottleneck of all the bottlenecks including land, power, shell, semiconductors, tools. I’d say the next bottleneck would be of course memory. And then following that, there’s a lot of bottlenecks around the desire to ramp optics to a really, really large degree, with co-package optics or near package optics. There’s humongous constraints in ramping that. We just simply cannot ramp, even if everyone wanted to do CPO in 2028 or even one major company wanted to do CPO in 2028, they could not satisfy their volumes in terms of lasers, in terms of fiber attach, in terms of all these different components that go into it.
And then, as far as land, power, shell, power is definitely a big constraint. But one thing that our research shows that I think is counterintuitive to much of the market is that next year there are data center constraints, but it’s a lot less than people realize because there are nearly 50 gigawatts of data centers being built globally, seven or eight gigawatts in China and then most of that, about 35 in the US, and then a bunch rest of everywhere else. Australia, India, Malaysia being some of the big places, but really all over the world data centers are being built at a tremendous pace.
And while power is a huge bottleneck, people can find creative ways around that. Whether it be turbines on site or reciprocating engines on site or fuel cells on site, all of these things are helping alleviate the power bottleneck. So, I think actually power, as far as what most people think, oh, data center and power are the biggest constraint. I think they’re constraints, but I think they’re actually a lot less than substrates, than memory, than lasers, and other things in the optics supply chain. Because there are things that people can do to ramp power capacity on a shorter time span. They are just significantly less complex than the semiconductor supply chain.
SR: Yeah, you bet. I think it’s really interesting on the power side. Jensen and Hock, Nvidia Broadcom, have given some pretty sizable guidances and Jensen said he’ll grow his revenues to at least 70%. Hock said 100 and those are actually constrained numbers and they’re looking primarily I think at land, power, and shell as their constraints. Maybe that’s because they’re semiconductor companies. They know what they can make I guess in terms of the semis. But on the land, power, shell side, I agree there are ways to get around this.
Although even for some of the things that are going on site, I don’t know what the lead time is on a turbine these days. Musk says he’s going to go start building his own turbine blades to get around that. I think on the memory side is really interesting. The memory stocks are all trading at two, three, four times price to forward earnings and pricing has gone through the roof. I don’t cover the memory stocks by the way. It’s a colleague of mine, but you take a look at like a SanDisk that, they guided to a quarterly EPS that’s higher than the stock price was 24 months ago. It’s really amazing because pricing has gone through the roof and they’re all trading at single digit PEs because that’s how you trade things at the peak of a cycle and everybody’s worried that especially as you get into 28, supply does start to come online and they’re worried that the pricing collapses.
And I sit here and I always wonder what happens if it doesn’t. You can look inside these racks. Look inside an AI rack. 70%, 80% or so of the silicon area inside that rack is memory. It’s mostly DRAM. There are other issues when you’re making high bandwidth memory. The memory that goes into the AI chips. There’s something called a trade ratio, but it takes three or four times as many wafers to make a gigabyte of high bandwidth memory as it does to make a gigabyte of standard DDR5 DRAM that would go into a PC or go into a smartphone.
So you’re sucking up wafers from that. We need a ton of memory. And this feeds to one of the other bottlenecks, which is semicap, the actual tools to build this stuff. We had a company report this week. They actually gave a preliminary outlook for WFE for wafer fabrication equipment for next year. And he said more than 200 billion. I think a lot of investor view was it was going to be a lot higher than that even, which I found really surprising because I just don’t think it’s possible. We don’t have the fabs.
There’s your other constraint is we don’t have clean rooms. They’re all full. They’ve actually been squeezing as much as they can out of the existing footprints. That’s why the WFE estimates beginning of this year were 120 maybe. We’ll probably end the year 155. That’s all finding other better ways to use the footprints that’s there but now they have to build and it takes years to put the stuff into the ground. The clean rooms do start to come online a little in 27, more into 28 and 29. And at that point, I guess we’ll know if there’s a supply demand mismatch or not.
But if the demand profiles continue at the rate that I think people seem to think they are, memory may still continue to be a constraint. You talked about substrates and PCBs and other things. Again, to go back to the tool vendors, they’re all complaining about short-term supply constraints that are actually impacting their own ability to upside even in the near term. And it’s exactly that kind of it’s PCBs and it’s actually semiconductor chips and it’s even other components, valves and mass flow controls, everything, but it’s simple stuff that’s actually affecting other parts of the space as well.
We saw a little bit of this sort of thing during COVID. I do think that the at least the semicap players and most of the folks in the supply chain did learn some lessons from COVID and that I do think they’re managing that sort of thing better now than they used to. But at the same time, we’ve got all kinds of constraints that are in the spaces that need to get fixed before the other constraints can actually get fixed. So, you have these sort of layers, I guess, to things. I actually don’t really see I think things are going to be tight for a while, frankly. I think this year is fine. I think next year is going to be the same thing. 2028 will probably still be tight. 2029, I don’t know. That turns into a demand question. But I suspect we’re going to be tight for a while.
Pricing Power and Capturing Value
AL: Okay. So if we’re tight for the next couple of years, and we talked about prices just skyrocketing, memory industries making margins in the 80%.
SR: Higher, higher.
AL: Yeah. Yeah. 88, 90, right? So do you just see all those players, everyone’s constrained up and down the whole stack. Do you see every industry in there just continue to kind of charge for their value?
SR: Well, it’s funny. This is an industry where pricing coming down was just sort of taken as a given, right? That was just and in fact the semiconductor industry as a whole historically was a massive value destroyer. It never covered its cost of capital. All of the value flowed to the consumer. I did a piece I used to work at McKinsey many, many years ago and I did a piece of work there. It was called value creation in the semiconductor industry. We defined value very classically. It was economic profit minus WACC times your average invested capital.
We aggregated it up. It’s like 150 different companies over like 12 different subsectors over a full cycle. I think the initial work was like 96 to 04. I think over that period collectively the industry destroyed $5 billion in aggregate economic profit. And if you took Intel out of the mix, it was 50 billion destroyed. It was which was a big number back then. So the industry didn’t actually really start covering its cost of capital, meaning that all of the economic profit you created was before that was flowing directly to the consumer. It was a wonderful time to be a customer for semiconductors.
But post 08, there was a lot of rationalization and consolidation and at least we started to kind of come in and meet the cost of capital. And then COVID happened and what the industry discovered in COVID was that they have pricing power, which I think was something that they did not feel in their bones before that. We had input costs going up and everything and I didn’t I cover a bunch of companies and I didn’t have a single one experience margin compression because of input costs going up. They all raised their prices to offset it and they raised their prices at their margin structure.
So if their cost went up by 100 because they raised it by 160 if they had a 60% gross margin and they had no problem passing it along. And right now we’re seeing and I’ve said this inflation saved the industry during COVID. If you look at industry unit shipments in 2023 or 2024, they were flat to 2019 and ASPs blended for the industry were 50% higher, 50, even if you take out memory. And we’re seeing that on steroids right now. So clearly memory pricing has gone through the roof. The industry, by the way, we’re going to do a trillion and a half dollars this year, maybe more. We’re going to be up probably 100%.
70 or 80% of the growth we’re seeing this year is memory ASPs. A big chunk of the rest of it is also like non-memory ASPs. And we got units are up 10% or something like that. Whatever it is, but the industry is now flexing that pricing power a little more. And by and large, I think customers are accepting it. They don’t really have any choice. So, yeah, it may stay tight. If it stays tight, we will probably continue to see this, I think.
DP: Yeah, I would say that the semiconductor industry is very interesting in that some people flex their pricing power in the good times and they suffer really bad in the bad times, but then others they just sort of eat margin the entire way through. So, if you look at TSMC over the last decade, they’ve risen from 45% gross margin to now 60. And it’s been a very steady climb up. Yes, they’ve taken more when times were good, but they haven’t given back much when times were bad. And you look at memory, right? They’ve gone from basically negative margins just a few years ago to now, you know,
SR: People forget we’re coming off the worst memory cycle since the tech bubble just like 24 months ago. Negative gross margins.
DP: And so when you look across the supply chain, some of these folks are taking tons of pricing because it is a structure of also what is the market like? And so TSMC, where they’re a near monopoly and they’re a very long part-time partner, has not really jacked up pricing even though they could. Whereas memory companies, if Apple’s going to screw them in the downturn, they’re going to screw Apple over in the upside of the cycle. Which is pretty clearly what’s happening.
So when you look at this market, to build a gigawatt in 2024, it was roughly $40 billion, across chips, land, power, shell, everything. Now it’s north of 60. So, the price to build a gigawatt of compute has skyrocketed. Now, of course, not many people were building a gigawatt or really no one was building a gigawatt in 2024 and a lot of people were building gigawatts in 2026.
SR: You’re getting more out of the gigawatt now, too, by the way.
DP: Yeah. And then that gigawatt’s profitability is huge. I think everyone is recognizing profitability extends. A lot of the mechanics I mentioned earlier, which is you can’t build a cluster as a cloud unless you have offtake from a hyperscaler, which means you’ve locked in pricing ahead of time. That is being exploited by Elon in the sense that okay, well if chips and land, power, shell are a constraint, if I just build these and then I don’t sign a contract until it’s actually up, I can instead of making 15 billion dollars per gigawatt, can make 50.
And then the AI labs who are making, Anthropic in their recent disclosures is making on the order of 70 to 80 billion a gigawatt off of their inference capacity. They’re kind of forced to eat it. Yes, their margins are 75, 80% when they’re spending 15 billion dollars a gigawatt for compute, but if they’re spending 50, their margins are much smaller. And so it’s not clear who’s going to eat all the cost. Nvidia’s had to keep their margin roughly the same, slightly decreased, but they’ve had to eat the memory pricing increase and other parts of the supply chain like substrates and optics and all these other things.
And so the prices that they charge their customers have soared despite the fact that their margins are flat. This is we’re seeing really interesting, who gets to keep their margin, who gets to expand it, and who’s going to eat margin. And I think it’s a very fun time to be on the pricing side. Because you’re real time discovering and yes, you can sign a deal at many different prices. But the best sales guys are going to get really insane pricing and hopefully they’re rewarded for it.
Performance Gains Versus Rising Costs
AL: Hilarious. So, okay, Stacy, I kind of like what you said about COVID. It was almost like a dress rehearsal for AI for the industry to start to realize like, hey, we can charge for our value here. And Dylan, you make an interesting point about like a system builder like Nvidia having to decide of all the components as their prices are going up. Do we pass it on or do we keep our margins the same? Now, I’m feeling a little tension of, okay, there’s still going to be buildout all the way through. We are not sure what’s going to happen in 2029, but clearly, supply is limited and demand is high for the next couple years. And yet, we started by saying that it’s getting harder and harder to finance. And so I’m wondering if there becomes a point where those people at the top of the stack, the Metas and the model lab companies, if they’re ever going to push back and say this is just getting too expensive or beggars can’t be choosers.
SR: Remember, you have to look at more than the margins and the cost. You also have to look at what they’re getting out of it. I’ll take Nvidia as an example. Their pricing is going with every generation. But what you’re getting out of it is going up more than the price, which I think is important. So you went from Ampere to Hopper.
AL: Yeah.
SR: ASP roughly doubled and the performance went up 10x or whatever. And again Dylan would have better numbers for exactly what the performance agree but it was a lot. Very NPV positive sort of thing. No problem. You can pass that along. You go Hopper to Blackwell, it’s a little harder to tell. The Blackwell architecture is much different, but call it a 50% roughly ASP increased per GPU performance went up anywhere from 3 to 30x and again maybe more given some of the later optimizations and software and everything. Again, no problem.
You go Blackwell to Rubin, same thing. The cost of the racks probably doubling, and cost per gigawatt. I think Nvidia said for their own reference it was like 25 billion for Blackwell and 40 for Rubin. But again, performance up 10x. So that’s what needs to happen. You need to keep giving your customer value because ultimately that’s what’s important. It’s how much performance you getting out of that gigawatt of compute relative to how much it costs. And as long as that gap is getting bigger, I think it’s still very not easy, but it’s palatable to do this.
Hypothetically just if they were to roll out Feynman and it’s only 20% better, then yeah, then I would start to worry a lot about pricing and margins and everything. This is always a huge controversy by the way for a company like Nvidia because they do have margins up in the 70s on a massive revenue base and so people just wonder you got competition, you’ve got ASICs and you got other GPU vendors and other things and how can they hold the margins in the wake of all that? And this is always in my view as long as they can keep improving performance at a significantly higher rate than their ASPs are going because they have to raise ASPs because their own costs are going up.
And things are getting more complicated. So you have to keep that performance to cost equation working for the customer and as long as they could do that, I don’t really worry about their margins. But if the pace of that innovation slows, then I would worry an awful lot. And they’re aware of this too. I went to the Hot Chips conference this would have been in 2023 and I didn’t understand anything there. I’m a process engineer by background so all the computer science and chip design stuff goes over my head. But they had Bill Dally there from Nvidia as their chief scientist. He gave the keynote. It was the one thing there I understood.
And what he was talking was exactly this GPU performance improvements. And he kind of made the point over the last 10 years, we’ve improved GPU performance by a thousandx. It was a whole bunch of different levers that they pulled, but they got there. And what he said was over the next 10 years, we want to improve performance by another millionx. And so far they’re on the trajectory. And if they can do that, then I don’t worry as much. If they fall off or if anybody else falls off, then I worry a lot because ultimately somebody has to pay for it. In a perfect world that somebody is the customer and there’s a lot of value from it. And if the costs go up at a rate that’s unsustainable, the suppliers have to start absorbing some of it and then it’s a dog fight to see who does and who doesn’t.
The Rise of Custom Silicon
AL: Yep. Gotcha, Dylan.
DP: That’s certainly the trend is all of these companies are trying to make chips. OpenAI showed, I think for the first time ever in the industry that you can make a first generation chip that’s really good. They let us into the lab. We tested on our open source InferenceX benchmark running all these open source Chinese models. And the performance was better than Blackwell. And that’s a first generation chip and they utilized AI heavily in the design and programming process. And Anthropic is starting their chip program and they want to do it in half the time that OpenAI did.
So they want to get a chip. These are aggressive goals, but if models are able to help them really accelerate this process. You’ve got Google selling TPUs externally. You’ve got Amazon’s Trainium really quite good now and fully usable in PyTorch and such. Meta’s very aggressive roadmap, Microsoft, etc. If you look at the, I said it went from $40 billion to build a gigawatt to 60. If you remove all the margin out of the chip design houses, the Nvidia and Broadcom, Marvell, etc., you’re actually at like $20 billion a gigawatt.
And so, if you were to able to do that, then you could pay for 5 gigawatts with 1 gigawatt worth of compute running inference. And so obviously the economics here are very aligned. And as Stacy said, these companies just have to run like crazy. You can’t afford to be 2x performance per dollar per year because you’re going to get completely smashed by these other companies. So if Nvidia can keep up this incredible pace, sure they deserve to continue to earn margin. But if the other competitors, and their main major customers are able to design chips, deploy them, scale their manufacturing, unmargined memory, TSMC, wafer, substrate, optics, etc. is really, really cheap.
So those guys I think have a lot of room to run on pricing especially if things get disaggregated more and Nvidia loses their position or Broadcom loses their position and that’s why these two stocks trade at their lowest multiples basically ever. A lot of people are worried about that. Is Meta going to disentangle Broadcom? Is Google going to disentangle Broadcom? Are both of these guys going to stop buying Nvidia and just insource completely? And so, I think the component vendors are going to be able to jack price up like crazy.
The question is if Broadcom and Nvidia can continue to earn these incredible margins and at least for the next two years it looks like yes. But beyond then is that off the table now? Because at the end of the day, while chip design is very, very complex, models have solved software and programming accelerators is actually really easy now. It took us six months to get Nvidia inference working for InferenceX, our open source, the largest open source inference benchmarking. It took us like six months or nine months to get AMD working.
We got Google working in a few months and we’ve got Amazon working in a few months. We got OpenAI working in a month and a half. So the pace of this from chip works to you’re able to run software on it is rapidly compressing and now these companies are all turning their eyes to chip design and can that actually work? It’s similar in the sense that it’s very verifiable like software. Obviously there’s a lot of challenges on data and scaling that, but this is the big question for the semiconductor industry is does design become more commodity like software in which case everyone can design chips?
Obviously there’s the pesky fact that you have to pay $50 million for all the fixed cost of masks and EDA tools and all this other stuff, but the number of people designing chips is going to explode just because programming them is much easier and then if designing them becomes much easier then margins have to fall. We’ve seen SaaS companies get obliterated in the market and we’ve seen Nvidia and Broadcom really trade sideways despite that their earnings are growing at 2x plus year-on-year. We’ll see what happens, but that’s the big question.
Nvidia Versus the Competition
AL: So, Stacy, I liked your point that it’s not about the fact that it’s going from 25 billion to 40 billion to 60 billion. It’s just are you squeezing out more tokens per dollar and that’s all that really matters. And then Dylan you brought up XPUs and OpenAI very interesting example. So how should we think about over the next two years the growth of GPUs, the explosion of XPUs? There’s all sorts of AI accelerator startups that are taking different architectural bets. Is it going to look any different in the couple years?
DP: Yeah. So for XPU companies, there’s sort of two paths. If you’re a hyperscaler, you can take a more conservative design approach and do similar things to Google or Nvidia and it works out. But if you’re a startup, you have to do just really crazy things. And that’s why in the first gen of AI hardware startups, the ones that did the craziest things, Cerebras and Groq were the only ones that were the most successful. As we look at the current crop of all these startups that are popping up, only the ones that do really, really crazy things that are very different and can’t immediately be copied by Nvidia or Google or whoever or AMD, those are the ones that are going to, those are the only ones who have a shot of succeeding.
The nice thing is because there’s so much capital flowing and there’s a lot of really crazy ideas being tested and companies that are trying to build some really insane formats for building XPUs. I think 23, 24 Nvidia swallowed up as a percentage of total HBM, total CoWoS, etc. in the industry. As we looked at 25, it started to moderate, but then in 26, Nvidia’s lost share. They’re using a lower percentage of the total CoWoS wafers, lower percentage of total HBM. And again, next year they lose share.
Now, ultimately, the number is growing so fast, their earnings is still doubling year-on-year next year or more. But ultimately, this is a big sort of, it’s a thing that’s already happening is Google’s taking share, Amazon’s taking share, AMD’s taking share.
AL: On a unit basis is what you’re
DP: Sorry?
AL: On a unit basis. Yeah.
DP: On a on a Yeah. Yeah. I mean units are always difficult. Who makes a big chip, small chip? But CoWoS wafers and HBM are kind of hard to and and revenue, right? Dollars spent on foundry wafers on HBM on these sorts of things. These guys are taking share. Ultimately, Nvidia again will the base is growing so fast that they’re still going to double earnings. And so, as we go up to 28, 29, maybe growth rates aren’t as high, maybe they’re not 100% increase year-on-year of revenue. And then what happens?
Does the growth moderate to the point where Nvidia actually loses a share entirely or are they, they’re making some crazy bets for their next generation architectures past Rubin. So, maybe that’s to Stacy’s point, right? They just have to run really fast and that’s how they defend themselves.
SR: This is one of the reasons Nvidia has moved to an annual cadence for chips. It wasn’t just for them, of course, it’s everybody else to try to run to keep up. They’ve got a scale that makes them hard to catch. Nvidia’s got a lot of people always talk about moats. They’ve got a lot of moats. There’s the CUDA software, although we can have a debate on whether or not CUDA is actually a moat or not anymore, but that’s there and the ecosystem and install base of infrastructure and then even their balance sheet and the financing is a moat.
The pace of their design, they’re leveraging AI to speed up their own design. They’re not the industry is not doing stuff there that they’re not aware of and they’re not using. But it’s a race. Jensen at least is paranoid. He took the old Intel lessons to heart. He’s looking over his shoulder every day. Yeah, this is absolutely why these stocks are trading where you’re absolutely right. They’re trading cheaper than they ever have. I wrote a note, oh god, this was probably nine or 10 months ago, but we were looking at Nvidia relative to the SOX index, which is a broad index of semiconductor stocks.
As of when I wrote the note, I think there were only 13 days in the last 10 years that Nvidia had traded cheaper relative to the broader semi industry and that was like nine months ago. That’s cheaper today. So these are the things that investors worry about. It’s mostly it’s beyond just the you’re worried about overall demand sustainability. If the demand’s not there, we’re all screwed regardless of whether it’s Nvidia or anybody else. Within the group itself, it’s clearly this competition is their biggest threat because they’re the forerunner by a mile and everybody’s trying to eat their lunch.
But at the end of the day, also that’s semiconductors. There’s always trying to somebody trying to eat your lunch. It’s up to you to make sure that you can stay ahead and that’s something that they’ll have to do that will probably get harder and harder as you go forward. Just naturally.
Enterprise Adoption and Productivity
AL: Sure. Sure. So when we’re talking about GPUs and XPUs and unit share and all these competitors gaining share, we’re really focused on the data center and hyperscalers pretty much. What do you think about enterprise on-prem as an opportunity for someone like Nvidia?
SR: Well actually even today Nvidia splits their revenue out and they split by hyperscale and non-hyperscale. Well, it’s not exactly clean, because they got the some of the labs are in the non-hyperscale, but some of the hyperscale spending is going to support or vice versa, so it’s not clean, but they’re about 50/50 right now. I think the labs directly and indirectly for Nvidia roughly 20% of the revenue right now. So, it’s sizable, but I would say with Nvidia, their revenue base is maybe even a little more diverse than maybe we had thought, which is why they’re giving that presentation now.
At the end of the day though, you clearly need enterprise adoption. That’s where this is. If we don’t have enterprise adoption, there’s no industry. It’s still early days, but I think certainly we’ve got at least one killer application for enterprise, which is coding. And, those revenues, agent coding revenues have gone vertical. Does anybody write software by hand, like artisanal code anymore? I don’t know that they do. I think everybody uses it.
AL: Yeah. What’s coding? Right.
[laughter]
SR: I don’t know. I also think on the enterprise side, people are looking at revenues, but some of the actual statement may not be revenues. It actually may be productivity. And it’s harder to measure, but I’ll give you just a dumb personal anecdote. I’m a financial analyst. We build models. One of the guys who works for me for his own personal account, he’s been using Claude to build models for himself. We tried this among the team a few months ago was gave us garbage but he said I think it’s working and I said okay I want you to we’re going to get on a Teams call and I want you to throw it up we’re going to have it build a for a company we already covered have it build a model for us from scratch.
It one-shotted a model in an hour and 20 minutes. 10 years of history, went through all the different resegmentations, it captured it all. It even forecasted it. It listened to what the CEO said on earnings calls and forecasted trying to use logic rather than random. It was something that would have taken one of my guys a week to build and would have taken me three months to build, in an hour and 20 minutes. And so I could do more with less. And there are other implications of that. Like are we going to be in a world where AI adoption in the enterprise is rampant, we have 20% unemployment? I don’t know. There are other factors to think about here, but I think we’re still early and I think most of the anecdotal evidence that we’re hearing right now because this is the overarching debate.
Is there ultimately a return to use this stuff or is there not? I think anecdotally we’re coming down pretty solidly on the sides of yes there is a return but it is absolutely still early. A lot of companies are still in the experimentation phase. They don’t have the expertise to know what to do or what they need to do and so it takes time. But I’ve been encouraged by what I’ve seen.
DP: The only thing I would say is for enterprise adoption is I don’t think that enterprises are going to adopt that much on the hardware side and run their own models and such. I think the efficiencies of running batch, the scale economics of models mean that most people are just going to be buying tokens and implementing them, combining them with their own data and ultimately that’s the real way that most people are going to consume AI.
And so it still leaves this sort of stratification where, I think and I think the industry’s gone towards that more and more. If you go back two decades ago, everyone bought their own infra and deployed it themselves and now most of the spend is in the cloud. And most of the spend is going to be buying tokens. Even pharma companies, even hedge funds, like all these guys, most of their compute and spend on AI is going to directly for models, not buying compute and running open source models. There will be providers who take that and do that, but I think yeah, that I would love to talk more about this, although we’re a bit over time. So I think it’s very fun.
AL: I see a big zero minutes left. Okay, I guess that means we’re done. Okay, guys. Well, this was awesome. I learned a lot. This was great. Thank you so much.
SR: You bet.