Michael Genovese, Senior Research Analyst, Rosenblatt Securities

It is really good to see the expectation for CPO laser revenues in the fiscal second quarter, because that would imply that you are either qualified or have line of sight to qualification and would kind of go against some of the fuzz, I think that is not coming from Wall Street, but more coming from Substack saying you guys are having trouble with that laser. Just any more color on the confidence that you will be ready for the customer to actually recognize revenue in that quarter and just progress that has been made recently.

Most people believe that the “Substack” being referred to here is who has been quite vocal about the Coherent’s lack of leadership in UHP lasers for CPO. You can see his recent earnings commentary here with a strong challenge to Coherent to present their laser data to prove to Wall Street that they are a viable InP laser alternative to Lumentum.

Irrational Analysis

Coherent Q4 FY26 Earnings

Irrational Analysis is heavily invested in the semiconductor industry…

Read more

6 days ago · 113 likes · 5 comments · Irrational Analysis

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

Also check out the Semi Doped podcast with and myself, and our daily free newsletter with latest semi news.

If you would like to engage independent research services for your project or need expert calls, check out my boutique consulting practice at SemiExponent.

This signals that Wall Street is becoming increasingly entrenched in industry commentary on Substack (this publication included). Earnings calls are no longer just financial numbers and promises. They are becoming events where company leadership needs to start bringing receipts to their claims, or risk getting called out on it by analysts who are increasingly technically savvy. These kinds of questions on earnings calls are only going to increase.

I have argued in the past that the know-how of building UHP lasers has existed for a long time, in the deep-dive articles below. These articles walk through step by step on how to design a UHP laser, and the information broadly exists for decades in the public domain.

Sure, there are refinements to the laser design that adds up over time to create performance advantages, but fundamentally, the challenge lies in making these devices at scale. There are too many things that can go wrong when hundreds of milliwatts are output from a laser. Knowing how to do this at scale is the real competitive moat.

From a pure performance perspective, does Coherent have a “good enough” CPO laser that will work? Their UHP laser specs match Lumentum’s at least on their website. Even if they don’t have the absolute best laser, can they build one that works reliably, at high volumes, while being on time?

I do believe Coherent can take some steps to build investor confidence going forward. If they would like to stop people from making guesses, then some technical data needs to be published sooner rather than later.

Showing Linewidth Data to Investors

To the first order, a lot of doubt can be alleviated if Coherent simply showed linewidth data. If a sub-500 kHz linewidth laser is what is being advertised, then that data should be easy to show. But that data point has been noticeably absent. Publishing linewidth, relative intensity noise, and mode-stability over a range of temperatures and electrical conditions should silence critics quite easily. There is no need to even publish the latest iteration of the laser, but show a “good enough” laser from a few years ago that is in the ballpark. This protects any strategic advantages going forward.

The danger of this strategy which likely worries Coherent Investor Relations is that anything short of what Lumentum has published would put them squarely at a number two position (possibly lower), and that messaging is not acceptable. In a way, this gives credence to what the critics have been saying all along and could become a dangerous narrative that is highly likely to be misconstrued on Wall Street, or even by technically savvy analysts.

What is even stronger is to actually have ELSFP modules at a conference like Lumentum did in ECOC 2025. See this short video, and notice how it includes a power meter showing nearly 24.5 dBm of power output, along with a spectrum analyzer showing the laser linewidth.

Coherent had a demo at ECOC 2025 too, but they only had a power meter and no spectrum measurements. The power reading was >24 dB alright, but that is a scalar measurement. A spectrum measurement of the laser output usually reveals a lot more – like linewidth and side-mode suppression ratio (SMSR).

This approach is even better than a published paper because critics can see the hardware working in real time and know that it is not a one-off result in a company’s lab environment. Several conference attendees will press the exhibitor on the show floor with questions, and then write on Substack and social media about it. Independent reports analyzing what they say serves as “proof-that-it-works” and builds investor confidence.

The only live demo of CPO with ELSFP was shown at OFC 2026. These numbers from the demo characterize the whole transmit chain: laser, coupling, split ratio, modulator, and driver, The laser-specific merits are either confounded or invisible. To judge the laser itself you need fiber-coupled power per port, wall-plug efficiency, RIN in dB/Hz, SMSR, temperature range, and reliability data.

The industry will be closely watching ECOC 2026 and OCP Global Summit for any information on this front.

The Counterargument

While laser measurements would be helpful, should Coherent really cave to the pressures of Wall Street or the blooming cottage industry of Substack industry analysts? From my own years of experience working within the semiconductor industry, it is common for companies to choose to not publish, or send representatives to conferences for fear of information leaks. Does lack of information necessarily mean that a product line is uncompetitive?

Broadcom has their Remote Laser Modules (RLMs) that they use for their CPO line of products. From what I can tell, I don’t see any published measurements of laser specs from them either. If you know how RLMs from Broadcom compare to Lumentum’s UHP lasers, please do let me know. But oddly enough, nobody questions Broadcom regarding their UHP laser capabilities.

This leads us to ask: is Coherent being held to an unfair standard? Should they publish data just because their competitor did? Nvidia invested $2B each into Lumentum and Coherent for their laser supply. I would assume basic due diligence by the highly competent optical networking experts at Nvidia (particularly the group from their Mellanox acquisition) would have revealed if there were fundamental flaws with their laser technology.

Ultimately what matters is whether they have a laser that works for CPO. In a highly supply constrained laser market, having a highly reliable product, with high yield, and competitive specs would make them as important a player as any in the high power laser market.

What to Watch For

In the absence of laser measurements, there are some industry signals that investors should watch for in the next 12-15 months, to understand where Coherent is headed.

Read more

Given its importance, the exact underlying technologies are still open to debate, sometimes even contradictory. The networking theme ranged from “Copper is not going anywhere” to “Optics everywhere,” and everything in between. We’ll go through some talks from OCP APAC Day 1 that dance around this theme.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

Also check out the Semi Doped podcast with and myself, and our daily free newsletter with latest semi news. If you would like to engage research services, reach me at SemiExponent.

Themes we cover in this report:

• Copper will not only survive, but thrive. SemiAnalysis about copper-optics coexisting for the foreseeable future.

• Copper is dead. Lets go to optics. Astera Labs talk that declares copper dead, and goes to NPO/CPO.

• Optics, but make it quick: Fast OCS switching for scale up by a company called Linque.

• Not copper, not optics. There may be a third door. Point2 e-tube technology had a talk and show floor demo. 4K video of hardware included.

Read more

Here are some key earnings takeaways:

• Revenue of $392.4 million, up 27% sequentially and 104% year over year, against a Q2 guide of $355-365 million given in May.

• PCIe 6 products, meaning Scorpio switches plus Gen 6 Aries retimers, crossed 50% of revenue for the first time, up from roughly a third in Q1.

• Q3 guidance of $540-560 million, non-GAAP gross margin of about 72%, and non-GAAP EPS of $1.16-1.21.

Their Scorpio line of switches is expected to become the largest product family in Q3, one quarter ahead of what management guided in May. Part of their uniqueness comes from their COSMOS software platform which does more than just telemetry and health monitoring.

Astera Labs should be viewed as a switch company – along the lines of Broadcom – going forward, as their 320-lane Scorpio X switch gains larger adoption, notably even with neoclouds and inference GPU providers.

CXL is an interesting segment that shows some promise, but is really a 2027 story. Optics is a 2027 story regarding NPO, while CPO lies even beyond that. The Q2 earnings call provided quite a bit of color regarding where things are headed.

For a deeper-dive on Astera Labs, see the post below.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement. For institutional research, contact SemiExponent.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

If you are not a paid subscriber, you can purchase just this article using the button below. You can find the whole catalog of articles for purchase at this link.

Buy the article

Scorpio X Acceleration

Around the time of IPO, the dollar content of Astera products per XPU was $50-$100. In the Q2 earnings call, COO Sanjay Gajendra said that he sees their business contributing multiple $1,000s per XPU with the 320-lane Scorpio-X scale-up switch itself contributing $1,000 per XPU.

In SemiAnalysis’ Trainium 3 deep dive, they estimate that the 320-lane Scorpio X has an attach rate of 0.25-0.28 per XPU depending on the switched rack Trn3 SKU. We’ll reproduce that table below for convenience.

At $1,000 of Scorpio X content per XPU, a 32-XPU Gen2 Trn3 rack carries $32,000 worth of 320-lane Scorpio X content, delivered as 8 scale-up switches. The 72-XPU variant Gen2 Trn3 rack carries $72,000 across 20 switches. All this points to an ASP of $3,600-$4,000 per Scorpio X switch.

This is consistent with Tomahawk 6 pricing. Broadcom priced Tomahawk 6 below $20,000 at launch, before volume discounts, for a total switch bandwidth of 102.4 Tbps. That works out to roughly $200/Tbps. Scorpio X has 5x lower switch bandwidth at 20.48 Tbps, and at $200/Tbps, that prices it right at $4,000.

Additionally, for the Gen2 variant, we are not counting the lower lane count Scorpio-P switches (144 32-lane, 72 64-lane, or 36 128-lane) that are on the PCB per rack. That would add even more incremental revenue for Astera Labs, and Aries retimers would be on top of that. You can see how much PCIe Gen 6 content exists in a rack, and how that can contribute to 50% of total revenue as stated in the earnings call.

Another bit of color in the Scorpio acceleration story stems from the fact that companies that are building add-in card GPUs like SambaNova or d-Matrix (for example) would also be interested in Scorpio X switches, and not just AWS. This becomes even more important if companies like these can buy NVLink Fusion interfaces in the future, so that they can work better with Nvidia hardware. This NVLink aspect will become important in Trainium 4. Similarly, neoclouds are a potential buyer of Scorpio scale-up switches too.

Gajendra said that there are 10 customers getting into pre-production and qualification cycles. It remains to be seen how many will materialize into revenue in future quarters.

COSMOS

A part of the switch story that we did not cover fully in the deep dive was their COSMOS software platform. Deep telemetry was an important part of Astera’s growth strategy in the early retimer days. Today, COSMOS is much more. Its Hypercast and in-network compute features are advanced hardware-accelerated networking technologies that offload data processing and multi-destination packet distribution directly onto smart fabric switches to speed up large language model (LLM) training and inference. This essentially speeds up collective operations during training and enables greater utilization of GPUs.

Thus, the Scorpio X switch story does not stand alone. It is closely coupled with the software to extract the maximum performance of switches and GPUs within a rack.

CXL

Compute eXpress Link (CXL) solutions are becoming increasingly popular in the industry now as memory shortages stay acute, and hardware makers scramble to provide alternatives for a variety of use cases. Gajendra points out a few in the earnings call:

1. High cost of DRAM is driving people to stick different kinds of memory behind a CXL bus

2. When CXL is attached to an XPU, KV cache acceleration drives lower inference latency

3. In general compute applications that are memory intensive, CXL provides a way of adding more memory to the CPU

CXL is a story that is just beginning for Astera Labs via their Leo series of products. In Q2, they closed a design win with a US hyperscaler. In 2027, they expect to ship CXL memory controllers to two US hyperscalers. But most of the volume and associated revenue is really a 2027 story.

After the paywall are some thoughts about where Astera Labs goes from here, what can fail, and what to watch for.

Read more

“Given the size of the opportunity and the associated dollar content, we would expect to see that Scorpio will become our largest product line by the end of the year, which is strong performance for the product line that was only a small percent of total company revenue last year.”

In addition, Astera Labs explicitly pointed out in their last earnings calls that they expect their dollar content per accelerator to go up.

“We do see our content continue to increase, and to that end we are expecting, and going forward with the design wins we have, over $1,000 worth of content per accelerator, and that is significant and growing rapidly for us.”

The consensus on Astera Labs is that Scorpio is the growth story and that part largely holds as Trainium 3 volumes ramp. What the consensus misses is that Scorpio X’s scale-up revenue has a narrow customer list, and a known end date when AWS moves Trainium 4 to NVLink Fusion. Astera Labs’ future hinges on their pivot to UALink in the next generation, and the potential to serve as NVLink Fusion connectivity partner to a broad cross-section of the AI accelerator industry.

In this post, we will focus on why Scorpio switches are the near term growth path, how AWS warrants plays in, and where NVLink Fusion and UALink land in Astera Labs’ future.

If you want to understand the history of Astera Labs from their early foray into retimers, check out our deep dive podcast on Semi-Doped on this. ALAB announces its 2Q2026 earnings call on August 4th 2026. We will publish an earnings review post as well.

Contents:

• Why it makes sense to go from PCIe retimers to switches

• What the choice of PCIe means as a protocol for scale-up

• Comparison of PCIe switches from Astera Labs, Broadcom, and Marvell

• How AWS warrants play into their relationship with Astera Labs

• What the move to NVLink Fusion in Trainium 4 means for Astera Labs

• Why UALink and its adoption defines the future of Astera Labs

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

If you are not a paid subscriber, you can purchase just this article using the button below. You can find the whole catalog of articles for purchase at this link.

Buy the article

Also check out the Semi Doped podcast with Austin Lyons and myself, and our daily free newsletter with latest semi news.

If you would like to engage independent research services for your project or need expert calls, check out my boutique consulting practice at SemiExponent.

PCIe: From Retimers to Switches

Astera Labs’ current networking strategy is entirely built on PCI express (PCIe), with a future goal to upgrade to native UALink. Their Scorpio line of switches is built on PCIe, which is a natural extension of their IP from the Aries line of retimer products that are broadly used to fix signal integrity of PCIe links that connect the XPU to CPU, NIC or SSDs.

The Scorpio product line has two SKUs, for different use cases in the rack:

• Scorpio P-Series: PCIe 6 switching for scale-out networking — fanning GPUs, NICs, and storage out of head nodes. It has been shipping since Q2 2025 with design wins at three hyperscalers and two more starting late 2026. Has product variants from 32 to 320 lanes.

• Scorpio X-Series: the scale-up fabric — the merchant silicon switch, running memory-semantic traffic over PCIe.

Astera Labs states that Scorpio is a bestseller for them, with 10%+ revenue within two quarters of launch, and is expected to become the largest product line by the end of 2026.

The Choice of PCIe

Regardless of their impending success with PCIe switches, it is interesting to ask why PCIe as a protocol of choice for scale up? PCIe has been the mainstay of connections of computer motherboards for decades, and what made Astera Labs a company with $1B ARR today is their prescient bet on PCIe in the AI era. Their massive design wins in the Hopper and Blackwell HGX boards within a server tray, was because of their first-to-market with a PCIe Gen5 retimer, which has become an incredibly sticky slot connecting XPU-CPU/NIC and is a big contributor to their success. The logical next move for Astera Labs is to move this IP up the stack into rack-scale.

PCIe itself is attractive as a protocol for one main reason: every device – XPU, CPU, NIC, SSD – already has a PCIe controller built in and natively speaks the same language. Because the protocol is designed to know the route directly to various devices, it routes by memory address and encodes destination in its upper bits. ESUN-T and UALink had to define specific transport layers to get the same thing. In addition, PCIe lanes run at a slower bit rate compared to Ethernet, which is useful for keeping scale-up in copper longer.

PCIe Gen5, for example, operates at 32 GT/s (giga-transfers-per-second), where the transfer refers to the movement of a bit. This relates to the modulation scheme as well: an NRZ modulated signal moves one bit per signal, while PAM4 moves two bits per symbol. The symbol modulation speed is measured in Baud: at 32 GT/s, it is 32 GBaud on NRZ, and 16 GBaud on PAM4. On a per-lane basis, the speeds of PCIe are slower than NVLink, ESUN (ethernet based) or UALink, which is oddly an advantage when it comes to interconnects for scale-up with copper (better reach). But, what PCIe lacks in per-lane speed, it must make up for in lane count.

The ultimate speed in Gbps of a PCIe link thus depends on: (1) speed in transfers/sec, (2) modulation format, and (3) number of lanes per link. Let’s work on a simple example to show this:

PCIe at 64 GT/s using PAM4 at 32 GBaud, and running four lanes in aggregate (x4) gives 64 GT/s x 4 / 8 bits = 32 GB/s unidirectional, or 64 GB/s of bidirectional bandwidth. The following table shows this calculation for several PCIe generations.

ALAB Scorpio X, AVGO PEX, Marvell Structura S

It is instructive to compare the Scorpio switch against the big dog in the merchant silicon switch market – Broadcom. Broadcom also offers PCIe 6 switches in its PEX line of switches, alongside its flagship Tomahawk 6 (TH6). We’ll gloss over Marvell too.

Read more

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

Also check out the Semi Doped podcast with Austin Lyons and myself, and our daily free newsletter with latest semi news. If you would like to engage independent research services for your project or need expert calls, check out my boutique consulting practice at SemiExponent.

Are you aware of the Situation?

Situational Awareness (SA) is a hedge fund run by ex-OpenAI upstart Leopold Aschenbrenner whose entire view on the AI trade is “hyper-bullish” to put it mildly. Their sole premise of investment is — AGI is here to swallow the world, and nothing short of it. In the era of bottleneck trades that we have been in the midst of for the last two years, this premise worked swimmingly, with the fund posting over 2,000% gains.

They did so with an insane amount of leverage — like 4x, where leverage is basically borrowing to invest, and not putting down your own money. You can make a lot of money with leverage, but when things go south like they did this week, you can also lose a lot of money. We’ll get to that.

SA become a global and FinX phenomenon, where everyone was waiting to see what they were investing in. Their 13F filings were more awaited than a Taylor album for a Swiftie. There were plenty of copy traders who waited to see what SA invested in and bought the same stocks. I imagine some of them were even more levered than SA because (a) blind faith/greed, and (b) they were not as early as SA, and had to “catch up.” More leverage = more money, especially if you already had bought into the “AGI is here and nothing else matters” kool-aid.

The Storm was Brewing

But the perfect storm of events were already brewing; one that you can probably only tell in hindsight.

• SK Hynix’s stellar numbers came in below consensus (a word the Street uses to mean “we all believe this is how much they will make in earnings”) — a sign that expectations had gotten ahead of reality

• China announced they are putting immersion DUV in production — this does not change anything immediately.

• CXMT IPO-ed and got a massive boost with people believing they will flood the market with cheap memory — they can make DRAM, not cutting edge HBM, and they definitely do not have capacity to supply the entire world with memory.

• Everybody getting very skittish about the amount of debt that the infrastructure companies had taken on.

There is one more factor that deserves its own section later, but let’s (briefly) talk about this debt: even earlier this week, there were fears that infrastructure builders are not going to be able to repay their debts if all the CapEx investments do not turn into revenue. Not to mention all the circular financing deals going on, where Nvidia is investing money into their own customers, and many more complex dealings tied to tranches of stocks vesting when they hit certain prices. It’s nuts, I agree, and hard to keep track of.

Anyway, investors often buy Credit Default Swaps (CDS) when they believe that companies will not be able to repay their debt; think of it as insurance against a bad investment. When this insurance premium goes up, you know that more people believe that companies will default of their lines of credit. CDS premium spreads were what what gave away the subprime mortgage crisis of 2008-2009 too, but at the time, they were insuring against Mortgage-Backed Securities. Only now, its against the AI infrastructure debt, not mortgages.

The Fuse Was Lit

In this supercycle, it is clear that memory investors are most skittish because of the rapid rise of profitability in the big-3 memory companies. These companies have made an order or magnitude more money on an annual basis than just a few years ago. Profit margins have reached 90%, which is an absurd number, and the entire tech industry is cracking under the burden of memory prices.

To put it mildly, investors have been looking for the memory top.

SK Hynix’s earnings showed an inflection point — one that investors have been waiting for. Korean investors, like so many others, have also been highly leveraged on memory companies; and when the first derivative hit zero, many investors found the top they had been looking for. This caused a massive unwind in the Korean market causing SK Hynix stock to have one of the greatest drops in Korean stock market history. The KOSPI index plummeted, and the Korean stock market had to limit short selling / hedged bets to even stabilize the carnage.

This essentially pulled the entire AI market down, because if we are at the top now, then future capex investments are going to decline, and that means there’s not a lot of money to be made. Better pick up the change and run. Even Corning, who unrelatedly made innocuous optical fiber faced a 20% drop in stock. Amkor — a packaging company — posted good earnings, but faced a steep sell off. The blast clearly has had a wide radius.

Pull the Trigger, But Aim First

In this midst of this devastation, SA had about $45B invested and mostly in AI trades and short software stocks. Their stars aligned, but for the worse — a Lovecraftian scene where the cruel moon shone its eerie white light on one hyper-bullish hedge fund. Its long positions in AI were crushed, while short software positions ripped.

In less than a week, SA lost tens of billions of dollars wiping out the entire public holding of the hedge fund. Its losses were about 75%, and what remained was its position in private funds, including a $5B stake in the Anthropic IPO which remained protected. This kept SA alive as a hedge-fund, but crippled and bleeding.

A letter posted by SA showed that they are still up 80% YTD, just not 2,000%+. But 80% is no big deal. Anyone who bought at the start of the year would have 45% in gains; you don’t need a genius to make 80%, let alone Leopold Aschenbrenner, who is still a smart man all said and done.

So what happened to SA’s public equities? Well, there is one aspect of the storm I haven’t told you about, remember? A few days ago, a note from Frank Flight, head of macro strategy at Citadel Securities — a hedge fund owned 80% by billionaire Ken Griffin — warned that the Fed would raise interest rates by 0.25% in an effort to fight inflation. This was likely a scare tactic because the fed did not end up raising the interest rates after all. But the damage was done.

Tech and AI stocks are highly sensitive to interest rates because their valuations rely heavily on projected future earnings. Higher rates make future profits less valuable today, and the scare was also a contributing factor to the sell off this week. One theory is that the scare tactic was planned, and intentional because of what happens next.

Citadel Securities comes in and BUYS SA’s ENTIRE PUBLIC STOCK PORTFOLIO for 40-50 cents on the dollar.

A lot of main news outlet’s claim that Citadel “bailed out” SA here, but make no mistake: Citadel aimed well, and shot SA straight in the head. Legal? Maybe. Market Manipulation? Maybe. Brutal? Yes, 100%.

Oh, and did I mention that Leopold was getting married this weekend, to the chief-of-staff of Anthropic — Avital Balwit? Dude.

After the paywall, a few thoughts on what happens next.

Where does that leave us now?

Read more

Since the video’s release on Youtube late last week, shock clips have appeared on social media that take the whole conversation out of context. What actually follows in the whole 35 minute discussion is an insightful discussion into the future of memory, how it is an exciting time for memory innovation, where memory is going in the next few years, and what the industry is looking at today.

This article discusses these points in greater detail, providing necessary color where helpful so that the whole discussion can be viewed in a more helpful and positive context for the future of memory.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

If you are not a paid subscriber, you can purchase just this article using the button below. You can find the whole catalog of articles for purchase at this link.

Buy the article

Also check out the Semi Doped podcast with and myself, and our daily free newsletter with latest semi news.

If you would like to engage independent research services for your project or need expert calls, check out my boutique consulting practice at SemiExponent.

The Three Fold Problem

Gelsinger points out that there are actually three major problems with HBM today: (1) insatiable demand, (2) larger capacity demand, and (3) greater memory bandwidth requirements.

When these three elements are put together, only one technology really emerges to satisfy the requirements of the industry: high bandwidth memory, or HBM. Stacking DRAM chips to make HBM ticks the higher capacity with high bandwidth boxes, while creating a massive appetite for memory. Today, AI is a memory-limited problem.

The consequence of this is that memory makers enjoy a near 90% margin, which Gelsinger points out is something that should never happen. Memory companies used to have one good year out of four as they usually built out capacity too quickly, and were left holding the bags in a downturn. Today, the memory cycle is looking strong several years into the run up, with no end in sight.

Unfortunately, the memory crunch is here for at least another 2-3 years, as Gelsinger points out, because capacity buildouts take time. He also points out that in that time frame, Apple may increase prices yet again. The consumer market will continue to be choked by AI demand in the near future. The hardest question to answer today is: When capacity does come online, will we still need all this memory?

Vijay Shilpiekandula from Dilation Capital makes an interesting point: token growth is exponential, while capacity buildout is linear. He mentions that memory intensity per token is on the rise - a framing that I particularly like. As context lengths continue to grow, and the user base (human and agent) increases, the amount of memory that the system has to support only increases, and from what we can tell today, there is demand inflection on the horizon.

HBM: The Ugly Baby

The said ugly baby was actually born out of a collaboration between AMD and SK Hynix as early as 2013, when they were trying to solve the memory bandwidth problem for supercomputing applications. HBM1 stacked four 1 GB DRAM chips for a total capacity of 4 GB, with a memory bandwidth of 128 GB/s. Today’s HBM4 stacks sixteen 4 GB dies for 64 GB capacity and a total bandwidth of 2TB/s.

The baby is all grown up now, but has become increasingly difficult to handle. Stacking to increase capacity causes thermal, yield, and capacity issues. DRAM chips generate heat, and it is becoming difficult to ensure that it is effectively removed. Yield is also becoming difficult as stack heights increase. Capacity-wise, HBM is incredibly inefficient because the bit-density of a DRAM die for HBM is 1/3rd of a standalone DRAM chip - a consequence of putting through-silicon-vias (TSVs) through the DRAM die. Making HBM now pulls all the capacity out of the DRAM supply chain because it takes 3x as many wafers to make the same HBM capacity, as you would with DRAM.

The problem gets worse with HBM5. JEDEC, the standard body that defines how high these stacks can physically be, has been relaxing the specs because keeping the physical dimension constrained would involve a shift to hybrid bonding - a technology still new to the memory industry at scale. Concept demonstrations for HBM5 from the big memory makers now include integrated cooling solutions co-packaged with HBM to extract heat. In addition, the base die that sits under the DRAM stack is going to a custom design that makes memory no longer a commodity, off-the-shelf product that is fungible between different designs.

The memory game has fundamentally changed, and Gelsinger’s main point is that the next few years will see the rise of new architectures and technologies if we were to fulfill our AI dreams, giving new life to a technology that has stopped scaling for the good part of a decade. It is the greatest time in history to be in the memory business.

After the paywall:

• The Era of Memory Innovations

  • Bandwidth solutions (SRAM, 3D-DRAM)

  • Capacity solutions (SOCAMM/LPDDR, Flash)

• Nobody knows how this memory cycle ends

Read more

In Part 1 of this series, we built up the fundamental physics of ultra high power (UHP) lasers. We concluded by identifying the four walls that make it challenging to implement a UHP laser: managing heat, maintaining narrow linewidth, getting light out, and catastrophic optical damage.

In Part 2, we will explore exactly how Lumentum overcame these challenges via their 2007 paper as JDS Uniphase, and via their later 2022 CLEO paper on UHP lasers. We will touch on reliability and single mode operation, and dig into how defensible their UHP laser platform is, including what it means for the competition.

Contents:

• Suppressing Re-Broadening for sub-MHz Linewidth: How to choose the key design parameter that makes UHP lasers work

• Precisely Engineering the Cavity Length: Are longer lasers necessarily worse? How do laser designers choose the laser cavity length?

• Pros and Cons for Longer Cavity Length: What does cavity length help with, and what it means for reliability and mode-stability

• The Moat isn’t the UHP Laser design: The engineering mindset is to out-design the competition in UHP lasers, but reality is different: is it good enough?

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

If you are not a paid subscriber, you can purchase just this article using the button below. You can find the whole catalog of articles for purchase at this link.

Buy the ebook

Also check out the Semi Doped podcast with and myself, and our daily free newsletter with latest semi news.

If you would like to engage independent research services for your project or need expert calls, check out my boutique consulting practice at SemiExponent.

Suppressing Re-Broadening for sub-MHz Linewidth

Even in 2007, Lumentum (then JDSU) realized that increasing the output power of a DFB past a few 10s of milliwatts resulted in the laser linewidth getting broader than 1 MHz even if the “Schawlow-Townes-Henry” equation predicts that it should get narrower as the laser power is increased (explained in part 1). Good lasers should have narrow linewidth because the transmission of an optical pulse on an optical fiber broadens the pulse anyway; no need to have that problem at the source.

Pulse broadens as power gets higher. Source: Gemini, for conceptual purposes only.

They explain that there were two things causing this:

1. With so much optical power in the laser, photons are not distributed uniformly in the physical structure under the DFB grating. We explained this effect as spatial hole burning in part 1. This non-uniformity also causes a non-uniform refractive index, which in turn makes the linewidth broaden.

2. Within the InP multiple quantum well (MQW, where electricity is converted to light), there are random charge recombinations always occurring. This added to the “phase noise” of the system and also caused linewidth to broaden.

The former mechanism was more dominant, but the latter is significant enough to not be ignored. It was already known through other published work at the time that the easiest way to avoid re-broadening is to make the cavity length (L) longer. This generally helps distribute the photons in the cavity more evenly and keep the refractive index as even as possible under the DFB grating.

But it is not quite as simple as just increasing length because how evenly the photons are distributed depends on the DFB grating design. In part 1, we explained how there is a quantity κ – the grating’s reflection strength per unit length – combined with the length of cavity L, determines how evenly the photons are distributed.

As a recap from part 1:

κL is the key knob that decides how light is distributed along the chip.

• If the κL is too low, then light spreads out and leaks towards the ends. Insufficient feedback to hold the light in the channel evenly, and wavelength selection is weak.

• If κL is too high, then light gets trapped and piles up in the middle of the chip and causes spatial hole burning.

Engineering the right UHP laser now came down to picking the right cavity length L, for a given grating reflection strength κ set by the grating design. There is a goldilocks range for κL where the photon field was the flattest – which JDSU determined to be 0.7. Look at the pink lines in the graphs below, for different phases of the optical signal. κL=0.7 provides a flatness over nearly 200 degrees of phase – much better than κL=0.5 or κL=1.

Precisely Engineering the Cavity Length

Now that κL is set, the crux of the problem is that both κ and L are design parameters that can be chosen to keep κL=0.7. How do we choose?

Read more

The goal of this article series is not to dethrone Lumentum or advocate for an alternative provider in any way. It is to understand the true difficulty of making UHP lasers for CPO from first principles, so that the reader is well informed enough to cut through the noise and think for themselves.

Instead of examining generic laser architectures, we will directly address the one that Lumentum says their structure is most similar to, in their CLEO 2022 paper.

In their CLEO 2022 paper, it seems interesting that Lumentum would quote a paper from 2007 as the basis for their high power laser design. These authors at the time were affiliated with JDS Uniphase, which spun off in 2015 to become the publicly traded, and well loved company that we know as Lumentum today.

Interestingly, none of those authors work at Lumentum any more. Another important takeaway is that Lumentum has been working on UHP lasers since before they were Lumentum. As it turns out, 20 years later, this precise laser architecture is what the world wants for CPO and Lumentum just happens to have a ton of experience with it.

This 2007 paper describes a 1310 nm InGaAsP/InP DFB doing 600 mW on-chip and 350 mW ex-fiber with sub-500 kHz linewidth, and is spec-for-spec the ancestor of the UHP laser Lumentum sells for CPO today. We’ll use this as a case study to understand UHP lasers and how its related challenges were solved, but is still by no means an endorsement for investments in Lumentum. Do your own research.

To understand how to make a good UHP laser requires a background in how these devices work. This first part is groundwork: how an InP DFB laser works and the physical walls that make high power hard. Where Lumentum’s advantage actually lies, and how deep the moat runs, needs this physics first, so that verdict waits for Part 2.

Contents:

• InP laser diode fundamentals

  • Multiple Quantum Well structure

  • L-I curve, spontaneous vs stimulated emission

• Distributed Feedback (DFB) grating design

  • Mode-hopping

  • How to space gratings

  • How DFB works

  • Refractive index changes

  • Reflection strength and length

• Laser Linewidth and DFB Cavity Length

  • Calculating linewidth: Schawlow-Townes-Henry equation

  • Impact of power, mirror loss and dependence on length

• Low Power DFB Lasers for Telecom/EML

  • Direct modulated lasers (DML)

  • CW lasers and EMLs

• High Power DFB Lasers for CPO

  • How to increase output power

  • The 4 walls of UHP lasers

    • Heat

    • Linewidth widening

    • Getting light out

    • Catastrophic optical damage

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you are not a paid subscriber, you can purchase just this article using the button below. You can find the whole catalog of articles for purchase at this link.

Buy the report

If you would like to engage boutique research services for your project, reach out to us.

Contact SemiExponent

The Indium Phosphine (InP) Distributed Feedback (DFB) laser is the fundamental component of optics today, so it is best we understand how it works. It consists of two basic parts built on the same piece of material: The InP laser diode and DFB grating.

The InP laser diode

This is often called the gain section and its job is to convert electrical current into optical power. The light is generated by Multiple Quantum Wells (MQW) in an “active region” which are essentially alternating layers of InGaAsP (Indium-Galium-Arsenic-Phosporus) sandwiched together. The alternating layers have slightly different compositions of these fundamental elements, so that they have slightly different energy levels. The purpose of this structure is to trap electrons in a low-energy layer between two high energy layers so that the electrons are trapped in a “well” and can’t get out. Multiple wells means more electrons, which means stronger laser output.

Source: Effects of multiple quantum well width on InGaAs/InP laser diode, doi:10.1088/1742-6596/2937/1/012007. (link)

If you want to understand more about what energy levels in a semiconductor mean, I have several articles on this.

To generate light, current is applied to the laser diode by applying a voltage across the anode and cathode. When the current applied is low, the emission of light is a messy stream of light waves out of phase with each other, each one behaving independent of the other. This is called spontaneous emission, and is not a focused beam of light; it is much like LED light. If photons were people, they would just wander around in a park without purpose.

Beyond a threshold of current, stimulated emission kicks in. The photons generated in the presence of a highly concentrated “well” of electrons kick up more photons themselves resulting in an avalanche of more photons. These photons now march in step with their phases correlated to each other. Now the photons are on a military parade in complete lockstep while being highly directional and focused.

Source: Wide-Band and Scalable Equivalent Circuit Model for Multiple Quantum Well Laser Diodes, Jae Hong Kim, PhD Thesis, Georgia Tech 2005.

This relationship between light and applied current is represented by the L-I diagram shown below. The threshold of current needed to make photons march together depends on the temperature; higher temperatures need more drive current. This is an important observation that will later drive a lot of our understanding of UHP lasers.

Source: Wide-Band and Scalable Equivalent Circuit Model for Multiple Quantum Well Laser Diodes, Jae Hong Kim, PhD Thesis, Georgia Tech 2005.

Read more

Beyond the midplane PCB, SemiAnalysis notes that:

• The NVL576 multi-rack system is also likely delayed or limited to small volumes due to CPO challenges, leaving Nvidia without a proven way to expand the scale-up world size for Rubin Ultra.

• The NVL72x2 back-to-back architecture was cancelled after pushback from hyperscalers over its odd design and operational burden.

• The Oberon rack will remain the mainstay, and Nvidia will sell more of them in the Rubin era to cover the Kyber shortfall. SemiAnalysis still sees data center compute revenue running 20 percent above consensus in the second half of FY27, so views are not uniformly bearish.

In this post, we will view all these claims through an engineering lens to understand the underlying physics, draw conclusions where possible, and suggest solutions that might be deployed.

Table of contents:

1. Is the midplane PCB really too hard to manufacture?

2. NVL72x2: Why It Might Have Really Been Cancelled

3. NVL72x2: How to Make it Work

4. CPO for Scaling up World Size

5. Does this really create an opening for Google and AMD?

At the end of this post, you will walk away with greater nuance as to what roadblocks are in the way of next generation hardware, how the big players are working around them, and where the pieces might fall in the future.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

You can find the whole catalog of articles for purchase at this link. This article will be available for paid digital download shortly after publication.

This is how we think about market claims at SemiExponent. If you have a bet you'd like tested this way privately, get in touch.

Contact SemiExponent

Is the midplane PCB really too hard to manufacture?

Let’s first introduce the midplane PCB and explain what problem it aims to solve. Here is a picture of it:

In all the generations of racks till date, the scale-up domain in NVIDIA racks has always been copper. There is a cable cartridge that contains just over 5,000 passive copper cables that hook up the GPUs in an all-to-all configuration. As the interconnect speeds in the scale-up domain increased in every successive generation, copper interconnect reach became an issue. Copper’s losses meant that you needed something much shorter than long copper cables running through a spine. Enter the PCB.

The idea of the PCB was simple: hook up all the GPUs and network switches on a board level, instead of cables running through a spine. The compute trays would hook into one side and the network switches on the other. Then, by simply running traces on the PCB, you can get GPUs hooked up to each other. Simple and elegant right?

No. Turns out that this is a routing problem of gargantuan proportions (literally) because you can’t just draw metal lines between GPUs like in a picture. As a result the routing problem only increases. You would need to route over or under metal connections to prevent them from intersecting.

Think of a freeway intersection: to keep the flow of traffic moving smoothly, you have different levels of roads, each level adding a significant amount of complexity to the routing problem.

It’s the same problem on the PCB, except that you have about 78 layers. In addition, the Kyber rack has 144 GPUs compared to the Blackwell era “Oberon” rack which has 72 GPUs. The exact PCB layer count seems to differ based on who you ask, but the takeaway remains: it is a very complex PCB. The midplane PCB makes the equivalent of 20,000 connections that would otherwise need to be copper cables which would be too many to fit in the spine. These boards are built on separate 26 layer boards each nearly a square meter in size and then laminated together to form a massive PCB required for Kyber midplane.

Anybody who has worked with high speed interconnects knows that PCBs are not automatically better. Sure, you can pack a higher routing density in a smaller space cutting the reach distance, but that says nothing about the losses. In general cables have lower losses for high speed signals than metal lines on a PCB. To keep the midplane PCB losses in check, it is built on an advanced high-frequency material consisting of an M9-grade copper clad laminate (CCL) with Quartz glass (Q-glass) reinforcement. The use of these specific materials is so that the dissipation factor (~0.0007) is as low as possible. Dissipation factor is a measure of how much of the signal is lost in the PCB dielectric material. When you have so many layers, it is best to lose as little as possible.

The sheer size, number of layers, and specialized materials required for the midplane PCB make it an engineering marvel. When deploying this in production, the yield of the PCB and reliability in the field have to be exceptionally high. The obstacles are plenty:

• No de-lamination can occur at any layer across a panel near a square meter.

• Quartz glass is abrasive and hard to drill cleanly; connections between layers (through what are called vias) and drill bit wear become yield limiters.

• Since yield compounds per layer, it becomes extremely difficult at 78 layers. A defect anywhere scraps the whole board.

• A board-level failure affects the operation of 144 GPUs (in an NVL144 configuration) all at once.

These factors make the engineering marvel hard to build in volume. The reason we examined the underlying engineering is to ground rumors of schedule delays in engineering reality. Yes, building these boards is hard but not impossible. AI PCBs already span tens of layers easily, the midplane PCB takes it to another level.

Confirming or denying the delay outright requires tracking the supply chain and orders closely, which is out of scope here. Additionally, Nvidia has responded saying that their roadmap is intact – so we’ll take their word for it. But, if you are NVIDIA, and your best engineers say that the midplane PCB is hard, what’s the backup plan? The answer is the NVL72x2.

NVL72x2: Why It Might Have Really Been Cancelled

144 GPUs are difficult to fit into a rack for a couple of reasons:

1. They draw a lot more power per rack which would require the use of 800V DC systems to handle the power distribution and copper cable weight within the rack (see earlier detailed post on this below)

2. The routing problem as we discussed is complex, and requires an enormous PCB that is hard to make.

The logical stopgap solution would be to instead put in 72 GPUs per rack, just like in the Blackwell era, and put in two such parallel racks back-to-back. This solves the scale-up interconnect problem within a rack because you only need to connect 72 GPUs in the rack just like in the Blackwell “Oberon” rack. NVIDIA can simply use existing racks and not have to worry about it. The 144 GPU implementation would then involve putting two such racks together.

The problem now is how to connect the two racks together so that they behave as if they were one rack.

While the Kyber issues were being sorted out, SemiAnalysis stated that NVIDIA considered a back-to-back configuration as shown below. Perhaps this configuration was known to their institutional clients, but a detailed check of my SemiAnalysis newsletter archives shows that this was not mentioned anywhere before – including in their Vera Rubin deep-dive. Perhaps it was not in consideration at the time of writing.

Regardless, this configuration was allegedly disliked by the hyperscalers for its “odd” configuration. This argument I do buy because the data halls are not laid out to accommodate racks in any orientation. There are cable raceways, cooling, and power all laid out to accommodate racks in a particular orientation, and going back-to-back will require changing all the data hall infrastructure. If this was actually suggested by Nvidia as a configuration, no wonder hyperscalers and CSPs pushed back. SemiAnalysis stated in their tweet that this configuration was cancelled.

However that does not mean the NVL72x2 will never work, or that there aren’t other ways around current challenges. Beyond the paywall, we discuss:

• How to make the NVL72x2 work

• Why CPO/NPO is the right answer for scaling to larger world size

• Does this really create an opening for AMD and Google?

Read more

HBC is a platform technology, the foundation Qualcomm means to run through every business unit and not only the data center, and the whole thing rests on one move in packaging: it does not remove the hard part, it shifts it from the advanced substrate bracket (CoWoS) to a stacked-die approach that brings its own complexities.

Contents:

• HBC as the Unit of AI Intelligence

• Memory-on-Logic Packaging as a Key Risk Factor

• Logic Die Function in HBM versus HBC

• The Power Budget Under the Stack

• Who Builds the HBC Stack

• What to Watch For

Disclaimer: I am an ex-Qualcomm employee but hold no MNPI regarding their line of datacenter products. I worked as an RF engineer dealing with transistor devices – far from the sexy world of AI. The facts in this article are from public sources, and include my own inferences. I currently do not hold any QCOM stock. Not financial advice. Do your own research.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

You can find the whole catalog of articles for purchase at this link. This article will be available for paid digital download shortly after publication.

If you would like to engage boutique research services for your project, reach out to us.

Contact SemiExponent

HBC as the Unit of AI Intelligence

The figure shows a standard packaging substrate with an System-on-Chip (SoC) and an low-power DDR (LPDDR) memory stack sitting on top of an XPU. At first glance it looks like an XPU next to HBM, but it is not that. The DRAM is stacked on the XPU, the die that runs the matrix multiplications, and the SoC beside it is a separate chip whose identity changes with the end market.

That second chip is the part Qualcomm has always built. It has been a modem, an automotive ADAS processor, an IoT SOC, and even a robotics chip. The XPU with the DRAM on top is the new part that provides Qualcomm with an AI accelerator. The main idea behind HBC for Qualcomm is for it to serve as the unit of intelligence, and pairing it with one of Qualcomm’s business-unit SoCs is what makes that line AI-enabled. As such, its applicability and importance extends to its entire business.

Memory-on-Logic Packaging as a Key Risk Factor

Qualcomm’s claim is that HBC removes the need for advanced packaging, whose supply chains are hard to get hold of right now, and that HBC therefore runs on standard packaging substrates, which are cheaper, more available, and easier to integrate on. HBC does avoid the 2.5D interposer that an XPU-plus-HBM design needs, and the gains it buys are real.

Stacking the memory on the logic die swaps the HBM path, a high-speed interface running along the die edge and out through an interposer, for short vertical links through the die area. That lifts the connection count from about 2,048 (in HBM4) along the edge to the order of 10,000-100,000 through the area, and it brings down the energy needed to move bits by an order of magnitude.

Qualcomm however glossed over the fact that packaging difficulty has not gone away. It moves from the advanced substrate (CoWoS) to 3D die-to-die stacking, which is also advanced packaging, and on a compute die, it is the harder version of it. Qualcomm is not removing the packaging step but instead trading a supply-constrained one for a harder and less proven one.

Stacking DRAM on logic is not new in itself. HBM already does it via a core logic die sitting under a stack of DRAM connected through silicon vias. HBC looks similar, but the logic die does a different job, and that difference is the whole risk.

Read more

Here’s what you might have missed this week:

• What AI Inference Actually Demands from a NAND SSD

I was traveling in the UK last week, and Austin was in NYC for Qualcomm’s event. We got to recording the podcast later than usual this week, but our entire breakdown of the Qualcomm investor event should be out on the podcast soon.

Reminder that you can keep up with the daily news on the Semi Doped Substack, for free.

Now, on to the news.

This edition of TWiC is presented by … SambaNova

Run frontier models. Skip the GPU rack.

SambaCloud runs MiniMax M2.7, the fastest provider for this model, at 435 tokens per second via API. M2.7 ranks alongside Claude Opus 4.6 and GPT-5.4 on agentic and software engineering tasks, at a fraction of the cost. No hardware procurement, no lead time, no sharding across hundreds of GPUs.

Texas Advanced Computing Center, OVH Cloud, and Hume.ai are already on the platform. If you’re running long-horizon agent workflows in production, M2.7 is available today on SambaCloud’s Developer and Enterprise tiers. Benchmark your workload against your current provider in an afternoon.

👉🏽 Start benchmarking today.

(I tried it for free, and its always using something while knowing what the hardware behind it looks like).

OpenAI’s Jalapeno shows that ASIC design is commoditized

There has been a lot of generic coverage about the fact that OpenAI has their own chips, but I want to point out one thing: this chip was made with Broadcom doing the ASIC design from start to finish in 9 months. That means one of two things: (a) Broadcom’s silicon prowess is what Hock says it is — amazing, or (b) ASIC design has become a commodity business, with no moat after all.

I am willing to bet on the latter. The process of wafer fab and packaging usually takes about 3-4 months from start to finish. If you can one-shot a cutting edge inference chip in a mere 5-6 months, it shows that its probably easier to make than most people think.

I have personally never done ASIC design so take my words with a grain of salt, and feel free to correct me. But I do know that engineers working on that largely deal with software only. Coupled with the fact that AI-enabled design flows are now being pushed by every major EDA company, maybe chip design is getting commoditized after all — the press release does say “accelerated by OpenAI models.” But, not all kinds of chip design are easy: optics, power, analog, and anything that does not rely on digital transistors turning on and off, are still challenging to make and require specialized expertise. CPU design is still challenging perhaps — I don’t hear of anybody doing that in 9 months. Just some observations.

What does Modular bring to the table for Qualcomm?

Qualcomm recently announced that they will be acquiring Modular in their recent Investor day. But why is a company like Qualcomm buying a software platform like Modular?

Qualcomm is betting on disaggregated inference, which means that the future of AI involves a mix and match of hardware, each suited to the best function and workload. Imagine that you can get a DragonFly rack from Qualcomm, a low latency inference rack of LPUs from Nvidia, and a CPU rack from AMD. How do you ensure that they all work together?

This is what Modular brings to the table. One software platform for different kinds of hardware. To work with the Modular layer means that the software stack is written with Mojo. If NVIDIA CUDA is like Apple’s iOS, then Mojo is like Android. No need for specialized hardware tied to the software, but instead work with every hardware provider possible. These kernels written up with Mojo is then served with MAX on your own cloud platform, or you could use Modular’s.

But if Modular is owned by Qualcomm, how does a purchaser of a Qualcomm DragonFly rack deploy actual services? Do they pay Qualcomm a royalty fee for the software layer? Are they even willing to adopt the “Android of AI Inference” in order to deploy Qualcomm racks? A lot of these questions remain and remains unclear until the hardware actually lands.

The Gatekeeping of Frontier Intelligence Models

ChatGPT 5.6 “Sol” was released this week, and the full capabilities of the model are only available to a select few. The Trump Administration has “requested” a staggered roll out, ask that it approve its use on a case-by-case basis, as reported by The Information. This is a much more favorable deal for OpenAI than what Anthropic got even with its nerfed Fable model. Mythos was never meant for public consumption from the get go.

Gatekeeping frontier intelligence models creates an opposing tension; on one hand, it creates a layer of resistance to monetize the newly trained model in an AI token factory, but on the other, the industry does not have enough compute to serve frontier intelligence at scale. The path to usefulness of AGI is always limited by how many chips we can actually make. These opposing tensions place frontier models in a neutral equilibrium: a restricted user base on a finite supply of compute.

This disturbing trend of AI governance opens the door to many future questions:

• Would OpenAI or Anthropic dedicate enough resources to develop the next frontier model knowing a-priori that its full capabilities are going to be restrained?

• Who decides who gets to use cutting edge models, and does that give them an unfair advantage over competitors?

• If model releases depend on the benevolence of political leaders, what levels of quid-pro-quo can we expect to see between Silicon Valley and the White House?

Model scaling laws still hold: adding more compute to training a model creates better models, but human laws are a limiter. The imperative now shifts to model labs to instead continue to monetize via AI inference where the real economic value lies. The bodes well for the smorgasbord of interference accelerator companies who rely on interference to be the main driver of silicon demand. Today, NVIDIA is the dominant provider of training chips, and a slow down in frontier model progress due to governance will also slow down the consumption related training hardware.

However, at the recent Computex event, I had an interesting chat with some investors who were interested in knowing if SLC NAND was making a comeback because write endurance of TLC/QLC NAND was worse, even if it had more capacity. The answer to this is nuanced because it is no longer sufficient to think of NAND as a single tier, but instead how NAND is starting to split into multiple sub-tiers based on its place within the memory hierarchy, each with its mix-and-match of technologies.

The broader question that it encompasses is what the performance expectations of NAND in these newly forming performance tiers are, and more importantly what products from which companies are well positioned to capture the upside.

In this post, we will explore how NAND SSDs fit into context memory storage (or CMX, as NVIDIA calls it), and look at which SSD vendors are best positioned to serve this market.

Here is what we will cover:

• Why NAND SSDs are a viable option for AI inference

• Tuning SSD Performance for Memory or Storage

• CMX-Specific SSD Optimizations

• I/O size and the indirection unit (IU)

• Asymmetric read/write bandwidth

• Write Amplification and FDP

• NAND SSD Competitive Landscape

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you are not a paid subscriber, you can purchase just this article. This post will be updated with a direct purchase link shortly; check back on Substack.

You can find the whole catalog of articles for purchase at this link.

If you would like to engage boutique research services for your project, reach out to us.

Contact SemiExponent

Why NAND SSDs are a viable option for AI inference

Relying solely on DRAM (including HBM) to hold context is becoming untenable in the agentic era of inference due to cost and capacity. As users delegate increasingly complex tasks to AI, inference is no longer a process that takes minutes; it is extending to hours and days, and if we could manage it, to weeks and months. “Hardware-utilization-factor” will become an increasingly important metric as neoclouds and hyperscalers grapple with large capital expenditure costs while trying to translate investments into revenue. Underutilization due to memory capacity constraints is an unacceptable scenario.

NAND SSDs are a viable alternative because their performance can be finely tuned to act more like memory, or more like storage, depending on the underlying technology used. What was once a means to just faster storage and an alternative to HDDs, is now becoming a memory tier that lies somewhere between storage SSDs and DRAM.

The primary function of NAND in the agentic AI era is to hold KV-cache, which can often balloon to tens of TBs over the course of a large agentic task, eliminating the possibility of storing it on any DRAM tier. During the decode step of inference, the KV-cache needs to be accessed as fast as possible to provide a high token throughput. Speed is especially important because the decode step generates one token at time due to the autoregressive nature of the transformer architecture. NAND SSDs, with the right modifications, are capable of serving as a KV-cache store while streaming data fast enough for decode to still work well.

DeepSeek v4, for example, has already made NAND a first class citizen by ensuring that context is offloaded to SSDs and cache-hits are maximized for the lowest possible token cost. It is highly likely that similar architectures will follow in other models.

Aside: CXL pooled DRAMs are equally viable, and some would argue a better alternative, but that’s a different post. In reality, it occupies its own tier in the memory hierarchy.

Tuning SSD Performance for Memory or Storage

The feasibility of NAND for AI inference stems from the design choices of the underlying flash cells. While my original NAND article covers the in-depth workings of the various types of NAND, we’ll address it briefly here, and focus more on how to slot them into different NAND performance tiers.

Flash memory essentially operates by storing or releasing charge, which represents the bit representation of 0 or a 1. An N-level NAND cell has 2^N states in which a memory cell can exist in. For a single level cell (SLC, N=1), there are just two states. For a quad-level cell (QLC, N=4), there are sixteen states. The capacity increases when more states are stored in the same cell because the cell physically does not take up any more space where it stores a 0/1 in SLC or 0000—1111 in QLC. In other words, you can store more bits in the same physical area.

Capacity increase comes with other trade-offs, namely endurance and speed – and both of which are tied to the number of cell states. It comes down to distinguishing between either just two cell states or sixteen, and how that can be done quickly and reliably over the life of the NAND SSD under all operating conditions it is subject to.

• Endurance: Over time, the voltage levels that distinguish between the cell states start to move due to constant cell writes and temperature effects over time. It is just part of the wear and tear induced in the oxide layers within the cell when it is subject to constant use over its lifetime. For a single voltage level that distinguishes between cell states (like in SLC), the shift can be higher and we can still tell if the cell holds a zero or one. For a QLC with 16 voltage states, it becomes much harder, and guessing wrong results in a bit error. Once bit errors increase beyond a certain threshold, the NAND SSD is deemed worn out.

• Write Speed: Writing to a cell involves finely adjusting the voltage applied to the cell, and closely monitoring the resulting current from the underlying cell. To determine what the state of the cell is, a process called incremental step pulse programming (ISPP) is used to recursively adjust the voltage to just the right level (more in the original NAND article). This iterative process of adjustment affects the write speed because it takes many recursive steps to set the right voltage. With fewer cell states, the iterative process is quicker which means SLCs are faster than QLCs.

By now it should start becoming clear how NAND SSDs can be fine-tuned for capacity and performance, at least to the first order. Use more cell states (higher level NAND) to get more capacity, but fewer cell states (lower level NAND) to get more speed/endurance. A hybrid approach is also possible: use a SLC-cache with a TLC NAND, for example, to get the best of both worlds with the right speed/endurance/capacity tradeoffs.

Next, we will discuss how NAND SSDs are better optimized for AI use-cases, and conclude with a competitive survey of the NAND landscape for agentic AI.

Beyond the paywall:

• CMX-Specific SSD Optimizations

• I/O size and the indirection unit (IU)

• Asymmetric read/write bandwidth

• Write Amplification and FDP

• NAND SSD Competitive Landscape

Read more

Other stuff in our universe has been centered around advanced packaging, in case you missed it.

As part of Semi Doped we also publish a daily newsletter with the hottest takes, straight from the oven — all for free. Sign up at semidoped.com.

Now, on to the this week’s explorations on TWiC.

How MEXT Makes Flash Work Like DRAM

This week AMD announced that they are acquiring mext.ai. The chart below is unrelated to the announcement, but also highly relevant. I’ll explain why.

This memory share estimate of CapEx is 36% two months ago, for 2027. Memory prices keep increasing, so let’s just round off to 40% for simplicity and assuming a $1T AI CapEx spend in 2027 because it makes math easy. That’s $400B dollars spent in just paying for memory. We had a whole episode on Semi Doped explaining why this crazy pricing is not useful to anybody. That much money can be spent on something much more useful — like trying to improve communication bottlenecks, or building better power delivery systems.

Any CFO worth their salt will try to reduce this spend by asking “Do you really use all that memory we pay for?” MEXT’s solution what is meant to answer this question.

MEXT is a software company that helps you deal with storage more effectively, whether it is DRAM or NAND. Their key insight is this: not everything is DRAM is immediately useful — some stuff is hot, some stuff goes cold. Then the sensible thing to do is to boot the cold data from DRAM to NAND flash which has much more storage capacity. Next, they predict using “AI” (it’s like AI ouroboros here) what needs to be in DRAM, and do the reverse as well — load data from NAND to DRAM before the compute asks for it. It’s the memory management method that sweeps data under the rug, and pulls out only what is needed at the right time.

NAND is much slower than DRAM, but when used together in a form of deliberate data dance, it becomes very effective. These kinds of storage optimizations are essentially bullish for NAND. This bottom line is: (a) Hardware builders cannot go on indefinitely adding DRAM without asking if we are using it effectively, (b) Memory companies cannot raise prices indefinitely without the industry trying to find ways around it, and (c) We will see the emergence of a lot more of software level optimizations like this in the future that will help with memory management in a clever manner.

PicoJool and VCSELs for AI Scale Up

This week a startup called PicoJool announced that they are introducing 200G VCSEL lasers for AI scale up. VCSELs have been famously difficult to make work beyond 100G due to modulation bandwidth limitations in the electrical-optical conversion. In simple terms, a high bandwidth means that the optical signal changes fast enough to track the changes in electrical signals.

The chart above is from a 2022 paper showing the modulation bandwidths of 850nm VCSELs. As you can see, VCSELs have been continuously getting faster, and capable of supporting higher modulation bandwidths. PicoJool’s stated bandwidth is better than anything on this chart — at nearly ~38 GHz. Their measurement is shown below.

The way to read this is to think of a continuously varying RF signal being input to the electrical-control of the VCSEL. At say 10 GHz, there is little loss from the conversion to optical — ie, the S21 (transmission scattering parameter) is 0 dB. That means just rapidly changing the electrical input doesn’t make the VCSEL light any dimmer. At ~38 GHz, the signal is so fast that only half the optical power is produced at the output. Beyond this speed, most of the energy is lost in the conversion because the VCSEL can’t keep up. At this point the link budget gets worse, and errors rates go up for a given reach.

The big thing about using VCSELs is that they are made from the widely available Gallium-Arsenide (GaAs), as opposed to the much sought after InP, and GaAs for scale up has a big impact on TAM of InP lasers should this technology actually work out (yet to be proven). However, note that not all VCSELs are Indium free. The quantum well could be engineered with GaAs/InGaAs too. It really depends on the laser implementation — so be sure to ask for device details before jumping into supply chain calculations.

One objection that people have around VCSELs is that they are not reliable. The degradation often comes from the oxidation of aluminum in the quantum junction (it is actually not purely GaAs — it is a mix of GaAs and AlGaAs). However, if you switch to 980nm/1060nm VCSELs (like Lumentum is doing), then the bandgap for that laser wavelength does not require aluminum — which inherently eliminates one major source of failure. The elimination of aluminum also helps with defect propagation, but we won’t get into that here.

There are a lot of different aspects to whether VCSELs will ultimately become feasible. It is interesting to follow the developments here.

Intel had a Very Good Thursday

Stock is up 10% into new all time highs. It’s mainly because of the news below, and also the fact that they hired Seok-Hee Lee, an SK Hynix veteran, who was ex-Intel incidentally, to lead advanced packaging and back-end engineering.

This is a major turnaround since Intel famously refused to manufacture iPhones for Apple because they didn’t think the market would be big enough. This primarily led Intel to miss the wireless boom, and handed over the keys to the mobile chips kingdom to TSMC.

Apple and TSMC have since built a massive business due to a long term partnership, where the business from just Apple alone was so much that it has made TSMC into what they are today. Apple going to Intel again for chips is opportunity knocking twice, at a time where Intel Foundry needs volume to pick up its foundry business whose near-death experience had many worried. TSMC’s capacity issues are now blowing fair winds Intel’s way again, and it’s time for Intel to orient their sails just right this time.

According to a recent TrendForce estimate, the industry’s CoWoS capacity could reach 200k wafers-per-month (wpm), with TSMC running ~120k wpm by the end of 2026, while ASE and Amkor get a seat at the table only as overflow, soaking up something like ~80k wpm. TSMC has been rapidly building out capacity to support CoWoS demand. Most of the big GPU chips are packaged by TSMC, while simpler flows like Vera CPUs and automotive parts overflow to Amkor/ASE facilities.

Intel’s EMIB is the first credible second source to show up with real orders behind it. AWS and Cisco are already shipping external silicon on EMIB. SpaceX and Tesla signed on around the Q1 2026 earnings call as partners in Intel’s Terafab project. Google’s TPU v8e is reported to land on EMIB in the second half of 2027, with Apple, Microsoft, and NVIDIA sitting in the talks pile. CFO David Zinsner moved Intel’s external advanced-packaging outlook from a few hundred million to north of a billion dollars across the last two calls, and told CNBC he expects billions of dollars per customer. For a business that small, packaging alone going past a billion is a step change.

However, as orders flow to Intel, TSMC is continuously expanding CoWoS capacity. The increasing sizes of advanced packages for AI accelerators puts the two competitors at a crossroads again -- and this time the outcome will be decided based on what happens in the next few years as the industry moves to adopt glass core substrates.

Contents:

1. EMIB Comes for the CoWoS Monopoly — the three-year packaging monopoly, and the orders that are breaking it.

2. Deeply Understanding CoWoS-L vs EMIB — the three CoWoS flavors, and how Intel’s bridge differs from TSMC’s.

3. Where the Size Limitations Lie — the round-wafer-versus-panel math that caps CoWoS-L and frees EMIB.

4. The Convergence on Glass Core Panels — why warpage and CTE force both foundries onto glass.

5. Where This Leaves Investors — the winners and losers, and the OSATs that win either way.

6. The Taiwanese Glass Partner Already Inside TSMC’s CoPoS — the name the glass maps miss.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you are not a paid subscriber, you can purchase just this article using the button below. You can find the whole catalog of articles for purchase at this link.

Buy the ebook

If you would like to engage boutique research services for your project, reach out to us.

Contact SemiExponent

Deeply Understanding CoWoS-L vs EMIB

Many comparisons online of CoWoS to EMIB are not precise about the exact flavors of each technology, and opens doors to misinterpretation. We’ll clarify that aspect first.

• CoWoS-S is the original, a single monolithic silicon interposer. It gets really large and expensive beyond a certain size. TSMC’s limit is about 3.3x reticle size (or 2,380 mm²).

• CoWoS-R swaps that silicon interposer for a cheaper organic RDL one, which is what AWS uses for Trainium 2/3 chips. They do not have the interconnect density of silicon interposers, and are rather limited in size too.

• The variant carrying the biggest AI packages today, the Blackwells and the Rubins, is CoWoS-L, which scales to 5.5x reticle size (or 4,720 mm²) today, with plans to expand to 9.5x reticle size (or 8,150 mm²) in 2027. TSMC’s roadmap extends to 14x reticle size (or 12,000 mm²) by 2029.

Both TSMC’s CoWoS-L and Intel’s EMIB are built on the same idea of giving up a full silicon interposer for silicon bridges only where you need fine pitch wiring, such as die-to-die and die-to-HBM, and use cheaper routing for everything else.

• TSMC calls its bridge an LSI (local silicon interconnect, and hence the ‘L’) and embeds it in an Re-Distribution Layer (RDL) interposer.

• Intel calls its bridge EMIB (embedded multi-die interconnect bridge) and embeds it straight into the organic package substrate.

Note the difference in where the bridges live. CoWoS-L’s RDL interposer is built on a wafer, so it stays bound to the 300mm wafer and the reticle limit (the whole reason CoPoS and glass panels are coming). EMIB embeds its bridges in the organic substrate with no wafer carrier, so it clears the wafer-size wall, and scales to bigger interposers due to the use of rectangular panels.

Here is Intel’s EMIB roadmap in terms of size.

Today, Intel’s EMIB holds a size advantage by being able to deliver on 8x reticle size (or 6,860 mm²), which is ahead of TSMC’s 5.5x capability, due to the choice of rectangular formats for organic substrates rather than RDL interposers on round wafers. Intel promises 12x by 2028 as opposed to TSMC’s 14x, but we’ll have to wait and see who actually delivers, and at what reticle-size multiple.

Meanwhile, we can try and guess.

Where the Size Limitations Lie

CoWoS-L runs into a size ceiling that comes straight from the round wafer it is built on. The LSI bridges are embedded into RDL, but the RDL itself is fabricated on a 300mm silicon wafer and then mounted on an organic package substrate. While the bridges themselves are small, the RDL interposer carrying them is one big rectangle, and a round wafer yields only a few per wafer.

Let’s actually calculate who is capable of delivering a 14x reticle by 2028. On the left is a 300mm round wafer with a total area of 70,600 mm^2, representative of TSMC’s process. On the right is a 310mm x 310mm organic panel, with a total area of 96,100 mm^2, representative of Intel’s EMIB.

The rough excalidraw picture above shows that a round wafer fits just about three 14x-sized packages, while the rectangular panel provides six. The area utilization goes from ~50% to ~75%. A transition to bigger 510mm x 515mm panels in the future will yield ~20 interposers.

It is clear that EMIB’s use of rectangular panels is vastly superior when it comes to large package sizes. It is the same round-versus-rectangular problem pushing TSMC toward CoPoS, except EMIB already uses the rectangular format. Intel is starting where TSMC is trying to get to. What keeps TSMC ahead in practice is volume, since it ships these packages today. Intel’s advanced packaging story is just getting started, but it holds a structural advantage in large packages in the future.

The Convergence on Glass Core Panels

Unfortunately for Intel, at 14x reticle sizes, it is not just the advantage of rectangular vs round formats that matter. For such large packages, organic substrates inherently become poor candidates due to their tendency to warp, and the co-efficient of thermal expansion (CTE) mismatch with silicon. Regardless of the format, the physical properties of the material starts causing yield issues which strips away some of Intel’s lead in large size packages, and instead levels the playing field with TSMC.

The answer for both foundries is the same: large rectangular panels on glass core substrates. It has unique properties that make well suited as a core material:

• It has a high degree of flatness

• It has a CTE that is closer to silicon than organic materials

• It has low dielectric loss (good for interconnect)

• It has a high Young’s Modulus, which means that it is very stiff and needs a high degree of force to stretch it even a little.

• In glass, through glass vias (TGVs) can be made much closer, which results in better power delivery due to lower parasitics.

• Light propagates through glass, which opens the doors to optical integration in the future.

Intel is the first mover on glass substrates, demonstrating a 78mm x 77mm, which is equivalent to 7x reticle size. This houses 2 full reticle-sized GPUs, for example, with room for interconnections with EMIB.

More importantly, Intel has solved the issue of “SeWaRe” - a Japanese term that refers to micro-cracking in glass substrates. It usually happens right through the lateral cross-section of glass -- kind of like when you pull apart the biscuit halves in an Oreo. Micro-cracking while drilling TGVs is an important problem that requires special techniques to deal with. One company holding a dominant position here is LPKF Laser and Electronics - a German company - whose Laser Induced Deep Etch (LIDE) process is capable of forming TGVs at scale. We’ll introduce another contender after the paywall.

The race to glass panels is just not an Intel vs TSMC race. Samsung and Rapidus are expecting to roll out glass interposer solutions as well.

For paid subscribers:

• What this means to investors, and how the advanced packaging scene will play out

• Two other supply chain names outside LPKF that are worth a look

Read more

It is arguably a provocative title, but going purely off the heading is akin to judging the book by its cover. The institutional note “automagically” landed in my inbox and I had a long, quiet read to process it. I also read all the notes by FundaAI around this, a few other Substack notes, and tried to keep up with the flurry of takes on X.

The now infamous note is largely post-Computex, where we identified optics and power as the major themes as well. Here is our coverage this week:

• 📚 Computex 2026 Debrief

• 🎙️ Computex 2026: Decoding Optics and Power Content

The long and short of it that CPO is coming eventually, as is 800V; nobody denies this, but the controversy is around the when. We’ll focus on CPO today.

The SemiAnalysis claim worth focusing on is that Spectrum-6 CPO is slipping by 2+ quarters due to: (1) an unusual level (>3.5 dB) of optical loss in Spectrum-6 that eats link margin and notably higher than Spectrum-5 that uses a similar port count forcing a redesign, and (2) a yield constraint that causes issues at scale.

We have no way of verifying actual numbers here, unless the reader is the rare case of the one actually working on it. However, technical discussions around these two claims are light, and rebuttals mostly come from the supply chain argument side. There is little dialogue around understanding why this could happen, and if the engineering side holds up.

We will try to fill in this gap today.

This edition of TWiC is presented by … SambaNova

Run frontier models. Skip the GPU rack.

SambaCloud runs MiniMax M2.7, the fastest provider for this model, at 435 tokens per second via API. M2.7 ranks alongside Claude Opus 4.6 and GPT-5.4 on agentic and software engineering tasks, at a fraction of the cost. No hardware procurement, no lead time, no sharding across hundreds of GPUs.

Texas Advanced Computing Center, OVH Cloud, and Hume.ai are already on the platform. If you’re running long-horizon agent workflows in production, M2.7 is available today on SambaCloud’s Developer and Enterprise tiers. Benchmark your workload against your current provider in an afternoon.

👉🏽 Start benchmarking today.

The industry agrees as a whole that scale-out is where CPO lands first. NVIDIA Spectrum-X CPO (Ethernet) and Quantum-X CPO (Infiniband) are slated to enter mass production in 2026. The SemiAnalysis claim is that the Ethernet version is due to slip by 2+ quarters, and is materially important because the Ethernet version ships more volume than the Infiniband one.

Both the increased optical loss and yield issues have their roots in the sub-micron accuracy alignment requirements of optical engines for CPO, and requires a procedure called “active alignment.” It involves optimizing the fiber attach unit (FAU) to the photonic chip across 6 degrees of freedom (position+angle across x,y,z). The alignment is done by actively monitoring the amount of optical power being coupled, and a few degrees of freedom are adjusted till maximum power is coupled.

This has to be repeated for each optical engine in the whole Spectrum-X switch, and is required at several points along the CPO build process. Even a single CPO module can require 100+ active alignment operations from start to finish, each one taking only a few seconds from start to finish in high volume production.

CPO testing is an interesting aspect that deserves its own deep-dive, but key takeaway is that co-packaging an optical engine next to the silicon is notoriously difficult, and the difficulty arises when all of them need to work when attached. Quantum X actually implements CPO differently from Spectrum X as shown in the pictures below.

Quantum-X is designed for routing supercomputing traffic with ultra-low latency. It has a total of 18 optical engines (6x optical subassembly with 3 engines each) to provide 18 ports in an NVswitch supporting Infiniband. Quantum X uses a replaceable module of 3 optical engines that can be swapped out as a whole if any one of the three engines in the module do not work. The complexity of active alignment is limited to just the optical subassembly now. If the yield of each attachment is 95%, the overall yield of the subassembly is 0.95³ = 0.853, or 85.3%.

Spectrum-X handles larger, multi-tenant cloud and Ethernet AI environments and thus requires more port counts, and more optical engines per switch. As a result, the “swappable module” approach does not work for this product line due to sheer lack of space. You can see from the picture that there are 32 optical engines directly co-packaged next to the silicon. A single non-functioning optical engine poses a danger to the entire switch.

Here the math is that if each one has a 95% chance of success, then the chance of all 32 engines being successful is (0.95³²=0.193, or ~19%). This makes it significantly harder to deploy Spectrum-X switches at scale if the yield is so low. If you want ~85% yield like we saw in the 3-engine optical subassembly, the required attachment yield rate skyrockets to 99.5%. There are some counter-arguments floating by to this method.

1. Is the yield really 95%? This number is genuinely difficult to come by, and if based purely on hearsay, is something to take with a grain of salt. The exact yield number matters a lot, arguably to the last decimal point, when you take a power of 32. Hand wavy napkin math does not work here.

2. Failing parts are binned. One post I read argues that the napkin math is bad because optical engines are binned and only functional ones will be attached and therefore the failure rate cannot possibly be that high. Yes, failing parts are binned, but the actual failures come during/after the attach process. As a result, there is no way to avoid the power of 32.

Most of the commentary on this matter has been a rebuttal on the basis that the bottleneck is on the supply-side, and not the demand side. Lumentum’s comments at the Mizuho conference also stated how much demand is waiting for them, if they could only ship. Morgan Stanley released a note with wafer shipments, while acknowledging TSMC yield numbers in SoIC and downstream assembly yield.

Subscribe now

The loss and yield argument makes “qualitative” engineering sense given how Spectrum-X and Quantum-X CPO architectures work, and the crux of the SemiAnalysis argument reduces to finding the right yield number. If there is a fundamental problem with the active alignment process, it is easy to drop half the optical power (which is what 3 dB of loss is) due to misalignment issues; remember that the alignment accuracy requirements are sub-micron for thousands of optical fibers per switch. The loss number itself is within the realm of reason given how sensitive CPO is to alignment issues.

Given how new the industry is to CPO at scale, loss low yield could very well be growing pains as the technology ramps. It is not an unsolvable problem; just one that takes its time as assembly processes for CPO mature. It is entirely reasonable that the alternative while the industry grows to adopt CPO, is NPO.

In the case of NPO, the assembly and packaging process is much simpler because only the optical engine in that NPO module needs active alignment — it is not an exponent-stacking problem like in CPO. This makes it much more conducive to the short/medium term while the CPO production process matures.

What the Street missed in the whole controversy is that whether it is NPO or CPO, the optical content remains unchanged. It is only a question of how the packaging works and the demand for optics is higher than ever, just in different form factors.

But first, Computex is a very large show, and impossible for one person to cover all what it has to offer in any reasonable depth. What I’ll discuss here is by no means comprehensive but I’ll provide a stream-of-consciousness download of the show. I hope you get something useful out of it.

I’ll discuss three major areas:

• Memory

  • The trade

  • HBM4 base die and cooling

  • What would actually break memory

• Optics

  • Why CPO is still early

  • The demos

  • NPO and pluggables

• Power

  • 800V and the conversion chain

  • Delta’s system play

  • When it’s real

  • Moving BBUs out of the rack

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you would like to engage boutique research services for your project, reach out to us.

Contact SemiExponent

Read more

What’s left of my grey cells have been deployed into thinking about interconnects. I have a deep dive piece on how power density is driving up active copper cable use; link below in case you missed it. I believe we’re still in the early phase on this one.

Today’s short post is about a strange contradiction in the argument about what the right laser source for CPO is: a single high power laser, or a wavelength multiplexer, multi-color laser. I believe either one is viable, and we will explore why.

A quick note: I will at Computex 2026 in Taipei next week, and posting might be light while I attend the conference. If you are attending and happen to see me, say hi! 👋🏽

There has been a lot of recent news about Sivers Semiconductor, whose stock has shot up about 2,000% based on their optics business lines for CPO. First thing to note is that their optics business is a minority; their main business is RF, wireless, and phased array antennas for Satcom. I’m surprised more people aren’t talking about them as a potential acquisition target by SpaceX post-IPO – because that’s where their real strengths lie. I’m still not convinced that there’s going to be a Satcom boom post-IPO, so let’s move on.

The other argument I’ve seen is that Sivers’ approach to CPO is inherently flawed, because the use of wavelength division multiplexing (WDM) is worse than using a single powerful external laser.

To explain this further, you should know that there are two schools of thought:

1. Ultra High Power Laser: The idea here is to just create a really powerful laser and keep it external to the rack. Once it reaches the optical engine, you split it up into parallel lanes and modulate each lane with data on a SiPho chip. Coherent showed a 400mW UHP laser, and Lumentum showed an 800mW-1W “Super High Power” (SHP) laser.

2. CW-WDM Laser: The other approach is what folks like Sivers and Ayar Labs are adopting. The light source itself is multi-color from the start with each color having about 50-65mW. Across 8 lanes, that is still 400mW total but you don’t have to split it up at the optical engine. Since you have different light colors, you can modulate each one with data.

But is the WDM approach to CPO really worse than UHP lasers? It is also important to note that Lumentum, often touted for their SHP lasers, also have a CW-WDM laser which is really very good: 8 wavelengths, each up to 100 mW. There is a whole multi-source agreement (MSA) around CW-WDM too.

If CW-WDM is so flawed from the start, why does Lumentum have a product here, and why are they part of the MSA? Note that Coherent is on here too. See the contradiction here?

The real answer is that UHP lasers vs WDM lasers are just two different ways of solving the same problem, and Lumentum is hedging its bets on which architecture will win by having both. Here’s why:

1. The SHP laser approach needs 800mW because it is going to be split up into multiple lanes on the other side. For this, there should be substantial power available for each lane after the split. Let’s take a more “ordinary” UHP laser like the one from Coherent for comparison running at 400mW, single color. Assuming no loss through the channel for simplicity, 400mW gives 50mW per channel across 8 lanes.

2. If you see Sivers’ WDM lasers across 8 lanes, each one has 50mW per channel. No difference at the modulator level compared to the single color UHP laser approach.

You can roughly see how they are equivalent. If you really want to take loss into account, remember that a single UHP laser needs a 1:8 which a WDM approach does not. This adds to the loss budget and requires a higher power laser to start with. WDM instead has to deal with multiplexer/demultiplexer loss. Problems on both sides.

Using a WDM approach means that you can transmit more data on a single fiber, whereas UHP lasers after splitting need different fibers. WDM has bandwidth density that split UHP lasers do not.

What matters is how much laser power is available at the modulator ultimately, and not how much power is available at the external laser source. Running 800mW in a single laser also presents cooling problems, while running 8 lanes at lower power makes thermals easier to manage on a per laser basis.

Higher laser power isn’t a virtue in itself as a lot of commentary on optics makes it out to be. Higher power is good for SNR and BER. But more power also means more heat, which is one thing that lasers explicitly dislike because their lifetimes drop like a rock at higher temperatures. Even if the laser is external to the rack, reliability is one thing that CPO adopters still worry about the most (data from conferences shows that lasers do okay, but no info is available at scale yet).

The real danger to Sivers is not the use of their WDM approach, but instead Lumentum beating them at their own game with higher quality WDM lasers.

Comparison table with reported numbers

Have a great weekend!

This can be done across many axes, with: hybrid bonding for advanced packaging, Unified Bus for memory protocols, near-packaged optics for interconnects and software optimizations overall. We will just focus on the transistor layer here, and address the question: What if the route to leading-edge density did not require a leading-edge transistor at all, but instead came from folding a 7nm-class design back on top of itself until the footprint density rivals a node three years ahead? This is what Huawei calls LogicFolding.

In this post, we will discuss:

• What Tau Scaling and LogicFolding actually are: chasing time as an axis instead of scaling dimension.

• Why hybrid bonding is the hard part: the challenges that decide whether fine-pitch logic stacking actually yields.

• Who builds the bonders: the short, concentrated supply chain and why its geography matters for China.

• How far along China’s own bonder makers are: Where is the current capability, and the gap that still leaves.

• What it means for the US lead: why the same stacking on an EUV-class node widens the gap instead of closing it.

If you need a refresher on hybrid bonding, see earlier post below.

We also have a deep dive on lithography on the Semi Doped podcast. See it on YouTube.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you are not a paid subscriber, you can purchase just this article using the button below. You can find the whole catalog of articles for purchase at this link.

Buy the ebook

If you would like to engage boutique research services for your project, reach out to us.

Contact SemiExponent

Tau Scaling and LogicFolding

Huawei’s Tau Scaling takes its name from the Greek Letter τ, which represents the delay through a signal path. Historically, making transistors smaller and faster has reduced this delay. But past the 7nm node, making the delay smaller has become increasingly difficult without the use of EUV lithography, which China does not have access to due to export control laws.

Huawei’s solution is practically sound: if you can’t improve the signal delay through the transistor without EUV, why don’t we optimize everything else we can? Their goal is to shrink that delay across four levels at once: the device, the circuit, the chip and the full system.

At the chip level, the way to do this is to take a chip built on a trailing DUV node, like SMIC’s 7nm-class process (incidentally, Huawei has never named the foundry, but everyone assumes it is SMIC). If you can stack logic chips built on this one on top of the other and hook them up, you’ve just doubled transistor density without EUV. Think of it like printing two chips on a napkin, and folding them in half so that they align, and then hooking them up. This is what Huawei calls LogicFolding, and hooking them up requires a packaging technology called hybrid bonding.

By folding chips like a taco, you’ve just reduced the interconnect distance massively, and the unwanted resistance and capacitance that come from longer wiring is dramatically reduced. This makes the silicon run faster with lower delay, and Huawei has effectively made chips faster and denser by stacking them on top of each other.

Huawei reports the Kirin 2026 part, its first LogicFolding product shipping in fall 2026, moving from 155 to 238 MTr/mm² of transistor density (roughly +53.5%), alongside a 41% gain in performance-core power efficiency and a clock bump to 3.1 GHz. The caveat here that many experts are pointing out is that Huawei spent twice the amount of silicon to double the transistor density, which is not the same as using EUV to make denser transistors.

Read more

The rise of power density within a single rack as hyperscalers pack in more GPUs is causing the networking domains to blur. As GPUs per scale-up domain climb, the domain outgrows a single rack and operators split it across racks to keep power density in check. Meta’s Catalina is a prime example where the scale-up domain spreads across racks at lower power density, linked with active copper.

A dense rack like NVIDIA Rubin pulls so much power that many brownfield datacenters (a term used to refer to existing datacenters repurposed for the AI age) cannot host one because the installed racks cannot support the power density. Short of swapping in NVIDIA reference racks as-is, the only way to deploy power-dense hardware is to spread the same GPUs across multiple lower-density racks.

Once power pushes scale-up across racks, passive copper does not have enough reach. But the gap is not wide enough for fully retimed copper with DSPs, which are too power hungry and too high latency, and optics for a few meters is overkill at this networking generation. Active copper fills the gap as a “goldilocks” choice to keep power budgets in check and opens up a new market for AI interconnects.

In this post, we will look at:

• A tour of the interconnect technologies inside an AI datacenter, and where passive copper, active copper, DSP cables, and optics each fit.

• Why rising rack power is forcing scale-up networks to split across multiple cabinets, with Meta’s Catalina as the worked example.

• Which interconnect actually wins the short links between those split racks, and why it turns out to be active copper.

• For paid subscribers, a full competitive teardown of the five vendors fighting for ACC: Semtech, Marvell, MACOM, Ciena/Nubis, and the conspicuous one sitting it out.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you are not a paid subscriber, you can purchase just this article using the button. You can find the whole catalog of articles for purchase at this link.

Buy the ebook

If you would like to engage boutique research services for your project, click below.

Visit SemiExponent

How AI Racks Move Data Today

All scale-up networking inside NVIDIA racks has been passive copper. The NVLink spine holds roughly 5,000 copper cables. Passive copper, or direct attach copper (DAC), works because its reach inside a single rack covers the generation it sits on. The speeds across three NVIDIA generations:

Hopper, at 100G per lane, gave DAC about 3m of reach which is plenty. Blackwell’s NVLink 5 went to 200G per lane and passive reach fell to about 1.5-2m, but NVIDIA kept the scale-up domain inside the spine by packing all 72 GPUs and the NVSwitch trays into one rack with a copper cable spine, so no link runs far.

Rubin’s NVLink 6 was labeled 400G per lane at GTC 2026, though the industry read is 200G bi-directional. Running a transmit+receive pair at once leans hard on echo cancellation and tightens cable performance demands, but DAC for scale-up is alive and well in Rubin. The added BiDi complexity is handled by the SerDes on the switch silicon, not with external redrivers. Rubin Ultra moves to a mid-plane PCB approach, which is still passive copper.

Scale-out is different: connecting a row of racks needs real reach. It has leaned on active electrical cables (AECs), which drove Credo’s purple cable mania a few years back. Marvell and Astera Labs play on the silicon side; Credo is vertically integrated. As speeds climb, scale-out is migrating to optics, pluggable transceivers first, then CPO for lower energy. For linking datacenters across a campus or beyond, optics is already the mainstay.

Today, a new space for interconnects is emerging as the scale-up domain spans multiple racks; one whose reach is not high enough for pure optics, but also latency and power sensitive for pluggable optics and fully-retimed copper solutions using DSPs. In the 1.6T generation and possibly beyond, this is the space that Active Copper Cable (ACC) is looking to fill.

Why Meta’s Catalina Spans Two Racks

The cleanest scale-up domain is the one that never leaves the rack because passive copper cables without active silicon is cheap, low-latency, and low-power. This is the sole reason why hyperscalers would all keep scale-up inside one rack if they could.

The fundamental problem is that a frontier scale-up rack like NVIDIA’s Blackwell NVL72 uses more power that traditional racks. Once you cross roughly 100 kW in a single rack, you run out of cooling headroom, busbar capacity, or both, and the only move left is to spread out the GPUs across two racks, and run the scale-up fabric between them.

A single GB200 NVL72 draws around 120-130kW in one rack. If most racks in an existing datacenter top out around 60-70 kW per rack, then NVIDIA offers the same 72-GPU domain repackaged as NVL36x2: two cabinets of 36 GPUs at roughly 67kW each. But what if NVIDIA’s customers have their own racks already?

Meta’s Open Rack v3 High Power Rack (HPR) supports 93.5 kW per-rack, and is not suited to directly deploying a single NVL72 GB200 domain within it. At this point, Meta has to either replace all the racks in its existing datacenter with NVIDIA’s reference architecture, or expand the 72-GPU domain across two racks. This latter is what Meta did with its Catalina racks.

Catalina stretches the 72-GPU NVLink domain across two ORv3 HPR cabinets, roughly 36 GPUs and ~67kW each, so both racks sit comfortably inside the 93.5kW power envelope Meta already knows how to cool and power. The Blackwell silicon is NVIDIA’s, but the rack around it is Meta’s implementation so that the cabinet drops into Meta’s existing data halls instead of forcing a new facility build.

The tradeoff lands on the scale-up traffic which has to jump across the split rack domains. In the picture below, you can see a thick bundle of black cables running between the two racks, tying the GPUs on both sides together through the NVLink switches into a single 72-GPU domain. The backend network within each rack is in-rack scale-up. Holding 72 GPUs non-blocking across that split takes 162 cross-rack links (18 ports/NVswitch × 9 NVswitches/rack). The saving grace is reach because the interconnect just has to go to the neighboring rack scale-up, and not run across an entire row like in scale-out networks.

Which Interconnect Wins Multi-Rack Scale-Up

The cable only has to reach the rack next door, which works out to about 3 meters once you allow for slack. That is a short, fixed reach, and at 200G/lane it is enough to rule most of the options in or out.

Passive copper is the first to go. At 200G/lane a plain DAC runs out of reach well short of 3 meters, so it cannot bridge the gap on its own. At the other extreme, a DSP-based AEC reaches far past 3 meters, but it burns around 20 watts per end and carries the latency of a full retimer, which is overkill when all you have to do is cross to the neighboring rack. Pluggable optics runs into the same wall from the other side. It clears the reach and data rate easily, but tens of watts of transceiver power per end is wasteful on a link that only has to span a few meters.

Co-packaged optics is the interesting one. It is the lowest-power and most future-proof option, and NVIDIA is pushing it for multi-rack scale-up in Rubin Ultra. But it is still unproven at scale and ties you to a handful of vendor ecosystems, so it reads as a next-generation answer rather than something to deploy at 1.6T today. CPO is not the only future bet either, as several players are investigating microLEDs and VCSELs for the same short-reach role, though none of these are in large-scale deployment yet.

That leaves the option built for exactly this gap. Active copper cables add a linear redriver, no DSP, which pushes copper out to 3 meters at 200G/lane for just a couple of watts per end. Most of the reach of an AEC, a fraction of the power, and none of the DSP latency.

After the paywall:

• Semtech’s proven design wins, including the marquee platform already running its silicon.

• Why Marvell is every bit as strong in ACC as Semtech, for completely different reasons.

• Why MACOM has the silicon and a standards seat but no ACC wins to show.

• How Ciena bought Nubis for optics and picked up a useful ACC play.

• Why the best-known copper-cable company is sitting ACC out.

Finally, a single table to summarize the competitive ACC landscape.

Read more

Thoughts swimming in my head this week have been entirely around the reality of “Orbital Compute” that Gavin mentioned on the podcast as the next big thing. I just wanted to pen down major themes swirling around this topic, on this week’s TWiC, so that I can revisit this when it becomes The Big Bottleneck™, but hopefully sooner than that.

In other news, I have content published around inference accelerators this week.

• 📚 Inside SambaNova’s Inference Architecture

• 🎙️ Cerebras IPO

Ok, lets get to the space stuff.

There has been a lot going on around the idea of putting datacenters in space. Some newsworthy items in no particular chronological order.

• Startcloud put an Nvidia H100 in space — they have a nice whitepaper worth reading.

• Elon Musk explains on X how datacenters should ideally go to space.

• Google’s moonshot Project Suncatcher explores putting Google TPUs in space.

• Axiom Space is a company working on orbital datacenters.

• Last week, Google and SpaceX are in talks to launch datacenters into orbit.

There’s probably a lot more (leave it in the comments), but 5 bullets will do for now.

With the impeding SpaceX IPO, and the fact that SpaceX has an absolute stronghold in the reusable rocket market while also running datacenters here on Earth, the interest in orbital compute is on the rise. In the interview, Gavin thinks orbital compute is just around the corner — in the 2027/28 timeframe.

The one thing Gavin clears up immediately is — “don’t think about it as a Pentagon-shaped datacenter in orbit.” Instead, he asks you to think about racks in space, not entire data centers floating in sun-synchronous orbit.

Today, the power onboard satellites is around 20 kW, and SpaceX engineers seem to think they can scale that up to 100 kW, which should fit a Blackwell-class rack in orbit. The networking aspect is most interesting. Scale up would remain in copper within the rack for sure, but scale out would use free space optics to connect between orbital racks. Most amazingly, SpaceX satellites today already have the ability to communicate with each other to operate in a mesh using free space optics. This technology isn’t in a lab; it’s actually up in space today.

For the industry itself, running data centers in space means that there are four markets that open up to make this happen.

1) High power free space optics

Connecting racks with free-space lasers isn’t something we do on earth today but constellation operators already have optical satellite links built into them, including SpaceX and Amazon, with Telesat and OneWeb still considering adding laser comms to their satellites. The use of free-space optics would open up an entire segment of the optics industry that isn’t directly involved with AI data centers today such as Tesat (privately held) who are widely known for inter-satellite optical communications, CACI () for their optical terminals for defense work, General Atomics, and others such as Skyloom (bought by IonQ) and Bridgecomm (acquired by Voyager Technologies). If optical comms in space are going to become a thing, only a few people really know how thats done today.

Imagine, if NASA’s Terabyte Infrared Delivery Subsystem (TBIRD) could run at 200 Gbps from Earth to orbit in a system the size of a tissue box, we can connect racks in space alright. I’ve written about this before; see post below.

2) RF SATCOM Companies

If you want to communicate with data centers in space from Earth at scale, the best-known approach is through satellite communication systems operating at Ku or Ka band. Starlink terminals currently use this frequency range and typically require their own satellite terminal to communicate with satellites at sufficient bandwidth.

Companies have tried going direct to cellular, but the bandwidth is usually not high enough. The idea of data centers in space could give second wind to companies working on RF-based SATCOM, especially since their forays into millimeter-wave 5G failed spectacularly.

The key technology here is mmW phased arrays, which use an array of antennas that can be cleverly controlled to form a pencil beam aimed directly at the satellite for maximum data throughput. There’s a GaN angle here too in power amplifiers for satellites. Qorvo, MACOM, Analog Devices are among some companies with this expertise.

Since I have actually worked in this field, I’ve written a bunch of posts about this, if you’re interested.

3) Radhard Specialists

If you’re not familiar with silicon in space, you need to know that silicon has to be space qualified and radiation hardened (Radhard) to work in space. Low Earth orbit deployments are easier from a radiation-hardening perspective because the Earth’s atmosphere shields a lot of radiation. As you move farther from Earth, radiation requirements get much worse.

Aside: I attended a talk by the Voyager mission’s chief engineer at IMS 2025, and he explained how insane it was to design electronics that work in the radiation environment of Jupiter. He was cracked AF and a great storyteller! At 15 billion miles away, the data rate from the Voyager is 160 bytes-per-sec. 😭 A 1-way message takes a day!

The ability to test and qualify parts for space is something a company builds over time, and it typically requires a unique end market. Not every silicon supplier can serve these markets, but defense providers and companies in the satellite space often have this capability, so it is worth watching for. Also, it would be wrong to assume it’s only the GPUs that need space qualification. Every component, from memory up through lasers, PCBs, connectors, and cabling, needs to be qualified for space operation. It’s a different ball game the industry is not entirely ready to play at the scale of orbital datacenters today.

4) Cooling Technology

A common mistake is assuming data centers are naturally suited to space because it’s so cold. Convection is the best way to extract heat from surfaces via air- or liquid cooling, and space naturally has neither.

The only way to remove heat from chips in space is through radiation, which is much less efficient. Cooling is a hard problem for data centers in space, even if liquid cooling is used because heat still has to be radiated away with large radiators to keep the chips cool. The ISS rejects on the order of 70 kW with very large panels, so cooling a single rack in space seems feasible in theory.

The best part is that space, well has “space”. You can make the radiators hundreds of feet in size, so a lot of heat can be exchanged with the cold sink outside kept at a steady 2.7K. NASA studies show radiators can account for more than 40% of total power system mass at high power levels. That is the real reason this is hard.

Some companies to watch for in this space are Sierra Space, Paragon Space Development, Redwire Space, ThermAvant Technologies, and Advanced Cooling Technologies.

If AI is going to space, then I’m going to write deep dives about its various aspects.

Meanwhile, have a good weekend!