The supply constraints issue for the H100 AI GPU is expected to be resolved.

According to a report by Digitimes, Terence Liao, the General Manager of Dell in Taiwan, stated that the delivery cycle for Nvidia's H100 AI GPU has been reduced from 3-4 months to just 2-3 months (8-12 weeks) over the past few months. Server ODMs have indicated that, compared to 2023, the supply has finally eased, a time when it was nearly impossible to obtain Nvidia's H100 GPU.

Despite the shortened delivery times, Terence Liao noted that the demand for hardware with artificial intelligence capabilities remains very high. Specifically, the purchase of AI servers is replacing the purchase of general-purpose servers in enterprises, even though the cost of AI servers is quite high. However, he believes that the procurement time is the sole reason for this situation.

The 2-3 month delivery window is the shortest delivery time for the Nvidia H100 GPU. Just six months ago, the delivery time reached 11 months, meaning that most of Nvidia's customers had to wait for a year to fulfill their AI GPU orders.

Since the beginning of 2024, the delivery times have significantly shortened. Initially, we saw a substantial reduction to 3-4 months earlier this year. Now, the delivery time has been shortened by another month. At this pace, we could see the delivery times completely disappear by the end of the year or even sooner.

Advertisement

This behavior might be the result of a chain reaction where some companies have an excess of H100 GPUs and resell part of their supply to offset the high maintenance costs of unused inventory. Additionally, AWS has made it easier to rent Nvidia H100 GPUs through the cloud, which also helps to alleviate some of the H100 demand.

The only Nvidia customers struggling with supply constraints are large companies like OpenAI, which are developing their own Large Language Models (LLMs). These companies require tens of thousands of GPUs to train their LLMs quickly and efficiently.Good news is, this should not become a long-term issue. If the delivery times continue to shorten exponentially, as they have over the past four months, Nvidia's largest customers should be able to get all the GPUs they need, at least in theory.

CoWoS Packaging Capacity is Key

The reduction in delivery times indicates that TSMC's increased CoWoS packaging capacity is starting to come online. It is reported that TSMC aims to double this capacity by the end of 2024 from the levels in mid-2023. From the current perspective, the progress of TSMC and its partners in expanding CoWoS capacity is faster than expected, significantly reducing the delivery times for high-performance GPUs, represented by the H100.

Industry insiders analyze that from July 2023 to the end of the year, TSMC actively adjusted its CoWoS packaging capacity, gradually expanding and stabilizing mass production. In December last year, TSMC's CoWoS monthly capacity increased to 14,000 to 15,000 wafers.

Although TSMC is actively expanding production, the capacity of this single company still cannot meet market demand. Therefore, Nvidia has already sought help from professional outsourcing semiconductor assembly and test (OSAT) factories other than TSMC in 2023, mainly including ASE Technology and Amkor. Among them, Amkor began to provide relevant capacity in the fourth quarter of 2023, and SPIL, a subsidiary of ASE Technology, also began to supply CoWoS packaging capacity in the first quarter of 2024.

In 2024, advanced packaging capacity for AI chips will still be in short supply. Professional OSAT factories, including TSMC, ASE Technology, Amkor, Powertech Technology, and KYEC, will increase their capital expenditures this year to layout advanced packaging capacity.

According to TSMC's expansion pace, it is expected that by the fourth quarter of this year, the leading wafer foundry's CoWoS monthly capacity will be significantly expanded to 33,000 to 35,000 wafers.

This year, ASE Technology's capital expenditure scale will increase by 40% to 50% year-on-year, with 65% of the investment used for packaging, especially advanced packaging projects. Wu Tianyu, Chief Operating Officer of ASE Technology, said that this year's advanced packaging and testing revenue ratio will be higher, and AI-related advanced packaging revenue will double, with related revenue increasing by at least $250 million this year. Powertech Technology is also expanding its advanced packaging capacity. Chairman Cai Dugong said that the company will actively increase capital expenditures in the second half of the year, with the scale expected to reach 10 billion New Taiwan dollars. Powertech mainly focuses on fan-out on substrate packaging technology, integrating ASIC and HBM advanced packaging. In the AI HBM memory field, Powertech is expected to mass-produce related products in the fourth quarter of this year. To meet the wafer testing needs after CoWoS packaging, KYEC will double its wafer testing capacity this year.H100 Resale Trend

As the delivery cycle shortens, some companies that previously hoarded H100 are considering reselling their excess inventory. This phenomenon is particularly evident under the influence of large cloud service providers such as AWS, Google Cloud, and Microsoft Azure. These companies offer convenient chip leasing services, allowing users to avoid the need for large-scale purchases and hoarding of hardware, thereby reducing costs and increasing flexibility.

Despite the improved availability of H100, the demand for AI chips remains strong, especially in the field of training large language models (LLM).

NVIDIA, as the world's leading GPU manufacturer, holds an important position in the AI chip market. However, with the continuous investment and development of companies like AMD and Intel in the AI chip sector, market competition is becoming increasingly fierce.

With the widespread application of AI technology, the AI chip market is entering a period of rapid growth. Although the supply issues of AI chips have been somewhat alleviated, market demand remains robust, and market competition remains intense. Companies like NVIDIA, while expanding production scales and improving supply chain efficiency, also need to pay attention to the dynamics of competitors and market changes to address potential challenges and opportunities in the future.

*Disclaimer: This article is an original creation by the author. The content of the article represents their personal views. Our reposting is solely for sharing and discussion and does not represent our endorsement or agreement. If you have any objections, please contact the backend.