All You Need To Know About Nvdia, and A Little On Semiconductors
The semiconductor space, who does what, how the landscape is affecting Nvidia
Here’s a early-release of a special edition coverage on Nvidia, reserved for subscribers only!
Podcast Link (Noted, recorded on 22nd May 2021). Some details may be slightly outdated, and note the technical details may have been watered down!
This post serves as an accompanying “FAQ” and analysis into key terminologies and highlights that may need further expansion beyond the 1-hour recording.
NOTE: As always information presented is for entertainment purposes only. Full disclosure can be found in the “About” section. It does not serve as any form of advice or recommendations for investment decisions, with information obtained from publicly available sources.
What is a Semiconductor? What’s Nvidia’s part in it?
Semiconductors (lets call them ‘Semicon’) conducts electricity in a coordinated way, between metals and non-conductors.
Semicons are shaped and formed into parts of other technologies down the semiconductor supply chain into more of an “end-product”.
These will be your common electrical appliances, such as your smartphones, computers, medical equipment, even light switches.
Nvidia uses Semicon to make Graphical Processing Units (GPUs), which are used in areas they focus on specifically.
They are:
Gaming and Graphical design
Such as on personal PCs to play high-performance games with high definition, where intense graphical computational required.
Additionally in designing in the 3D space (and soon to be 4D with the AR/VR trend), more specialized processing is needed to render designs, videos.
This software is used by a lot of design businesses, such as your interior design, engineering design companies (think aircraft engines/layouts, simulations, and the ones we keep hearing, metaverses) to do their daily works, which requires lots of processing to render results.
Cloud computing.
Lots of machine learning workloads require specially designed processing units.
We’ll expand more on this later.Cryptocurrency mining
Mostly number crunching, similar to cloud computing but can be customized further.
Whats the difference between a GPU and CPU?
A little geeky here, you can skip if you are getting woozy at this point :)
CPU
CPUs are traditionally used in desktop computers and laptops. They process instructions throughout the computer, which most of the “normal” workloads people have are reading and writing to a disk.
You open files, save them somewhere, these are all data retrieval and storage.
GPU
GPUs accelerates the processing in creating and rendering of images of any kind. There are integrated GPUs in your PC/Laptops’ motherboards:
There are more “independent” GPUs, also known as Graphic Cards. These are commonly attached to a motherboard with a dedicated slot:
Both are loosely referred to as GPUs, but the ones requiring slotting into a motherboard are commonly known as Graphic Cards, as they are outside of the main motherboard (i.e. non-integrated).
Your CPU can do the same processing GPUS do, but much slower.
It’s not optimized to do image processing at a high volume. It’s only optimized to actually streamline your usual workloads on your operating system such as your macOS, your Windows, such as opening, and saving files.
But when you run some very intensive computation such as a game, or machine learning jobs on your computer, there are better ways to optimize the same kinds of underlying technologies.
In short, GPUs can be seen as an accelerator to certain workloads, that can either be together on the same motherboard/architecture or be independent in itself.
How Does This Acceleration Work?
An oversimplification: A GPU works closely with 2 main components.
Motherboard - for data and power.
A very fast graphics card is still limited by a motherboard’s capacity to deliver the data. The card’s connection to the motherboard and the speed at which it can get instructions from the CPU will affect GPU performance.Video memory (VRAM) - holds information about each pixel and to have temporary storage of completed pictures.
GPUs create the images, but it needs somewhere to hold information and the completed pictures - that’s where the VRAM on the graphic card’s own “motherboard” comes into play, storing data about each pixel, colour and location on the screen.
The VRAM also holds the completed images, until it is time to display them.
Graphics Card vs GPU
A Graphics Card (not entirely the same as a GPU) acts as if it has its own motherboard.
It has its own “printed” circuitry, having its processor and video memory, and inputs/outputs dedicated to doing the job of processing millions of pixels.
This helps offload the work from the operating system (Windows, macOS etc) and software (Adobe AfterEffects, 3D software such as AutoCAD etc) to interpret and understand the images.
In an integrated card’s motherboard, these same workloads are channeled to the GPUs instead of the regular CPU. Apple calls is their “Unified memory architecture”.
Apple has their own architecture design to optimize how all these parts are working optimally with each other. Similar, Nvidia has these too, but optimized for the verticals they focus on, such as gaming/design (home-use, plugged in), and integrated (data centers, crypto mining).
The Current Situation - A Semiconductor Shortage Since 2020
Before going into the meat of the discussion, it is worth understanding the layers of complexity to appreciate the situation of Semicon shortages.
Understanding The Value of Semicon
You may hear “nanometer” being used often when you come across news about semiconductor developments.
Nanometer refers to how many transistors (i.e. electrical signals) you can etch onto a Semicon surface. We’re talking about the microscopic level.
Semiconductors are all about this metric, as it’s about how many electrical connections you can squeeze into that nanometer. Ultimately, that affects how many is being packed into space affects use cases widely.
From how much computations a graphic card can do for your game, or in the case for AI or cryptocurrencies, how much compute it can churn in a second, and that matters for the end use-cases, as products can be priced higher.
Think in terms of a shorter amount of time to finish completing a large data set for machine learning, or to decrypt another block on some crypto’s blockchain - you would need less, but it can do more. Or, you can stack more of these into the same space, and it’ll be more performant.
Each Nanometer configuring has lots of R&D done over years of work, before putting them into a production line. Thus, improvements were not “discovered” overnight, but from incremental innovation added to production in a timely fashion.
Understanding The Processes
Let’s start with the biggest player, TSMC. They are manufacturing the chips. They are doing everything possible, supporting automotive, headphones, microphones, smartphones. Lots of production lines, and are built in such a way to run 24/7 to meet output KPIs.
The gist is that for TSMC, there are lots of clients, and they would not allocate all their operations for a single client. So, multiple of these are run at once.
Say 60% of their capacity of production is in Apple, 20% to automotive, and so on. At any one time, TSMC’s machines are producing different components for different clients. This diversifies revenue sources, and assists in managing risk.
These processes are highly automated and need to be very finely tuned in setup. It’s not something that can be changed overnight or even a course of a few months if they want to change a production line.
As such, even if there were innovations in Semicon production or to squeeze in extra Nanometers, these will not occur overnight. There is no technology miracle implemented in a few weeks to “save the situation”.
Now, circling back to the original question: Why are semiconductors in a shortage?
For every end-product or use case you know, such as smartphones, cars, computers, every small component was manufactured by someone, somewhere. It is very rare to have only one entity providing all parts.
Nvidia is a designer, so they are not producing all these parts - instead, they come up with architecture designs, make orders, and have the finished modules assembled into products like Graphics cards and so on.
Thus, one must understand that it is a complex process, supply chain and operation that just 1 player undergoes. There are also interlocking dependencies - one Semicon supplier produces for another, who is dependent on a certain part before fabricating an end-product, that end-users can use.
And suddenly, there is a high demand for certain kinds of chips and there’s just not enough of these being able to produce on the foundries itself, on the manufacturers itself. McKinsey gives a great breakdown on identifying the vulnerabilities, for phones as an example end-product:
In Step 1, you can see there are various modules, processors and chipsets that make up some parts of the phone we know, like a Camera. Further down in each module/chip, there will be smaller components, and so on.
Here are some of the companies you may not have heard of:
Pegatron Corporation
Cheng Uei Precision Industry
LuxShare Precision Industry
All these stacks up when demand rises across the board, and supply cannot keep up - some companies producing these small modules are also smaller companies, and can only output so much.
Then, add on the complexity of international logistics being affected because of COVID closures - you get even longer delivery times, and thus supply is hard to get.
Why is there a high demand?
With more people staying at home, they may start to see ways to get their basic WFH setup, and improve their quality of life at home.
People may want a larger fridge, to store as many vegetables and fruits as possible. Maybe a higher quality TV to watch than Netflix, or gaming console. This is your demand pool.
So what you are seeing in the prices of most commodity goods from 2020-2021 are due to a perfect storm affected from the supply side as well as on the demand side, until it reaches a kind of equilibrium.
We can see this similarly to the Lumber market, which has its prices coming back down to more ‘healthier’ levels:
What Can Be Done About It?
Most Semicon producers are at max capacity. They are still trying to roll out more scalable ways to actually produce these microchips, but there are constraints.
A new improvement to fabrication processes can take up to 18 months, and until then, if demand does not ease, prices can continue to rise.
Until then, even the largest companies have no choice but to delay releases, such as Samsung’s foldable phone, and for Nvidia, their GeForce series have been very difficult to buy. Apple have also announced they will be raising prices for the iPhone 13, weeks away from their first release, passing down the costs from TSMC.
And there there is a lack of supply, companies would also raise prices as they have limited inventory. Thus, this is how costs and supply chain dynamics have an impact down the chain, and to the end consumers.
Nvidia’s End-Products and Their Use Cases
Gaming & Design
The more familiar portion - imagine there is this high-specced game or a video/3D simulation that needs rendering. This needs the latest in graphics processing technology to load, and won’t work on older graphics cards. Or, you could do it in the lowest resolution possible…
Most people opt to purchase the newest, to experience the “best” just like how people would also buy the latest iPhone even though they have one!
As for Design-use cases, things are a little different - it is necessary for them to have this, else turnaround time is too slow, and deadlines could be missed.
Compute
As of 2021, Nvidia earns more from data centers than gamers. Data Centers are related to “cloud compute” as you heard it - those massive server farms you may have seen in art like those above.
They do a lot of computational workloads - number crunching, such as your data science workloads, graphics processing, even crypto-mining but done at scale, rather than on your own, individual machine.
Amazon Web Services (AWS), Azure by Microsoft and Google Cloud Platform (GCP) are the main players. And Nvidia is the number one leading supplier of GPUs to these players, by about a market share of 90%.
As a result, Nvidia can call the shots here. An example: Nvidia is advising data centers to use its highly specced ones rather than the cheaper GeForce products.
Growth Opportunities
Data Center Workloads
The trend is that amount of data is exploding, and the compute required to process all these for data science use cases, is also increasing.
Just looking at Fortune 100-500 companies, they will look to increase AI workloads as a means of offering differentiated services to customers, while reducing costs.
Your retail names like Walmart, or Insurance companies such as Aviva, all have computing workloads - recommending you items to buy on the way to checkout online, or in insurance, estimating the likelihood for you to make a claim.
Traditional companies are increasingly looking to move to cloud, and in both cases, they would need to sign up with a cloud provider, and they, in turn, will need more GPUs.
Shift from Home-compute, to Cloud-compute and Mobile
There is less need for high-powered GPUs at home and more need for them on a server farm somewhere. As a retail user, you tap on a centralized source of compute, likely powered by GCP/Azure/AWS, which relies on Nvidia’s GPUs.
Thereafter, you just need a decent display and a fast internet connection, which is also where the rest of these telco industries are going towards. So the shift towards these data centres, it’s a lot more other than just the workloads, but it’s also because of the gaming part of things. There’s a gaming added component to their existing workloads.
This ultimately affects Nvidia as there would be less graphics card demand, but more so on their servers segment to make up for it.
Gaming is going towards mobile, and most workloads are going to be in the mobile platform, rather than on PC as a platform. That’s where Nvidia is not very strong at, and that is where the ARM discussion comes into play.
The ARM Acquisition Play
ARM owns patents is many devices, especially mobile. You can find ARM architecture in both Androids and iPhones, in which Apple had recently built further in their release of the M1 architecture in phones, and coming soon to a Macbook near you.
They are so prevalent, ARM takes up to 90% market share in smartphones and tablets, and are now making a “push into servers, networking infrastructure, embedded intelligence markets”. Perfect, that’s where Nvidia would like to be as well.
Should the acquisition go through, Nvidia+ARM will be the vast “supplier” of processing, both on servers, home and mobile. The opportunities are endless:
A new market in the Internet of Things (in each device), may have its own computation capability on the device itself. This is called “edge computing” in some circles.
Main Markets of mobile/data center, edge and IoT, Nvidia+ARM can work together to produce new architectures that take market share away from rival technologies (e.g. QCOM, Intel).
This comes from offering alternative processsors to common types such as Intel’s, and offering lowered-energy usage and emissions to help companies go green.
Why does it matter?
ARM is an architecture mainly for smartphones (dubbed “mobile systems-on-a-chip or SOC), with heavy usage throughout the world. ARM is all about how efficient and scalable they made their architecture, compared to other alternatives.
Companies such as Apple, Samsung, Google license from ARM to develop into products such as the iPhones, Galaxys and Pixels
If you compare that to Intel, you always see “Intel Inside” because that’s your processor on your motherboard, your PC powering everything in some sense, but Intel isn’t as good and has a cap on performance, due to design and supply chain decisions made previously, causing TSMC to leapfrog Intel in terms of market share, and product performance.
With ARM innovations, and Apple having their own architecture piggybacking ARM Apple’s just going to say “We’re going to build it ourselves for less cost, and higher performance.”
Now, if Nvidia really gets the acquisition without all these antitrust lawsuits going on, even if they managed to pass it, this is a very big boost for them. Because one is that free business. Apple, whatever device you sell, Nvidia could get a cut of it.
The second part of it is that, for ARM’s architecture, is that its remarkable performance of the M1 chip, is its ability to be used in very small spaces or devices. It’s not just in your Mac or your MacBooks, it’s also in your iPhones and in that factor already, you can actually power games very well with such a good efficiency to energy ratio and the heat disbursement is just phenomenal.
Compared to a traditional graphics card, it does not need a fan or water cooler, and heat is relatively low. This is game-changing, disruptive stuff.
Here’s what CEO Jensen Huang says:
“We were delightful licensees of ARM. As you know we used ARM in one of our most important new initiatives, the Bluefield GPU. We used it for the Nintendo Switch — it’s going to be the most popular and success game console in the history of game consoles. So we are enthusiastic ARM licensees.
There are three reasons why we should buy this company, and we should buy it as soon as we can.
Number one is this: as you know, we would love to take Nvidia’s IP through ARM’s network. Unless we were one company, I think the ability for us to do that and to do that with all of our might, is very challenging. I don’t take other people’s products through my channel! I don’t expose my ecosystem to to other company’s products. The ecosystem is hard-earned — it took 30 years for Arm to get here — and so we have an opportunity to offer that whole network, that vast ecosystem of partners and customers Nvidia’s IP. You can do some simple math and the economics there should be very exciting.
Number two, we would like to lean in very hard into the ARM CPU datacenter platform. There’s a fundamental difference between a datacenter CPU core and a datacenter CPU chip and a datacenter CPU platform. We last year decided we would adopt and support the ARM architecture for the full Nvidia stack, and that was a giant commitment. The day we decided to do that we realized this was for as long as we shall live. The reason for that is that once you start supporting the ecosystem you can’t back out. For all the same reasons, when you’re a computing platform company, people depend on you, you have to support them for as long as you shall live, and we do, and we take that promise very seriously.
And so we are about to put the entire might of our company behind this architecture, from the CPU core, to the CPU chips from all of these different customers, all of these different partners, from Ampere or Marvell or Amazon or Fujitsu, the number of companies out there that are considering building ARM CPUs out of their ARM CPU cores is really exciting. The investments that Simon and the team have made in the last four years, while they were out of the public market, has proven to be incredibly valuable, and now we want to lean hard into that, and make ARM a first-class data center platform, from the chips to the GPUs to the DPUs to the software stack, system stack, to all the application stack on top, we want to make it a full out first-class data center platform.
Well, before we do that, it would be great to own it. We’re going to accrue so much value to this architecture in the world of data centers, before we make that gigantic investment and gigantic focus, why don’t we own it. That’s the second reason.
Third reason, we want to go invent the future of cloud to edge. The future of computing where all of these autonomous systems are powered by AI and powered by accelerated computing, all of the things we have been talking about, that future is being invented as we speak, and there are so many great opportunities there. Edge data centers — 5G edge data centers — autonomous machines of all sizes and shapes, autonomous factories, Nvidia has built a lot of software as you guys have seen — Metropolis, Clara, Isaac, Drive, Jarvis, Aerial — all of these platforms are built on top of ARM, and before we go and see the inflection point, wouldn’t it be great if we were one company.
And so the timing is really quite important. We’ve invested so much across all of these different areas, that we felt that we really had to take the opportunity to own the company and collaborate deeply as we invent the future. That’s the answer.”
So…Why?
Firstly, Nvidia could work with ARM to build smaller, but more powerful CPUs AND GPUs. These can then be used to expand Nvidia’s use cases - to not just home desktops for gaming or design, but to computer manufacturers themselves, replacing Intel at some point.
Nvidia has already been working with ARM architecture, but they do not have direct influence over its evolution, and thus cannot influence the technology roadmaps of others (yet).
Secondly, looking at the data center market, less for more is all the rage - real estate is expensive.
For example (hypothetical, just based on M1’s form factor) instead of needing to get a 1m x 1m rack of 16 GPUs, Nvidia’s new designs (of course proprietary) with ARM can slot in 96 - that’s a 6x improvement in space utilization, and also 6x the compute power. Very difficult to beat, and cementing Nvidia’s place in supplying data centers.
Thirdly, the same designs can be used to enter mobile as a new market for Nvidia. ARM is currently the industry standard for mobile processors. In doing so, Nvidia can address the trend where compute is going towards mobile and edge devices (Internet of Things), which is covered more below.
In all cases, Nvidia can license designs for both the CPU and GPU market, in high-growth markets. That’s a lot of market dominance, of which poses risks to regulatory bodies for monopolistic considerations. Risks such as these are expanded below.
Risk Factors
The shift in trends - Gaming moving to the cloud, and Edge Computing is coming soon
People on the way back from home, even people at home and all don’t switch on their PC to play games. They play mobile games instead. These mobile games also either need the AI on the device itself or the graphics and the AI can be on the cloud.
Recently, Microsoft have announced their push for XBOX cloud gaming, whereby the gaming device itself may not matter in the future- but more of a platform that distributes games. If this takes off with other game device makers such as Sony and Nintendo, it could be disruptive for hardware makers’ margins, and beneficial for the game developers.
In either case, Nvidia doesn’t address that well yet. If it’s in the cloud, probably, but it’s quite indirectly because it goes through your cloud vendors, your cloud providers. It’s not Nvidia themselves. So it’s not something that they can exactly bank on, but they know that they are the key supplier for these data centers. Fine. But what if?
Edge compute will become more prevalent once 5G is commonplace. This is a matter of time. For both mobile and edge compute, the GPUs need to take on smaller form-factors, and perhaps even sit on the same architecture/board. If this shift happens too soon, Nvidia may not be able to ride on this trend, while seeing market share taken away by other competitors.
This is heavily dependent on their ARM acquisition, covered below.
ARM-wrestling the anti-trust (pun intended)
i-Micronews has a great visual summary of the opportunities and challenges:
A big one will be Intel, and another by the UK. Because once this acquisition goes through, the R&D part is already settled, and then they managed to merge these ARM architecture as well as the GPU together. There’s no need for the other players riding on their past innovations, e.g. Qualcomm, Intel, AMD and Xilinx. That’s just eating into the business, big time.
Wait, But Why So Impactful To Other Companies?
Here’s why.
Qualcomm: Has major risks to its business model, if Nvidia becomes a “gatekeeper to ARM’s technology”, i.e. refusing to license important technologies to be used in conjunction with Qualcomm tech. In doing so, Nvidia can influence product and technology development roadmaps heavily. It is so impactful, Qualcomm wants to invest in ARM for itself too.
Intel: ARM is a direct competitor for the CPU market, and many other Semicon players came onto the space to innovate out of inefficient Intel designs.
Intel has also not gotten a meaningful share of the mobile segment, and is not as performant in either CPU or for mobile. Nvidia+ARM combining would be a blow to completely knock Intel out of the market as it has not innovated since its heydays.
AMD: AMD does not have great solutions using the ARM architecture, and has continuously lost in product bake-offs to Nvidia. AMDs market share in PC is already small, and has high dependency on x86 (in which Intel also uses), which limits their product performances. Nvidia could ‘block’ progress of AMD in innovating to use ARM architecture to improve their product lines.
Xilinx: Acquired by AMD, and both Xilinx and AMD are competing with Nvidia. The risk factors for Xilinx are the same as of AMD, and the acquisition may not have as much synergies as they thought.Apple: Apple has a perpetual license to ARM, but we know that Apple can extend the ISA for its own uses. Apple does not have the motivation to acquire ARM itself even if Nvidia may not get along well with Apple in developing architectures unsuitable for Apple devices. And that’s fine, because Apple is charting their own technology roadmap and are happy to pay royalties to achieve the performance/design they want. Nvidia has no incentive to block Apple if they do own ARM, since they would be getting paid.
It uncertain if Nvdia, owning ARM, would continue licensing the technologies to these companies - that will throw a really large spanner into the works of these companies, and is something regulatory bodies are concerned with.
The UK competition authority has some weight here, stating national security concerns if Nvidia does so, and possibly “stifling innovation” of markets. CEO of Nvidia, Jensen Huang, has stated that Nvidia has “every intention to add more IP tools and to…accelerate the development of server CPUs”.
Thus, how this acquisition plays out will be a significant factor in Nvidia’s future growth trajectory, and also to incumbent players.Tech companies building their own GPUS
We’ve already seen this done by Apple, but they too, are a designer and focus primarily on hardware.
That would not stop Google or Amazon from doing so, as they can reduce reliance on Nvidia for their data center businesses.If this happens, Nvidia loses revenue in their largest segment and will be dependant on other opportunities such as ARM to secure another growth area.
For more in-depth content and weekly updates right in your inbox, subscribe at the button below, or share the goodness with someone you know!
Here are key sources which have shaped my understanding and analysis:
https://www.nvidia.com/en-sg/
https://computer.howstuffworks.com/graphics-card.htm
https://fortune.com/2021/06/15/why-companies-are-spending-more-on-a-i/
https://www.sierrawireless.com/iot-blog/5g-waves/
https://stratechery.com/2020/chips-and-geopolitics/