Technology Report

The Untapped Value at the Intelligent Edge

The need to process data close to its source is creating a new computing architecture and a growing market.

By David Crawford, David Deming, and Bill Radzevych

First published on Σεπτεμβρίου 18, 2023
min read


At a Glance The market for intelligent edge computing is greater than previously understood, with as much as $127 billion in spending on embedded silicon forecast through 2027. This increase reflects the rise in the intelligent edge paradigm, which puts computing resources close to data sources to reduce latency, power usage, and bandwidth costs. Succeeding at the intelligent edge requires strong capabilities to integrate software, digital hardware, and analog hardware since customers give higher satisfaction ratings to fully integrated solutions. This article is part of Bain's 2023 Technology Report. Explore the report Edge computing—that is, moving processing from central data centers to regional and local computing clusters closer to the source—represents a major shift in technology. Because of constraints on power and latency, many applications benefit from data processing at the source—for example, close to sensors (see Figure 1). For these applications, a new architecture is emerging: intelligent edge devices. Figure 1 Power constraints, latency requirements, and limits on connectivity shift computing toward an intelligent edge paradigm Sending vast amounts of data to regional or local cloud hubs for processing is inefficient at best and has the potential to overwhelm existing transport and computing architectures. Furthermore, much of the data generated is noise, and filtering and processing it right at the source can cut down dramatically on transport and processing requirements. Many applications also have a very low tolerance for latency because they are unable to wait for transfer times or delays, requiring resolution in real time. Even traditional edge processing nodes, including local data centers and on-premise gateways, are not efficient enough for the real-time feedback loops these modern devices require. Intelligent edge devices represent a paradigm shift as they perform most data inference and processing directly at the device itself and offer distinct advantages in terms of latency, power consumption, and use of bandwidth. Noise-canceling headphones are a good example: They analyze sound signals in the surrounding environment and counter with opposite frequencies locally, reducing background noise without the need to transmit data to the cloud. Video surveillance is another example: Instead of sending a raw video pixel by pixel to a central processor, which would require tremendous bandwidth and central processing power, an intelligent edge camera can perform a first-order structuring of the data—for instance, looking for humans—and send this structured data to a central hub for decision making. Not all devices will use this architecture, of course. For example, large medical datasets that search for genetic abnormalities require comprehensive processing and inference, and they need access to the massively parallel computational power offered in cloud architectures. Additionally, many smart building management systems take advantage of a central hub in the building to make decisions based on sensor data coming from all over the building. But for the devices at the intelligent edge, the market opportunity may be larger than previously understood. Bain’s analysis forecasts that by 2027, $127 billion of the $277 billion total market for embedded silicon could come from the intelligent edge. It’s a huge opportunity, but most ecosystem participants aren’t fully ready to address this market. Given how rapidly this space is evolving, companies need to react now to serve this market effectively. Develop new capabilities Tapping the intelligent edge opportunity requires new technical capabilities and business models. First, companies will need to further develop their ability to integrate software with digital and analog hardware to deliver fully integrated solutions. Bain research finds that when customers choose solutions, integration of hardware and software is the most highly ranked purchase criteria and that customers award a higher Net Promoter Score℠ for fully integrated products, though even here there is room for improvement (see Figure 2). Given their starting points, this will mean finding the right partnerships or acquisitions while also building up internal capabilities. But our research is clear: Delivering an integrated, end-to-end solution will produce the product most likely to win in these markets. Figure 2 Developers prioritize integrated full-stack solutions These companies will also need to strengthen their abilities to assess and determine which solutions merit full integration and where it makes more sense to deliver parts of solutions, leaving integration to the customer or third-party integrators. As markets mature, customers may develop a preference for open solutions that they can more easily tailor to their needs or that fit better into their software ecosystem. Understanding the determining factor among digital, analog, or a combination can help shape the best offer for a given application and customer. Developing a better understanding of customers and their commercial needs will also be key. It’s likely that as the market matures, customers will want more control over the elements of a solution. Capture opportunities Given the importance of integration and the nascent nature of the solutions in this space, traditional lines of competition are blurring in intelligent edge computing. Most vendors do not provide full-stack solutions, so to win in this market, companies have to find partners across the edge solution stack. For example, application-specific solutions that integrate digital and analog components bring together microcontroller and analog players. Gateway players such as Nvidia and Qualcomm are investing in integrated computing and software solutions as well as developer-friendly tools (for example, Nvidia Jetson and Qualcomm Drive Data Platform). We see many participants in the market starting from their position of strength, pushing into adjacent segments, and blurring traditional lines of competition (see Figure 3). Figure 3 The intelligent edge requires more integrated solutions, pushing players to expand their capabilities Companies in different tech sectors have different avenues to success in the intelligent edge market. Digital semiconductor: Digital and connectivity players should continue to work toward the integration of analog components, strengthening the integrated software stack and building purpose-specific applications. Analog players: Anchor in analog strengths while leveraging the ecosystem and partnerships to build full-stack solutions and software capabilities. Software: Meet developers in their ecosystem of choice, and embrace growing industry standards and open-source offerings. Original equipment manufacturers: Focus on strategic issues and holistic use cases, and let vendors absorb the hardware and software complexity. Hyperscalers: Recognize where cloud-tethered solutions have limitations, and partner with hardware vendors—for instance, as NXP partnered with Amazon Web Services. Intelligent edge devices represent a new computing paradigm that will enable a new class of devices that react in real time to the world around them, without needing the computing and connectivity requirements of the cloud or local data center processing. But success will also require a shift in capabilities and new linkages within the ecosystem. Vendors in this space should assess their readiness for this shift. Read the Next Chapter After the Chip Shortage, Fears of a Capacity Glut Are Overblown Read our 2023 Technology Report Download the PDF Explore the report

At a Glance

The market for intelligent edge computing is greater than previously understood, with as much as $127 billion in spending on embedded silicon forecast through 2027.
This increase reflects the rise in the intelligent edge paradigm, which puts computing resources close to data sources to reduce latency, power usage, and bandwidth costs.
Succeeding at the intelligent edge requires strong capabilities to integrate software, digital hardware, and analog hardware since customers give higher satisfaction ratings to fully integrated solutions.

Edge computing—that is, moving processing from central data centers to regional and local computing clusters closer to the source—represents a major shift in technology. Because of constraints on power and latency, many applications benefit from data processing at the source—for example, close to sensors (see Figure 1). For these applications, a new architecture is emerging: intelligent edge devices.

Power constraints, latency requirements, and limits on connectivity shift computing toward an intelligent edge paradigm

Sending vast amounts of data to regional or local cloud hubs for processing is inefficient at best and has the potential to overwhelm existing transport and computing architectures. Furthermore, much of the data generated is noise, and filtering and processing it right at the source can cut down dramatically on transport and processing requirements. Many applications also have a very low tolerance for latency because they are unable to wait for transfer times or delays, requiring resolution in real time. Even traditional edge processing nodes, including local data centers and on-premise gateways, are not efficient enough for the real-time feedback loops these modern devices require.

Intelligent edge devices represent a paradigm shift as they perform most data inference and processing directly at the device itself and offer distinct advantages in terms of latency, power consumption, and use of bandwidth. Noise-canceling headphones are a good example: They analyze sound signals in the surrounding environment and counter with opposite frequencies locally, reducing background noise without the need to transmit data to the cloud. Video surveillance is another example: Instead of sending a raw video pixel by pixel to a central processor, which would require tremendous bandwidth and central processing power, an intelligent edge camera can perform a first-order structuring of the data—for instance, looking for humans—and send this structured data to a central hub for decision making.

Not all devices will use this architecture, of course. For example, large medical datasets that search for genetic abnormalities require comprehensive processing and inference, and they need access to the massively parallel computational power offered in cloud architectures. Additionally, many smart building management systems take advantage of a central hub in the building to make decisions based on sensor data coming from all over the building.

But for the devices at the intelligent edge, the market opportunity may be larger than previously understood. Bain’s analysis forecasts that by 2027, $127 billion of the $277 billion total market for embedded silicon could come from the intelligent edge. It’s a huge opportunity, but most ecosystem participants aren’t fully ready to address this market. Given how rapidly this space is evolving, companies need to react now to serve this market effectively.

Develop new capabilities

Tapping the intelligent edge opportunity requires new technical capabilities and business models.

First, companies will need to further develop their ability to integrate software with digital and analog hardware to deliver fully integrated solutions. Bain research finds that when customers choose solutions, integration of hardware and software is the most highly ranked purchase criteria and that customers award a higher Net Promoter Score℠ for fully integrated products, though even here there is room for improvement (see Figure 2). Given their starting points, this will mean finding the right partnerships or acquisitions while also building up internal capabilities. But our research is clear: Delivering an integrated, end-to-end solution will produce the product most likely to win in these markets.

Developers prioritize integrated full-stack solutions

These companies will also need to strengthen their abilities to assess and determine which solutions merit full integration and where it makes more sense to deliver parts of solutions, leaving integration to the customer or third-party integrators. As markets mature, customers may develop a preference for open solutions that they can more easily tailor to their needs or that fit better into their software ecosystem. Understanding the determining factor among digital, analog, or a combination can help shape the best offer for a given application and customer. Developing a better understanding of customers and their commercial needs will also be key. It’s likely that as the market matures, customers will want more control over the elements of a solution.

Capture opportunities

Given the importance of integration and the nascent nature of the solutions in this space, traditional lines of competition are blurring in intelligent edge computing. Most vendors do not provide full-stack solutions, so to win in this market, companies have to find partners across the edge solution stack. For example, application-specific solutions that integrate digital and analog components bring together microcontroller and analog players. Gateway players such as Nvidia and Qualcomm are investing in integrated computing and software solutions as well as developer-friendly tools (for example, Nvidia Jetson and Qualcomm Drive Data Platform). We see many participants in the market starting from their position of strength, pushing into adjacent segments, and blurring traditional lines of competition (see Figure 3).

The intelligent edge requires more integrated solutions, pushing players to expand their capabilities

Companies in different tech sectors have different avenues to success in the intelligent edge market.

Digital semiconductor: Digital and connectivity players should continue to work toward the integration of analog components, strengthening the integrated software stack and building purpose-specific applications.
Analog players: Anchor in analog strengths while leveraging the ecosystem and partnerships to build full-stack solutions and software capabilities.
Software: Meet developers in their ecosystem of choice, and embrace growing industry standards and open-source offerings.
Original equipment manufacturers: Focus on strategic issues and holistic use cases, and let vendors absorb the hardware and software complexity.
Hyperscalers: Recognize where cloud-tethered solutions have limitations, and partner with hardware vendors—for instance, as NXP partnered with Amazon Web Services.

Intelligent edge devices represent a new computing paradigm that will enable a new class of devices that react in real time to the world around them, without needing the computing and connectivity requirements of the cloud or local data center processing. But success will also require a shift in capabilities and new linkages within the ecosystem. Vendors in this space should assess their readiness for this shift.

Read the Next Chapter

After the Chip Shortage, Fears of a Capacity Glut Are Overblown