AI Appliances Pop Up at the Edge

Companies in a variety of industry sectors are looking for ways to deploy artificial intelligence (AI) in Internet of Things (IoT) applications. So in manufacturing, retail, healthcare, and other verticals, devices are starting to appear that combine AI processing power with edge compute capabilities and connectivity for multiple IoT devices.

Many of these AI-equipped edge appliances were introduced at tradeshows and conferences this year, including Embedded World and Computex. Based on emerging AI chipsets designed for edge, PC, and laptop implementations (including Intel’s Lunar Lake, AMD’s Strix Point, and Qualcomm’s Snapdragon X Elite, for example), these devices gather information from machinery located at the network edge for AI inferencing and analysis. The result is lower latency and faster response for applications designed for the likes of industrial machinery or connected cars, where instantaneous responses to sensor input is essential.

These AI-based edge appliances can interact with cloud services or datacenter applications. Since they monitor and control physical devices, they can unify operational technology (OT) oversight with IT (information technology). Some also are equipped with WiFi and/or cellular connectivity to 4G LTE or 5G networks, which allows them to gather information at the edge and respond quickly in retail, healthcare, or industrial environments.

For the internals of AI edge appliances, there’s a growing array of components designed for edge and PC implementations. Some of these, such as Qualcomm’s Snapdragon X Elite, are augmented with CPUs, GPUs, and neural processing units (NPUs). NPUs are chips that handle deep learning algorithms, allowing AI to be supported on devices instead of in public clouds or AI factories. The performance capabilities of NPUs are measured in trillions of operations per second (TOPS), and according to Microsoft, 40 TOPS is the minimum required for an AI PC, so it follows that AI-based edge units must have at least as much horsepower.

A Sampling of AI Edge Devices

Following is a sampling of some devices designed for the AI-enabled IoT edge:

HPE Aruba 700 Series WiFi 7. This unit is an access point designed to bring AI to IoT edge applications. While not specifically geared to processing AI but instead capturing and routing data to AI systems, the vendor says that extra processing power and storage makes it possible for container-based apps to run on the access points, speeding up the ability to fetch responses in real time. Link-level encryption and a private network mode add security.

Lanner Electronics EAI-I133. This product is described by the vendor as “a 5G Edge AI Gateway with NVIDIA Jetson Orin NX/Nano.” Designed for edge AI inferencing, it features up to 100 TOPS of performance, contains an NVIDIA GPU, and features 4G LTE, 5G, and WiFi connectivity. It also supports HDMI for video applications.

Nokia MX Grid. This unit is based on Nokia’s MX Industrial Edge (MXIE) platform. It adds AI/ML to the vendor’s on-prem industrial system, analyzing data closest to its source in factory environments. The MX Grid brings AI apps to networks of so-called micro-edges, which are field devices designed to take input from sensors and other equipment. Micro-edges are linked by Nokia private networks or WiFi. The addition of AI allows these micro-edges to respond immediately or pass data to MXIE for further analysis.

Supermicro SYS-E403-13E. Designed for advanced AI at the edge, this device contains an Intel Xeon Scalable processor and room for up to three Intel accelerator cards. Connectivity options include two 10-Gb/s ports and three PCIe 5.0 with 16 lanes. The unit is small, allowing it to be used as a portable device or mounted in cabinet on the wall.

Thundercomm EB5G2 Edge AI Station. Thundercomm started as a joint venture with Qualcomm in 2016. Its EB5G2 is based on the Qualcomm QCS8550 processor and delivers 48 TOPS of AI processing power. According to the vendor, the unit’s proprietary edgeOS supports “device management, video management, AI algorithm configuration, application upgrades, edge-cloud collaboration, customization, and multi-cloud connectivity.” It is recommended for use in smart retail, smart building, smart factory, smart transportation, and other AI edge scenarios.

Vecow EAC-4000. Designed by an embedded system vendor for entry-level edge AI use cases such as delivery robots, object recognition, or smart traffic vision, the Vecow EAC-4000 is based on NVIDIA Jetson Orin NX and Jetson Orin Nano components, along with NVIDIA Ampere GPUs, Arm-based CPUs, and other elements that support up to 100 TOPS of AI processing. Among its connectivity options are a LAN port, four USB ports, two RS-232/422/485 ports, one Nano SIM card socket, and two antennas for WiFi/4G connectivity. A small form factor makes this unit suitable for small spaces, the vendor says.

The Vecow EAC-4000. Source: Vecow

Futuriom Take: Increased focus on AI and edge compute has vendors delivering AI-capable edge appliances for use in industrial, healthcare, automotive, and other IoT settings. This is a niche worth watching.