Boost AI Performance with This Miniature Coprocessor
In the realm of AI edge computing, Femtosense has made a significant stride with their Sparse Processing Unit (SPU). This hardware, optimised for handling sparse data and consuming under 1 mW, is set to redefine low-power AI edge computing, particularly for audio applications like always-listening keyword detection.
The SPU, as found in some earbud applications, accelerates AI inference by focusing on sparsity in data and models. This strategy reduces the number of active computations and memory accesses compared to dense processing units, resulting in significant energy savings without compromising the performance required for tasks like keyword spotting in audio streams.
The SPU-001, a key component of Femtosense's hardware, plugs into a PMOD connector found on many processor evaluation boards and utilises an SPI interface to connect to a host processor. The company's hardware is based on this SPU, with a notable example being the AI-ADAM100, a single-chip solution incorporating a Cortex-M0+ core and an SPU.
Sam Fok, CEO at Femtosense, discussed the development of this ground-breaking hardware. He highlighted that the SPU's focus on improving performance is not limited to high-end, cloud-based solutions but is also applicable to low-power embedded solutions.
One of the methods for improving performance is addressing model sparsity. Sparse matrices are common in machine-learning models since these weights are often zero or close to it. By leveraging this sparsity, the SPU can eliminate the need to perform arithmetic operations, reducing overhead by a factor of 100 or more.
Femtosense's hardware is designed to work seamlessly with popular AI/ML frameworks like PyTorch and TensorFlow. The company's software tools can accept models from these frameworks, making integration straightforward for developers.
Moreover, the low-power requirements make it possible to implement an always-listening mode even when using battery power. This is particularly beneficial for audio applications, enabling always-listening devices to operate with extended battery life and responsiveness at the edge, without reliance on cloud processing.
The topic of adding artificial intelligence acceleration to a host microcontroller is a hot discussion in the industry. Femtosense's SPU is at the forefront of this conversation, offering a viable solution for low-power, always-listening keyword detection using less than 100 mW.
In conclusion, Femtosense's SPU is set to revolutionise low-power AI edge computing, particularly for audio applications. With its focus on sparsity and low-power consumption, the SPU offers a promising solution for always-listening devices, enabling extended battery life and responsiveness at the edge.
The SPU, a component of Femtosense's hardware, aims to revolutionize low-power AI edge computing, especially for audio applications, by reducing energy consumption through a focus on sparsity and operating under minimal power consumption, as low as 1 mW. This cutting-edge technology, such as the SPU, is poised to integrally involve artificial intelligence (AI) and cloud computing in various hardware solutions, like the AI-ADAM100, which incorporates a Cortex-M0+ core and an SPU.
