Edge Impulse Launches Open Source ESP-EYE Firmware and Announces Official Espressif ESP32 Support

With modifications, the first firmware gives a 3-4x speedup over vanilla TensorFlow Lite for Microcontrollers, with more to come.

Edge machine learning specialist Edge Impulse has announced official Edge Impulse Studio support for the Espressif ESP32 family, with an emphasis on the ESP-EYE artificially intelligent camera board.

"We chose the ESP-EYE as a reference for Edge Impulse Studio integration since it comes with an OV2640 camera, microphone, 8MB of PSRAM, and 4MB of flash memory, allowing us to construct speech, vision, and other sensor data-enabled machine learning applications," says Dmitry Maslov of Edge Impulse. "However, both pre-built firmware and source code for connecting additional ESP32-based boards to the Edge Impulse ecosystem might be valuable."

The board's support includes the release of an open source Edge Impulse-specific firmware for the ESP-EYE, which includes data collection and inference capabilities for the on-board microphone and camera — but can easily be expanded to other ESP32-based boards, including the ESP-CAM, according to Maslov.

"Not only does the ESP32 constitute a new milestone for Edge Impulse in terms of community appeal, but it is also the first significant non-Arm processor board to be supported," Maslov notes. "Working on firmware for the ESP32 has presented some unique issues – the CMSIS-NN and CMSIS-DSP optimizations could not have been employed for the Tensilica Xtensa LX6 CPU."

Edge Impulse on ESP32 devices, on the other hand, makes advantage of Espressif's ESP-NN neural network acceleration technology, which provides a 3-4x performance gain over vanilla TensorFlow Lite for Microcontrollers. "This allows us to perform our keyword detecting project in real-time with excellent accuracy," Maslov explains.

Support for Omnivision OV2640, OV3660, and OV5640 image sensors, the ESP-on-board EYE's I2S microphone, an STMicroelectronics LIS3DHTR accelerometer linked to the I2C bus, and "any analogue sensor" attached to the ESP32's A0 pin are all included in the first firmware version.

Although no plans for further sensor support have been announced, Maslov has stated that ESP-DSP will be included in the next firmware version to speed up spectral analysis, MFCC, and MFE digital signal processing blocks in Edge Impulse Studio applications.

The Edge Impulse blog has further details, and anyone interested in trying out the updated firmware may access it on the company's GitHub site under a permissive open source licence.