te-comp: A Complete MLIR Compiler Generating Self-Contained C Inference Engines for MCUs
te-comp: A Complete MLIR Compiler Generating Self-Contained C Inference Engines for MCUs
This research presents te-comp, the first complete MLIR-based compiler that converts standard model formats (.tflite or .safetensors) into self-contained C code for microcontrollers. The system eliminates runtime dependencies and framework links by generating inference engines, kernels, memory layouts, and RTOS integration layers. Benchmarks on 10 MCUNet CNN models show te-comp reduces SRAM usage by 38% (up to 5.9x) and Flash by 15% compared to MCUNet TinyEngine. The authors demonstrate deployment of an 8.3M-parameter GPT-Neo transformer on a single-core Cortex-M7 at 160MHz, achieving ~1 token/s text generation with 249KB SRAM and 3.62MB Flash using Q4_0 quantization, int4 KV cache, and vocabulary pruning—capabilities not supported by existing MCU frameworks.
The paper introduces te-comp, an MLIR-based compiler that generates self-contained C inference engines for microcontrollers from standard model formats, achieving significant reductions in SRAM and Flash usage on MCU benchmarks while enabling deployment of transformer models like GPT-Neo on Cortex-M7 hardware.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
CNN benchmarks, te-comp outperforms MCUNet TinyEngine by using 38% less SRAM (up to 5.9$\times$) and 15% less Flash, while generating complete C code rather than linking a pre-written kernel library. To demonstrate, we use te-comp to compile an 8.3M-parameter GPT-Neo transformer (TinyStories-v2) and deploy it on a single-core Cortex-M7 at 160MHz. The full firmware uses 249KB of 512KB SRAM and 3.62MB of 4MB Flash and
Open source S4
achieves approximately 1 token/s interactive text generation via Q4_0 weight quantization, a stateful int4 KV cache, and vocabulary pruning -- a capability no existing MCU inference framework supports.
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org