The elastic AI ecosystem

hanging light bulbs

AI is everywhere

AI algorithms can now be found in a broad range of technical environments, from the cloud to embedded systems. Our contributiuon to that is an open-source ecosystem for adaptive AI that will enable everyone to quickly experiment with their own distributed and embedded intelligent systems.

Grafik des elasticAI Ökosystems, inklusive elasticAI.explorer, elasticAI.creator, elasticAI.runtime und elasticAI.hardware

Anyone can experiment with AI

Admittedly, artificial intelligence (AI) is not a particularly simple subject. But with a little practice, it is possible to create and train the first AI model in pyTorch, for instance. But what comes next? How do you integrate this AI into your own system? How do you run it on a small embedded system? How do you link it to incoming data and send the results to a consumer? Who will design the necessary embedded hardware? And how do you program an intelligent application that can dynamically switch between AI in the cloud and on-device AI?

For all of this, we are developing an open-source AI ecosystem we call elastic AI. It consists of four parts:

  1. the elastic AI.creator, for automatically generating optimised AI accelerators for embedded FPGAs,
  2. the elastic AI.runtime, for running distributed AI systems; connecting AI in the cloud with edge and embedded AI, 
  3. the elastic AI.hardware, for developing your own intelligent end devices, and
  4. the elastic AI.explorer, for automated development, optimization, and hardware-adaptive deployment of AI models.

The elastic AI Ecosystem

An open source solution for adaptive and distributed AI

Creator

​

​​​​​Elastic AI.creator (Schema)
​​
​With the elastic AI.creator it is possible to design AI models especially optimised for FPGAs, to train them and to translate them into a hardware description.

First of all, it provides custom optimised layers from which a suitable model is assembled.

The finished model then undergoes its training and is then translated into a VHDL representation of the corresponding FPGA target configuration.

These VHDL files can be compiled using third-party software such as Vivado Xilinx for the Elastic Node or other devices.

The Creator is based on the work in the LUTNet project. The source code and information on how to use it can be found on the associated Github page.

Contact: Lukas Einhaus, M.Sc.

Runtime

​​Schema elastic AI.runtime
​The elastic AI.runtime is a continuation of the Elastic IoT Platform and in a sense represents the infrastructure of the ecosystem. It is a platform for controlling, managing and monitoring distributed AI systems. Its structure consists of an embedded runtime, a cloud runtime and the monitor, a cloud application for supervisory purposes.

The embedded runtime is used on the respective end devices of the distributed embedded systems and acts as an interface between the server and the respective devices. It takes care of general control and communication, resource management as well as the selection of FPGA configurations and – by means of the Elastic Node Middleware – their deployment and their operation.

The individual end devices have their counterparts on the server side in the form of Digital Twins. These simultaneously serve as a model representation of the physical network elements, as load balancers and as intermediaries to them. The Digital Twins also provide abstractions that allow greater flexibility in dealing with concrete hardware which is dependent on specific protocols and structures.

Finally, the Monitor is designed to oversee the entire assemblage and, with the help of the Digital Twins, creates an overview of the distributed system and the status of its elements.

Full source code and technical details on Github:

Contact: David Federl, M.Sc.

Hardware

​Elastic Node mit Beschreibung

​The accompanying hardware is the fifth iteration of our own Elastic Node.

It is a small, very energy-efficient endpoint device for distributed sensor networks that optionally connects to cloud or edge AI through its wireless module, or relies on on-device AI.

Since its initial release, the hardware has been steadily refined. The Elastic Node v5 is now less than 5.8 cm high and only 3.4 cm wide, making it suitable for very space-constrained deployments.

Its heterogeneous design essentially combines a power-saving microcontroller (MCU), a cache for data and an embedded FPGA which is primarily intended for AI applications, together with memory for additional AI models.

The microcontroller takes over comparatively simple tasks such as general controlling and reading out of sensor data, while the FPGA takes care of more specific tasks for which suitable neural networks were previously assembled by means of the Creator.

Thanks to its model storage, the Elastic Node can dynamically reconfigure itself in situ and thus always have the appropriate neural network ready for varying requirements.

Other features include a power consumption monitor and the possibility of expanding functionality by adding extension boards.

Contact: Dr.-Ing. Andreas Erbslöh

Explorer

​

​​ElasticAI.explorer is the central component for the automated development, optimization, and hardware-adaptive deployment of AI models. The elasticAI.explorer combines hardware-sensitive neural architecture search (HW-NAS) with hardware-specific generation and deployment processes (Generator), thereby enabling the efficient transfer of AI models to embedded target systems.

The integrated NAS engine systematically searches for model architectures that meet the requirements of the machine learning task and the physical constraints of the target hardware. To achieve this, the engine combines configurable search spaces, multi-criteria optimization methods, and flexible sets of criteria for accuracy, latency, and resource consumption. Currently, classic search strategies, such as random, grid and evolutionary search are supported. Extensions such as quantization-aware NAS or generative search approaches are under development.

Through the generator pipeline—consisting of the ModelBuilder, ModelCompiler, Host, and HardwareManager—the elasticAI.explorer automatically produces hardware-optimized models and converts them into deployable artifacts. The integrated deployment system enables reproducible cross-compilation processes, transfer to target systems, and automated evaluation of runtime metrics directly on the target device. Deployment is currently supported for Raspberry Pi 4/5 and the Pico microcontroller, with additional platforms under development. Users can also define their own custom generators.

Additional components include a Reflection API that manages and validates supported layer types and quantization schemes, as well as a Knowledge Repository for the structured storage of architectures, models, and measurement data, serving as a basis for training future AI-based hardware cost estimators. Both components are under active development.

Further resources can be found on GitHub.

Contact: Robin Feldmann, M.Sc.

End-To-End Data Processing

​

​​DeNSPP.offline (denspp.offline) is an open-source Python framework for analyzing transient offline measurement data and exploring complete signal processing pipelines. The focus is on a seamless workflow—from raw data handling and preprocessing, including configuration, to training and evaluating deep neural networks—with a view to later implementation on embedded and hardware target platforms. The goal is to consolidate the entire processing chain, from raw data to model training and evaluation, into a reproducible end-to-end workflow. For creating digital hardware accelerators (e.g., for microcontrollers and FPGAs/ASICs via Verilog/VHDL), the framework can be combined with elasticAI.creator.

Further resources can be found on GitHub.

Contact: Dr.-Ing. Andreas Erbslöh, Florian Hettstedt, M.Sc.

Publications related to the ecosystem

2026

  • Ling, Tianheng; Qian, Chao; Schiele, Gregor
    Resource-Aware Mixed-Precision Quantization for Enhancing Deployability of Transformers for Time-Series Forecasting on Embedded FPGAs
    In: Mobile and Ubiquitous Systems: Computing, Networking and Services; Proceedings / 21st EAI International Conference (MobiQuitous 2024); Oslo, Norway; November 12–14, 2024 / Soylu, Ahmet; Liu, Fan; Mitra, Karan; Zhang, Yan; Grønli, Tor-Morten (Hrsg.). Cham: Springer Nature Switzerland, 2026, S. 361 – 381
    DOI

2025

  • Ling, Tianheng; Qian, Chao; Klann, Theodor Mario; Hoever, Julian; Einhaus, Lukas; Schiele, Gregor
    Configurable Multi-Layer Perceptron-Based Soft Sensors on Embedded Field Programmable Gate Arrays: Targeting Diverse Deployment Goals in Fluid Flow Estimation
    In: Sensors, Jg. 25, 2025, Nr. 1, 83
    DOI
  • Ling, Tianheng; Qian, Chao; Hasler, Lukas Johannes; Schiele, Gregor
    Automating Versatile Time-Series Analysis with Tiny Transformers on Embedded FPGAs
    In: IEEE Computer Society Annual Symposium on VLSI: ISVLSI 2025 / ISVLSI 2025: 28th IEEE Computer Society Annual Symposium on VLSI, 6-9 July 2025, Greece / Institute of Electrical and Electronics Engineers (IEEE) (Hrsg.). Piscataway: Institute of Electrical and Electronics Engineers (IEEE), 2025
    DOI
  • Ling, Tianheng; Singh, Vipin; Qian, Chao; Biessmann, Felix; Schiele, Gregor
    Automated Energy-Aware Time-Series Model Deployment on Embedded FPGAs for Resilient Combined Sewer Overflow Management
    In: Proceedings of the IEEE International Smart Cities Conference (ISC2 2025) / International Smart Cities Conference (ISC2 2025); 06-09 October 2025; Patras, Greece. Piscataway: Institute of Electrical and Electronics Engineers (IEEE), 2025
    DOI
  • Qian, Chao
    Leveraging Application-Specific Knowledge for Energy-Efficient Deep Learning Accelerators on Resource-Constrained FPGAs
    In: Architecture of Computing Systems: Proceedings / 38th International Conference (ARCS 2025); April 22–24, 2025; Kiel, Germany / Tomforde, Sven; Krupitzer, Christian; Vialle, Stéphane; Suarez, Estela; Pionteck, Thilo (Hrsg.). Cham: Springer, 2025, S. 459 – 468
    DOI

2024

  • Ling, Tianheng; Qian, Chao; Schiele, Gregor
    On-Device Soft Sensors: Real-Time Fluid Flow Estimation from Level Sensor Data
    In: Mobile and Ubiquitous Systems: Computing, Networking and Services; Proceedings, Part II / 20th EAI International Conference, MobiQuitous 2023, November 14–17, 2023, Melbourne, Australia / Zaslavsky, Arkady; Ning, Zhaolong; Kalogeraki, Vana; Georgakopoulos, Dimitrios; Chrysanthis, Panos K. (Hrsg.). Cham: Springer Nature, 2024, S. 529 – 537
    DOI
  • Qian, Chao; Ling, Tianheng; Schiele, Gregor
    Exploring energy efficiency of LSTM accelerators: A parameterized architecture design for embedded FPGAs
    In: Journal of Systems Architecture, Jg. 152, 2024, 103181
    DOI
  • Ling, Tianheng; Qian, Chao; Schiele, Gregor
    Towards Auto-Building of Embedded FPGA-based Soft Sensors for Wastewater Flow Estimation
    In: 2024 IEEE Annual Congress on Artificial Intelligence of Things (AIoT): Proceedings / IEEE Annual Congress on Artificial Intelligence of Things (IEEE AIoT), 24-26 July 2024, Melbourne / IEEE (Hrsg.). New York: IEEE, 2024, S. 248 – 249
    DOI

2023 

  • Burger, Alwyn
    FPGA-Augmented Intelligent Devices for the Internet of Things
    Duisburg, Essen, 2023
    DOI
  • Qian, Chao; Ling, Tianheng; Schiele, Gregor
    Energy Efficient LSTM Accelerators for Embedded FPGAs Through Parameterised Architecture Design
    In: Architecture of Computing Systems: 36th International Conference, ARCS 2023, Athens, Greece, June 13–15, 2023, Proceedings / 36th International Conference, ARCS 2023, Athens, Greece, June 13–15, 2023 / Goumas, Georgios; Tomforde, Sven; Brehm, Jürgen; Wildermann, Stefan; Pionteck, Thilo (Hrsg.). Cham: Springer, 2023, S. 3 – 17
    DOI
  • Ling, Tianheng; Schiele, Gregor
    On-Device AI: Quantization-Aware Training of Transformers in Time-Series
    In: Proceedings of the 2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops 2023) / International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops); 13-17 March 2023; Atlanta, USA. Piscataway: Institute of Electrical and Electronics Engineers (IEEE), 2023, S. 235 – 236
    DOI
  • Buron, Leo; Erbslöh, Andreas; Seidl, Karsten; Schiele, Gregor; Seidl, Karsten; Ur-Rehman, Zia; Klaes, Christian
    Deep.Neural.Signal.Pre-Processor-Towards Development of AI-enhanced End-To-End BCIsIn: Current Directions in Biomedical Engineering, Jg. 9, 2023, Nr. 1, S. 471 – 474
    DOI
  • Qian, Chao; Ling, Tianheng; Schiele, Gregor
    Enhancing Energy-Efficiency by Solving the Throughput Bottleneck of LSTM Cells for Embedded FPGAs
    In: Machine Learning and Principles and Practice of Knowledge Discovery in Databases: International Workshops of ECML PKDD 2022, Grenoble, France, September 19–23, 2022, Proceedings, Part I / International Workshops of ECML PKDD 2022, Grenoble, France, September 19–23, 2022 / Koprinska, Irena; Mignone, Paolo; Guidotti, Riccardo (Hrsg.). Cham: Springer, 2023, S. 594 – 605
    DOI
  • Qian, Chao; Ling, Tianheng; Schiele, Gregor
    ElasticAI: Creating and Deploying Energy-Efficient Deep Learning Accelerator for Pervasive Computing
    In: 2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops) / PerCom 2023, 13-17 March 2023, Atlanta, GA, USA. Piscataway: IEEE, 2023, S. 297 – 299
    DOI

2022

  • Qian, Chao; Einhaus, Lukas; Schiele, Gregor
    ElasticAI-Creator: Optimizing Neural Networks for Time-Series-Analysis for On-Device Machine Learning in IoT Systems
    In: Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems (SenSys 2022) / 20th ACM Conference on Embedded Networked Sensor Systems (SenSys 2022): 6 - 9 November 2022; Boston, USA / Gummeson, Jeremy; Lee, Sunghoon Ivan (Hrsg.). New York: Association for Computing Machinery (ACM), 2022, S. 941 – 946
    DOI

2021

  • Cichiwskyj, Christopher; Schmeißer, Stephan; Qian, Chao; Einhaus, Lukas; Ringhofer, Christopher; Schiele, Gregor
    Elastic AI : System support for adaptive machine learning in pervasive computing systems
    In: CCF Transactions on Pervasive Computing and Interaction, Jg. 3, 2021, Nr. 3, S. 300 – 328
    DOI
  • Einhaus, Lukas; Qian, Chao; Ringhofer, Christopher; Schiele, Gregor
    In-Situ Artificial Intelligence for Self-∗ Devices: The Elastic AI Ecosystem (Tutorial)
    In: 2nd IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion, ACSOS-C 2021: Proceedings / 2nd IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion, ACSOS-C 2021, Virtual, Washington, 27 September - 1 October 2021. Piscataway: IEEE, 2021, S. 320 – 321
    DOI
  • Burger, Alwyn; Schiele, Gregor; King, David W.
    Developing Action Policies with Q-Learning and Shallow Neural Networks on Reconfigurable Embedded Devices
    In: ACM Transactions on Autonomous and Adaptive Systems (TAAS), Jg. 15, 2021, Nr. 4, 14
    DOI

2020

  • Burger, Alwyn Johannes; Cichiwskyj, Christopher; Schmeißer, Stephan; Schiele, Gregor
    The Elastic Internet of Things - A Platform for Self-Integrating and Self-Adaptive IoT-Systems with Support for Embedded Adaptive Hardware
    In: Future Generation Computer Systems, Jg. 113, 2020, S. 607 – 619
    DOI

2019

  • Schiele, Gregor; Burger, Alwyn Johannes; Cichiwskyj, Christopher
    The Elastic Node: An Experimentation Platform for Hardware Accelerator Research in the Internet of Things
    In: Proceedings of the IEEE International Conference on Autonomic Computing / ICAC 2019; Umea; Sweden; 16 - 20 June 2019. Piscataway Township: IEEE, 2019, S. 84 – 94
    DOI

2018

  • Burger, Alwyn Johannes; Schiele, Gregor
    Demo Abstract: Deep Learning on an Elastic Node for the Internet of Things
    In: IEEE International Conference on Pervasive Computing and Communications Workshops / PerCom Workshops 2018; Athens; Greece; 19-23 March 2018: Institute of Electrical and Electronics Engineers Inc., 2018, S. 424 – 426
    DOI

2017

  • Burger, Alwyn Johannes; Cichiwskyj, Christopher; Schiele, Gregor
    Elastic Nodes for the Internet of Things: A Middleware-Based Approach
    In: Proceedings - 2017 IEEE International Conference on Autonomic Computing, ICAC 2017 / 14th IEEE International Conference on Autonomic Computing, ICAC 2017; The Ohio State University Columbus; United States; 17 July 2017 through 21 July 2017 / Wang, Xiaorui (Hrsg.). Piscataway, NJ: Institute of Electrical and Electronics Engineers Inc., 2017, S. 73 – 74
    DOI