💎 ONNX

A developer guide to the Open Neural Network Exchange

💎 Part 1: ONNX Origins

  1. Introduction to ONNX and ONNX Runtime – What ONNX is, why it was created, and its significance in ML model deployment.
  2. The Origins of ONNX and Industry Involvement – The companies behind ONNX, how it evolved, and its relationship with Microsoft.

Open Neural Network Exchange (ONNX) is an open-standard model format that aims to make machine learning models more portable, and interoperable across different frameworks. In the first part of this report, we trace the history of ONNX, examining the motivations behind its creation, and highlight the roles played by Microsoft and other industry collaborators.

By detailing how an open specification has accelerated development and adoption, we set the scene to understand why ONNX has become such a prominent standard in the AI community.

💎 Part 2: A Developer Guide to ONNX

  1. How ONNX Runtime Works – Core concepts, including execution providers, inference workflows, and model optimization.
  2. Deployment Considerations and Linking Strategies – Statically vs. dynamically linking ONNX Runtime, implications for different platforms and architectures.
  3. Execution Providers and Performance Trade-offs – Overview of CPU, CUDA, TensorRT, and other execution providers, with a focus on deployment flexibility.
  4. ONNX Runtime in Practice – How libraries and applications integrate ONNX Runtime, challenges, and best practices.

In part two we get into the technical aspects of ONNX Runtime, focusing on how execution providers, linking strategies, and platform compatibility influence real-world deployment. Readers will learn about the process of integrating ONNX Runtime into Python or Rust projects, as well as the considerations that arise when targeting different architectures or hardware accelerators.

We'll cover everything from choosing between CPU or GPU execution providers, to best practices for balancing performance and maintainability. Developers will hopefully come away from reading this with greater clarity on ONNX and the confidence to build on it.

A Deep Research series