ONNX Runtime
A cross-platform library for running and training machine learning models quickly.
- Stars
- 22.2k
- Forks
- 4.4k
- Licence
- MIT
- Language
- C++
- Last update
- 4 days ago
- Type
- Library
What it is
ONNX Runtime is a library that runs machine learning models on many kinds of hardware and operating systems. It supports models built in frameworks such as PyTorch and TensorFlow/Keras, as well as classical libraries such as scikit-learn, LightGBM and XGBoost. According to the README, it aims for faster performance and lower costs by using hardware accelerators and graph optimizations, and it also offers a training feature for transformer models on NVIDIA GPUs.
- Run an exported ONNX model inside a C++, Python or other supported application to make predictions
- Deploy a model from PyTorch, TensorFlow/Keras or scikit-learn in a production service
- Accelerate transformer model training on multi-node NVIDIA GPUs with a one-line addition to a PyTorch script, per the README
Great for
- Developers who want to deploy a trained model inside an application or service
- Teams running the same model across Windows, Linux, macOS, or ARM devices
- Projects that need to speed up inference on available hardware accelerators
- Engineers with existing PyTorch training scripts who want to test the README's training acceleration
Not ideal for
- People who only want to train or experiment with models in a notebook, since the README focuses on production inference and training acceleration rather than beginner workflows
- Readers without some programming background, because the README assumes familiarity with ML frameworks and model formats
How to get it
- 1
Install with a package manager
Clone the repository
git clone https://github.com/microsoft/onnxruntime - 2
Run it
- Read the documentation at onnxruntime.ai/docs to choose a language binding and install it
- Obtain an ONNX model, either exported from your framework or downloaded
- Load the model in an inference session using your chosen language API
- Run the session with your input data and read the output
- 3
Get the most out of it
- Follow the companion inference examples repository linked in the README
- Check the training examples repository if you want to try the training feature
- Review the graph optimization and hardware accelerator options described in the documentation
- Watch the official YouTube tutorials linked in the README
Downloads
Strengths
- Supports many hardware and operating system targets, according to the README
- Works with models from several major frameworks through the ONNX format
- Open source under the MIT license with active development and published releases
Trade-offs
- The README is truncated and gives few concrete setup details, so getting started requires reading the linked documentation
- Performance benefits depend on hardware and model, and the README does not give specific numbers
- Training acceleration as described targets NVIDIA GPUs and transformer models
How it compares
| Project | Stars | Type | Level | Updated | Best for |
|---|---|---|---|---|---|
| onnxruntime (this) | 22.2k | Library | intermediate | 4 days ago | Developers who want to deploy a trained model inside an application or service |
| nni | 14.4k | Library | intermediate | 2.3 years ago | Researchers comparing hyperparameter tuning algorithms such as TPE, Hyperband, or Bayesian methods |
| wandb | 11.3k | Library | beginner | 4 days ago | Machine learning engineers who want to compare training runs over time |
| petals | 10.7k | Library | intermediate | 2.2 years ago | Developers who want to experiment with large open models like Llama 3.1 or BLOOM without owning the hardware to run them alone |
| annotated_deep_learning_paper_implementations | 67.6k | Learning resource | intermediate | 4 days ago | Students learning how transformers, GANs, or optimizers work internally |
| netron | 33.6k | App | beginner | 4 days ago | Engineers who need to check a model's layers and tensor connections before deploying it |
Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.