Skip to content
Get packs

JARVIS

Uses ChatGPT as a controller to plan tasks and route them to Hugging Face AI models.

GitHub Download source
Stars
25.5k
Forks
2.3k
Licence
MIT
Language
Python
Last update
1.3 years ago
Type
AI model / agent

No commits in over a year — check for forks or alternatives before adopting.

What it is

JARVIS (also published as HuggingGPT) is a research system where a large language model acts as a controller. It reads a user's request, breaks it into tasks, picks expert AI models from the Hugging Face Hub based on their descriptions, runs them, and then writes a final response combining their outputs. The project is released alongside an arXiv paper and is aimed at researchers and developers exploring how LLMs can coordinate other AI models.

  • Send a multi-step image request, such as combining the pose of one image with the content of another, and see the planned task list
  • Run the command-line mode and ask questions about a set of images to see which models are chosen
  • Use the web API endpoints /tasks or /results to inspect intermediate planning and model-selection output
  • Try the lite configuration, which relies on Hugging Face Inference Endpoints instead of local models

Great for

  • Researchers studying LLM-driven task planning and tool use
  • Developers who want to experiment with chaining Hugging Face models through a language model
  • Users who can supply an OpenAI key and Hugging Face token and want a lightweight lite-mode trial
  • Teams comparing agent designs against the related TaskBench or EasyTool projects

Not ideal for

  • Non-developers looking for a finished consumer app, since setup involves conda, model downloads and API configuration
  • Machines without a large GPU if they want the default local setup, which the README says needs VRAM of at least 24GB and over 284GB of disk
  • Production use cases needing guaranteed uptime, since the README notes the project is being reworked and a new version is planned but not yet released
Step-by-step guide: how to use JARVISInstall, first run, examples, tips and FAQs
JARVIS vs LangChain

How to get it

  1. 1

    Clone and build from source

    You'll need: Conda · Python 3.8 · Git LFS · Node.js and npm (only for the web client) · NVIDIA GPU with CUDA for the default local setup

    Create and activate the conda environment

    cd server && conda create -n jarvis python=3.8 && conda activate jarvis

    Install PyTorch with CUDA 11.7

    conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia

    Install Python requirements

    pip install -r requirements.txt

    Download models (needed for local or hybrid mode)

    cd models && bash download.sh
  2. 2

    Run it

    • Set your OpenAI key and Hugging Face token in server/configs/config.default.yaml or as environment variables
    • Start the model server with python models_server.py --config configs/config.default.yaml
    • Start awesome_chat.py in server mode with python awesome_chat.py --config configs/config.default.yaml --mode server
    • Send a POST request to the /hugginggpt, /tasks or /results endpoint on port 8004
    • Or try the CLI with python awesome_chat.py --config configs/config.default.yaml --mode cli
  3. 3

    Get the most out of it

    • Start with configs/config.lite.yaml to try the system without downloading local expert models
    • Use the /tasks and /results endpoints to inspect each planning and selection stage
    • Run the web client from the web folder with npm run dev for a browser chat interface
    • Switch between OpenAI and Azure OpenAI or GPT-4 as described in the release notes
    • Read the linked TaskBench and EasyTool projects for related evaluation and tool-instruction work

Downloads

Source code (.zip)main branch · straight from GitHub

Strengths

  • Clear research basis with a linked paper and related benchmarks from the same project
  • Offers several run modes: server, web, Gradio, and command line
  • Lite configuration avoids large local model downloads

Trade-offs

  • Default setup is heavy: the README lists 24GB VRAM and hundreds of gigabytes of disk
  • The README notes the project is in a planning and rebuilding phase with a new version to come
  • Requires an OpenAI key and Hugging Face token, and the lite mode depends on remote endpoints

How it compares

ProjectStarsTypeLevelUpdatedBest for
JARVIS (this)25.5kAI model / agentadvanced1.3 years agoResearchers studying LLM-driven task planning and tool use
langchain149kFrameworkintermediate4 days agoPython developers building LLM-powered applications
LaTeX-OCR16.6kAI model / agentintermediate1.8 years agoStudents who want to turn a handwritten or printed formula photo into editable LaTeX
annotated_deep_learning_paper_implementations67.6kLearning resourceintermediate4 days agoStudents learning how transformers, GANs, or optimizers work internally
Made-With-ML49.9kLearning resourceintermediate8 months agoSoftware engineers who want to learn how ML models get deployed and maintained
pytorch-tutorial32.5kLearning resourceintermediate3.2 years agoResearchers or engineers who already know Python and want to learn PyTorch

Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.