Running Agentic AI Offline on your Linux Machine
Tutorial Room A | Wed 21 Jan 1:30 p.m.–3:10 p.m.
Presented by
-
Sophie is an avid open-source user and current treasurer of Linux Victoria. (more details to be added soon)
Sophie is an avid open-source user and current treasurer of Linux Victoria. (more details to be added soon)
Abstract
Most people assume they need cloud APIs or powerful servers to run large language models. The good news: you can run a capable agent entirely offline, on your own Linux laptop, using open-source tools.
In this hands-on tutorial, we’ll begin by installing a small but powerful open-source model (such as Mistral or Llama) using Ollama. Once your local LLM is up and running, we’ll wrap it with JanAI to give it agentic capabilities. Finally, we’ll extend the agent with MCP tools so it can interact with your Linux environment.
By the end of the session, you will:
- Understand what “agentic AI” means and how it differs from simple prompting.
- Have a working local agent that runs completely offline.
- Know how to install, configure, and swap between open-source models.
- See how to extend your agent with new MCP tools.
This session is beginner-friendly, no prior AI experience is required, only comfort with using the Linux terminal. Attendees will follow along on their own laptops.
Come learn how to take control of AI on your own hardware, with open tools, and no cloud dependency.
Most people assume they need cloud APIs or powerful servers to run large language models. The good news: you can run a capable agent entirely offline, on your own Linux laptop, using open-source tools.
In this hands-on tutorial, we’ll begin by installing a small but powerful open-source model (such as Mistral or Llama) using Ollama. Once your local LLM is up and running, we’ll wrap it with JanAI to give it agentic capabilities. Finally, we’ll extend the agent with MCP tools so it can interact with your Linux environment. By the end of the session, you will: - Understand what “agentic AI” means and how it differs from simple prompting. - Have a working local agent that runs completely offline. - Know how to install, configure, and swap between open-source models. - See how to extend your agent with new MCP tools.
This session is beginner-friendly, no prior AI experience is required, only comfort with using the Linux terminal. Attendees will follow along on their own laptops. Come learn how to take control of AI on your own hardware, with open tools, and no cloud dependency.