Skip to main content
Preparing workspace

Local AI Chat (WebGPU)

Chat with Llama 3.2, Qwen 2.5, Gemma 2 or DeepSeek-R1 Distill on your own GPU through WebGPU. The model downloads once; prompts stay on your device.

Still loading. If the tool doesn’t appear, reload this page.

About this tool

Local AI Chat: Open Language Models on Your GPU

Chat with open-weight language models such as Llama 3.2, Qwen 2.5, Gemma 2 and DeepSeek-R1 Distill running on your own GPU in the browser with WebGPU. The model downloads once; prompts stay on your device.

Local AI Chat runs open-weight language models inside your browser tab with WebLLM and WebGPU, so the replies are computed by your own graphics card instead of a cloud service. Pick a model, let its files download once (the weights from huggingface.co, from about 200 MB for SmolLM2 360M to about 4 GB for the 7B models, and a compiled model library from raw.githubusercontent.com), and chat. Your prompts and replies are not sent anywhere; the cached model loads from disk on later visits. Small models answer quickly but make more mistakes than large hosted chatbots.

How to use Local AI Chat (WebGPU)

  1. 1

    Open the model picker at the top of the chat and choose a model; the badge shows its approximate download size, or a green dot when it is already cached.

  2. 2

    Press Load followed by the model name and wait while the files download and compile; the progress bar shows each stage.

  3. 3

    Type in the Message box and press Enter to send (Shift+Enter adds a new line); use Stop generation to cut a reply short.

  4. 4

    Open Settings & System Prompt to change the system prompt, temperature or response length, and Export chat as Markdown to save the conversation.

Why use Local AI Chat (WebGPU)

  • Inference on your device

    WebLLM runs the model with WebGPU inside the page. Prompts, replies and chat history never leave the browser; only the model files are downloaded.

  • Nine open models to pick from

    From SmolLM2 360M and Qwen 2.5 0.5B up to Mistral 7B and DeepSeek-R1 Distill Qwen 7B, with the GPU memory each one needs shown before you load it.

  • Downloads once, then cached

    Model files stay in the browser's Cache Storage, so a second visit loads from disk. The model picker can delete cached weights to free space.

Common use cases

  • Drafting or rewording private text you would rather not paste into a hosted chatbot
  • Trying small open models such as Llama 3.2 1B, Qwen 2.5 Coder or Gemma 2 2B side by side on your own hardware
  • Asking quick coding or general questions on a laptop with a capable GPU, with no account or API key

Frequently asked questions

Does anything I type get sent to a server?

No. The model runs in your browser with WebGPU, so prompts and replies are computed on your device. The page downloads the model weights from Hugging Face and the compiled model library from GitHub, but nothing you type is included in those requests.

Which browsers and devices work?

You need a browser with WebGPU turned on: recent Chrome or Edge on Windows, macOS and ChromeOS, Safari 26 or newer on Macs, iPhones and iPads, or Chrome on many Android phones. Firefox support is limited. The smallest models need well under 1 GB of GPU memory; the 7B models need about 5 GB.

How big are the downloads?

From roughly 200 MB for SmolLM2 360M to roughly 4 GB for Mistral 7B and DeepSeek-R1 Distill Qwen 7B. The sizes shown are approximate. After the first download the files are cached by the browser and load from disk.

How good are the answers?

These are small open models compressed to 4 bits so they fit in a browser. They are useful for drafting, summarising and simple coding questions, but they make more factual mistakes than large hosted chatbots. Check anything important.

Why is it slow or failing to load?

Speed depends on your GPU. Integrated graphics run the 1B to 3B models best; 7B models need a dedicated GPU or a recent Apple Silicon Mac. If loading fails, pick a smaller model, close other GPU-heavy tabs, or check that hardware acceleration is on in your browser settings.

How do I free the storage again?

Open the model picker and use the delete button next to a cached model, use Purge Cache in Settings to delete every downloaded model (unfinished downloads included), or clear this site's data in your browser settings.