From the box to your first prompt

Set up local AI on your PC

A safe, understandable starting path for Windows and Linux: verify the system, choose a runtime, load a suitable model, then optimize.

The practical path

Six steps to a local LLM

The interface is personal preference. Drivers, memory headroom and a compatible model format are the technical foundations.

  1. 01

    Update firmware and drivers

    Install operating-system updates plus current graphics and chipset drivers from official sources. On unified-memory systems, also check BIOS/UEFI and available UMA allocation.

  2. 02

    Check memory and free SSD space

    Keep useful capacity free beyond the model itself. Several quantizations of a large model can quickly consume hundreds of gigabytes.

  3. 03

    Choose a runtime

    Ollama is convenient for a local service and command-line workflows. LM Studio provides a graphical interface. llama.cpp gives experienced users extensive control.

  4. 04

    Start small

    Begin with a well-documented 7B or 8B instruct model from your runtime's catalog. Verify drivers, GPU offload and operation before starting very large downloads.

  5. 05

    Watch utilization

    Monitor GPU/UMA memory, system RAM, temperatures, fan behavior and time to first token. If heavy offloading occurs, a smaller or more strongly quantized model is often better.

  6. 06

    Secure access

    Initially bind a local API to 127.0.0.1 only. Do not expose it to your LAN or the internet without review. Sensitive documents stay local only if plugins, web search and telemetry are configured accordingly.

Choose your tool

Which runtime fits you?

Ollama

A simple local model service and useful foundation for apps and automation. Models are managed through the runtime.

Good for:beginners, developers, local APIsOfficial documentation

LM Studio

Graphical model discovery, chat interface and local server in a desktop application.

Good for:visual setup, comparing modelsOfficial documentation

llama.cpp

A lightweight open-source backend with extensive controls for offloading, quantization and platforms.

Good for:control, experiments, custom integrationsProject page

Windows

Start native, add WSL when needed

Native installation is usually enough for Ollama and LM Studio. WSL2 helps when your development stack expects Linux. Avoid parallel services consuming the same GPU memory.

  • current GPU driver
  • check performance mode under sustained load
  • configure local-server autostart deliberately

Linux

Flexible for services and development

Linux works well for headless operation and reproducible environments. Check which kernel, GPU driver and backend are officially supported for your hardware.

  • install runtimes only from trusted sources
  • do not bind the API publicly
  • keep models on a sufficiently large partition

Private does not automatically mean secure

Four security rules

Check offline behavior

Verify whether your chosen application uses online features or telemetry.

Limit the API

Bind locally, or protect it with authentication and a firewall.

Read the license

Model licenses can impose different rules on use and redistribution.

Monitor sustained load

Watch temperatures, dust, power consumption and noise.

Hardware still undecided?

Choose the right memory class first

The finder translates your desired model into a realistic system direction.

Open the PC Finder