Local LLM tools: AI tools compared (6)

What local LLM tools tools are for

Local LLM tools let you download a language model and run it on your own computer. They suit people who want to work offline, experiment with models or keep selected processing on their machine. Depending on the application, you can chat with a model, ask questions about documents or call it from other software. Previously, preparing a summary might have involved copying passages from a folder into a remote service. With a suitable local setup, you select the files, ask your question and review the answer without sending those documents to a remote model. GPT4All, for example, supports conversations without a cloud service and questions about local documents. Results depend on the selected model, available hardware and configuration. You also need to check which surrounding features still connect to external services before treating the whole workflow as offline.

Example uses:

• You summarize local documents and compare the resulting claims with passages actually present in the source files.
• You draft text offline from notes that you want to keep stored on your own computer.
• You test different models on your own questions to compare their French comprehension and recurring mistakes.
• You call a local model from your application to explore a feature without using a remote model API.
• You compare responses to the same request and identify disagreements that need checking against reliable source material.

Our picks by need

These picks follow simple rules applied to the verified data, with no partnership or affiliate link:

• Best scored : LM Studio. 90/100 on our access and transparency grid.
• Cheapest paid option : Msty. First plan at $149 per year.
• To keep data in Europe : PyGPT. Processing on your device.

The main tools at a glance

LM Studio 90/100

LM Studio downloads language models and lets people chat with them on their computer. It serves users who want to query documents offline or serve a model through a local API. Features: Download language models; Chat with a model on your computer; Query documents offline. Entry price: $20 per month, with a free tier. Runs on: Windows, macOS, Linux, and API. Data hosting: Several regions.

PyGPT 73/100

PyGPT lets users chat with local or remote models and query their files on Windows, macOS and Linux. It also serves people creating agents, generating images or videos and using speech features. Features: Chat with local or remote models; Query your files; Create agents that use tools. Free tier available. Runs on: Windows, macOS, and Linux. Data hosting: On device.

Ollama 71/100

Ollama lets people using models download and run them on their computers. It provides an application for conversations, API access, remote hosted models, and support for coding assistants using open models. Features: Download and run models on your computer; Chat with models in an application; Call models through an API. Entry price: $20 per month, with a free tier. Runs on: Windows, macOS, Linux, and API. Data hosting: Several regions.

Jan 68/100

Jan helps people run models on their computers and chat with local or remote models. It offers model downloads, an OpenAI API-compatible server, and connections between assistants and external tools. Features: Run models on your computer; Chat with local or remote models; Download models from a catalog. Free tier available. Runs on: Windows, macOS, Linux, and API. Data hosting: On device.

GPT4All 65/100

GPT4All helps people download and run models on their own computer and chat without a cloud service. It supports querying local documents, adjusting models and exposing a local API server. Features: Download and run models on your computer; Chat without a cloud service; Query local documents. Free tier available. Runs on: Windows, macOS, Linux, and API. Data hosting: On device.

Msty 63/100

Msty lets people using local or remote models chat with them and ask questions about their documents. It also compares responses, organizes conversations and reuses custom instructions. Features: Chat with local or remote models; Ask questions about your documents; Compare multiple responses. Entry price: $149 per year, with a free tier. Runs on: web browser, Windows, macOS, and Linux. Data hosting: On device.

The category in figures

Of the 6 tools in the category, 6 (100%) offer a free tier, 67% have a public API and 3 have a French interface. The median entry price, per month, is $20 (across 3 tools with a published price). 6 disclose where data is hosted, 4 of them in Europe or on device.

Also useful

These tools belong to another category but also cover this need:

• LMQL LMQL lets developers write queries for language models and impose constraints on generated responses. It provi
• Unsloth Unsloth serves people running and adapting models on their computers. It provides visual dataset preparation,
• Continue Continue lets people programming ask questions about project files and edit code from an editor or terminal. I
• HammerAI HammerAI is for people who want to chat with custom characters or write interactive stories. It supports conve
• DiffusionBee DiffusionBee is an open-source macOS tool for people creating and editing images. It generates images from pro
• LLM LLM lets developers query and chat with local or remote models from the terminal. It stores conversations in S
• Falcon Falcon lets users chat with Falcon models to write and summarize texts or ask technical questions. Users can a
• BrainSoup BrainSoup lets Windows users create agents with specialized roles and have them work together. Agents can acce

Choosing well and avoiding pitfalls

Check operating system support, available memory and the storage required for model files. Try a representative task rather than relying on a casual conversation. Compare French quality, document handling and responsiveness on your computer. Review supported model formats, API access and conversation exports. If you plan to question files, inspect how the application reads them and retrieves relevant passages. Separate local features from remote connections. Finally, read model licences and the conditions attached to paid features. Include the hardware resources you may need when comparing the practical cost of different setups.

Running a model locally does not make its answers accurate. It can invent references or misread a document, so check claims against source files. A desktop application may also offer remote models or connected extensions. Inspect settings and network activity when confidentiality matters. Download models from identifiable sources, review their licences and restrict access to folders. Monitor storage and resource use as well. Keep a human review step for consequential work, and do not assume that an offline workflow removes errors in the underlying material.

To compare with the cost of using an AI model directly, see the API price calculator.

Free or paid?

Some applications and models are available without a subscription, subject to their licences. Your computer still needs enough resources, and individual components can have separate conditions. Paying may be useful for an interface, support or additional features. You may also need different hardware if your intended tasks exceed what the current machine can handle comfortably.

Tools in the category with a free tier: LM Studio, PyGPT, Ollama, Jan, GPT4All, and Msty.

For which jobs?

The profiles that use these tools most, based on the use cases documented by publishers: developers, researchers, students, freelancers, and data analysts.