Definition:
Ollama is an open-source platform for using generative artificial intelligence models from applications, agents and development tools. It can download, run and manage models on a person’s own computer and also provide access to cloud-hosted models through the same environment.
Ollama is not a particular model or chatbot. It is an execution and integration layer that provides a desktop application, command-line interface, API and model library. Privacy, cost and technical requirements depend on whether the model runs locally or through a remote service.
Table of contents
Ollama history and development
Ollama appeared in 2023 with a focus on simplifying the local execution of open models. Its repository acknowledges the contribution of projects such as llama.cpp, which support inference with optimised models on different types of hardware.
The platform has developed from local execution into an environment that also connects applications, coding agents and cloud models. Available functions and data flows therefore need to be checked for each configuration instead of assuming that every use of Ollama is necessarily local.
Main Ollama features
Particular functions change with versions and the selected model. Main features include:
- Model management: Models can be downloaded, run, updated and removed through the application or terminal commands.
- Local execution: Compatible models can be processed on the device, subject to its available memory, storage, CPU and GPU.
- Cloud models: The platform can also provide access to hosted models that do not run on the device.
- Model library: It includes options for conversation, coding, reasoning, vision and embedding generation.
- Modelfile: It can define a derived model, parameters, system messages, templates and compatible adapters.
- API: Ollama provides its own API and compatibility with certain clients that use OpenAI or Anthropic interfaces.
- Libraries and integrations: It provides Python and JavaScript libraries and can connect with compatible applications, editors and agents.
- Model import: It supports certain formats and quantised variants according to the documentation and engine capabilities.
Support for a function does not mean that every model provides it. Context, tools, vision, embeddings and other behaviours depend on the model and integration being used.
Ollama advantages and limitations
Ollama can simplify local control, prototyping and model comparison without requiring an inference infrastructure to be built from scratch. A model that has already been downloaded can run without a connection, although installation, downloads, cloud models and some services require a network. The ability to change models within a similar workflow also helps compare capabilities and requirements.
These advantages have limits. Large models may require substantial memory, storage and computing capacity, and speed varies with hardware and quantisation. Running a model through Ollama does not remove errors, hallucinations, bias or risks arising from untrusted instructions or data. Exposing the API to a network or connecting tools that can act on files and systems requires authentication, isolation and additional controls. Local execution uses the owner’s resources, while hosted services may be subject to current pricing, quotas and limits.
Local does not automatically mean private or secure. On-device execution can keep inputs and outputs on the computer when no external services or integrations are involved, but logs, telemetry, permissions and every external service in the workflow need to be reviewed.
Common Ollama use cases
The platform can act as a technical component in applications that provide generation, classification, extraction or conversation through an API. It can also connect with coding agents that explain, generate or review code, and with internal assistants that retrieve information or execute workflows under defined controls.
Other uses include generating embeddings and answers supported by sources retrieved through RAG, comparing models, parameters and prompts, and running locally in environments with limited connectivity or requirements governing where information is processed.
In every case, Ollama provides access to the model but does not replace the evaluation, observability, security or data governance required by a production application.
Best practices for using Ollama
The following aspects should be reviewed before Ollama is incorporated into a project:
- Choose an appropriate model: Compare its licence, size, language, context, capabilities and resource requirements.
- Distinguish local and cloud use: Document which model is used, where it runs and which data leave the device.
- Protect the API: Avoid exposing the service directly to the Internet and apply authentication, segmentation and limits when it is accessible over a network.
- Manage resources: Monitor memory, disk space, concurrency, loading times and models that remain active.
- Evaluate outputs: Test the system with representative cases and review accuracy, safety and behaviour before decisions are automated.
- Maintain versions: Record Ollama, model, parameter and Modelfile versions so that changes can be reproduced.
- Review licences: Check model and data conditions, particularly before commercial use or redistribution.
The official Ollama documentation contains current instructions for installation, models, the API and integrations.