mrkeyoor.com_
Wed 05 Aug 05:05 UTC
npmAI / MLupdated 05 Aug 2026

ollama

ollama is the official JavaScript client for the Ollama server, the tool most people use to run open-weight LLMs locally. It wraps Ollama's REST API in typed methods: chat and generate (with async-generator streaming), embeddings, tool calling, model management (pull, create, delete, list, ps), and newer additions like web search and Ollama's hosted cloud models. The default export talks to localhost:11434; a custom client takes any host, custom headers, or an ollama.com API key. It is a thin transport layer by design: no prompt templates, no retries, no agent framework, just the API with types.

ollama · v0.6.3 · 721k/wk · 2.6 kB min+gz · ★ 4,330 · pushed 18 Feb 2026 · ollama/ollama-js · docs · reviewed by Keyoor

Verdict

The right client if and only if Ollama is your runtime: thin, typed, and matches the server API closely. If there is any chance you will switch model providers, put an abstraction like the Vercel AI SDK in front of it.

API stability3/5Version 0.6.3, so no semver stability promise, and the surface grows whenever the Ollama server adds features (thinking levels, logprobs, image generation params are all recent). Core chat/generate calls have stayed compatible in practice.
Docs3/5The README is the documentation: a complete parameter reference for every method plus an examples folder, which is enough for a thin client, but there is no dedicated docs site, no migration notes, and browser/CORS setup is underdocumented.
Maintenance3/5Official Ollama org project, but the JS client is clearly second priority: last npm publish November 2025, last repo push February 2026, 84 open issues and PRs, and README features can land ahead of releases.
Ecosystem3/5721k weekly downloads and first-party status within the large Ollama ecosystem, but few libraries build on this client directly; most frameworks integrate Ollama through their own adapters instead.

Use it if

  • You run models through an Ollama server (locally or on your own box) and want typed access from Node or the browser instead of hand-rolled fetch calls
  • You are building a privacy-sensitive feature where inference must stay on-device or on-prem
  • You want one client that covers chat, streaming, embeddings, tool calling, and model management against the same server
  • You prototype against local models and occasionally offload to Ollama's cloud models with the same code path
Skip it if

Setup reality

The npm install is trivial (one dependency, whatwg-fetch for older runtimes), but the library is useless without the Ollama server itself: install that separately, pull a model with ollama pull, and make sure the daemon is running on port 11434 before any call works. First model pulls are multi-gigabyte downloads. Browser use needs the ollama/browser import and a server configured to accept cross-origin requests (OLLAMA_ORIGINS). Cloud models add an account plus API key. Errors when the daemon is down are plain fetch failures, so wrap calls if you want friendly messages.

Patterns

Basic chat completionchat-basic

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'llama3.1',
  messages: [{ role: 'user', content: 'Why is the sky blue?' }],
})
console.log(response.message.content)

The default export assumes a local server on 127.0.0.1:11434; a connection refused error means the Ollama daemon is not running.

Stream tokens as they generatechat-streaming

const response = await ollama.chat({
  model: 'llama3.1',
  messages: [{ role: 'user', content: 'Write a haiku about builds' }],
  stream: true,
})
for await (const part of response) {
  process.stdout.write(part.message.content)
}

With stream: true the return type changes to an AsyncGenerator, so the same call site cannot handle both modes untyped.

Point the client at a remote servercustom-host

import { Ollama } from 'ollama'

const client = new Ollama({
  host: 'http://192.168.1.50:11434',
  headers: { Authorization: 'Bearer ' + process.env.OLLAMA_API_KEY },
})
const res = await client.chat({
  model: 'llama3.1',
  messages: [{ role: 'user', content: 'hi' }],
})

Use the named Ollama export for custom clients; headers apply to every request, which is how ollama.com cloud auth works too.

One-shot generation without chat historygenerate-prompt

const res = await ollama.generate({
  model: 'llama3.1',
  prompt: 'Summarize: build tools should be fast.',
  system: 'You answer in one sentence.',
})
console.log(res.response)

generate() returns res.response (a string), not res.message like chat(); mixing the two shapes is a common bug.

Generate embeddings for searchembeddings

const { embeddings } = await ollama.embed({
  model: 'nomic-embed-text',
  input: ['first document', 'second document'],
})
console.log(embeddings.length, embeddings[0].length)

embed() accepts a string or string array and always returns an array of vectors; the older embeddings() call is the legacy path.

Let the model call a functiontool-calling

const res = await ollama.chat({
  model: 'llama3.1',
  messages: [{ role: 'user', content: 'What is 12 * 34?' }],
  tools: [{
    type: 'function',
    function: {
      name: 'multiply',
      description: 'Multiply two numbers',
      parameters: {
        type: 'object',
        properties: { a: { type: 'number' }, b: { type: 'number' } },
        required: ['a', 'b'],
      },
    },
  }],
})
const calls = res.message.tool_calls ?? []

Only some models support tools; unsupported ones just answer in text, so always check tool_calls before assuming a call happened.

Ask about an imagemultimodal-images

import { readFileSync } from 'node:fs'

const res = await ollama.chat({
  model: 'llava',
  messages: [{
    role: 'user',
    content: 'What is in this picture?',
    images: [readFileSync('photo.jpg').toString('base64')],
  }],
})

images accepts base64 strings or Uint8Array; sending a filesystem path does not work through the JS client.

Pull a model with progresspull-model

const progress = await ollama.pull({ model: 'llama3.1', stream: true })
for await (const part of progress) {
  if (part.total) {
    console.log(part.status, Math.round((part.completed / part.total) * 100) + '%')
  }
}

Pulls are gigabytes; without stream: true the promise just hangs silently until the whole download finishes.

List installed and loaded modelslist-models

const installed = await ollama.list()
console.log(installed.models.map((m) => m.name))

const loaded = await ollama.ps()
console.log(loaded.models.map((m) => m.name))

list() shows what is on disk, ps() shows what is in memory right now; they answer different questions.

Cancel in-flight generationsabort-request

const client = new Ollama()
setTimeout(() => client.abort(), 5000)

try {
  const stream = await client.chat({ model: 'llama3.1', messages, stream: true })
  for await (const part of stream) process.stdout.write(part.message.content)
} catch (err) {
  if (err.name === 'AbortError') console.log('\ncancelled')
}

abort() kills every stream on that client instance, so use one client per cancellable request.

Force JSON-formatted responsesjson-output

const res = await ollama.chat({
  model: 'llama3.1',
  messages: [{ role: 'user', content: 'List 3 colors as JSON with a colors array.' }],
  format: 'json',
})
const data = JSON.parse(res.message.content)

format: 'json' guarantees syntax, not schema; also say JSON in the prompt or some models emit whitespace forever.

Control how long the model stays loadedkeep-alive

await ollama.chat({
  model: 'llama3.1',
  messages: [{ role: 'user', content: 'warmup' }],
  keep_alive: '30m',
})

Default unload is a few minutes; keep_alive: -1 pins the model in memory, 0 unloads immediately after the call.

Alternatives

PackageRegistryPick it when
openainpmYou want hosted models, or you point the OpenAI SDK at Ollama's OpenAI-compatible endpoint to keep one client style
ainpmYou want a provider-agnostic layer (Vercel AI SDK) so switching between local and hosted models is a config change
node-llama-cppnpmYou want in-process inference with no separate server daemon to manage