Ollama connector quickstart

Give your Sprites the full power of Ollama with the Ollama Connector.

Ollama started off as the easiest way to run Llama, Gemma, Qwen and friends on your own machine. Now it's way more than that. Ollama Cloud serves a broad catalog of models, including ones far too large to run locally, over the same API. One key connects you to the models available to your account.

The Ollama Connector brings Ollama to your Sprites. Configure the connector once, then grant access to any or all of your Sprites. Your Ollama key is encrypted at rest, never returned by the connector API, and never reaches a Sprite. Your Sprites only need a connection ID and a URL.

Both of Ollama's API surfaces work through one connector: the native /api/* routes and the OpenAI-compatible /v1/* routes.

Set up the connector Read the docs

Setup

Add your key. Choose your Sprites. Make a call.

Before you start, you'll need an API key from an existing ollama.com account and at least one Sprite.

Where the key lives

  1. Your Sprite

    Sends a request using the connection ID. No Ollama key inside.

  2. Sprites gateway

    Checks the Sprite's access and attaches your stored key.

  3. Ollama Cloud

    Runs the model. The response comes back through the gateway.

  1. 1

    Add the connector.

    Go to the Fly.io dashboard, select your organization, then Sprites in the sidebar. In the Connectors tab, click Add connector and choose Ollama.

    Paste your key and click Connect. Fly.io checks it against ollama.com before saving, so an invalid key fails here rather than on your first call.

    If Ollama isn't listed, it isn't available for your organization yet.

  2. 2

    Grant Sprite access.

    A new connector denies every Sprite. Until you grant access, calls from your Sprite return 403.

    Open the connector and use the Access Configuration card. Grant access by Sprite Labels (the Sprite must carry all the labels you list), by Name Prefix, or with Allow all sprites. If you combine labels and a prefix, both must match. The Authorized Sprites list updates as you edit so you can review who gets in. Click Save.

  3. 3

    Test reachability.

    Each row in Authorized Sprites has a Test reachability button. It makes a real request from that Sprite to Ollama's model catalog through the gateway. This checks connectivity, not credentials or model access. Make an inference call next to check those.

  4. 4

    Call it from a Sprite.

    The connector page provides examples with your connection ID filled in. For this example, replace YOUR_CONNECTION_ID and choose an Ollama Cloud model available to your account.

    Shell 
    # Run inside an authorized Sprite. Replace both placeholders.
curl https://api.sprites.dev/v1/gateway/ollama/YOUR_CONNECTION_ID/api/chat \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "YOUR_OLLAMA_CLOUD_MODEL",
    "messages": [{"role": "user", "content": "Hello from my Sprite!"}],
    "stream": false
  }'

    There's no Authorization header by design. The gateway knows which Sprite is calling and attaches the key itself. This only works from inside an authorized Sprite; a direct call from your laptop returns 401.

    Set "stream": false on native /api/chat calls. The gateway buffers native newline-delimited JSON responses, so this gives you a single JSON response to parse. OpenAI-compatible event streams are handled separately.

Using the OpenAI SDK

Point your OpenAI client at the gateway's /v1 base URL. The api_key argument is required by the SDK and replaced by the gateway, so use a placeholder. Ollama's OpenAI compatibility documentation lists supported features.

Python 
# Install the client first: pip install openai
from openai import OpenAI

client = OpenAI(
    api_key="sprites",  # Placeholder; the gateway supplies your real key.
    base_url="https://api.sprites.dev/v1/gateway/ollama/YOUR_CONNECTION_ID/v1",
)

response = client.chat.completions.create(
    model="YOUR_OLLAMA_CLOUD_MODEL",
    messages=[{"role": "user", "content": "Hello from my Sprite!"}],
)
print(response.choices[0].message.content)

Coding agents

Sprite base environments include the sprite-api-gateway skill for Claude Code, Cursor, Codex and Gemini. An agent inside an authorized Sprite can discover its connectors with the command below. If your environment doesn't have the skill, the same discovery endpoint works directly.

Shell 
curl -s https://api.sprites.dev/v1/gateway/list

Gotchas

Good to know

  • Bring your own key.

    There is no Fly.io-managed Ollama option. Ollama bills you for model usage; Sprite compute is separate.

  • Ollama sets your limits.

    Model access and usage limits come from your Ollama account, subscription or credits. See Ollama pricing.

  • Cloud inference, not local model management.

    This connector calls ollama.com, not a local Ollama server. Don't rely on local-only features or embedding endpoints; use the cloud API's supported routes.

  • Rotate the key in place.

    Use Edit API key, then Update Key on the connector. Your connection ID and gateway URL stay the same.

Putting Ollama to work

Fuzzing with a model fleet

Fuzzers are little programs whose job is to mutate bytes. They are great at creating malformed garbage. Models can complement them with inputs that are structurally valid and semantically vicious: a date range that parses cleanly but runs backwards, two Unicode names that normalize into one, a YAML document that two parsers read differently.

Valid but wrong is where interesting bugs hide, and it is exactly what a language model can help invent. A model knows what a plausible date looks like, which lets it write one that is plausible and wrong. Ollama makes it easy to generate a large pile of that test data.

Start with a loop inside a Sprite. Ask a model for a batch of hostile inputs through the Ollama Connector, then run your target code against each input in a subprocess. Save anything that crashes, hangs, or disagrees with a second implementation. The workload runs in a hardware-isolated VM, and the connector keeps your Ollama key outside it. Still set resource limits and avoid putting sensitive data in the test environment.

  1. Generate.

    Fan the work out across fast models available to your account, such as nemotron-3-nano:30b. Give each one a different attack axis: encoding, boundary, contradiction, locale, or resource limits. Ask for one base64 line per case, because the payloads contain exactly the characters that break text protocols. Validate and decode each line before running it.

  2. Execute.

    Run every case in a subprocess with a timeout and resource limits inside the Sprite. You have untrusted generated input hitting a real process in a hardware-isolated VM you can checkpoint and restore.

  3. Judge.

    Crashes are the easy case. A useful additional check is differential: run each input through two libraries that should produce equivalent results and flag disagreements. A mismatch is a lead to investigate, not proof that either library is wrong.

  4. Triage.

    Deduplicate on exception type plus top stack frame, then hand the clusters to a larger model such as kimi-k3 to suggest likely root causes. Reproduce and verify those suggestions before treating them as bugs.

  5. Keep the corpus.

    Generation uses model tokens; execution uses Sprite compute. Write the corpus to the Sprite's disk and it becomes a regression suite you can replay on every change.

The generators and the triager use the same connector. Switch the model in your request, while the machine running unreviewed input stays separate from your Ollama credential.

Set up the Ollama connector

Frequently asked questions

Do I need to put my Ollama API key inside a Sprite?

No. You add the key once to your Fly.io organization. It is encrypted at rest and never returned by the connector API. The Sprite uses a connection ID and gateway URL; the gateway attaches the stored key after checking access. The connector does not place your Ollama credential inside the Sprite.

How do I give a Sprite access to Ollama?

Open the connector in the Fly.io dashboard and set an access policy on the Access Configuration card. Grant access by Sprite labels, by name prefix, or to every Sprite in the organization. A new connector denies every Sprite until you grant access. If you combine labels and a name prefix, the Sprite must satisfy both.

Why does my Ollama call return 403 when the connector is set up?

Check the response body first. A new connector denies access by default, and an existing policy may reject the Sprite's labels, name prefix, or requested endpoint. Check the Authorized Sprites list and endpoint rules, save the policy, then retry. An upstream Ollama error can also be forwarded through the gateway.

Can I call the Ollama gateway from my laptop or from CI?

Not directly. The gateway requires a verified calling Sprite identity to authorize the request and attach the stored key. Requests without that identity return 401. To try a call by hand, run it inside an authorized Sprite using sprite exec. CI can execute the call inside a Sprite, rather than calling the gateway directly.

Where does the Ollama API key come from?

You supply it from your ollama.com account. There is no Fly.io-managed Ollama key. Ollama handles your model usage and billing; Fly.io stores the key and controls which Sprites may use it. Sprite compute usage is billed separately.

Can I use the OpenAI SDK with the Ollama connector?

Yes. Point the client at your connector's gateway URL with /v1 appended. The SDK requires an api_key argument, but the gateway replaces it with the stored Ollama key, so use a placeholder. Ollama supports parts of the OpenAI API; check Ollama's compatibility documentation for the endpoints and features your client uses.

Do structured outputs work with Ollama models on Fly.io Sprites?

Ollama's documentation currently says Ollama Cloud does not support structured outputs. Do not rely on format or response_format for schema enforcement. For models that support tool calling, it is an alternative way to request arguments, but validate those arguments in your own code before using them.

Which Ollama models can a Sprite call?

Use the models available to your Ollama Cloud account. Access and rate limits come from Ollama, not from Sprites, and may depend on your subscription or usage credits. The model catalog is not a guarantee that a call will succeed under your account's current limits. Check Ollama's pricing and test the model you intend to use.

Can I restrict which Ollama endpoints a Sprite is allowed to reach?

Yes. A connector access policy can include allowed and blocked endpoint lists using exact paths or a trailing wildcard. Block rules take precedence over allow rules. Restrict access to the inference and catalog endpoints your application needs.

How do I rotate the Ollama key used by my Sprites?

Open the connector, choose Edit API key, enter the new key, and click Update Key. The key is validated before it replaces the old one. The connection ID stays the same, so callers keep using the same gateway URL. Then revoke the old key with Ollama when you no longer need it.