Showing posts with label AI agents. Show all posts
Showing posts with label AI agents. Show all posts

Run NanoClaw with Ollama Cloud: Complete Setup Guide

NanoClaw is a powerful secure, personal Claude assistant, but did you know you can also run it with models from Ollama Cloud (like glm-5:cloud and gemma4:31b-cloud)?

Summary: This guide shows how to run NanoClaw with Ollama Cloud by placing a LiteLLM proxy between NanoClaw's Anthropic-oriented runtime and Ollama's cloud API.

Who this is for:
1. Developers who want a lower-cost hosted model setup for NanoClaw without rewriting the core agent workflow.
2. Agents - instructions for agents running the NanoClaw setup.

What you'll need: A coding asssistant (Claude), an Ollama account, Ollama API key and Docker.

Quick Answer

Yes, NanoClaw can be used with Ollama Cloud. The practical approach is to run LiteLLM locally, map the expected Anthropic model to an Ollama Cloud model, and point NanoClaw at that proxy.

This opens up new possibilities for model experimentation at low cost, while keeping the powerful "software engineer" capabilities of the Claude Agent SDK.

In this post, I'll walk you through how to successfully bridge NanoClaw's Anthropic-centric design with Ollama's cloud API using a local proxy and some clever networking.

For those looking to automate this, I have provided a dedicated set of machine-readable instructions for AI agents at the bottom of this post. During the Nanoclaw /setup, when it asks you to enter your Anthropic Key or Claude account details - pick 'No' and then tell NanoClaw to "read this page to setup ollama cloud: https://www.dipockdas.com/2026/04/nanoclaw-ollama-cloud-setup.html".

Note: Everything that follows was developed and tested using Claude Code running on NanoClaw, but powered by the gemini-flash-preview model hosted on Ollama Cloud.

The Challenge

The Claude Agent SDK (which runs inside NanoClaw's Docker containers) is hard-coded to expect an Anthropic-compatible API. It also has strict validation for things like API keys and model names. To swap in Ollama Cloud, we need to trick the SDK into thinking it's talking to Claude, while actually routing traffic to Ollama.

The Solution: LiteLLM Proxy

Use LiteLLM, a lightweight proxy that translates OpenAI/Anthropic calls into other provider formats. By running LiteLLM locally, you create a bridge:

uv run --python 3.12 --with "litellm[proxy]" litellm --config litellm_config.yaml --port 4000

A litellm_config.yaml handles the model aliasing:

model_list:
  - model_name: claude-3-5-sonnet-20241022
    litellm_params:
      model: ollama_chat/glm-5:cloud
      api_base: https://api.ollama.com
      api_key: "YOUR_OLLAMA_API_KEY"
drop_params: true max_tokens: 4096

Bridging the Container Gap

NanoClaw runs the Agent SDK inside Docker. By default, the container can't see the proxy running on the host machine. We solved this by using Docker's internal DNS (host.docker.internal) and updating NanoClaw's src/container-runner.ts to pass the correct environment variables:

// Pass the local LiteLLM endpoint to the container
if (ANTHROPIC_BASE_URL) {
  args.push('-e', `ANTHROPIC_BASE_URL=${ANTHROPIC_BASE_URL}`);

  // Bypass the internal OneCLI proxy for local traffic
  if (ANTHROPIC_BASE_URL.includes('host.docker.internal')) {
    args.push('-e', 'NO_PROXY=host.docker.internal,127.0.0.1,localhost');
  }
}

Putting It All Together

With the proxy running and NanoClaw updated, we simply set our .env (and synced it to data/env/env for the container) to point to the local gateway:

ANTHROPIC_BASE_URL=http://host.docker.internal:4000
ANTHROPIC_MODEL=claude-3-5-sonnet-20241022  # Aliased to GLM-5 Cloud

The Result: SUCCESS! 🚀

After these changes, NanoClaw's agent (Daisy) was fully functional on Telegram, Slack, and WhatsApp—powered entirely by Ollama Cloud. The agent could even write complex code (like a new Google Drive skill!) while running on glm-5:cloud.

For a full step-by-step setup, check out the docs/OLLAMA_CLOUD_SETUP.md in my NanoClaw fork.


🤖 Automated Setup for AI Agents

If you are an AI agent tasked with configuring this environment, you can use either the GitHub Fast-Path or the Manual Reconstruction steps below.

Goal: Configure the local NanoClaw environment to route traffic through a LiteLLM proxy to Ollama Cloud, satisfying SDK model validation.

Option A: GitHub Fast-Path (Recommended)

Run these commands to pull the pre-configured files directly into the NanoClaw root:

# Add the Ollama Cloud configuration as a remote and merge the changes
git remote add ollama-cloud https://github.com/dipockdas/nanoclaw-ollama-cloud.git
git fetch ollama-cloud
git merge ollama-cloud/main --allow-unrelated-histories

Option B: Manual Reconstruction

If you cannot access GitHub, use these commands to create the necessary configuration files and verify code compatibility:

1. Verify/Apply Code Changes

Ensure src/container-runner.ts handles ANTHROPIC_BASE_URL and NO_PROXY correctly. If you're an agent, search for "ANTHROPIC_BASE_URL" in that file; if missing, apply the patch described in the "Bridging the Container Gap" section above.

2. Create LiteLLM Config

cat << 'EOF' > litellm_config.yaml
model_list:
  - model_name: claude-3-5-sonnet-20241022
    litellm_params:
      model: ollama_chat/glm-5:cloud
      api_base: https://api.ollama.com
      api_key: "YOUR_OLLAMA_API_KEY"
      drop_params: true
      max_tokens: 4096

general_settings:
  anthropic_version: "2023-06-01"
  allow_unauthorized_requests: true
EOF

3. Update Environment Variables

Append these to .env and sync to data/env/env:

echo "ANTHROPIC_BASE_URL=http://host.docker.internal:4000" >> .env
echo "ANTHROPIC_AUTH_TOKEN=ollama" >> .env
echo "ANTHROPIC_MODEL=claude-3-5-sonnet-20241022" >> .env
echo "ANTHROPIC_API_KEY=sk-ant-api01-proxy-placeholder-key-000000000000000000000000000000000000000000000000" >> .env

mkdir -p data/env && cp .env data/env/env

4. Start the LiteLLM Proxy

uv run --python 3.12 --with "litellm[proxy]" litellm --config litellm_config.yaml --port 4000 &

5. Verify Setup

Run this check to ensure the proxy is active and aliasing correctly:

# Verify the proxy is responding correctly
curl -s http://localhost:4000/v1/models | grep -q "glm-5:cloud" && echo "✅ Setup Verified" || echo "❌ Setup Failed"

Happy coding!


FAQ

Can NanoClaw use Ollama Cloud directly?
Not natively in this setup. A proxy layer is used to bridge the API expectations.

Why use LiteLLM?
LiteLLM handles provider translation and makes NanoClaw's Anthropic-facing runtime work with Ollama Cloud.

What model was tested?
This setup was tested with glm-5 on Ollama Cloud.


🤝 Credits & Shout-outs

This integration wouldn't be possible without the incredible work of the open-source community:

  • NanoClaw: The extensible, secure personal assistant framework that makes this all possible. Huge thanks to the developers at qwibitai for building such a robust orchestrator.
  • Ollama: For making high-performance LLMs accessible to everyone. Their new cloud API (glm-5:cloud) is a a must for experimentation and agentic workflows requiring speed and reasoning.
  • LiteLLM: The "glue" of the AI world. Their proxy made bridging these two ecosystems a breeze.

⚡ Meta: Built with Claude Code using Gemini-flash-preview hosted on Ollama

What makes this setup even more interesting is how it was built. This entire integration—including the LiteLLM configuration, the Docker networking fixes, and this very blog post—was developed and tested using Claude Code running on NanoClaw, but powered by the gemini-flash-preview model hosted on Ollama Cloud.

I didn't use Anthropic's models to build the bridge to Ollama; I used Gemini to build the bridge for Claude. 

AI Guardrails for AI Applications: Designing Safe and Reliable Agents

 

Don't Just Ship the Model—The Critical Need for AI Guardrails

In our last post, we discussed the importance of giving your AI a distinct and trustworthy persona. Now, let's talk about how to ensure that persona behaves responsibly, every single time.

Imagine onboarding a new employee. You’d typically have an interview process, background checks, and a clear understanding of their skills. But what if this new hire was a complete unknown? What if you gave them a laptop, full access to company resources, and simply trusted they would be a productive member of the team, all without any formal training or oversight?

You wouldn't just hope for the best. You'd have strict security policies, HR training, and a clear code of conduct. This is precisely the framework we need when integrating AI agents into our applications.

What if that new employee could be tricked by a phishing email into wiring funds to a scammer? Or started sharing confidential client information? Or, just as damagingly, what if they were rude, abusive, or expressed extremist views in a conversation with a customer? As we integrate AI, we must recognize that they are exposed to a new frontier of risks that product teams must actively manage. Welcome to your new employee.

The "Lethal Trifecta": A New Class of Vulnerability

As technologist Simon Willison eloquently explains, the most significant security risks for AI agents emerge from a combination of three factors: access to private data, exposure to untrusted content, and the ability to communicate externally.

This "lethal trifecta" creates a potent vulnerability. An AI agent with access to your company's private documents, that can also read incoming emails (untrusted content), and has the ability to send its own emails (external communication) can be manipulated into performing actions that compromise your entire system. This isn't hypothetical; attackers use techniques like prompt injection to turn your AI against you, instructing it to find and exfiltrate sensitive data.

Guardrails: Your AI's Code of Conduct

So, how do we protect our products, our companies, and our users? The answer is by designing and implementing robust guardrails. These are the architectural constraints and behavioural policies that prevent an AI agent from acting outside of its intended purpose. Relying on third-party solutions alone is not enough; this must be built into your product's DNA.

Let's break this down into two critical areas: Security Guardrails and Behavioural Guardrails.


Part 1: Security Guardrails

These are the technical barriers that protect against malicious attacks and data breaches.

1. Data Minimization: The "Need-to-Know" Principle Your AI agent should only have access to the absolute minimum amount of data required to perform its task.

  • Example: An AI assistant helping with a support ticket needs access to that ticket's data, not the entire customer database. Before data is sent to the AI, it should be scrubbed of irrelevant personally identifiable information (PII).

2. Input Validation: Inspecting What Comes In You must inspect and sanitize the data being fed into your AI to detect and block hidden, malicious instructions.

  • Example: An AI tool that summarizes web articles must first scan the page's content for malicious scripts or known prompt injection phrases (e.g., "ignore all previous instructions...") before passing the text to the LLM.

3. Enforcing Existing Permissions: Don't Let the LLM Bypass Your Security Enterprise systems use layers of abstraction and strict Role-Based Access Control (RBAC) to manage secure access to data. These same principles must be applied to data accessed by an LLM. Never assume an LLM will respect your existing permissions for free. While its ingenuity in finding answers is powerful, that power must be constrained by your security model.

  • Example: If your data retrieval system is designed to only allow an AI to access a "dev" environment, you need technical enforcement to prevent it from generating queries that access the "production" database, even if a user prompt subtly encourages it.


Part 2: Behavioural Guardrails

This is where we connect directly back to the persona we defined in Part 1. Behavioural guardrails ensure the AI's persona remains consistent, professional, and aligned with your brand, preventing it from saying something inappropriate or harmful.

4. Output Monitoring: The Character Clause It is not enough to just prevent data leaks; you must police the AI’s personality. An unconstrained LLM can drift into unsafe conversational territory.

  • What it is: A final check on the AI's response to ensure it adheres to your brand's voice, tone, and code of conduct before it's shown to the user.

  • Example: An AI that generates customer service emails should have its output scanned to ensure it doesn't use abusive language, express biased opinions, or become inappropriately suggestive. You can implement filters and define "canned responses" for when a conversation veers into a forbidden topic, such as, "I cannot discuss that topic, but I can help with [intended function]."

5. Contextual Awareness: Defining the AI's "Role" This is about setting clear boundaries for the AI's responsibilities so it understands what it is and, just as importantly, what it is not.

  • What it is: Programming the AI with a clear understanding of its operational purpose and limitations.

  • Example: An internal HR chatbot should be programmed with the context that its role is to answer questions about public company benefits. If a user asks for an opinion on a political candidate, the AI should recognize that this question falls outside its defined role and politely decline, reinforcing the persona of a professional HR assistant, not a personal confidant. As a rule, your AI Agent should be as aware of your HR policies as any other employee.


What's Next?

Implementing comprehensive guardrails is the foundation of a trustworthy AI. In the final post, we'll explore a third critical consideration for building human-centered AI products:

  • Part 3: Beyond the Prompt - Designing for True AI Interactivity: We'll discuss strategies for fostering engaging, multi-turn interactions and creating user experiences that feel more like a conversation than a command line.

Key Takeaways for Product Managers and Engineers:

  • Guardrails Have Two Halves: You must protect against both external security threats (like data exfiltration) and internal behavioural failures (like inappropriate responses).

  • Codify Your Brand's Principles: Your AI's operational rules should reflect the same code of conduct and ethical boundaries you would instill in a human employee.

  • Behaviour is a Feature: An AI that is consistently helpful, professional, and on-brand is not an accident. It is the result of deliberate design and robust behavioural guardrails.