Skip to main content
Bumblebee is local-first: the default path runs open weights on your hardware via Ollama, without shipping prompts to a third-party inference API. That ethos is unchanged in the open-source project. This page documents an optional inference mode for a different job: exercising the harness as a product—routing, memory, tools, platforms, and cognition loops—against the most capable hosted models you choose (often called “frontier” or “bleeding edge” APIs). It is a test and evaluation lever, not a replacement for self-hosted inference as the recommended default.

​
Why this exists

The implementation is intentionally thin: the same OpenAICompatibleTransport and provider surface used for Ollama and the home gateway. No separate “cloud edition” of the harness—only a different BUMBLEBEE_INFERENCE_PROVIDER and API key.

​
Supported providers

The harness recognizes two named presets (you can override the base URL if a vendor changes endpoints): Both speak the OpenAI-compatible surface the harness already uses: /v1/chat/completions, /v1/embeddings, /v1/models. Health checks use GET /v1/models (not the home gateway’s GET /health on the tunnel root).

​
Base URL convention

The transport builds API paths as {base_url}/v1/..., same as local Ollama (http://127.0.0.1:11434 + /v1/...). For hosted vendors, BUMBLEBEE_INFERENCE_BASE_URL must be the prefix that ends right before /v1—for OpenRouter that is https://openrouter.ai/api, not a URL that already contains /v1. If you omit BUMBLEBEE_INFERENCE_BASE_URL, the preset defaults above apply.

​
Bearer token and custom key variable

Requests send Authorization: Bearer <key> using:
  1. OPENROUTER_API_KEY or VENICE_API_KEY by default, or
  2. The env var named in BUMBLEBEE_INFERENCE_API_KEY_ENV, or
  3. Harness inference.api_key_env in YAML when it is not the generic gateway name BUMBLEBEE_INFERENCE_GATEWAY_TOKEN (that name stays reserved for the home tunnel gateway).
This keeps gateway tokens and hosted API keys from colliding conceptually, even though both are Bearer headers.

​
Configuration

Or for Venice:
Optional overrides: Full tables: Environment variables. Copy-paste examples also live in the repo .env.example under “Hosted brain”.

​
Harness YAML

In configs/default.yaml or an overlay, you can set:
BUMBLEBEE_INFERENCE_PASS_NUM_CTX=true|false still overrides pass_num_ctx when set.

​
Models must match the host

models.reflex, models.deliberate, models.embedding, and any per-entity cognition.*_model overrides must use IDs your host actually serves (OpenRouter model slugs, Venice model ids, etc.). The harness will call /v1/models where supported; if your chosen id is wrong, you will see errors from the API, not from Bumblebee-specific logic. Embedding models in particular must exist on that provider if you rely on vector memory features.

​
Local vs hybrid Railway

Using hosted inference on Railway does not change Postgres, volumes, or platform behavior—only where /v1/chat/completions is executed.

​
Setup wizard

After you choose hybrid or local in bumblebee setup, the wizard offers an optional step: configure OpenRouter or Venice, write keys into .env, set BUMBLEBEE_INFERENCE_PASS_NUM_CTX=false, and optionally prompt for a custom base URL. For cloud bodies, mirror those variables in Railway yourself (the wizard explains this). Structured onboarding: Setup & onboarding.

​
Relationship to other docs

  • Portable inference — mental model: inference as a replaceable endpoint; hosted presets are another endpoint, not a fork.
  • Gateway — home appliance for your Ollama; orthogonal to OpenRouter/Venice.
  • Hybrid Railway — default hybrid brain is still tunnel + gateway; this page is the alternative brain for evaluation.

​
Operational notes

  • Cost and data handling are between you and the provider; read their terms. This mode is opt-in and key-driven.
  • Tool calling and long contexts depend on the model and provider, not on Bumblebee’s local-first defaults.
  • If a host returns errors on extra JSON fields, keep pass_num_ctx off and avoid provider-specific assumptions beyond OpenAI-compat chat/embeddings.

​
Summary

Open-source posture: defaults stay local and self-hosted; the repo remains Apache 2.0 and fully runnable without any hosted API. Product testing posture: flip BUMBLEBEE_INFERENCE_PROVIDER to openrouter or venice, set the key, align model ids, and run the same harness to see how your entity behaves on the strongest models you subscribe to—without changing the core architecture or licensing story.