Economics: Keeping A Public RAG Assistant Sustainable

The cost architecture of a public assistant is also a safety architecture. If a system waits for a bill or provider rejection before limiting work, its fallback behavior is already too late. Ask Zico reserves scarce capacity before model generation and keeps a useful non-model path available.

The observability article explains the event stream used for cost triage.

The cost surface

A request can consume Worker CPU, vector queries, KV reads, D1 reads and writes, embedding work, model input tokens, model output tokens, and provider platform fees. The useful monthly model is therefore additive:

monthly cost = worker requests and CPU
             + vector queried and stored dimensions
             + KV and D1 operations and storage
             + direct-model input and output tokens
             + fallback-provider inference and fees

This is more actionable than "cost per chat" because each term has a different control.

flowchart TD
    request["Public request"] --> gate["Access and exact quota reservation"]
    gate --> retrieval["Bounded retrieval"]
    retrieval --> context["Compact context"]
    context --> provider["Capacity-aware provider route"]
    provider --> counters["Usage counters and alerts"]
    gate -->|"No capacity"| fallback["Sources or search without model call"]
    provider -->|"Unavailable"| fallback

Pricing snapshot: July 21, 2026

The table records public list-price inputs used for architectural reasoning. It does not disclose Ask Zico traffic, spend, budgets, or configured thresholds. Prices and free allowances can change, so the linked primary pages remain authoritative.

Cloudflare's July 7, 2026 Workers pricing page lists the paid minimum as USD 5 per month, additional requests at USD 0.30 per million, and additional CPU at USD 0.02 per million CPU milliseconds after included usage. Vectorize's April 21, 2026 page lists USD 0.01 per million queried dimensions and USD 0.05 per 100 million stored dimensions after its paid allowances. Google lists gemini-2.5-flash-lite at USD 0.10 per million paid text-input tokens and USD 0.40 per million output tokens, including thinking tokens. These are publication-date inputs, not promises: reprice the calculator at deployment and whenever a provider changes a model or plan.

Exact quota reservation

Checking a counter and incrementing it later creates a race: concurrent requests can all pass the same remaining-capacity check. Ask Zico uses D1-backed reservation so a request consumes an allowed slot before attempting generation.

flowchart TD
    request["Public request"] --> reserve["Reserve request and estimated capacity in D1"]
    reserve --> capacity{"Reservation succeeds?"}
    capacity -->|"Yes"| generate["Generate from compact evidence"]
    generate --> result{"Valid provider result?"}
    result -->|"Yes"| record["Record outcome and usage metadata"]
    result -->|"No"| recordFailure["Record provider failure"]
    capacity -->|"No"| fallback["Return source or search fallback"]
    recordFailure --> fallback

Failing closed when the quota store is unavailable avoids a denial-of-wallet path. Public and evaluator callers also use separate credentials and capacity, so a regression run cannot silently consume the public assistant's allocation.

Provider-capacity routing

The configured Gemini ladder has per-model request and token windows. The Worker selects a model only after reserving capacity, records attempts, and can use OpenRouter for qualifying provider-layer failures or exhausted direct capacity. This is an economics and resilience decision, not a retrieval decision: the provider route may change, but every provider must satisfy the same grounded-evidence and citation-validation contract.

The exact model list is a July 2026 deployment detail. The durable design is the sequence: estimate, reserve, attempt, record, and fall back without weakening citation validation.

flowchart TD
    estimate["Estimate request capacity"] --> direct{"Direct model slot reserved?"}
    direct -->|"Yes"| modelCall["Call selected Gemini model"]
    direct -->|"No"| secondary{"Fallback-provider capacity?"}
    modelCall -->|"Provider failure"| secondary
    modelCall --> primaryValidate["Validate primary answer and citations"]
    primaryValidate -->|"Passes"| done["Return and count"]
    primaryValidate -->|"Fails"| safe
    secondary -->|"Yes"| route["Call configured OpenRouter model"]
    secondary -->|"No"| safe["Source or search fallback"]
    route --> fallbackValidate["Validate fallback answer and citations"]
    fallbackValidate -->|"Passes"| done
    fallbackValidate -->|"Fails"| safe

Compact context and search fallback

Compact context reduces input tokens and often reduces distraction at the same time. It is not free optimization: context selection must keep the evidence required for correctness, so the compact path is evaluated rather than assumed. The same applies to the output cap and reasoning budget: they are set to the smallest values that pass the grounded-answer evaluation, not to a provider maximum by default. A route that gains no measurable quality from extra reasoning tokens should not purchase them.

Search fallback has a different economic role. It preserves utility after quota or provider exhaustion without triggering another model call. Existing Algolia/global search and the library pages are therefore first-class product paths, not degraded leftovers: they answer navigational and source-finding needs without model inference. This makes the cost ceiling visible in product behavior instead of turning it into a silent outage.

Illustrative cost calculator: July 21, 2026

This scenario is a sanitized teaching calculation, not Ask Zico traffic, spend, or quotas. It uses paid-tier text rates for gemini-2.5-flash-lite, excludes cached tokens, tools, embeddings, OpenRouter fees, storage, and tax, and must be repriced before use.

monthly questions = users/day x questions/user/day x 30
model cost = questions x ((input tokens x input price) + (output tokens x output price)) / 1,000,000
vector dimensions queried = (indexed vectors + questions) x embedding dimensions
vector dimensions stored = indexed vectors x embedding dimensions
Cloudflare billing measures = Worker requests; KV reads; D1 reads/writes; Vectorize queried and stored dimensions

For each scenario, record the Cloudflare billing measures beside the model subtotal rather than hiding them inside an average chat price. Compare Worker requests with the free daily envelope, KV reads with the cache-read allowance, Vectorize queried dimensions with its allowance, and D1 writes with the reservation/event pattern. When the expected or burst case crosses an allowance, price that individual surface from the dated Cloudflare page and add it to the subtotal.

Budget gates, counters, and alerts

The Worker records quota attempts and estimated provider cost at an architectural level, then evaluates configured daily/monthly and budget alert conditions. Four gates make the policy explicit: a daily quota per caller, a global monthly cap, a model_enabled switch, and a fallback_only switch. The last two are operational controls: model_enabled=false prevents any provider request, while fallback_only=true sends every request directly to search, sources, and library pages. Public documentation intentionally omits the actual limits and usage. Publishing control design is useful; publishing the exact abuse and budget envelope is not.

An illustrative forecast should use scenarios, never production traffic. The example below is a model-inference forecast, not an all-in platform total:

scenario model cost = requests x (input tokens x input rate + output tokens x output rate) / 1,000,000
scenario Vectorize queried dimensions = (indexed vectors + queries) x embedding dimensions
scenario Vectorize stored dimensions = indexed vectors x embedding dimensions

Run low, expected, and burst scenarios with current provider prices. Price each non-model surface separately from its dated provider rate, for example Workers requests and CPU, KV operations and storage, D1 operations and storage, Workers AI embeddings, and Vectorize queried and stored dimensions, rather than adding unlike usage units. Then verify that the quota ceiling and source-only path keep every scenario operationally acceptable.

What shipped

Ask Zico launched with exact quota reservation, provider-capacity routing, compact context enabled, progressive context disabled, source/search fallback, usage counters, and budget alerts. The product does not depend on a future optimization to remain bounded.

The final lesson is simple: cost controls should select a safe response before expensive work begins. When economics and trust share the same fallback path, sustainability becomes part of correctness.

Return to the Ask Zico project hub or begin the series again with the production architecture.

References

• Cloudflare Workers pricing
• Cloudflare Vectorize pricing
• Cloudflare D1 pricing
• Google Gemini API pricing
• OpenRouter pricing
• OpenRouter models and pricing API