Budget a website AI feature: tokens, context and outputs are different limits

Draw the complete visitor journey

List the calls made for one successful task. Include the instructions, conversation history, source passages, tool definitions, tool results and final answer. Note any automatic retries, answer validation or extra summarization.

Also map failure paths. A timeout can produce a retry; a retrieval gap can produce another search; a long answer can be regenerated. The budget needs a bound on those paths, not only an average for a clean demo.

Use our automation workflow guide to separate preparation from the action. Approval can add another turn, and an authorized operation may involve a separate service charge. Neither disappears because the visible interface calls the whole task one request.

Read thinking and completion limits together

Google’s thinking documentation illustrates why a generation cap can affect internal reasoning as well as visible output for supported models. A very small cap can leave an incomplete or empty answer. The exact interaction is model-specific.

Inspect completion status and the reported usage categories. Do not label a truncated response successful because the request returned a transport-level success. The page should explain an incomplete result and offer a bounded retry or alternative route.

Set limits around the task rather than making every answer as short as possible. A necessary explanation, citation or warning can require space. Evaluate usefulness and correctness alongside consumption so the budget does not reward misleadingly incomplete answers.

Do not assume repeated context is free

A conversation can send earlier information again or use provider-managed state. Review what the actual service meters. The presence of an earlier message in the interface does not tell you whether its cost is repeated, discounted or represented differently.

Google’s caching guidance describes eligible repeated input and observable cache-hit counts. Use actual cache evidence when calculating savings. Support, conditions and minimums differ; a repeated paragraph is not sufficient proof of a billed cache hit.

Retain a conservative uncached estimate while validating the workflow. If a business case depends on cache savings, document the hit rate under representative use and the conditions that reduce it. Do not present the best cached demonstration as the normal cost for every visitor.

Choose limits that bound the whole feature

Define maximum calls per task, input size, output size, retries and any tool-search count. Add appropriate user or session limits and a route for legitimate requests that exceed them. An alert can notify an operator without actually preventing further spending; verify the stop mechanism separately.

Keep credentials on the supported server side and enforce limits there where relevant. A browser-only counter can be reset or bypassed. Do not expose an unrestricted provider key to make a prototype easier to call.

Our AI selection checklist helps assess the operating commitment. A model with more capacity is useful only when the task benefits and the resulting cost and latency fit the service.

Compare billing units honestly

A hosted assistant may meter queries, messages, credits or source allowances instead of passing through raw token charges. Use the hosted-assistant workload guide as a separate example of plan-level units. Do not assume one hosted query equals one underlying model call.

Keep the provider usage ledger and the platform invoice categories distinguishable. When a charge cannot be allocated precisely, label it shared or unresolved rather than inventing a per-answer rate.

Include abandoned and unsuccessful tasks in the usage review. Dividing only by successful answers can conceal the expense of repeated failures, while dividing by all button clicks can make a costly incomplete journey look efficient. Report both the total consumption and the number of useful completed tasks under the stated conditions.

Verify before extrapolating

Run a small authorized pilot using synthetic questions, including a long input, tool turn, failure and retry. Retain the usage fields, completion status and task result privately. Then estimate a realistic range for the expected workload and its bounded worst case.

Your next step is a call-level ledger and a tested stopping rule. Those two artifacts make the budget reviewable without confusing capacity, visible answer length and total consumption.