LLM token metering

A full walkthrough of the reserve / commit / release pattern for LLM calls, including per-unit input quantities, output topup at commit time, and reservation safety buffers.

1. Set up a plan

In the dashboard, create (or edit) a plan and add an LLM row for the model you want to gate - for example, llm.gpt-4o. LLM rows support stacked unit caps: add a tokens cap (e.g. 1,000,000 tokens/month) and a cents cap(e.g. 500 cents = $5/month). Either cap can fire first depending on the model's price - the user is blocked as soon as any matched group hits its quota.

Expand the Reservation safety collapsible on the LLM row and set a threshold - 85%is a sensible default. This leaves 15% of the quota reserved for unknown output tokens so the cap isn't breached mid-call.

Assign the plan to your users with upsertSubscription() from your backend.

2. Reserve with input tokens

Count the input tokens before calling the model (e.g. via tiktoken for OpenAI models or the model provider's token counter). Pass the count as inputTokens - the server resolves the right quantity for each matched limit group automatically.

import { vevee } from '@/lib/vevee';

async function callWithMetering(userId: string, prompt: string) {
  const inputTokens = countTokens(prompt);      // e.g. tiktoken
  const inputCostCents = Math.ceil(inputTokens * GPT4O_INPUT_PRICE_PER_TOKEN_CENTS);

  const r = await vevee.reserve(userId, 'llm.gpt-4o', 0, undefined, {
    inputTokens,
    inputCostCents,
    prompt,                                     // optional: stored in event_logs
  });

  if (!r.allowed) {
    if (r.reasons?.includes('reservation_headroom_exceeded')) {
      // User is inside the safety buffer - not yet at the hard cap.
      // The buffer is protecting headroom for unknown output cost.
      // Show a "close to your limit" warning; do not offer credits as an alternative.
      throw new NearLimitError({
        gate: r.groups?.[0]?.headroomGate,
        quota: r.groups?.[0]?.quota,
        current: r.groups?.[0]?.current,
        reasons: r.reasons,
      });
    }
    // limit_reached or no_subscription - hard block
    throw new LimitError(r.reasons);
  }

  return r.reservationId!;
}

3. Call the LLM, capture output

Make your standard provider call. The response contains the output token count and you can compute the output cost from your pricing table.

const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: prompt }],
});

const outputTokens = response.usage?.completion_tokens ?? 0;
const outputCostCents = Math.ceil(outputTokens * GPT4O_OUTPUT_PRICE_PER_TOKEN_CENTS);

4. Commit with the actual output

Pass the output units in extra - the server adds them on top of the reserved input amount. Check the overflow flag to detect whether the commit pushed the counter past the quota (which blocks the next reserve).

const c = await vevee.commit(reservationId, {
  response: response.choices[0].message.content ?? '',  // optional: stored in event_logs
  extra: {
    outputTokens,       // added on top of reserved inputTokens → total token count
    outputCostCents,    // added on top of reserved inputCostCents → total cost
  },
});

// Detect overshoot: the commit succeeded, but the counter is now past quota.
// The NEXT reserve() for this user will return limit_reached.
if (c.groups.some(g => g.overflow)) {
  console.warn('LLM commit exceeded quota - next call blocked until period resets');
}

return response;

5. Handle errors mid-call

If the LLM call throws, release the reservation so the reserved input tokens are refunded to the user's quota. The commit never happens, and the counter goes back to where it was before the reserve.

let reservationId: string | undefined;

try {
  reservationId = await callWithMetering(userId, prompt);
  const response = await openai.chat.completions.create({ ... });

  const outputTokens = response.usage?.completion_tokens ?? 0;
  const outputCostCents = Math.ceil(outputTokens * GPT4O_OUTPUT_PRICE_PER_TOKEN_CENTS);

  await vevee.commit(reservationId, {
    extra: { outputTokens, outputCostCents },
  });

  return response;
} catch (err) {
  if (reservationId) {
    // Refund the reserved input tokens - do not charge the user for a failed call.
    await vevee.release(reservationId, {
      errorCode: 'provider_error',
      reason: err instanceof Error ? err.message : String(err),
    }).catch(() => {});   // release after TTL is harmless - just swallow the error
  }
  throw err;
}

6. Why two caps?

Tokens give you a hard ceiling on volume; cents give you a hard ceiling on cost. They measure different things and either can fire first:

• A power user hitting 1M tokens with a cheap model maxes out the token cap before spending $5.
• A user sending one expensive prompt (e.g. a long document) might hit the $5 cent cap while still well under the token limit.

Both caps share the same reservation - a single reserve() call checks all matched groups in parallel. If any group blocks, the reserve fails and no counters are incremented. This means adding a dollar cap to an existing token-capped plan is a dashboard-only change: no code change neededbeyond the per-unit input/output pattern you're already using.