Skip to content
Meterry Docs | Meter every Run, Value every Result

LLM Gateway

LLM Gateway integrations usually need two layers:

  1. Configure an ExtractorRuleSet per model provider to extract metrics from the provider-native response.
  2. Report request-level prices through top-level x-billing.pricing_hints when the gateway sends the ingest request.

Extract provider usage

Section titled “Extract provider usage”

Provider response shapes vary:

  • OpenAI may return usage.input_tokens, usage.output_tokens, and usage.input_tokens_details.cached_tokens.
  • Anthropic may return usage.input_tokens, usage.output_tokens, usage.cache_read_input_tokens, and usage.cache_creation_input_tokens.
  • Gemini may return usageMetadata.promptTokenCount, usageMetadata.candidatesTokenCount, and usageMetadata.cachedContentTokenCount.

Normalize those provider fields into metrics such as input_tokens, output_tokens, cached_tokens, and cache_creation_tokens.

{
  "subject": {
    "default_type": "api_key",
    "type_path": "$.meta.subject_type",
    "id_path": "$.meta.subject_id",
    "required": true
  },
  "dimensions": [
    {"name": "provider", "path": "$.provider", "type": "string", "required": true},
    {"name": "model", "path": "$.model", "type": "string", "required": true},
    {"name": "project_id", "path": "$.meta.project_id", "type": "string", "required": true}
  ],
  "charge_item_extractors": [
    {
      "name": "input-tokens",
      "metric": "input_tokens",
      "path": "$.usage.input_tokens",
      "type": "number",
      "unit": "token",
      "skip_if_missing": true,
      "currency": "CREDIT",
      "unit_price_expr": "dec('0')",
      "amount_expr": "dec_mul(dec_div(dec(item['quantity']), dec(item['pricing_unit'])), dec(item['unit_price']))"
    },
    {
      "name": "output-tokens",
      "metric": "output_tokens",
      "path": "$.usage.output_tokens",
      "type": "number",
      "unit": "token",
      "skip_if_missing": true,
      "currency": "CREDIT",
      "unit_price_expr": "dec('0')",
      "amount_expr": "dec_mul(dec_div(dec(item['quantity']), dec(item['pricing_unit'])), dec(item['unit_price']))"
    }
  ]
}

Create the rule set with this configuration, then use the returned id for testing, publishing, and ingest URLs.

The amount_expr references item['unit_price'] and item['pricing_unit'] so same-metric x-billing.pricing_hints can override the default price.

Report dynamic prices from the gateway

Section titled “Report dynamic prices from the gateway”

If your platform has membership tiers, customer-specific prices, coupons, or marketing campaigns, prefer selecting the price in the gateway and reporting it with the raw event.

Amount does not have to mean fiat money. Its meaning follows your currency: USD, CREDIT, POINT, TOKEN_CREDIT, or any platform-defined unit.

{
  "source": "llm-gateway",
  "external_event_id": "resp_01JZ8E7K9S3K4N2V9A6Y0B1C2D",
  "idempotency_key": "openai:resp_01JZ8E7K9S3K4N2V9A6Y0B1C2D",
  "occurred_at": 1783008000,
  "raw_json": {
    "type": "response.completed",
    "provider": "openai",
    "id": "resp_01JZ8E7K9S3K4N2V9A6Y0B1C2D",
    "model": "gpt-4.1-mini",
    "usage": {
      "input_tokens": 2400,
      "output_tokens": 520,
      "total_tokens": 2920,
      "input_tokens_details": {
        "cached_tokens": 800
      }
    },
    "meta": {
      "project_id": "proj_gateway",
      "api_key_id": "key_live_001",
      "subject_type": "organization",
      "subject_id": "org_001",
      "membership_tier": "pro"
    }
  },
  "x-billing": {
    "pricing_hints": {
      "input_tokens": {
        "unit_price": 0.75,
        "pricing_unit": 1000000,
        "currency": "CREDIT"
      },
      "output_tokens": {
        "unit_price": 3,
        "pricing_unit": 1000000,
        "currency": "CREDIT"
      }
    }
  }
}

Use x-billing.items for sparse gateway-known metrics such as tool calls. If the same metric is produced by both extractor output and x-billing.items, the reported x-billing item wins.

Section titled “Related pages”

To compute per-token supplier cost and total cost in the same event, see the LLM Gateway use case section of the Analytics-Only page.