Qaf · AI cost audit · 7–13 October 2026What drives our AI costs
Last week's AI bill was $22.2k, about 14 times net revenue. Most of it comes from a large number of free users asking ordinary questions. There is no abuse behind it. The cost of each answer is driven by what we send to the model. Three quarters of the Gemini bill is uncached input, mostly search passages sent back to the model. Long threads and power users matter, but neither is the main problem. Gemini prices double on 1 January 2027, so the priority is to cut the cost of each turn.
Spend grew from $2.0k to $3.2k a day during the week as daily chat turns went from 24k to 63k. Treat these figures as a snapshot of a growing product.
Where the money goes
Gemini 3.7 Flash is 87% of the bill, Cohere rerank 9% and GPT‑5.6 terra 4%. By feature, the chat answer agent alone is three quarters.
What Gemini charges for
An average answer takes 2 model steps and sends about 33k input tokens. Roughly two thirds of that is search results sent back on the second step. Only 15% of input hits the cache. Agent loops don't run away: turns with 5 or more steps are under 1% of turns.
So the biggest lever is how much text goes into each turn: how many passages, how long, how much is cached and how much history is resent. Restricting users saves much less.
Free vs paying
Free users drive the total because there are so many of them. Each Pro user costs about 11 times a free user: $2.60 a week against $0.23.
- Pro users send 6.9 times as many messages (42 against 6 a week), and each message costs 1.6 times as much ($0.062 against $0.038), mostly because their threads are deeper.
- About half of Pro cost is in threads of 41 or more messages. For free users it is 7%.
So changes to the free tier move the bill. Changes to Pro move the margin.
Logged in vs logged out
Guests cost $1.3k (6.7%), or about $0.11 each. They are cheap because they send few messages (2.8 on average). Their messages cost the same as anyone else's. Counting chats that move to an account at sign-up, guest-originated cost is $1.6k (8.4%).
Requiring sign-up would save about $1.1k a week, or about $0.5k with a one-message-a-day guest cap, before counting lost sign-ups. It's a small saving and it puts the funnel at risk.
Device
Cost follows the user base. Android and iOS users look the same, at about $0.25 and 6.5 messages a week each. Desktop web costs the most per user, mainly because Pro users concentrate there: registered users average $0.68 against $0.27 on Android.
Our tracing records almost only Android traffic, so iOS cost and most web cost are modelled from Android at the same thread depth and reconciled to the real bill. The bill limits the error to about 6–8% per turn.
Device-specific controls wouldn't save anything meaningful. The tracing gap is worth fixing.
Long threads
A single turn gets much more expensive deep in a chat, because the full history is resent every turn. Long chats are rare, though: 300 chats of 50 or more turns cost $1.5k (7.7%).
Cost of one turn by its position in the chat
Share of chats vs share of cost, by thread length
Sending only the last 10 turns of history would save about $1.2k a week ($0.9–1.4k). It's a contained fix that mostly protects Pro margin.
Power users
Cost is moderately concentrated, and we found no bot traffic: only 10 of 3,704 heavy users look automated. The top 0.1% (76 users) are mostly paying users in very long chats, with a median of 140 messages a week. Below them, 96% of the top 1–10% are free users.
What a per-user cap would save
| Cap | Saves / week | Users hit | Of them paying |
|---|---|---|---|
| 50/day | $95 | 60 | 30 |
| 30/day | $337 | 194 | 80 |
| 23/day | $566 | 821 | 118 |
| 10/day | $2,360 | 5,977 | 265 |
| 100/week | $498 | 118 | 60 |
Loose caps save very little and mostly hit paying users. Only tight caps such as 10 a day save real money, and those hit about 6k users.
Other drivers
- Welcome bonus. Free users get 20 one-off messages plus 3 a day. Usage beyond the daily 3 costs $2.8–5.6k a week, so the bonus funds roughly $1.8–5.2k of it.
- Quota hitters. 5.7k users hit the limit, cost $4.3k (22%), and about 3% of them started a trial or Pro that week.
- One-off batch job. $2.0k of untraced Gemini usage on 11–12 October, most likely catalog generation.
- Hadith detail panel. $625 a week (72% of the terra bill), or $0.19–0.32 per panel opened. That is 5 to 8 times the cost of a chat message.
- Old app versions. Clients below 1.3.0 weren't quota-limited until 13 October. That cost about $0.3k extra and is now fixed.
- Not drivers: voice, images, filters, incognito, edits, regenerations, errors, country or language. An Arabic turn costs 0.95 times an English one.
Unit economics
Paying users roughly break even on AI: $1,606 of revenue against $1,477 of cost while paid, a margin of +8% (range −5% to +21%). The top 10% of active payers cost $880 against $171 of revenue, while the other 90% run at +50%. Annual plans are the weakest:
| Plan | Payers | AI cost | Net revenue | Margin |
|---|---|---|---|---|
| Stripe annual | 105 | $356 | $201 | -77% |
| Apple annual | 191 | $298 | $307 | +3% |
| Google annual | 109 | $177 | $177 | +0% |
| Apple monthly | 87 | $219 | $354 | +38% |
| Stripe monthly | 50 | $194 | $228 | +15% |
| Google monthly | 35 | $117 | $135 | +13% |
An annual Pro user breaks even at 26–31 messages a week, and the median Pro user sends 23. Free usage cost about $160 per new payer last week, against about $12 a month of net revenue per payer.
What to do
Savings are first-order: they assume users behave the same. Some overlap: items 1, 3 and 4 all cut input tokens, so they don't add up fully.
| Action | $/week | Confidence | |
|---|---|---|---|
| 0 | Plan for the 1 January Gemini price change: switch model, negotiate, or reprice | avoids +17.3k | high |
| 1 | Send half as much search text per turn (fewer or shorter passages), after testing answer quality | ~4.1–4.4k | medium‑low |
| 2 | Shrink the 20-message welcome bonus (to 10 / to 0) | ≤1.0k / 1.8–5.2k | medium |
| 3 | Raise cache hits by 10 points (stable prompt prefix, explicit caching) | ~1.3k | medium |
| 4 | Send only the last 10 turns of history | ~1.2k | medium‑low |
| 5 | Require sign-up for guests (or cap them at 1 message a day) | ~1.1k / ~0.5k, minus lost sign-ups | medium |
| 6 | Run catalog jobs on the Batch API, with a budget and tracing | ~1.0k per run | low |
| 7 | Cache hadith-detail answers per hadith, or use a cheaper model | ≤0.7k | low |
| 8 | Fair-use limit for the heaviest Pro users (100 or 50 a week) | 0.3–0.6k | medium |
| 9 | Cap agent steps per turn | ≤0.55k | medium |
Not worth doing for cost: daily caps above about 20 messages, device or country controls, voice or image limits, anti-bot work.
Data and method
- Window. 7–13 October 2026, 7 full UTC days, all on the Cloudflare AI Gateway path (before the move to direct Vertex on 14 October). Costs are at list price. Vertex and Cohere usage draws on credits.
- True bill. Provider billing per model per day: Gemini $19.3k, Cohere rerank $1.9k, terra $0.9k, other $0.1k. Non-AI infrastructure is not included.
- Attribution. Langfuse traces split cost by user, chat and turn. They are joined to the production database (accounts, guests, message counts), Autumn, Stripe and RevenueCat (plan at time of use, revenue net of store fees and tax) and Mixpanel (device, quota hits, app version). Langfuse misses thinking tokens and prices cached input at $0, so every dollar figure is scaled day by day to the true bill for each model. 87% of the bill is attributed to users. The rest is the batch job, the playground and detail panels logged without a user.
- Coverage. Langfuse records about 98% of Android cost but almost no iOS and only about 12% of web. iOS and web cost is modelled from real message counts and Android's cost at the same thread depth. The true bill limits how much more an average iOS or web turn could cost: about 6–8%. Findings about plans and thread depth also hold on fully observed Android data.
- Thin samples. Threads deeper than 200 turns, the Max plan (2 users) and observed desktop-web turns (about 32 users).
- Verification. 20 analysis agents. Every claim was independently recomputed by two adversarial verifiers, and a critic reconciled the corrections. All figures are aggregates; no user-level data is published.