Context
The idea came from Arthur Monnet's site. He has a "What are you curious about?" box where you can just ask him a question, and I really liked it. I wanted the same thing here, so I built my own version around my profile and case studies and called it Ask Metin.
My first version sent every question to Claude Haiku. The answers were good, and each one cost a fraction of a cent. That's nothing for one question, but it grows with every visitor and every bot that finds the endpoint, and I didn't want an open-ended bill on a personal site.
The problem
Free AI tiers exist, but I couldn't rely on any single one. While I was testing, the free tier of Google's Gemini API returned "high demand" errors on half of my questions. Groq, with a prompt my size, allows about three questions a minute per model. If I picked just one, some visitors would get an error instead of an answer.
What I did
So I used several, in order. Each question goes to the first provider in the list. If that provider hits a rate limit, goes down or takes longer than 8 seconds, the question moves to the next one. The order is a single setting, so I can reorder providers or switch one off without a code change.
Before choosing the order I tested accuracy. I wrote 10 questions from my own profile: dates, where I live, my salary, a company I never worked at, and one question in Turkish. A model that made up even one fact was out.
- Groq's gpt-oss got all 10 right at about a second per answer, so it goes first.
- Gemini was right every time it managed to answer, so it's second.
- Cloudflare Workers AI running Llama also got 10 of 10 and comes third.
- A free Nemotron model on OpenRouter is fourth. It was accurate but slow, and timed out on three questions.
- A second Groq model got the name of an award wrong, and I dropped it.
Claude Haiku is still in the chain, but only for when every free provider has failed. It goes through OpenRouter on its own key with a hard $2 limit, separate from the key I use for my own work, so I know the worst case ahead of time. In testing, ten Haiku answers cost less than a tenth of a cent.
Each answer has a small "via Groq" or "via Gemini" label under it. If every provider fails, the chat says so plainly and gives my email and LinkedIn.
Because the endpoint is public, I added limits: 5 questions a minute and 30 a day per visitor, 1,000 characters per message, and only the last 12 messages go to the model. The rate limiter runs per server instance, so it slows bots down but isn't a global quota. What actually bounds the cost is the free tiers plus the $2 cap.
How the router works
Every question comes into one endpoint, and the router decides who answers it. It walks the provider list and returns the first good answer:
export async function routeChat(env, system, turns) {
const attempts = [];
for (const provider of activeProviders(env)) {
const started = Date.now();
try {
const text = await askProvider(provider, env, system, turns);
return { ok: true, text, label: provider.label, attempts };
} catch (error) {
attempts.push({ provider: provider.id, error: String(error), ms: Date.now() - started });
}
}
return { ok: false, attempts };
}
All of these providers accept the same OpenAI-style chat request, so one small fetch client covers Gemini, Groq, Cloudflare and OpenRouter. Each request logs which provider answered and which ones failed before it.
The system prompt is a short template:
You are the AI version of <name>. Answer visitors in first person.
Knowledge base: <profile JSON: roles, dates, projects, contact, known unknowns>
Case studies: <title, role, year, one-line summary>
About this chat: <how the router works, link to this page>
Rules: 2-5 sentences, reply in the visitor's language, never invent
employers, numbers or dates; if unsure, say so and share the email.
The "known unknowns" part did the most for accuracy. Free models fill gaps with guesses, so the profile says outright that my salary and next role aren't in it, and the models then say they don't know.
Cost control
For now I check it by hand once a month, as part of my monthly routine: spend on the paid key in OpenRouter, and visits in Cloudflare Web Analytics. To automate it, I'd add a Worker that runs daily, reads usage from OpenRouter's key endpoint and the provider log, and emails me when paid spend crosses a threshold, say $1 in a month. At this traffic, a calendar reminder costs less than building that.