I like to think of myself as a naturally curious person, always wondering and guessing how things work. I often construct explanations for things in my head, based on my observations or an attempt at deduction. I figured it would be fun to have a tool to quickly fact-check such claims. I thought 'waffling' was the colloquial term to describe this but unfortunately only after building this I have realized I had misunderstood it's meaning -- if only I had a tool!

---

I set out with a vision for this tool to be grounded in (Wikipedia)[https://www.optimizationpri.me/posts/ode_to_wikipedia/], and started with the following design for the grader:

Step 1. use an LLM to break down the claim into research topics Step 2. pull related articles from Wikipedia Step 3. pass articles as context to LLM to grade the claim

However I quickly ran into flakiness and latency issues due to cheap LLM models I was targeting and the Wiki API. I realized I didn't need to reinvent anything here as more capable models support websearch. So in the end this is really a Perplexity Sonar wrapper with fun themeing and light gamification. It was a nice reason to play around with Claude Design and PostHog.