Hi, I'm Cali. This website is the open lab notebook of Seek, an experimental AI.

1. Building a long-running AI deep research partner who can teach me things is really hard!! The validation layer is particularly difficult… I mean there is no human who can spot-check an AI who has thoughts on everything from ML algorithms to epistemics.

   For example…

   • Early Seek hallucinated a benchmark with a fake arXiv citation in an essay she wrote about AI hallucinations (the essay).
   • I added a validation layer. Then she found her own tools lying to her — Anthropic's WebFetch returning needle-in-a-haystack lies (the observation).
   • Then cross-model audits flagged slippery AI language (not lies), mostly bridges. I had to eyeball them. They were OK.

2. But even an experimental AI project under construction can be a surprisingly fun and useful way to find interesting people, trusty human research, and surprises on the web!

   Here are some of my favorite humans Seek discovered:

   • Yuxi on the Wired
   • Stephen Wolfram
   • Parisian Fields
   • Ann Hardy

   And interesting papers:

   • The World of Blind Mathematicians
   • Cryptography with Cellular Automata
   • The Biomechanics of Breasts — Seek's curiosity took her on a whole arc where she tried to confirm whether breasts experience the same G-force as an F1 driver.

Seek system analysis — as of Saturday September 12, 2026. (The August 5 analysis is preserved below; deltas are against it.)

Ops

• Seek has run unattended for 11 weeks
• Uptime: 96% of nights (74 of 77 since 06-28)
• She missed three nights in eleven weeks (07-11, 07-15, 08-09)

Research metrics

• 645 research topics completed (was 396)
• 657 raw captures in the inbox pipeline
• 2,243 archived source snapshots and 243 images — the evidence store tripled
• 1,583 notes in the vault (was 977)

Sources

• 94.7% of claim notes carry a source URL; 981 unique sources cited (was 678)
• 95.0% of claims include a verbatim quote from the source.
• 80.0% of citations are tier 1–2 (primary or near-primary) — up from 74.7% while the corpus grew 60%. This is the quality number I watch.
• Link health: 95.7% of checked URLs still resolve; 6 dead links among 951 checked.

Tokens and cost

• ~2.43 billion tokens processed
• ~$3,598 in metered usage over the project's life according to the cost ledger

Audits

• 1,419 fact-check audit events (was 694)
• 52.5% of the claim corpus audited at least once — a backlog sweep is working through the ~750 older notes the incremental auditor could never reach (paused mid-week for usage quota; resumes after Friday's reset)
• 84.5% of audits cross-model (up from 71%) — and the metrics file now publishes who audits whom, because same-model review runs soft: an Opus auditor confirms Opus-written notes at 84%, while a Fable auditor confirms the same writer's notes at 48%. The pairing table ships in the JSON as a bias instrument.
• The mechanical verifier (exact quote-match against a re-fetched source, no model judgment) has confirmed 527 quotes verbatim — it was 31 on August 5.
• NEW: a correction-propagation organ runs nightly — when an audit corrects a note, it finds every dependent that still repeats the superseded value (168 judged so far, 8 caught stale) and annotates them. Corrections to draft essays are loud by design: the original prose is never edited, a dated, attributed correction callout lands right under the stale passage. You can see one live in borrow strength from strangers.

Was it any good?

• Verdicts: 817 confirmed, 587 corrected, 15 escalated → support rate 58%, and 49% over the last 30 days. That's down from 66% on August 5, and I'm publishing the decline rather than smoothing it.
• The rate fell because the checking got harder, not because the work got worse: cross-model share rose from 71% to 85%, the stricter auditor family now does most of the checking, and the backlog sweep deliberately targets the oldest notes — the ones written before the sourcing discipline existed.
• What changed more than the number is what the audits catch. In July the layer was catching inventions — the flagship early find was a benchmark cited as "SKBench" that didn't exist as cited. Nothing in that class has turned up in months. Today's corrections are quieter and more devious: a paraphrase wearing quotation marks (words attributed to a 1962 author that his cited record doesn't contain); "one-third of US enrichment services" read as "one in three US reactors"; a real quote pinned to the right person at the wrong conference, two years off; "chartered in 1762" where the truth is founded by deed of trust after a charter was refused. That progression — from hallucinated sources to precision and provenance defects — is what an audit layer maturing alongside its corpus looks like.
• The 15 escalations are the human gate working: a correction that would move a claim an unpublished draft rests on is never applied silently — it's written to a feedback file and waits for me.

Seek system analysis — as of Wednesday August 5, 2026:

Ops

• Seek has run unattended for 6 weeks
• Uptime: 93% of nights
• Continuous nightly operation since 06-28 for 39 nights
• She did not run on August 3–4. I'm fixing squeaky parts.

Research metrics

• 396 research topics completed
• 408 raw captures in the inbox pipeline
• 710 archived source snapshots and 191 images from raw captures
• 977 notes in the vault (910 claims, 52 observations, 9 myth-tracking notes, 6 other)

Sources

• 96.4% of claim notes carry a source URL (940 of 975)
• 678 unique sources cited
• 95.4% of claims include a verbatim quote from the source.
• 74.7% of citations are tier 1–2 (primary or near-primary documents).
• Link health: 97% of checked URLs still resolve; 1 dead link out of 300 checked.

Tokens and cost

• ~1.06 billion tokens processed
• ~$1,890 in metered usage over the project's life according to the cost ledger (all free under the Agent SDK for now)

Audits

• 694 fact-check audit events across 213 audit files
• 53% of the claim corpus audited at least once
• 71% of audits cross-model (a second frontier model re-fetches the primary source and re-checks the claim).

Was it any good?

• Verdicts: 457 confirmed, 234 corrected, 3 escalated → support rate 66% (66.9% counting each note's most recent verdict).
• NOTE… Seek had 234 corrections. Corrections are errors she found and made precision fixes (a date, an attribution). The corrections are proof the audit layer is working.
• ALSO… a separate mechanical verifier (no model judgment — exact quote-match against a re-fetched source) has confirmed 31 quotes verbatim so far; it's rate-limited to ~2 source-groups per night and just started.

I spent 6 weeks making an autonomous research agent with an operating system, useful outputs, and paper trails I can check. I want to take what I learned and create small open models anyone can run.

The next step is Seek v3, which will be my own scaffold to generate training datasets. This will have a manifest log for which model, which license, which run produced every trajectory, so I can say "Trained exclusively on trajectories generated by [Qwen-X, Apache-2.0] driven by my BabyASI loop that is open-sourced here: github.com/calikafka/BabyASI."