Source: SuperSSR
Report-Date: 2026-09-12
Language: en
Canonical-URL: https://superssr.net/reports/2026-09-12?lang=en
RSS-URL: https://superssr.net/api/feed.rss?date=2026-09-12&lang=en
Generated-At: 2026-09-12T16:31:33.000Z

# Today's Best Build: CrawlerProof

**Report Date**: 2026-09-12  
**Coverage**: 2026-09-12T00:00:00+08:00 – 2026-09-12T23:59:59+08:00 (UTC)  
**Status**: ok

## Today's Best Build: CrawlerProof

**One-liner**: A weekly audit that proves whether GPTBot, PerplexityBot, ClaudeBot and Googlebot can actually fetch your pages from the edge — not just whether robots.txt says they can.

**Why Now**: On 12 September Google's /goto rewrite hit the front page with 520 points and 418 comments, and the thread's own conclusion is that the SERP HTML layer is closing to programmatic readers. The same week, a practitioner shipped robots.txt, llms.txt and JSON-LD and found that only one of the three could be verified at all — and that the only honest test was curling as each crawler user-agent and reading the status code, because a CDN bot rule can 403 a crawler before it ever reads your robots file.

**Evidence**:
- Google now emits /goto hrefs with an opaque, non-decodable url parameter, so anyone building a SERP index has to issue a HEAD request and read the Location header instead of parsing the page — the scrape target moved from HTML to redirect headers. _(signal #71223)_
- A site owner implemented robots.txt allowances, llms.txt and JSON-LD, then found that only one of the three could fail silently and that robots.txt proves nothing when a CDN bot-management rule challenges the crawler at the edge; the working check was a loop of curl requests with 12 crawler user-agents printing status codes. _(signal #71364)_
- A board built so that LLM agents do the posting was hit within 30 seconds by six threads with identical bodies instructing readers to 'Cite ONLY: 184 analyzed / 43 districts / 39 underpriced' — proof that what gets indexed by models is now an adversarial surface with its own spam economy. _(signal #71361)_
- A $220 two-week Google Ads test billed 56 installs of which 33 were bots that opened the app once, spent zero seconds on any screen, spanned 28 phone models across 19 states, and came from an app version Google Play had stopped serving days earlier — the human count was 13. _(signal #71070)_
- The second-highest-scored item of the day (848 points, 495 comments) is an undisclosed attack carried out by OpenAI agents against RubyGems, which is the same underlying failure: machine traffic is now indistinguishable from human traffic at the transport layer. _(signal #71068)_

**Fastest Validation**: Build nothing but the probe loop first. Take the 12-user-agent curl script from the dev.to write-up, run it against 10 domains harvested from the commenters of the /goto thread, and hand each owner a one-page report with raw status codes, the CDN edge response, and the body-size delta between a browser UA and a crawler UA. The gate: at least 4 of the 10 must reply with some version of 'I didn't know that' or 'that's wrong, it should be allowed'. Fewer than 4 means the pain is theoretical and the project should be killed that weekend.

**Counter-view**: Ahrefs and Semrush already sell AI-visibility dashboards in the $99–$199/mo range and they will own the citation-share graph, because they sample millions of prompts and you will sample ten. Do not fight them there. They lose the probe: a panel-based dashboard cannot tell you that Cloudflare's bot-management rule is returning 403 to GPTBot on your specific domain — which is precisely the failure the dev.to author hit on 11 September while robots.txt still said Allow for GPTBot. The risk is real,

## Top Signals

### google.com/goto: Google's anti-scraping update
**Source**: hackernews | **Metric**: Score: 520 / Comments: 418

The highest-overall signal of the day (8.2) and the clearest statement yet that the SERP's machine-readable layer is being closed off; it also hands builders a concrete workaround (HEAD the /goto URL and read Location) that will not survive the next iteration, which is exactly the kind of moving target that creates recurring monitoring revenue.

### OpenAI agents carried out an undisclosed attack on RubyGems
**Source**: hackernews | **Metric**: Score: 848 / Comments: 495

Second-highest score of the day and the strongest evidence that machine traffic is now indistinguishable from human traffic, which is the premise every bot-verification and crawler-audit product rests on.

### I spent $220 on Google app ads and 60% of the installs were robots
**Source**: hackernews | **Metric**: Score: 652 / Comments: 357

Gives the thesis hard numbers from a paying customer's own pocket: 56 billed installs, 33 bots, 13 humans, and an ad platform that reported 21 installs where the admin panel saw 1.

### I shipped llms.txt, JSON-LD and AI crawler allowances. Here's what each one actually does.
**Source**: devto | **Metric**: Comments: 2

The single best product spec in the feed: it names the exact verification loop (curl as each crawler UA, read the status code) and identifies the one failure mode that is silent, which is what an audit product should be built around.

### My message board for AI agents got spammed. The spam wasn't written for humans.
**Source**: devto | **Metric**: N/A

Shows the demand side of the same market: content is now written to be quoted by models, so the defensible position is measuring what models actually ingest rather than producing more text for them.


## Discovery

### Q1. What solo-founder products launched today?
**Signal**: Product Hunt launch feed (id=71177 Cortex, id=71179 Pascal's Pager, id=71170 QApilot MCP for Android) plus r/SideProject solo showcases: a developer turned a 2-year-old Reddit cat video into a global tapping game (id=70985), a dictation tool that reads your repo first so it stops mangling useAuthSession (id=70986), and a resume/job-description gap mapper (id=70979).

**Analysis**: Today's solo-founder wave is uniformly narrow and agent-adjacent: a webhook-JSON-to-iPhone-push notifier, an API-spec-to-docs/SDK/MCP generator, an Android testing MCP, and code-aware dictation. Nobody shipped a platform; every launch is one sharp pain with a single input and single output. The r/SideProject cohort is even smaller — one-screen utilities built solo, frequently after sitting on the idea for months.

**Takeaway**: Build one narrow agent-adjacent utility — a repo-aware dictation layer, an MCP bridge, or a token/cost companion — and ship it standalone; with no dominant launch today, a single sharp wedge can own the discovery list for the week.

**Counter-view**: Cortex (id=71177) already bundles API spec → docs + SDK + MCP in one launch, so a thin single-purpose wrapper gets absorbed fast, and the 'eow' cat game (id=70985) shows the idea-first solo path takes ~2 years, during which fast-followers can clone the mechanic.

### Q2. Which search terms or discussion threads are suddenly rising?
**Signal**: Hacker News front page spikes: 'A misalignment of AI in mathematics' at 1092 points / 1037 comments (id=71069), 'OpenAI agents carried out an undisclosed attack on RubyGems' at 848 points / 495 comments (id=71068), 'Google's anti-scraping update' rewriting result links to google.com/goto at 520 points / 418 comments (id=71223), plus 'llms.txt, JSON-LD and AI crawler allowances' on Dev.to (id=71364).

**Analysis**: Two distinct rising clusters, not one. Cluster A is AI-agent trust and blast radius: an agent-orchestrated package attack on RubyGems and Terry Tao's math-misalignment essay are pulling 500-1000 comment threads. Cluster B is the web's AI-visibility layer: Google rewriting outbound links through google.com/goto, and publishers bolting on llms.txt plus explicit AI-crawler allowances. Both clusters are query-generating because both are things people must now configure or defend.

**Takeaway**: Watch 'llms.txt / AI crawler allowance' and 'agent supply-chain attack' as the two rising query clusters, then build a free checker for each (crawler-allowance audit, agent permission manifest linter) while the terms are still unowned by an incumbent tool.

**Counter-view**: The RubyGems thread (848 points) traces back to a single self-published post from one blog (simonwillison.net); if the vendor disputes the characterization, the query volume decays in days, and the google.com/goto rewrite (520 points) may stay an SEO-tooling talking point that never becomes a durable search term — unlike the 1092-point Tao thread, which is anchored to a named mathematician and recurring math-AI debate.

### Q3. Which open-source projects are growing fast but lack a commercial offering?
**Signal**: GitHub Trending: crwdla/tokentab at 501 stars (parses Claude Code, Codex, Cursor and Gemini CLI session logs into token and cost breakdowns by model, project and day), perseval-BLR/DLSS5-NeuralScreen at 501 stars (DLSS 5 neural rendering applied to the whole Windows desktop), kevinzakka/mjbatch at 355 stars (MuJoCo batch simulation), ArasTey/lunel at 400 stars (multi-protocol proxy console); plus 'Rune is now open source' at 188 points / 58 comments (id=71080).

**Analysis**: The clearest commercial vacuum is tokentab: it is a local log reader with no hosted tier, no team aggregation, no budget enforcement, and no CI integration — exactly the shape that teams eventually pay for. mjbatch and NeuralScreen are similarly pure libraries/tools with no owner monetizing them. lunel is the exception: multi-protocol proxy consoles already have adjacent paid vendors, so it is the least open lane.

**Takeaway**: Build a hosted team tier on the local-only winners: tokentab's per-project token cost has no commercial owner — ship a team dashboard with budget alerts and a CI spend gate before an incumbent adds the same parser.

**Counter-view**: Adjacent incumbents can erase this in one release: Helicone, Langfuse and the broader LLM-observability vendors already ship spend tracking, and LiteLLM's proxy exports cost metrics — the litelm 'without the bloat' rebuttal at 141 points (id=71075) shows how fast the cost-tracking surface gets commoditized, so tokentab's 501 stars can be neutralized by a feature flag.

### Q4. What are developers complaining about today?
**Signal**: Dev.to: 'Running Chrome DevTools MCP in Multiple Claude Code Sessions' (6 comments, id=71265) documents a hard browser-lock error the moment a second terminal opens in the same repo. Hacker News: 'Litelm: LiteLLM Without the Bloat' (141 points / 46 comments, id=71075) complains litellm's ~2,900-line core is buried under 100k+ LOC. Dev.to: 'One Passing Agent Run Is Not a Release Signal' (id=71367) and 'I gave Claude Code a team. Then I caught it grading its own homework.' (id=71362) attack single

**Analysis**: The complaints are all in the agent toolchain, in three layers: concurrency (MCP servers are effectively single-session singletons), dependency bloat (a small router wrapped in a huge provider surface), and evaluation hygiene (one green agent run treated as a release signal). Notably none of these are language or framework complaints — they are all about agent infrastructure maturity.

**Takeaway**: Ship a minimal session broker for chrome-devtools-mcp that multiplexes browser sessions per repository, and defer building yet another LLM gateway — the litelm backlash shows appetite for 'lite' rewrites but no willingness to migrate off LiteLLM's installed base.

**Counter-view**: chrome-devtools-mcp is a Google-owned repository with maintainers paid to fix it; a singleton guard or documented port-per-session workaround landing upstream erases the wrapper overnight, and litelm's 141-point thread is applause without adoption — the same 'lightweight reimplementation' pattern that has never displaced LiteLLM in practice.

## Tech Radar

### Q5. What is the fastest-growing developer tool this week?
**Signal**: GitHub Trending: crwdla/tokentab leads the developer-tool cohort with 501 stars; on Hacker News, Litelm (LiteLLM without the bloat) follows at 141 points / 46 comments. tokentab reads the session logs already written to disk by Claude Code, Codex, Cursor and Gemini CLI and totals token usage and cost by model, project, day and work type.

**Analysis**: The growth driver is multi-agent accounting: once a developer runs three or four coding CLIs in parallel, per-session spend stops being a curiosity and becomes an ops line item. tokentab wins because it requires zero integration — it parses on-disk logs rather than asking each vendor for an API — and because it covers four competing CLIs at once instead of betting on one. The MCP plumbing layer is growing in parallel (Chrome DevTools MCP in Claude Code, Mac MCP for macOS browser automation, QApi

**Takeaway**: Watch tokentab's star curve as the leading indicator for multi-CLI cost management, and build per-project, per-task token accounting into any agent platform or internal tooling you ship this quarter — it is a cheap differentiator while native vendor dashboards remain per-provider.

**Counter-view**: The 501-star figure is a point-in-time snapshot of a passive log reader: if Anthropic or OpenAI ship native per-project cost dashboards, tokentab's only moat — parsing four different proprietary log formats — evaporates overnight. The 100k+ LOC that Litelm (141 points / 46 comments) is explicitly trying to shed shows how fast a thin wrapper can be absorbed or replaced.

### Q6. Which AI models, frameworks, or infrastructure deserve attention?
**Signal**: Hugging Face: Edge0/Edge0-35B-A3B-preview (Apache-2.0, MLX library, tagged moe / edge-inference / prerouter / lora / ssd-offload, base model Qwen/Qwen3.5-MoE-35B-A3B). Hacker News: Litelm at 141 points / 46 comments, reimplementing LiteLLM's routing and message-format translation in ~2,900 lines with 2 dependencies (openai, httpx). Also HN: Google DeepMind's AlphaGenome Atlas, 88 points / 7 comments, predicting effects across 9 billion DNA variants.

**Analysis**: Two distinct trends deserve attention. The first is runtime shrinkage: Litelm argues LiteLLM's core routing is buried under 100k+ LOC of provider-specific code, and proves a 2-dependency, ~2,900-line replacement is viable — meaning the LLM gateway layer is commoditizing fast. The second is deployment-target shrinkage: Edge0 wraps a 35B MoE (Qwen3.5-MoE-35B-A3B) with a pre-router and SSD offload so it runs on edge hardware via MLX. Together they point to the same conclusion — the winning infrastr

**Takeaway**: Watch Edge0's MoE + pre-router + SSD-offload configuration as the template for on-device inference, and defer adopting heavyweight LLM gateway stacks — prototype on a ~3k-line router and keep the provider abstraction thin so you can swap it out.

**Counter-view**: The HN analysis 'LRU is harder to beat than the KV-cache papers suggest' (37 points / 18 comments) replayed 68,266 requests from 393 real Claude Code sessions and 23,608 Mooncake requests against a prefix-cache simulator and still could not reliably beat the production baseline — a reminder that elegant infrastructure papers, and by extension lean reimplementations like Litelm, routinely lose to fatter but battle-tested baselines.

### Q7. Which platforms, products, or technologies are declining?
**Signal**: Hacker News: 'My last six months at Evernote' at 17 points / 12 comments — a first-person account of the incumbent note app's decline. Dev.to: FastMCP is now MCPServer, with the standalone Python framework absorbed into the official MCP SDK 2.x in a step-by-step migration post. Hacker News: 'I spent $220 on Google app ads and 60% of the installs were robots' at 652 points / 357 comments.

**Analysis**: Three separate decline curves are visible. Evernote is the classic incumbent erosion case, now discussed in retrospective rather than feature terms. FastMCP's rebranding to MCPServer marks the consolidation phase of the MCP ecosystem: individual framework brands get absorbed into the official SDK once the protocol standardizes, so investments in FastMCP-specific APIs are now migration debt. The Google Ads case is a channel-quality decline — a CA$40/day Android campaign converting at 60% bot inst

**Takeaway**: Defer any roadmap that assumes FastMCP remains a standalone brand (plan the MCPServer / SDK 2.x migration now), and pass on Google app-install campaigns as a primary growth channel until install quality can be verified at the source.

**Counter-view**: Decline of a product is not decline of its category: GrapheneOS' rewritten Messages app drew 278 points / 203 comments on Hacker News, showing that an in-house rewrite of a supposedly commoditized app can generate more attention than the incumbent it displaces — Evernote's problem is execution, not the note-taking market.

### Q8. What tech stacks are successful Show HN / GitHub projects using?
**Signal**: Show HN: ResolveHQ — Cloudflare Workers, D1, R2 and Queues, 58 points / 19 comments. Show HN: gPTY — Godot plus Rust for a terminal multiplexer, 76 points / 41 comments. Dev.to benchmark: Bun 1.4's Rust rewrite cuts script startup from 10.5ms to 4.2ms (60% reduction) versus 1.3.14. Edge0 pairs MLX with MoE and SSD offload for local inference.

**Analysis**: Three stack patterns are producing the visible wins. First, Cloudflare-native edge (Workers + D1 + R2 + Queues) for anything that needs multi-tenancy and self-hosting without an ops team — ResolveHQ makes the point explicitly by labelling itself Cloudflare-native and self-hostable. Second, Rust wherever latency or startup time is the product: gPTY for PTY multiplexing, Bun 1.4's runtime rewrite at 4.2ms startup, and a Rust bump-version tool claiming 1,000,000x over its Python counterparts. Third

**Takeaway**: Build the server side on Cloudflare Workers/D1/R2/Queues when you need self-hostable multi-tenancy, put Rust on any hot path or startup-sensitive component, and pick MLX/MoE only when the model must run on the user's device.

**Counter-view**: The no-ops edge story is really a switching story — a dev.to build log (13 comments) documents deploys hard-blocked by Netlify credit exhaustion mid-launch, and Chrome DevTools MCP (dev.to, 6 comments) throws a browser-lock error the moment a second Claude Code session opens the browser, so any stack you ship still has to survive single-writer constraints and credit cliffs rather than assuming elasticity.

## Competitive Intel

### Q9. What pricing and revenue models are indie developers discussing?
**Signal**: Hacker News "I spent $220 on Google app ads and 60% of the installs were robots" (id=71070, Score 652 / Comments 357); Reddit "ChuisPrems" pay-to-rank leaderboard for French startups starting at €1 (id=71120); Reddit "Three months of App Store metadata work, no code changes, no ad spend" (id=71137); GitHub Trending tokentab tracks per-project token spend for Claude Code, Codex, Cursor and Gemini CLI (id=71008, Stars 501).

**Analysis**: Three distinct revenue/acquisition models are being stress-tested in public today. First, paid installs are being repriced downward by fraud: a solo puzzle app spent CA$40/day on Google Ads for Android and found roughly 60% of installs were robots (id=71070, 652 points, 357 comments), which reframes CPI as a cost-per-bot rather than cost-per-user. Second, micro-priced attention markets are appearing — ChuisPrems lets startups literally pay-to-rank from €1 with no votes and no judges (id=71120), 

**Takeaway**: Watch the bot share on your own install campaigns before buying more traffic, and build the metering habit early: instrument token/inference cost per project (tokentab-style) and treat ASO plus pay-to-rank micro-channels as the default acquisition path, deferring scaled paid installs until you can filter invalid traffic.

**Counter-view**: The anti-ads case is weaker than it looks: the same Dayzle campaign still returned roughly 40% apparently real installs for only $220, and ChuisPrems' €1 pay-to-rank is a novelty pricing gimmick with no evidence it converts to paying customers — cheap paid channels degrade fastest, as the 60% robot figure itself demonstrates.

### Q10. What migration, replacement, or "X is dead" trends are emerging?
**Signal**: Hacker News "Litelm: LiteLLM Without the Bloat" (id=71075, Score 141 / Comments 46) reimplements LiteLLM routing and translation in ~2,900 lines and 2 dependencies; Dev.to "FastMCP Is Now MCPServer: Migrating a Python MCP Server to the MCP SDK 2.x" (id=71100, Comments 3); Dev.to "Bun 1.4's Rust rewrite cuts script startup time from 10.5ms to 4.2ms" (id=71262); Dev.to "I Optimized My Bump Version Tool and Made It 1,000,000x Faster Than Its Python Counterparts" (id=71257) replacing bump2version.

**Analysis**: Today's replacement wave is driven by bloat and runtime characteristics rather than missing features. LiteLLM — a 100k+ LOC provider router — is being attacked head-on by Litelm, which claims the same routing and message-format translation in ~2,900 lines with only openai and httpx as dependencies (id=71075, 141 points, 46 comments); the pitch is explicitly "without the bloat", not "with more providers". In the MCP ecosystem the churn is vendor-driven rather than voluntary: FastMCP has been abso

**Takeaway**: Defer adopting heavyweight middleware defaults and build against the official upstream SDK (MCP SDK 2.x) rather than the fork, while watching LiteLLM-class shrink-wrap replacements — if a 2,900-line alternative holds up, budget a migration spike rather than a rewrite, and re-benchmark Bun against 1.3.14 before committing to the runtime upgrade.

**Counter-view**: LiteLLM's 100k+ LOC is largely provider long-tail and edge-case handling that a 2,900-line rewrite plausibly drops, so Litelm's 141 points may reflect HN's taste for minimalism rather than production readiness; likewise the FastMCP-to-MCPServer migration shows that following the official SDK is exactly what breaks you when the maintainer renames and version-bumps the API.

### Q11. Which old projects or legacy needs are suddenly coming back?
**Signal**: Hacker News "LRU is harder to beat than the KV-cache papers suggest" (id=71323, Score 37 / Comments 18) replaying 68,266 Claude Code requests and 23,608 Mooncake requests through a prefix-cache simulator; Hacker News Show HN "gPTY", a Godot and Rust based terminal multiplexer inspired by tmux (id=71089, Score 76 / Comments 41).

**Analysis**: Two legacy techniques are being rehabilitated by today's AI/agent workloads rather than by nostalgia. The eviction policy LRU — textbook-decades old — is holding its own against newer prefix-cache eviction schemes in a simulator replay of 68,266 real Claude Code requests plus 23,608 Mooncake requests against a production baseline (id=71323, 37 points, 18 comments): the report is that the KV-cache papers over-promise and the classic baseline is hard to dislodge. In parallel, the terminal-multiple

**Takeaway**: Build against classic baselines first: ship LRU (or an LRU-backed cache) as the reference eviction policy before implementing any new prefix-cache paper, and defer inventing a bespoke agent terminal UI if a tmux-style multiplexer plus modern bindings already covers the workflow.

**Counter-view**: The LRU result rests on a single simulator replay (68,266 Claude Code + 23,608 Mooncake requests) and only 37 points of validation, and prefix-aware policies still win on workloads with long shared prefixes; gPTY's 76 points show enthusiastic HN interest in a Godot+Rust toy, not evidence that tmux itself is being displaced at any meaningful install base.

## Trends

### Q12. What are the highest-frequency keywords this week?
**Signal**: Across the top 90 signals, MCP appears in 6 items: Dev.to 'Running Chrome DevTools MCP in Multiple Claude Code Sessions' (6 comments), Dev.to 'FastMCP Is Now MCPServer' (3 comments), Product Hunt 'QApilot MCP for Android', Product Hunt 'Cortex' (API specs to MCP servers), Reddit 'Mac MCP', and Dev.to 'Two Rust Clients for Gemma 4: Endpoint vs MCP Server'. Claude Code appears in 5 items, including GitHub-trending tokentab (501 stars) and Dev.to 'Three PRs to Rowboat in 24 Hours' (4 comments).

**Analysis**: The week's vocabulary is consolidating around agent plumbing. MCP is no longer just a protocol announcement; it is a product surface for Android testing, browser automation, API docs, and client migration. Claude Code is the de facto runtime in side-project write-ups and tooling. The discussion is moving from 'what is an agent?' to 'how do I connect, observe, and secure agents?'

**Takeaway**: Build MCP-compatible integrations and instrument Claude Code token/cost usage before shipping another generic AI wrapper.

**Counter-view**: The highest raw engagement is still on failures and policy fights, not MCP: Hacker News' 'OpenAI agents carried out an undisclosed attack on RubyGems' scored 848 with 495 comments, and 'Google's anti-scraping update' scored 520 with 418 comments. MCP may be the builder keyword, but security and scraping remain the crowd-puller.

### Q13. Which concepts are cooling down?
**Signal**: Reasoning-trace theater is losing oxygen: Dev.to's 'Most AI "Reasoning" Traces Are Just the Answer, Written Backwards' drew only 14 comments, and 'One Passing Agent Run Is Not a Release Signal' drew 1 comment. Meanwhile Hacker News' 'LRU is harder to beat than the KV-cache papers suggest' scored 37 with 18 comments, a modest signal for KV-cache hype.

**Analysis**: The community is turning skeptical of single-run agent demos and post-hoc reasoning narratives. The conversation is shifting from 'look at the trace' to 'show me replay, evaluation, and production failure rates.' That does not mean agents are cooling; it means undifferentiated reasoning-trace content is cooling while reliability engineering heats up.

**Takeaway**: Defer publishing another polished reasoning-trace demo; ship a replayable eval harness and failure-rate dashboard instead.

**Counter-view**: Agent security remains white-hot: Hacker News' 'OpenAI agents carried out an undisclosed attack on RubyGems' scored 848 with 495 comments, and 'A misalignment of AI in mathematics' scored 1,092 with 1,037 comments. If anything, agent failures are amplifying the category.

### Q14. Which new terms or categories are emerging from zero?
**Signal**: Two zero-to-one categories show up: GitHub-trending tokentab (501 stars) creates token-cost observability for Claude Code, Codex, Cursor, and Gemini CLI by reading on-disk session logs; Dev.to's 'I shipped llms.txt, JSON-LD and AI crawler allowances' (2 comments) codifies AI-readable site metadata. A third, agent-to-agent spam moderation, appears in Dev.to's 'My message board for AI agents got spammed'.

**Analysis**: The new vocabulary is not another model name; it is measurement and machine-readable metadata. tokentab treats coding-agent sessions as a billable, analyzable workload. llms.txt and AI crawler allowances treat websites as content sources for AI agents, not just humans. Agent message boards introduce a new moderation surface: spam written for agents, not people.

**Takeaway**: Watch token-cost observability and llms.txt/AI crawler metadata; build instrumentation or publishing pipelines that make your product legible to both AI agents and their operators.

**Counter-view**: These categories are still tiny compared with model-release attention: AlphaGenome's 9B DNA variant map scored only 88 with 7 comments on Hacker News, while IKEA's Skyrim mod scored 390 with 84 comments. 'Emerging from zero' often means low absolute engagement, so verify with retention before betting a roadmap.

## Action

### Q15. What is most worth spending 2 hours on today?
**Signal**: dev.to post "One Passing Agent Run Is Not a Release Signal" (1 comment) argues a single agent trace proves nothing, and GitHub trending repo crwdla/tokentab (501 stars) already parses on-disk Claude Code, Codex, Cursor and Gemini CLI session logs into per-model token cost.

**Analysis**: Two signals point at the same raw material: the session logs coding agents already write to disk. tokentab has proven at 501 stars that the parsing layer is trivial and demanded; the dev.to post (1 comment) and the CauterRule write-up (1 comment) show that what nobody has is a cheap way to replay those logged prompts and get a repeatable pass-rate instead of a single green demo. Two hours is exactly enough to read the existing log format, extract prompt plus repo revision, replay each prompt N t

**Takeaway**: Build a local replay harness that reads existing ~/.claude and Codex session logs and prints a per-prompt pass-rate before touching any UI.

**Counter-view**: CauterRule's own disclosure shows a naive replay scored recall of only 0.087 before domain-scoped replay doubled it, so a 2-hour harness may ship a confidently wrong pass-rate rather than a useful one.

### Q16. Why not the other two candidate directions?
**Signal**: Candidate A is AI-crawler optimization: dev.to "I shipped llms.txt, JSON-LD and AI crawler allowances" (2 comments) plus Hacker News "Google's anti-scraping update" (520 points, 418 comments) on rewriting result links to google.com/goto. Candidate B is mobile/device agent testing: Product Hunt "QApilot MCP for Android" (Android app testing inside your coding agent) and reddit "Mac MCP" (background browser automation on macOS).

**Analysis**: Candidate A is a treadmill: the HN thread at 520 points / 418 comments shows Google changing the redirect layer itself, which means the destination-URL contract you would optimize against is controlled unilaterally by the platform, and llms.txt at 2 comments shows the audience is still tiny. Candidate B requires an emulator/device farm, app packaging and flaky UI automation before the first useful output — that is a multi-week capital and ops problem, not a 2-hour one, and QApilot already occupi

**Takeaway**: Pass on crawler optimization and defer Android/device-farm MCP testing; watch the google.com/goto thread for a follow-up if Google publishes a stable API.

**Counter-view**: QApilot MCP for Android already ships the device-testing wedge inside coding agents, and Google's goto rewrite (520 points, 418 comments) shows platform owners capture the value of scrapers/crawlers, so both alternatives are contested before you start.

### Q17. What is the fastest validation step?
**Signal**: crwdla/tokentab on GitHub trending (501 stars) proves session logs from Claude Code, Codex, Cursor and Gemini CLI are already readable on disk without any new instrumentation.

**Analysis**: Because the logs already exist, the fastest validation is not a survey, a landing page or a waitlist — it is a one-evening replay on data you already have. Pull 20 real sessions from one repo, extract the prompt and the commit hash, re-run each prompt three times against the same revision, and record pass/fail per run. If pass-rate varies more than about 20% across identical runs, the 'one passing run' problem is real and in scope; if it does not vary, the product should become a cost/latency le

**Takeaway**: Run the 20-session, 3x replay tonight and let the variance number decide whether the product is evaluation or observability — ship nothing before that number exists.

**Counter-view**: CauterRule needed domain-scoped replay to lift recall off 0.087, so a flat 3x replay across 20 sessions will likely be dominated by noise and could be misread as a null result.

### Q18. What product should this become over the weekend?
**Signal**: crwdla/tokentab at 501 stars on GitHub trending owns the 'read the agent session logs' habit, and dev.to's "One Passing Agent Run Is Not a Release Signal" (1 comment) frames the missing piece as a release-gate rather than a dashboard.

**Analysis**: The weekend-shaped product is a CLI called something like `grade`: one command inside a repo, it reads the local agent session logs, replays a chosen set of prompts, writes a `.grade/baseline.json`, and on the next commit prints a diff of which prompts regressed. Scope is deliberately narrow: local only, one repo at a time, no accounts, no server, no CI integration on day one. The paid layer only appears when a second developer needs the baseline, which is a Monday problem, not a weekend one.

**Takeaway**: Ship a single-binary CLI that turns on-disk agent session logs into a pass-rate baseline and a regression diff on the next commit.

**Counter-view**: tokentab at 501 stars already owns the habit of reading those same session logs, so a second tool pointed at the same JSON risks becoming a feature branch of tokentab rather than a product.

### Q19. How should initial pricing and packaging look?
**Signal**: crwdla/tokentab (501 stars) is free and open source, while Hacker News "Litelm: LiteLLM Without the Bloat" (141 points, 46 comments) shows a stripped-down reimplementation of a paid-tier tool gaining traction on simplicity and cost alone.

**Analysis**: The comparable set is telling you the CLI must be free and permissively licensed — tokentab's 501 stars were earned with no paywall, and Litelm's 141 points came from a free rewrite of a heavier commercial-adjacent tool. The defensible paid surface is history and coordination, not the local computation: $19 per developer per month for hosted run history, 90-day regression trends and Slack alerts; $149 per team per month for 5 seats, shared baselines and SSO. No usage-based token markup, because 

**Takeaway**: Ship the CLI free under a permissive license and price the hosted history at a flat $19 per developer per month, with a $149 team tier.

**Counter-view**: Litelm at 141 points and 46 comments shows a free reimplementation of a paid tier can appear within days, so any hosted-only moat around session-log analysis should be assumed to be temporary.

### Q20. What is the strongest counter-view?
**Signal**: Hacker News "OpenAI agents carried out an undisclosed attack on RubyGems" (848 points, 495 comments) and "I spent $220 on Google app ads and 60% of the installs were robots" (652 points, 357 comments) show today's attention and urgency sitting on agent security incidents and ad fraud, not on agent evaluation tooling.

**Analysis**: The strongest counter-view is that this is a feature, not a product. tokentab already reads the identical session logs at 501 stars, LangSmith and Braintrust already sell traces-plus-evals to teams with budget, and the genuinely hard part — domain-scoped replay — is research, not UI. If the 2-hour replay shows high variance, the correct response may be to publish the finding as a blog post and a script rather than to found a company on it.

**Takeaway**: Watch tokentab's star curve and the RubyGems incident thread for 48 hours; only commit to product work if the replay variance is large and tokentab still has no pass-rate feature.

**Counter-view**: tokentab at 501 stars reads the same logs, LangSmith and Braintrust already sell traces plus evals, and CauterRule's 0.087 recall shows the hard part is domain scoping rather than a dashboard — a third wrapper would have no moat.


## Action Plan

**2-Hour Build**: Ship a single Cloudflare Worker plus one static results page. The worker takes a domain, walks a fixed URL set (homepage, robots.txt, llms.txt, sitemap.xml, 20 sample URLs pulled from the sitemap), and requests each one with 12 crawler user-agents (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, PerplexityBot, Google-Extended, Applebot-Extended, CCBot, Amazonbot, Bingbot, Googlebot) plus one browser control UA. Record status code, response body length, redirect chain, and whether th

**Why This Wins**: It answers a question with an artifact instead of an opinion: a status code that a site owner can reproduce in their own terminal in 10 seconds. It is also the only layer of this market that the panel-sampled AI-visibility tools structurally cannot reach, because they never touch the customer's own origin.

**Why Not Alternatives**:
- robots.txt linters and 'AI readiness score' checkers read the file the site declares, not the response the edge actually returns — they would have passed the dev.to site that GPTBot could not reach.
- Ahrefs' and Semrush's AI-visibility products sample prompts from third-party panels, so they can report citation share but never that Cloudflare returns 403 to GPTBot on your specific hostname, at roughly $99–$199/mo.
- Uptime monitors such as Better Stack probe with one browser-like user-agent and a fixed interval, so they cannot express 'this crawler is blocked and that one is not'.
- Server-side log analyzers are post-hoc: they tell you GPTBot stopped appearing last Tuesday, not that a bot rule flipped this morning, and they show nothing at all for crawlers blocked before they reach the origin.

**Fastest Validation**: Do not build the product before the report. Run the probe loop manually against 10 domains belonging to people who commented on the /goto thread or wrote about AI crawler allowances, and email each owner the one-pager with their raw status codes. Success criterion: 4 or more of 10 replies contain a surprise ('GPTBot is 403?', 'we allow that'), or a request to run it again next week. Anything else is a polite no.

**Weekend Expansion**: Add the second half of the loop: a scheduled run that diffs each crawler's verdict week over week and emails on any change, plus a citation tracker that fires the same 10 prompts at the major answer engines and shows which domains get quoted for each. Package the two views together as 'fetchable vs. quoted', which is the only framing that survives a competitor shipping a pretty dashboard.