Has this been asked on Mindscape?

Sean Carroll’s Mindscape podcast includes monthly Ask Me Anything (AMA) episodes with listener questions in their show notes. I used those questions and the corresponding answers to build a searchable index of 2,639 pairs from 64 episodes. Before submitting a new question, a listener can check whether something similar has been asked. Each match shows the original question, who asked it, and where to find the answer. The page usually responds in about 150 ms because it retrieves existing text without asking a model to generate an answer.

In a businessSupport teams save work by finding an existing answer before they open a new ticket. On 50 paraphrased questions with known sources, the right source ranked first 49 times. That supports suggesting matches for a person to review. Closing duplicates automatically would need its own evaluation.

Technical skills: Python · FastAPI · Text embeddings · Semantic search · Retrieval evaluation

My contribution

I designed and built the project. The public version retrieves existing questions without generating an assessment, keeping the original wording available to compare.

Result and limit

The known source ranks first for 49 of 50 paraphrased test questions. That does not establish a threshold for deciding whether a new question is equivalent.

The current index includes public AMA transcripts through 2026-09-14, refreshed on 2026-09-27.

Evaluation

The measurements below are from the earlier 2,581-pair index. The refresh adds new questions; these scores have not been rerun on the expanded index.

The source of each indexed question is known, so I can test whether the search returns that source when given the original wording or a paraphrase. This measures how well it finds a known question. It does not, by itself, show that the system can reliably decide whether a genuinely new question has already been answered.

The two source-recovery columns are recall@1 and recall@5: 0.98 means the known source was found in 98% of tests.

The nearest other question has a median cosine similarity of 0.635 and a 90th percentile of 0.74. Cosine similarity compares the directions of numerical representations of the questions; higher values mean closer representations, not a probability that the questions are equivalent. Those values help describe how much questions overlap, but they do not establish a reliable boundary between a similar question and an equivalent one. The public page shows the matches and similarity scores for the reader to assess.

The fuller version

The private version also retrieves the passage containing Sean’s answer, extracts the relevant precedent, and classifies the new question as already answered, a close variant, or new. It uses a LangGraph workflow over Heart of Gold. On 20 questions for which the source was withheld, retrieval found the source AMA every time, the extracted precedent was correct 85% of the time, and the status was correct in every case. This took about 73 seconds per question. The public page leaves out that generated assessment to keep the response fast and let the reader compare the original questions directly.

Stack

FastAPI, Qwen3-Embedding-0.6B, a precomputed index; served from a Mac under launchd through a named Cloudflare tunnel.

My role

I designed and built the project. For the public version, I kept the original listener questions available to compare and left out the slower, generated assessment of whether a question has already been answered.

Where this could be useful

A similar approach could help people find an existing answer in a support archive or internal FAQ. The important next test would be whether it distinguishes a related question from one with the same answer on that collection. These are potential applications of an independent project.

If you are hiring, the experience page says which business problem this project shares its mechanics with.