How I built Mindscape Explorer

Archive refresh, 2026-09-27. The live Explorer now covers 441 episodes through 2026-09-21. Search and Ask Mindscape cover 441 episodes through 2026-09-21, and the AMA checker contains 2,639 questions from 64 AMA episodes. The refreshed topic model selects 40 middle-resolution topics rather than 50; current charts are in the Explorer’s Findings view. The build history and evaluation figures below describe the earlier snapshot.

I’ve listened to Sean Carroll’s Mindscape since it began in 2018. The range of subjects is part of what I enjoy about it, but it also makes the archive difficult to explore through episode titles alone. I wanted to know which subjects recur, which subjects rarely appear together, and whether solo lectures move through their material differently from interviews. I also wanted to find discussions when I could remember the idea but not the words used to describe it.

I built Mindscape Explorer (opens in a new tab) to investigate those questions, along with an AMA checker (opens in a new tab) for finding earlier listener questions. I ran the analyses on my own machine. This post explains the main decisions and what I think the results do and do not show.

The Explorer now makes 439 episodes searchable by discussion and topic. For example, someone who remembers a conversation about “dark matter” can find passages and their timestamps without remembering the episode title. Recommendations can also follow an episode’s less prominent subjects to other parts of the archive.

The topic-based system predicted Sean’s subject tags better than the full-text baseline tested here, and recommendations through minor topics were more varied than the similar-episode lists. One expectation did not hold up: I did not establish that solo lectures stay more focused than interviews. The findings depend on how topics and changes of subject are measured; the methods below show those choices.

Technical methods and evaluation

Jump to topic selection, the unexpected result, or search.

Getting the transcripts without being a nuisance

The transcripts are included in the WordPress posts available through the site’s public API, so collecting the archive takes twenty or thirty requests rather than a separate request for every episode page. The scraper checks robots.txt, identifies itself with a user agent containing my email address, waits three seconds between requests, and honors Retry-After. It also caches responses so that changing the parser does not require fetching the same material again. A full run took about two minutes, largely because of the delays between requests.

Parsing required handling five conventions for marking speakers and timestamps across the archive. Of 444 posts, 439 have transcripts and 436 can be parsed into labeled utterances. The 5 without transcripts are a welcome post, two holiday messages, a Patreon-only AMA, and a post linking to an outside interview. The 3 transcripts without speaker labels are kept with unknown speakers and a flag explaining why. An unrecognized format is also flagged, which gives me a way to identify parser failures instead of silently losing episodes. The resulting corpus contains about 7.4 million words.

A pipeline that knows what changed

The analysis has sixteen stages, several of which depend on the outputs of others. To avoid reusing stale results, each stage records the corpus fingerprint, its source-code hash, its parameters, and the versions of its inputs. A change causes the affected stage and its dependents to run again. The website is the final build stage, so rebuilding through the pipeline updates it from the analysis outputs. Publishing still requires deploying that build.

Deciding how many topics there are

I need groups of words that describe recurring subjects, at a level of detail that remains reasonably stable when the archive changes. The topic model starts with TF-IDF features over one- to three-word terms and uses non-negative matrix factorization. Including multiword terms matters here because phrases such as “dark matter” and “wave function” occur frequently. Choosing the number of topics requires a separate decision. The pipeline compares values of k using coherence, stability across random restarts, and reconstruction error.

In the tested range, coherence increased with k, so selecting its maximum would simply select the end of the range. Combining coherence with stability produced a peak, but its location was sensitive to changes in the corpus. Adding two episodes shifted the maximum from 20 topics to 50. The selection therefore uses the one-standard-error rule. It chooses the smallest candidate within one standard error of the best combined score, treating differences within that band as insufficient grounds for selecting a larger model.

The selected anchor is 50 topics. The other two resolutions are 27 and 92, derived by dividing and multiplying the anchor by approximately 1.84, the tribonacci constant used in the larger project. The anchor is selected from the data; the ratio between resolutions is a design choice inherited from that project. The analysis then compares adjacent resolutions using similarity matrices and best-parent tables, rather than assuming that the finer topics necessarily fit inside the coarser ones.

Checking the model against the person who made the show

Topic labels can look plausible without showing that the representation is useful. Sean’s own subject tags provide one way to check this. The topic model was fit without those tags, and a classifier then uses the topic loadings to predict them. Classifiers are tuned on inner folds and scored on held-out outer folds. The comparison covers 17 subject tags, excluding format labels such as “Podcast” and “Solo.”

A gradient-boosted classifier using the topic loadings achieved macro average precision of 0.649, against a chance baseline of 0.078. Logistic regression on the full TF-IDF matrix scored 0.549. The topic-based system performs better in this comparison. It is tempting to attribute the improvement to the topics alone, but the classifier changes too: one system uses gradient boosting and the other logistic regression. The result supports choosing between these two tested systems; identifying what the topic representation contributes would require holding the classifier fixed. Physics, Biology, Astronomy, Economics, and Environment scored between 0.81 and 0.90. Humanity, World, and Psychology were harder to predict, at 0.37 to 0.42. Differences in how Sean assigns the categories could contribute to that pattern. The scores alone do not tell us whether that is the explanation, since they also depend on what the representation and classifier can distinguish.

The finding I got wrong in advance

I expected solo lectures to stay more focused than interviews. To test that expectation, the analysis divides each transcript into tenths, projects each part onto the fixed topic basis using non-negative least squares, and adds the Hellinger distances between consecutive parts. This produces a measure of the episode’s route through topic space.

Solo lectures have a median route of 3.12 (95% bootstrap interval 2.66 to 3.90, n = 28), compared with 3.54 for interviews (3.38 to 3.74, n = 347). The difference is −0.42, with an interval from −0.90 to +0.44 and a permutation p-value of 0.22. The interval includes both a shorter route for solos and a longer one, so the lower observed median is insufficient to support my original expectation. AMAs have a median of 6.44 (6.34 to 6.63, n = 64), or 2.91 above interviews (2.66 to 3.16, p < 0.001). Moving among many listener questions seems a plausible reason for that longer route. However, the comparison groups whole episode formats together; it does not separate the effect of changing questions from their other differences.

I also checked whether the conclusion survives a different way of representing meaning. Repeating the analysis in sentence-embedding space gives a different pattern. The medians are 2.14 for AMAs (2.07 to 2.21), 2.19 for solos (2.01 to 2.41), and 2.36 for interviews (2.32 to 2.39). The rank correlation between lexical and semantic route lengths is 0.04. If both measures captured the same notion of changing subject, I would expect them to agree more closely about which episodes travel furthest. Their disagreement makes the interpretation depend on the representation: the first result describes movement among word-based topics, and cannot simply be extended to every sense in which a conversation changes subject.

A recommender scored on more than accuracy

Without listening histories or listener ratings, I cannot evaluate whether someone would actually enjoy a recommendation. I use topical affinity as a proxy for relevance, measured by the Bhattacharyya coefficient between episode topic distributions and averaged over the three resolutions. Each episode gets three lists. “More like this” ranks by that affinity. The serendipity list follows the approach of Ge and colleagues by requiring relevance while excluding the obvious top-N results. It ranks the remaining candidates through the seed episode’s minor topics, reports the connecting topic, and diversifies the list with maximal marginal relevance. The third list deliberately selects episodes with little topical overlap.

The evaluation uses measures from Kaminskas and Bridge, with percentile-bootstrap intervals over 439 seed episodes. The similar-episode list has intra-list diversity of 0.49, compared with 0.82 for the serendipity list. The paired difference is +0.33 (0.32 to 0.34), and the latter list covers 99.8% of the catalogue. There is no clear difference in novelty, with a paired difference of +0.001 and an interval from −0.007 to +0.008.

The “least likely” list reaches only 37% of episodes and has a Gini concentration of 0.91. Many seeds therefore point to the same relatively unusual episodes. Choosing the least similar episodes therefore tends to send many starting points toward the same unusual part of the archive. The serendipity recommendations reach a broader range of episodes. This is a useful difference between the lists, but a listener’s surprise also depends on what they already know and have heard. Without that information, the archive-level measures cannot settle which list feels more surprising.

Subjects that no episode has joined

Another way to examine the archive is to ask which topics seldom appear together. I measure the co-presence of two topics by summing the smaller of their two shares across episodes. Among frequently discussed topics, the lowest co-presence is between cancer and genetics and party politics and elections. Other low pairs include cancer and genetics with scalar fields, and cancer and genetics with the AMA cluster of wave functions and black holes.

For each pair, the analysis identifies past guests whose topic profiles might connect the subjects. For the first pair it identifies Andrew Leigh, or James P. Allison and Ezra Klein as a possible pair of specialists. These are suggestions based on topic distributions, not a judgment about whom Sean should invite or whether a conversation would work.

A second representation

Word counts and embeddings capture different aspects of a transcript, so I used a second representation to check how much the results depended on the first. Qwen3-Embedding-0.6B encodes 33,522 passages of about 220 words and 49,760 interview turns. The main analyses are then repeated using those embeddings. The Mantel correlation between lexical and semantic episode distances is 0.644 (p = 0.0005 over 2,000 permutations), and the two representations share an average of 46% of each episode’s ten nearest neighbors.

The Explorer lists pairs that are close in embedding space but distant in word-count space, as well as the reverse. Other analyses compare conversational echo and changes in concept use. Guests echo Sean’s words more than he echoes theirs in 85% of interviews. “Emergence,” “intelligence,” and “complexity” have some of the largest contextual shifts across three time periods, with lexical and semantic drift rankings correlated at ρ = 0.75.

Searching by what you meant

The search is the part I use most. It accepts a question and returns passages with an episode, timestamp, speaker information, and a source link. It uses Heart of Gold’s retrieval floor over 20,842 transcript chunks. A dense cosine search collects candidates, a lexical channel adds candidates containing rare query terms, and a cross-encoder reranker scores the passages against the question. The lexical channel helps recover names and technical terms that the embedding search may miss.

Reranking takes most of the time. The model scores about 32 chunks a second on my machine, and the candidate pool includes 376 dense candidates plus up to 376 lexical additions. A full pass can take 12 to 24 seconds. I compared shorter budgets with the fully reranked result over twelve blind queries to see how much of that result each budget retained.

The budget applies only to reranking. Total wall time also includes query embedding, candidate retrieval, and completion of the batch of 64 candidates being scored when the deadline passes. The service uses a 12-second budget, which retained an average of 99% of the floor in this evaluation. That result is specific to these queries and this corpus.

The page requests an initial cosine-only result as well as the reranked result. The first usually arrives in about a second, and the page replaces it when reranking finishes. The model service runs on my Mac through a Cloudflare tunnel. If it is unavailable, the Explorer falls back to a keyword index over the same passages and identifies the result as a keyword search.

The companion page

The AMA checker uses the listener questions preserved in Sean’s show notes. The index contains 2,581 question-and-answer pairs from 63 AMA episodes. It embeds a new question and finds similar indexed questions without generating an answer, usually in about 150 milliseconds.

Because the source of each indexed question is known, I can test whether the system retrieves it from the original wording or a paraphrase. The correct source ranked first for all 200 verbatim questions. For 50 paraphrases it ranked first 98% of the time and within the top five in every case, with a worst rank of 2. The nearest other question had a median cosine similarity of 0.635. This evaluation measures retrieval of known questions. It does not establish a threshold that reliably separates a new question from one already answered, so the public page leaves that judgment to the reader.

The transcripts are Sean’s

The Explorer displays short passages with links to the source episodes, rather than reproducing complete transcripts. I collected the transcripts through the public API and have told Sean about the project and offered to change or remove material if he prefers. This is an independent project, not an official Mindscape site.

If you have a question about the archive or an idea for another analysis, you’re welcome to get in touch.