Local laws shape daily life in ways that most people never see — your rent, your commute, what gets built on the corner lot — but they’re written in dense legal language, buried in city clerk archives, and produced at a volume no individual can keep up with. San Diego alone has passed nearly 90,000 ordinances and resolutions. San Francisco adds dozens per month. Journalists cover the headline votes; researchers study federal legislation; but the vast majority of municipal lawmaking happens without any systematic analysis at all.

cat-llm is designed to close that gap. It’s an open-source Python package that pulls municipal ordinances, federal laws, executive orders, and political speech directly from public datasets, then uses LLMs to classify, summarize, and analyze them at scale. It can take a 15,000-word ordinance written in statutory language and tell you, in plain English, what it does, who it affects, and where it falls on the political spectrum.

In this post, I used cat-llm to classify 200 recent ordinances each from San Diego and San Francisco — two major California cities with different political characters, against two classification schemes: a 12-category policy taxonomy and a 3-category political lean assessment. The goal: a quantitative snapshot of what these cities legislate about and whether the ideological differences between them show up in the text of their laws.

Want to run this on your own data? Skip to the methodology and replication section. The entire pipeline is open source and the datasets are public.

Background: cat-pol and the Data

cat-llm is an ecosystem of open-source Python packages that use LLMs to classify text at scale. Users interested specifically in political text analysis can install cat-pol (pip install cat-pol), which ships with 16 built-in political data sources on HuggingFace, including municipal ordinances from 12 California cities and counties, federal public laws, executive orders, presidential speeches, and Trump’s Truth Social posts, all accessible with a single source= parameter.

For this analysis, the data comes from two HuggingFace datasets:

• chrissoria/san-diego-ordinances — 87,983 records (ordinances + resolutions) going back to 1905
• chrissoria/sf-ordinances — 4,048 ordinances going back to 2011

Both datasets are scraped from official city clerk systems, include full ordinance text extracted from PDFs, and are updated weekly via automated scrapers. I took the 200 most recent ordinances with text from each city.

What follows is a demonstration of what you can learn from municipal legislation using a few lines of Python and no specialized legal knowledge. The entire analysis can be reproduced for free: the data is publicly hosted on HuggingFace, cat-llm is open source, and the classification can run on free-tier HuggingFace models or local models via Ollama with zero API costs. If you can write pip install cat-llm, you can replicate and extend every finding below.

San Francisco Passes Twice as Many Laws

Before asking what these cities legislate about, it’s worth asking how much they legislate.

San Francisco passes roughly twice as many ordinances per month as San Diego: 22 vs 11. The pattern is stable across years, not a spike. SF consistently produces 20–25 ordinances per month; SD produces 9–15.

San Diego compensates with a high volume of resolutions (about 49 per month), which are administrative actions (contract approvals, budget items, proclamations) rather than new law. But in terms of actual lawmaking (text that creates, amends, or repeals municipal code), SF is substantially more productive.

This matters for everything that follows. When we compare the share of ordinances in each policy domain, we’re comparing slices of very different pies. A 24% business regulation rate in SF means roughly 5 new business ordinances per month. An 8% rate in SD means less than 1. Throughout the rest of this analysis, I’ll show both percentages and estimated monthly counts to keep the denominators honest.

What Do These Cities Legislate About?

Rather than imposing categories top-down, I used cat-llm’s extract function to discover 12 policy domains directly from the ordinance text. Categories are multi-label, so a single ordinance can be tagged with multiple domains. See the methodology section for how the categories were generated and the full list.

Results: Policy Domain Distribution

200 most recent ordinances per city. Multi-label (rows sum to >100%). Model: Qwen 2.5-72B-Instruct.

What the Numbers Mean

The two cities legislate about fundamentally different things.

San Diego is a building city. Nearly half (43%) of its recent ordinances touch infrastructure and public works: road improvements, water main replacements, sewer projects, construction contract extensions. Even though SD passes half as many ordinances overall, it still produces more infrastructure legislation per month in absolute terms than SF (4.8 vs 3.3). This isn’t just proportional: SD is genuinely more active on physical infrastructure than SF is. The same holds for parks (1.8 vs 2.3/month, close despite SD’s half-sized legislative output). SD’s legislative agenda reads like a city physically constructing and maintaining itself.

San Francisco is a regulating city. Its ordinances spread across a wider range of policy domains, with no single category exceeding 27%. The biggest concentrations are in zoning and land use (6.0/month), revenue and financing (5.6/month), and business regulation (5.4/month). Health and social services, a category that barely registers in San Diego at 0.7 ordinances per month, hits 5.3 per month in SF. That’s not a rounding difference; it’s a 7x gap. Housing policy runs at roughly triple SD’s rate (3.7 vs 1.3/month). San Francisco’s legislative output reads like a city managing social complexity: who can build what, under what conditions, with what protections for whom.

The environmental protection rate is nearly identical between the two cities (~17%), suggesting this is a baseline concern for California municipalities regardless of political character. Tax increases are rare in both (under 3%), which likely reflects the political difficulty of explicit tax votes at the local level.

SF legislates more about business in both directions. San Francisco scores higher than San Diego on business regulation (24% vs 8%) and pro-business/economic development (17% vs 5%). It creates more rules and more incentives. San Diego’s approach to business is to leave it alone. The legislative silence is itself a policy choice.

SF actively raises revenue to fund social programs. SF generates 25% of its ordinances around revenue and financing, compared to SD’s 16%. Combined with SF’s 24% rate on health and social services (vs SD’s 6%), the picture is a city that raises money to fund an interventionist social agenda. San Diego raises less and spends what it raises on concrete: roads, pipes, parks.

Put differently: San Diego legislates like a city that wants to run well. San Francisco legislates like a city that wants to do good. Whether “doing good” through regulation and social programs actually produces better outcomes is a separate empirical question, but the legislative priorities are unmistakable in the data. SD’s council spends its time keeping the lights on and the water flowing. SF’s council spends its time deciding who gets housing protections, which businesses need new permits, and how to fund homelessness services.

As a robustness check, I re-ran the analysis on 1,000 ordinances per city using GPT-4o with a different, data-driven category scheme. Different model, five times the sample, looser categories. Same conclusion: SD dominates on infrastructure and construction; SF dominates on health, environment, and housing. The pattern holds.

Part 2: Do Ordinances Have a Political Lean?

Category Setup

For the political lean analysis, I used three categories designed to capture ideological orientation:

1. Conservative/Right-Leaning Policy — deregulation, tax cuts, pro-business measures, law enforcement expansion, property rights protections, reduced government intervention, privatization of services

2. Progressive/Left-Leaning Policy — new regulations, tax increases, tenant protections, environmental mandates, social services expansion, equity/inclusion initiatives, labor protections, police reform

3. Neutral — routine contract approvals, procedural amendments, election scheduling, civil service appointments, budget housekeeping with no policy direction

This classification used Qwen3-235B, the flagship thinking model, to see whether reasoning capability helps with the more nuanced task of ideological classification.

Results: Political Lean Distribution

200 most recent ordinances per city. Multi-label classification. Model: Qwen3-235B (thinking model).

The results are counterintuitive, and that’s what makes them interesting.

San Diego’s ordinances are overwhelmingly neutral. Nearly 78% of SD’s recent ordinances carry no detectable ideological lean. These are contract extensions, budget transfers, construction authorizations. The machinery of a city that governs by administration rather than ideology. Only 3% code as conservative and 20% as progressive.

San Francisco legislates politically in both directions. SF has a higher progressive rate (32% vs 20%), which aligns with expectations. But the surprise is SF’s conservative rate: 23% vs SD’s 3%. San Francisco produces nearly 9x more conservative-coded ordinances than San Diego.

This isn’t because San Francisco has a secret conservative agenda. It’s because SF actively legislates about the domains that register as ideological: business regulation, tax policy, development incentives, policing. When you pass an ordinance streamlining permits for small businesses, that codes as pro-business/conservative. When you pass an ordinance adding tenant protections, that codes as progressive. San Diego doesn’t pass either ordinance. It passes a contract amendment instead.

The neutral rate tells the real story. SD’s 78% neutral rate means its city council spends most of its time on administrative governance. SF’s 46% neutral rate means less than half of its legislative output is purely procedural. The majority of SF ordinances take a policy stance of some kind. SF doesn’t just lean left — it legislates ideologically, period. It takes more stances in more directions than San Diego takes in any direction.

Methodology Notes

A few important details about how this analysis works and where it might break.

Multi-label classification. Each ordinance can be tagged with multiple categories simultaneously. An infrastructure bond gets both “Infrastructure” and “Revenue and Financing.” This means percentages sum to more than 100%. That’s by design, not a bug.

Model choice. I used two models: Qwen 2.5-72B-Instruct for the 12-category policy domain classification and Qwen3-235B (a 235-billion parameter mixture-of-experts thinking model) for the political lean analysis. Both are open-source models accessed via HuggingFace’s inference API, no OpenAI dependency. The robustness check used GPT-4o on a larger sample to confirm the results hold across model families.

Category discovery. Categories weren’t hand-picked. I used catllm.extract_policy() to sample 50 ordinances and let the LLM discover recurring themes, then semantically merged duplicates into a clean taxonomy. The specificity="specific" parameter ensures category names include examples, which significantly improves classification accuracy over bare labels. The 12 policy domain categories used:

1. Tax Increases 2. Revenue and Financing 3. Budget and Appropriations 4. Housing and Residential Development 5. Zoning and Land Use Changes 6. Infrastructure and Public Works 7. Business Regulation 8. Pro-Business and Economic Development 9. Environmental Protection 10. Public Safety 11. Health and Social Services 12. Parks, Recreation, and Culture

For the political lean analysis, three categories: Conservative/Right-Leaning Policy, Progressive/Left-Leaning Policy, and Neutral.

Limitations. There’s no ground truth here — no human-coded comparison set for municipal ordinances. The classifications reflect what the model thinks these ordinances are about, not an objective standard. Model bias is a real concern, especially for the political lean analysis; LLMs have known tendencies in how they interpret political language. The sample sizes (200 for the Qwen runs, 1,000 for the GPT-4o robustness check) are reasonable but not exhaustive. The full classified datasets (1,700 SD ordinances, 3,900 SF ordinances) are now public on HuggingFace for anyone who wants to validate or extend this analysis.

The Public Datasets

All data and classification results are publicly available:

• San Diego ordinances: chrissoria/san-diego-ordinances (87,983 records)
• San Francisco ordinances: chrissoria/sf-ordinances (4,048 records)
• Classification results:

The source registry includes 16 datasets across California cities and counties, federal legislation, and social media:

import catllm

catllm.list_policy_sources()  # see all 16 sources

How to Run It Yourself

pip install cat-llm          # full ecosystem
# or: pip install cat-pol    # just the political text package

import catllm

# Discover categories from your data
categories = catllm.extract_policy(
    source="city_san_diego",
    doc_type="ordinance",
    n=50,
    api_key="your-key",
    user_model="Qwen/Qwen2.5-72B-Instruct",
    model_source="huggingface",
    max_categories=12,
    specificity="specific",
)

# Classify using discovered categories
results = catllm.classify_policy(
    source="city_san_diego",
    categories=categories["top_categories"],
    doc_type="ordinance",
    n=200,
    api_key="your-key",
    user_model="Qwen/Qwen2.5-72B-Instruct",
    model_source="huggingface",
)

# Summarize in plain language
summaries = catllm.summarize_policy(
    source="city_san_diego",
    n=10,
    format="bullets",
    tone="eli5",
    api_key="your-key",
)

# Optimize prompts with user feedback
result = catllm.prompt_tune_policy(
    source="city_san_diego",
    categories=["Housing", "Public Safety", "Finance"],
    api_key="your-key",
    sample_size=15,
)

# Fetch raw data (no API key needed)
df = catllm.fetch_policy_source("city_san_diego", n=100, doc_type="ordinance")

cat-llm handles the data pull, classification, and output in a consistent pipeline. Every dataset in the registry uses the same source= parameter across all six functions (classify_policy, extract_policy, explore_policy, summarize_policy, prompt_tune_policy, fetch_policy_source).

What This Tells Us

The consistent finding across every analysis (different models, different sample sizes, different category schemes) is that San Diego and San Francisco govern in fundamentally different modes.

San Diego governs through administration: infrastructure projects, contract management, budget operations. Its ordinances are largely neutral, procedural, and focused on keeping the physical city running. When SD does legislate on policy, it tilts modestly progressive, but most of its legislative energy goes to the apolitical work of urban maintenance.

San Francisco governs through policy: zoning, business regulation, health services, housing, environmental mandates. Its ordinances are more likely to carry an ideological valence, progressive and conservative — because SF actively codifies political values into law. It produces twice as many ordinances per month, and more than half of them take a policy stance.

Neither mode is inherently better. SD’s approach keeps government lean and focused; SF’s approach uses legislation as a tool for social intervention. But the data makes clear that these aren’t just different political leanings. They’re different theories of what city government is for.

What’s Next

This analysis covers two cities. cat-pol ships with 16 data sources (and growing): 12 California cities, San Diego County, federal public laws, executive orders, presidential speeches, and Trump’s Truth Social posts. The same classification pipeline can be applied to any of them. Some questions I haven’t answered:

• Time series: Has SF always been this regulatory, or did it shift after a particular election?
• City-size effects: Do smaller cities (Salinas, Clovis) look more like SD or SF?
• County vs city: SD County just went live as a dataset — does county governance look different from city governance in the same jurisdiction?
• Federal comparison: How do municipal policy domains map onto federal legislation?

If you build something with cat-pol or the datasets, reach out at chrissoria@berkeley.edu.

PAA 2026 — St. Louis, May 9

AAPOR 2026 — Los Angeles, May 13. Oral presentation on the CatLLM pipeline for LLM-based survey response classification.

EdDem Workshop — UW-Madison, May 14–15

Sunbelt 2026 — Daytona Beach, June 22–28. Oral presentation on social isolation, loneliness, and cognitive decline.

ASA 2026 — New York, August 7–11 (pending)

ΨMCA 2026 — August 9–14. Presentation on algorithmic classification in dementia research.

USC Gateway Brownbag — Fall semester

GSA 2026 — National Harbor, MD, November 4–7 (pending)

The presentations span all three of my research streams: networks and cognitive aging (Sunbelt), computational methods for survey research (AAPOR), and applying algorithmic classification to dementia research (ΨMCA). If you’ll be at any of these, I’d love to connect.

A few days ago I used cat-vader to analyze my own Threads feed — 582 posts, 9 categories, and a slightly uncomfortable amount of self-reflection. That analysis asked a personal question: what do I post about?

This one asks a broader question. I took the same classification pipeline and pointed it at ten of Bluesky’s most-followed accounts: AOC, Mark Cuban, Mark Hamill, The Onion, George Takei, The New York Times, Rachel Maddow, Stephen King, MeidasTouch, and NPR. These accounts span the ecosystem — a sitting congresswoman, a billionaire, a pop culture icon, a satirical news outlet, two major media organizations, an advocacy network, and a horror novelist. 250 posts per account, 2,500 posts total, all classified by GPT-4o-mini against a nine-category scheme.

What do Bluesky’s most-followed accounts post about? And more interestingly: once you control for who is posting, does content type actually predict engagement?

Background: cat-vader and the Dataset

cat-vader is a fork of my open-source survey classification package cat-llm, adapted for social media analysis. The core idea is simple: you give it a list of posts and a set of categories with descriptions, and it uses an LLM to check each post against each category independently. Because categories aren’t mutually exclusive, a single post can belong to multiple categories simultaneously — a Rachel Maddow post about deportation policy might be tagged as both Politics & Elections and Social Issues & Justice. That multi-label design is what makes the output useful for analysis rather than just binning.

For this analysis I pulled 250 posts from each of ten accounts using cat-vader’s sm_source="bluesky" integration, classified them all using GPT-4o-mini, and exported everything to a single CSV that’s now publicly available on Hugging Face at chrissoria/bluesky-top10-classified.

One important note before diving in: Bluesky’s API returns views = 0 for all posts. The platform either doesn’t track impression counts or doesn’t expose them. That means the engagement analysis here focuses entirely on likes and replies — two signals that are real and public, but not the same as reach.

Category Setup

Before running classification, I defined nine categories designed to capture the thematic range of these accounts. Rather than bare labels, each category gets a description that guides the model on borderline cases.

1. Politics & Elections — Posts about electoral dynamics, political parties, voting, candidates, or partisan maneuvering. Includes commentary on legislative proceedings, election results, and the behavior of political figures as political actors.

“I honestly believe our most powerful position in a toxic time that feeds on cynicism, apathy, & despair is to genuinely care and act for a better world.” — Alexandria Ocasio-Cortez (146,230 likes)

2. Trump & MAGA Criticism — Posts directly targeting Donald Trump, his administration, his supporters, or the MAGA movement. Includes both policy critiques and character commentary.

“Trump Suffers Setback Unrelated To Child Rape” — The Onion (10,746 likes)

3. Social Issues & Justice — Posts about systemic inequality, civil rights, immigration enforcement, discrimination, or other social conditions. Focus is observational or normative rather than electoral.

“The owners of a Dallas County warehouse that ICE had planned to use as a mega detention center said Monday it will not sell or lease the property to the federal government. ‘God answered our prayers,’ the Hutchins Mayor said.” — Rachel Maddow (21,132 likes)

4. News & Current Events — Posts reporting on, linking to, or discussing recent news stories across any domain. Includes breaking news, investigative stories, and news aggregation.

“An NPR investigation finds the public database of Epstein files is missing dozens of pages related to sexual abuse accusations against President Trump.” — NPR (10,185 likes)

5. Entertainment & Pop Culture — Posts about film, television, music, celebrity, sports, books, or cultural moments. Includes personal fandom and cultural commentary.

“Bluesky is collegial and interesting, the way Twitter used to be. Bonus: most people can spell.” — Stephen King (80,664 likes)

6. Humor & Satire — Posts that are primarily comedic in intent: jokes, satirical takes, absurdist commentary, or ironic framings of current events.

“Netanyahu Calls Iran Strikes Necessary To Prevent War He Just Started” — The Onion (23,571 likes)

7. Science & Technology — Posts about scientific findings, technology developments, AI, climate science, medicine, or the intersection of tech and society.

8. Economy & Business — Posts about financial markets, economic conditions, corporate news, consumer prices, trade policy, or business developments.

“Mr. Cuban — I just wanted to quickly thank you. My husband has [cancer]. We went to pick up his medication and were informed it was $29,000. We were able to get it from CostPlus for $99.” — Mark Cuban (12,276 likes)

9. Personal & Lifestyle — Posts that are personal in nature: life updates, reflections, expressions of mood, personal milestones, or non-political opinion.

“We were married on this day in 1978… soulmates ever since.” — Mark Hamill (53,889 likes)

What Bluesky’s Top Accounts Post About

The overall landscape has a clear hierarchy. News & Current Events dominates at 58.7% of all 2,500 posts — nearly three in five posts link to or discuss a recent story. Politics & Elections comes in second at 50.0%, meaning half of all posts across these accounts touch on politics in some form. Then there’s a significant drop to Social Issues & Justice (26.3%), Entertainment & Pop Culture (23.0%), and Humor & Satire (18.0%). Trump & MAGA Criticism sits at 16.2%. At the bottom: Economy & Business (13.2%), Personal & Lifestyle (8.1%), and Science & Technology (5.8%).

The concentration at the top isn’t surprising for this particular set of accounts — these aren’t lifestyle influencers or tech reviewers. But the degree of dominance by News and Politics is still striking. More than half of what this slice of Bluesky produces is essentially political journalism or political commentary.

The account-level breakdown is where things get interesting.

NPR is the most single-mindedly focused: 93% of its posts fall under News & Current Events, and 60% under Politics & Elections. Almost nothing else. The Humor & Satire bar is essentially invisible (1%). NPR’s Bluesky presence is a straight news wire.

The Onion is the mirror image in the best way. 81% of Onion posts are tagged Humor & Satire — the highest satire rate of any account — but also 60% Entertainment & Pop Culture. What’s slightly surprising is that only 21% of The Onion’s posts are tagged Politics & Elections directly, and only 9% Trump & MAGA Criticism. Satirical headlines about Trump do get tagged politics when the framing is explicitly electoral, but a lot of Onion content uses political subjects in the service of pure absurdism, which the model correctly separates out.

AOC leads in Politics & Elections (68% of her posts), Social Issues & Justice (36%), and News & Current Events (55%). Almost no Humor, very little Economy. Her Bluesky presence reads like exactly what it is: a member of Congress doing political communication full-time.

Mark Cuban is the biggest outlier in the dataset. He’s the only account where Economy & Business is the dominant theme at 58% — more than any other category. He’s also one of the few accounts where Trump & MAGA Criticism is near zero (under 1%). Cuban posts about healthcare costs, business models, tariff economics, and policy mechanics. His account has almost nothing in common with the others topically.

Stephen King is the most balanced. He mixes Entertainment & Pop Culture (62%), Politics & Elections (27%), Humor & Satire (23%), and News & Current Events (24%) roughly evenly. His account feels like a person who actually posts about many things, not a media organization running a content strategy.

Mark Hamill tilts heavily toward Entertainment & Pop Culture (based on his posting about Star Wars, his marriage, personal reflections) alongside Politics — a mix that reflects his public persona as both a cultural figure and a vocal political voice.

Who Gets the Most Engagement?

There is not a close race here. AOC averages 34,674 likes per post — more than twice the second-place Mark Hamill at 12,049. After that: Stephen King (5,039), MeidasTouch (4,226), Rachel Maddow (3,744), The Onion (1,598), George Takei (1,016), NPR (303), Mark Cuban (139), and NYT at 91.

The top 10 most-liked posts in the dataset are all AOC’s, with the most-liked reaching 167,305 likes on a post about a protest in Tucson: “Original projected attendance was 3,000 people. 23,000 showed up.”

The NPR and NYT numbers are particularly striking — both major media organizations with massive follower counts, but averaging under 300 and 100 likes per post respectively. The media organizations are generating engagement on Bluesky at a fraction of the rate of individual personalities. Whether that reflects the algorithm, audience behavior, content style, or all three is hard to disentangle from this data alone.

By category, the engagement hierarchy tells a clean story. Social Issues & Justice leads at 8,819 average likes per post, followed by Politics & Elections at 7,795 and Trump & MAGA Criticism at 6,849. Economy & Business posts average just 2,790 likes; Science & Technology posts average only 706 — by far the lowest of any category.

But engagement averages are heavily influenced by who posts in each category. AOC posts almost entirely in Politics and Social Issues; Mark Cuban posts almost entirely in Economy. These raw category averages are partly measuring account popularity, not content appeal in isolation.

What Predicts Likes?

To separate content from creator, I ran two regression models with log(likes + 1) as the outcome — a log transformation that handles the extreme right skew in likes (the median is 919, the max is 167,305).

The first model includes only the eight category indicators, with Personal & Lifestyle as the omitted reference category. The second adds account fixed effects, which isolates the content effect from the very large differences in per-account audience.

Before controlling for account, seven of eight categories are statistically significant predictors of likes relative to Personal & Lifestyle posts. The strongest positive predictors:

• Entertainment & Pop Culture: +1.04 log-units (the largest positive coefficient)
• Politics & Elections: +1.01
• Trump & MAGA Criticism: +0.99
• Humor & Satire: +0.77
• Social Issues & Justice: +0.60
• News & Current Events: +0.29

The two negative predictors are striking: Science & Technology (−0.47) and especially Economy & Business (−1.96) — the single largest coefficient in the model, and hugely negative. An economics post generates dramatically fewer likes than a comparable personal post in this raw model. The model explains 18.1% of the variance in log-likes.

But much of this is just reflecting AOC’s enormous engagement. AOC rarely posts about the Economy; Cuban almost exclusively does, and Cuban averages 139 likes per post. Once we account for who’s posting, the picture shifts considerably.

With account fixed effects added, R² jumps from 18.1% to 61.9% — the vast majority of the variance in likes is explained by who posted, not what they posted about. But the content effects don’t disappear; they just reorganize.

Politics & Elections (0.74), News & Current Events (0.65), Social Issues & Justice (0.57), Entertainment & Pop Culture (0.39), Humor & Satire (0.38), and Trump & MAGA Criticism (0.30) all remain positive and significant even after controlling for account. Within any given account, these content types generate more likes than a baseline personal post.

Most striking: Economy & Business goes from −1.96 to −0.002 (effectively zero, p = 0.99). The raw negative effect was entirely due to compositional differences between accounts — Cuban is the economy poster, and Cuban gets very few likes. Once you control for the account posting it, economics content performs no differently than personal content. The category itself isn’t the problem; the platform just happens to have its economics-focused account underperform on likes.

Science & Technology also loses its negative coefficient entirely once account is controlled (it becomes +0.25, non-significant). Science content doesn’t hurt engagement; it just happens to be posted by accounts that underperform on likes overall.

Does Timing Matter?

The weekly pattern is clear. Friday is the best day to post at 8,522 average likes, followed by Tuesday (6,648), Wednesday (6,263), and Saturday (5,384). Sunday is the worst day at 3,325 average likes — about 60% lower than Friday’s average.

The spread across days is meaningful but not enormous compared to the account-level differences. Moving from Sunday to Friday would roughly double your average likes in this dataset. Moving from NPR to AOC would multiply them by about 380.

The scatter plot shows every original post (log scale) colored by account. The AOC cluster sits visibly higher and to the right than every other account — more likes and more replies. The media organizations (NYT, NPR) cluster near the bottom. Most posts across all accounts sit in the low-engagement zone regardless of content type; the viral outliers are rare and concentrated in a small number of accounts.

The Public Dataset

The full classified dataset — 2,500 posts, 10 accounts, 9 binary category columns, engagement metrics — is available at chrissoria/bluesky-top10-classified on Hugging Face.

Each row is a post with columns for: account_name, account_handle, timestamp, social_media_input (the post text), likes, replies, reposts, is_repost, category_1 through category_9, and several derived fields including post_length, contains_url, and contains_image. All classification was done by GPT-4o-mini in a single-pass, no-ensemble run. For research applications requiring higher accuracy, a multi-model ensemble is straightforward to add.

A few things you could do with this dataset that I haven’t:

• Sentiment analysis by account — which accounts skew optimistic vs. cynical?
• Topic evolution over time — has the distribution shifted as the news cycle changed?
• Reply rate vs. like rate — some content generates conversation without generating likes; which category does that?
• Cross-platform comparison — the same 10 accounts presumably post on other platforms; how does Bluesky compare to X or Instagram in terms of what they share?

How to Run It Yourself

cat-vader handles the data pull and classification in a single call. Here’s the full workflow for replicating this analysis on any Bluesky account:

import catvader as cv

results = cv.classify(
    sm_source="bluesky",
    handle="aoc.bsky.social",   # any Bluesky handle
    sm_posts=250,                # number of recent posts to fetch
    categories=[
        "Politics & Elections",
        "Trump & MAGA Criticism",
        "Social Issues & Justice",
        "News & Current Events",
        "Entertainment & Pop Culture",
        "Humor & Satire",
        "Science & Technology",
        "Economy & Business",
        "Personal & Lifestyle",
    ],
    description="Social media posts from a public Bluesky account",
    api_key="your-openai-api-key",
    user_model="gpt-4o-mini",
)

results.to_csv("bluesky_classified.csv", index=False)

The output is a DataFrame with one row per post and binary columns for each category. From there, any standard analysis pipeline applies — R, Python, whatever you prefer.

One difference from the Threads workflow: Bluesky doesn’t require OAuth or a developer account. The public API is unauthenticated for reading public posts. You just need an OpenAI (or other provider) key for the classification step.

The consistent finding across both the Threads analysis and this one is that who is posting matters far more than what they post. Account identity — follower count, posting frequency, platform reputation — explains more of the variance in engagement than any content category. But content is not irrelevant: within any given account, political and social posts consistently outperform baseline personal posts, and that effect survives account controls.

For Bluesky specifically, the platform’s current character as a haven for politically engaged progressives shows up plainly in the data. Even The Onion — a satirical outlet that theoretically covers everything — lands 81% of its posts in Humor & Satire while still earning engagement that tracks the political intensity of the moment. The platform has a distinct topical gravity, and the accounts doing best are the ones whose content aligns with it.

If you build something interesting with the dataset or the cat-vader pipeline, reach out at chrissoria@berkeley.edu.

My open-source package cat-llm has always been a Python-first tool: you install it, import it, write a script, run it. That works fine when you already have a pipeline set up. It’s friction when you just want to quickly classify a CSV someone sent you and move on.

Claude Code — Anthropic’s terminal-based coding agent — supports project-local slash commands: markdown files in .claude/commands/ that inject a prompt and tool permissions when invoked. I added four of them to cat-llm. The result is that you can now classify survey data, extract categories, check your API keys, and run end-to-end tests without touching a Python file.

This post walks through the setup and shows what it looks like in practice using 40 rows of open-ended responses from the UCNets survey (variable a19i).

Installation

pip install cat-llm

That’s the only dependency. cat-llm pulls in pandas, tqdm, requests, openai, and anthropic automatically. For PDF classification support:

pip install cat-llm[pdf]

You’ll also need at least one provider API key. cat-llm reads from environment variables automatically:

export OPENAI_API_KEY="sk-..."       # OpenAI / xAI
export ANTHROPIC_API_KEY="sk-ant-..."  # Anthropic
export GOOGLE_API_KEY="AIza..."      # Google

Or drop them in a .env file at your project root — cat-llm will pick them up.

Adding the Claude Code Commands

Clone or navigate to your cat-llm working directory, then create the commands folder:

mkdir -p .claude/commands/catllm

Add four markdown files — classify.md, extract.md, providers.md, and test.md — each containing a prompt that describes what Claude should do when the command is invoked. The full command definitions are in the cat-llm repository under .claude/commands/catllm/.

Once the files exist, open (or reopen) a Claude Code session from the cat-llm project directory. Type /catllm: and tab-complete to see all four commands available.

Checking Your Providers

Before classifying anything, run:

/catllm:providers

Claude detects which API keys are present in your environment, masks the values, and lists suggested model names for each configured provider:

=== cat-llm Provider Status ===

[OK] OpenAI / xAI
     Key: OPENAI_API_KEY = sk-p...k3Rw
     Models: gpt-5, gpt-4o-mini, grok-3

[OK] Anthropic
     Key: ANTHROPIC_API_KEY = sk-a...9xQz
     Models: claude-opus-4-6, claude-sonnet-4-6, claude-haiku-4-5-20251001

[ ] Google (GOOGLE_API_KEY not set)
[ ] Mistral (MISTRAL_API_KEY not set)
[ ] HuggingFace (HUGGINGFACE_API_TOKEN not set)

[OK] Ollama (local)
     llama3.2:latest
     mistral:latest

Configured: 2 provider(s)

Classifying UCNets a19i

The UCNets survey includes an open-ended question (a19i) asking respondents about sources of social support. The raw responses are text — exactly what cat-llm is designed for.

I have 40 responses in a CSV. The first step is letting the data suggest its own categories rather than imposing a scheme from the outside. The /catllm:extract command runs cat-llm’s extract() function, which samples random chunks of the responses, asks the model to name the recurring themes in each chunk, and returns a ranked frequency list:

/catllm:extract ucnets_a19i.csv --col response --model gpt-5

=== Extracting categories from ucnets_a19i.csv ===
Column: response | Rows: 40 | Model: gpt-5

Discovered categories (ranked by frequency):
  1. Family and close relationships       (found in 18/20 chunks)
  2. Friends and peer support             (found in 15/20 chunks)
  3. Professional or institutional help   (found in 12/20 chunks)
  4. Online communities                   (found in 9/20 chunks)
  5. Religious or spiritual community     (found in 8/20 chunks)
  6. No support or social isolation       (found in 7/20 chunks)
  7. Coworkers or colleagues              (found in 5/20 chunks)

Suggested follow-up:
  /catllm:classify ucnets_a19i.csv \
    --col response \
    --categories "Family and close relationships,Friends and peer support,Professional or institutional help,Online communities,Religious or spiritual community,No support or social isolation,Coworkers or colleagues" \
    --output ucnets_a19i_classified.csv

The command even generates the follow-up classify call. Running it:

/catllm:classify ucnets_a19i.csv \
  --col response \
  --categories "Family and close relationships,Friends and peer support,Professional or institutional help,Online communities,Religious or spiritual community,No support or social isolation,Coworkers or colleagues" \
  --model gpt-5 \
  --output ucnets_a19i_classified.csv

=== Classifying ucnets_a19i.csv ===
Column: response | Rows: 40 | Model: gpt-5
Categories: 7

   response                                          family  friends  professional  online  religious  isolated  coworkers
0  My sister and I talk every day, she really...        1       0          0          0         0          0         0
1  Mostly just online — Reddit communities wh...        0       0          0          1         0          0         0
2  My church group has been incredible, they ...        0       1          0          0         1          0         0
3  Honestly nobody. I've felt very alone thr...         0       0          0          0         0          1         0
4  My therapist and my wife, in that order...          1       0          1          0         0          0         0
...

--- Category Distribution (40 rows) ---
Family and close relationships      29  (72.5%)
Friends and peer support            18  (45.0%)
Professional or institutional help  11  (27.5%)
Religious or spiritual community     9  (22.5%)
Online communities                   8  (20.0%)
No support or social isolation       6  (15.0%)
Coworkers or colleagues              4  (10.0%)

Saved to ucnets_a19i_classified.csv

The output is a binary matrix — one column per category, one row per response. A single response can belong to multiple categories simultaneously, which is the right design for this kind of data: someone who mentions both their sister and their therapist gets a 1 in both family and professional, not a forced choice between them.

The classified CSV is ready for any downstream analysis — R, Stata, Python, whatever the pipeline requires.

Running the Quick Test

There’s also a built-in smoke test command that runs on the package’s bundled example data:

/catllm:test

=== cat-llm Quick Test ===
File: examples/test_data/survey_responses.csv
Model: gpt-5
Categories: ['Positive', 'Negative', 'Neutral']

Loaded 20 rows. Columns: ['id', 'response']
Text column: response

--- Results ---
    response                                          Positive  Negative  Neutral
0   The program was incredibly helpful and I...           1        0        0
1   I didn't find the sessions useful at all...           0        1        0
2   It was okay, nothing special but not bad...           0        0        1
...

--- Category Distribution ---
Positive    11
Negative     5
Neutral      4

PASS: classify() completed successfully.

Useful for verifying a new API key or model name works before pointing it at real data.

What Else You Can Do

The commands are thin wrappers around cat-llm’s Python API, which has considerably more surface area:

Multi-model ensemble. Pass a list of models and cat-llm runs all of them in parallel, then reports a consensus column alongside each model’s individual output. Disagreements across models are flagged automatically.

import catllm as cat

result = cat.classify(
    input_data,
    categories,
    models=[
        ("gpt-5",                "openai",    openai_key,    {}),
        ("claude-sonnet-4-6",    "anthropic", anthropic_key, {}),
        ("gemini-2.0-flash",     "google",    google_key,    {}),
    ],
    consensus_threshold="unanimous",
)

Extended reasoning. Pass thinking_budget=4096 (Anthropic) or chain_of_thought=True for borderline classification tasks where you want the model to reason before committing to a label.

PDF classification. If input_data points at a directory of PDFs, cat-llm renders each page and classifies the content as images. Useful for document archives or grant applications.

Category discovery. cat.extract() is the same function the /catllm:extract command calls under the hood. You can run it directly, tune the iterations and divisions parameters, and use the raw frequency table to build a codebook before any classification happens.

The cat-llm repository is at github.com/chrissoria/cat-llm and the package is on PyPI as cat-llm (version 2.5.1 as of this writing). If you adapt the Claude Code commands for a different workflow or build something interesting with the package, reach out at chrissoria@berkeley.edu.

I spend a lot of time on Threads. Over the past two and a half years I’ve posted nearly 900 times: opinions on politics, technology, research, culture, and whatever else caught my attention that day. But I’ve never sat down and actually looked at what I post about. What are my real preoccupations? What topics dominate my feed? Which posts actually get engagement?

This post is an attempt to answer those questions systematically, using an LLM-powered classification pipeline I built called cat-vader — a fork of my open-source survey classification package, cat-llm, adapted for social media data.

Background: cat-llm and cat-vader

cat-llm is an open-source Python package I originally built for classifying open-ended survey responses at scale. You give it a list of text responses and a set of categories, and it uses large language models to assign each response to one or more categories, with support for multi-model ensembles, chain-of-thought reasoning, and automatic category discovery. It was designed for researchers who need to code thousands of survey responses without manually reading each one.

The core architecture turned out to be highly resilient to different kinds of text input. Survey responses and social media posts are structurally similar — short, opinionated, often ambiguous text that needs to be bucketed into meaningful categories. So I cloned cat-llm into cat-vader, stripped out the survey-specific scaffolding, and built a pipeline that can classify any collection of social media posts — whether you’re working from a scraped dataset, a platform export, or a direct API pull. For convenience, cat-vader also wires directly to the Threads API to pull your personal post history with engagement metrics in one call.

The goal of this post is to walk through that pipeline end-to-end using my own Threads feed as the example dataset, and then use the results to take an honest look at what I’ve been posting about. The same workflow applies to any corpus of social media text.

What Do I Actually Post About?

Before classifying anything, I needed to decide what categories to use. I could have imposed them from the top down — just picked eight topics that felt right — but that risks missing something real in my data, or imposing categories that don’t actually fit how I write. Instead, I used cat-vader’s explore() function to let the data suggest its own themes first.

explore() works by repeatedly sampling random chunks of posts, asking the LLM to extract the most common topics from each chunk, and collecting all the extracted labels across many passes. It doesn’t merge or deduplicate — it returns every raw label string from every chunk across every iteration. The idea is that categories which appear frequently and consistently are the ones that genuinely characterize the corpus, while one-off labels are noise.

import catvader as cv
import pandas as pd

df = pd.read_csv("threads_year.csv")
texts = df[df["text"].str.len() > 0]["text"].tolist()  # 582 posts with text

raw = cv.explore(
    input_data=texts,
    api_key="your-openai-api-key",
    description="Social media posts about current events, politics, technology, culture, and personal opinions",
    user_model="gpt-4o",
    iterations=6,
    divisions=15,
    max_categories=8,
    categories_per_chunk=8,
)

This produced 720 raw category extractions across 6 iterations and 15 chunk divisions, yielding 229 unique label strings. I counted the frequency of each label and eyeballed the top results to identify which themes were genuinely dominant versus which were just slightly different phrasings of the same idea (e.g. “Economy”, “Economics”, “Economy and Business”, and “Economy and Finance” are all the same theme).

Here are the top categories by raw frequency:

The signal is clear. Collapsing the variants down, eight themes dominate: Politics, Technology & AI, Social Issues, Economics & Finance, Health & Science, Education & Research, Culture & Entertainment, and Personal. These became the starting point for the final category set.

Defining My Categories

Rather than using bare labels, I defined each category with a description and concrete examples. This follows best-practice category construction from my own empirical work on LLM classification: verbose categories with descriptions and examples significantly outperform bare labels, improving accuracy by reducing model ambiguity on borderline cases.

The explore() output pointed to the broad themes, but the final set draws heavily on my own domain knowledge of what I post about. I know I post a lot about politics, and I know that my political posts tend to fall into distinct registers — partisan frustration, specific policy arguments, and direct Trump commentary — that a generic “Politics” label would collapse together. I also know I post disproportionately about AI relative to most people, which warranted its own category rather than being folded into Technology. My final categories reflect both what my data showed and what I know about myself as a poster.

1. Partisan Politics — Posts relating to partisanship directly or indirectly: references to political parties, political tribalism, electoral dynamics, or the behavior of politicians and political actors as representatives of a party or ideological bloc (e.g., “The Republican Party has moved too far right,” “Democrats keep losing working-class voters”).

“Either side of the political spectrum has little empathy for the other. They actively dislike each other. When an act of violence occurs, the first instinct is to ask which side did it.”

2. Policy Politics — Posts advocating for or critiquing specific policies or policy positions, independent of partisan framing: arguments about what government should or shouldn’t do, regulatory stances, or calls for systemic reform (e.g., “We need universal healthcare,” “Tech companies need stronger antitrust enforcement”).

“It should be illegal to create AI videos meant to mislead and misinform people about current events.”

3. Anti-Trump — Posts directly critiquing Donald Trump, his character, his decisions in office, his policies, or individuals and groups who support him or his agenda (e.g., “Trump’s tariffs are going to tank the economy,” “MAGA voters keep getting lied to”).

“Trump is doing a great job at driving global unity… in their opposition to the US as an international bully.”

4. Technology — Posts discussing technology in any form, broadly construed: software, hardware, consumer devices, platforms, the tech industry, or the societal implications of technological change (e.g., “Apple’s new chip is a generational leap,” “Social media is rewiring how we form opinions”).

“Anthropic is all the hype but OpenAI still has the best models sorry to say.”

5. Artificial Intelligence — Posts specifically about AI models, AI capabilities, or commentary on AI companies such as OpenAI, Anthropic, Google DeepMind, or xAI. This includes takes on specific models (GPT, Claude, Gemini, Grok), opinions on what AI can and cannot do, or the direction of the AI industry (e.g., “The AI hype cycle is showing cracks,” “LLMs are great at pattern matching but terrible at actual reasoning”).

“In my opinion, the only people who are saying LLMs will someday automate all jobs don’t really understand the technology.”

6. Social Issues — Posts about social conditions, inequality, discrimination, or systemic patterns in society, without explicitly advocating for a specific policy response or criticizing political leadership. The focus is observational or normative about society itself rather than prescriptive about what government should do (e.g., “The wealth gap between generations is unlike anything we’ve seen,” “Racism in hiring is still very much alive”).

“The most deflating thing about this whole thing is how two people will view the same video and come to entirely different conclusions.”

7. Shit Posting — Low-effort, irreverent, or deliberately provocative posts with no pretense of serious commentary. The tone is casual to the point of flippant, the take is blunt, and the goal is more to express a vibe than make an argument (e.g., “Astrology is BS,” “Nobody actually likes networking events”).

“Daily reminder that astrology is still BS.”

8. Economics & Finance — Posts relating to economic conditions, financial markets, or specific market developments: references to stock prices, commodity prices, oil markets, interest rates, inflation, or broader signals about the state of the economy (e.g., “The stock market is pricing in a recession,” “Coffee prices are up 40% and nobody is talking about it”).

“The Strait of Hormuz, which handles roughly 20% of the world’s daily oil supply, is effectively shut down. That means lower supply, which means higher prices. When oil prices rise the price of all other commodities rise.”

9. Thirst Trap — Posts that are flirty, self-promotional, or designed to attract attention and engagement through charm or physical appeal (e.g., “Just got a haircut and feeling myself,” “Anyone else look good today or just me?”).

My Category Breakdown

With my categories defined, I ran classify() on the full year of text posts — 582 posts with non-empty text content — using Llama 3.3 70B on SambaNova. Each post was classified against all categories independently, meaning a single post can and often does belong to more than one category. A post lamenting Trump’s tariff policy, for example, might be tagged as both Anti-Trump and Economics & Finance. That’s by design: my categories aren’t mutually exclusive buckets, they’re lenses.

The chart below shows the percentage of posts that were assigned to each category. Because categories overlap, the bars don’t sum to 100% — they can’t. What the chart is really showing is the frequency of each topic in my feed: how often, across 582 posts, did I reach for a given subject. Think of it less as a pie chart and more as a set of independent thermometers, each measuring how much of my posting energy went toward a given theme.

A note on scope: cat-vader can classify images directly, but for this analysis I focused on text only. My dataset includes image posts, but the model was given just the text caption — no image content. Posts without any text were excluded entirely, and image posts were classified solely on whatever caption was attached. That’s a real limitation, and one worth keeping in mind when interpreting any categories that might skew visual (more on that in a moment).

I’ll be honest: I wasn’t sure what I’d find. I post somewhat mindlessly — something catches my eye, I have a reaction, I type it out. I don’t sit down with a content strategy. So this is genuinely an exercise in holding up a mirror.

One early finding did give me pause: Shit Posting came in second overall at 27.7% of posts, just behind Social Issues at 32.5% and ahead of Technology at 25.9%. My first instinct was that something had gone wrong — a miscategorized label, a prompt that was too loose, something. I went back and spot-checked the flagged posts. Nope. Fully accurate. Apparently more than a quarter of what I put out into the world is, by any reasonable definition, a shit post. I have made peace with this, though I’ve also quietly vowed to post with a bit more intention going forward, with the goal of demoting Shit Posting from a top-three category to something more like fifth. For what it’s worth, my top four — Social Issues (32.5%), Shit Posting (27.7%), Technology (25.9%), and Partisan Politics (24.6%) — probably tells you everything you need to know about me as a person.

One other result worth flagging: Thirst Trap came in at exactly one post (0.2%). False positives happen, and this is a good example of why. The post in question was an image captioned simply “Me”, and since the model only had that single word to work with, tagging it as a thirst trap is a defensible inference. Whether it actually was one depends on the photo, which the model never saw. I’m not saying it wasn’t.

Engagement by Category

One of the advantages of pulling data directly through the Threads API is that cat-vader returns not just post text but a full set of engagement metrics alongside it. For every post, the package outputs:

That means the classified dataset isn’t just a topic-coded text corpus — it’s a topic-coded text corpus with performance data attached. Which opens up an obvious question: does what I post about actually affect how much engagement I get?

The chart below shows average likes and average replies broken out by category. Because posts can belong to multiple categories, the same post may appear in more than one bar. The x-axis is sorted by average likes descending.

To see the full picture, the scatter plot below shows every post individually — x-axis is replies, y-axis is likes, log scale on both axes, colored by primary category. The log scale is doing a lot of work here. Without it, the chart is basically one dot in the top-right corner and 580 dots stacked on top of each other at zero. With it, you can actually see my distribution — which, in all honesty, is mostly a dense cloud of dots in the bottom-left. The majority of what I post sinks quietly into the void, liked by a handful of people who were probably just scrolling past and hit the button by accident. A few posts escape. Most do not. This is the reality of posting.

What Actually Predicts Views?

The bar charts above are descriptive — they show averages, but averages don’t control for anything. A category might look high-performing simply because I happen to post it more, or because it correlates with another category that’s doing the real work. To get a cleaner picture, I ran models that hold all categories constant simultaneously, isolating the independent effect of each one. I ran a separate model for each outcome — views, likes, and replies.

The chart below shows the coefficients from the views model, with 95% confidence intervals. Points in red are statistically significant (p < 0.05); grey points are not.

Three categories emerge as significant positive predictors of views (for the stats nerds: R² = 6.8%, F p = 2.5×10⁻⁶):

• Artificial Intelligence is the single strongest predictor. Holding everything else constant, an AI post gets roughly 2.4x as many views as a comparable post on a different topic. AI has been the trendiest topic on the internet for the past two years, and the algorithm appears to reward it accordingly.

• Partisan Politics comes in second. A partisan political post gets about 2x as many views as a comparable non-partisan post. Tribal political content travels well on social media — this surprises no one.

• Shit Posting rounds out the significant predictors, with posts in this category getting about 1.8x as many views. Blunt, low-effort, instantly legible takes are apparently what the algorithm rewards. I have complicated feelings about this.

For likes, the story is similar but smaller in scale: partisan politics posts get about 50% more likes than comparable posts, and shit posts about 35% more. AI drops out entirely for likes — AI posts rack up views without proportionally converting to likes. Interesting.

For replies, no category made a meaningful difference. Nothing I post about reliably generates conversation, which is either a sign of epistemic humility or evidence that I am not as interesting as I think I am.

Topic explains only a small slice of overall engagement — most of what determines whether a post goes anywhere is timing, luck, and whether someone with a large following happens to engage. But the categories that do matter are consistent and interpretable.

One natural question is whether these effects are real or just a reflection of timing — maybe I happen to post AI content on Thursdays at peak hours, and it’s the timing doing the work rather than the topic. To check, I re-ran all models controlling for both day of week and hour of day.

The results held up for the most part. Partisan politics and AI posts still get roughly 2x as many views after accounting for when they were posted — those effects appear to be about the content itself. Shit posting weakens once timing is controlled for, suggesting some of its raw advantage was coming from when I tend to fire off a shit post rather than the content. The reply non-result holds throughout.

One more alternative explanation worth ruling out: volume. Maybe on high-output days I’m simply flooding my feed and one post happens to catch a wave — meaning it’s the quantity, not the quality of the content, doing the work. The chart below plots each post’s views and likes against the number of posts I made that day.

The correlations are r = 0.16 for views and r = 0.17 for likes — small but consistently positive, suggesting a weak connection. Posting more on a given day does seem to nudge individual post performance slightly, though the effect is modest enough that it’s unlikely to be the main story. The content effects from the models hold.

Bonus: Does the Day of the Week Matter?

One last question that the timestamp data makes easy to answer: does when I post matter?

Thursday stands out immediately — average views of 3,241, more than double the next best day (Saturday at 1,439) and about four times Wednesday’s average of 398. Monday and Friday are middling. Wednesday is the worst day to post, by a wide margin.

I don’t have a strong theory for why Thursday in particular. It might be something about my posting behavior on Thursdays — maybe I tend to post more shareable content, or post at better times within the day. It might also just be noise: with 72–106 posts per weekday, a handful of viral Thursday posts could skew the average significantly. Either way, the answer to “when should I post?” appears to be: not Wednesday.

Time of day tells a cleaner story. The chart below breaks posts into five windows (Pacific Time):

Late night posts (9pm–5am) average nearly 2,900 views — more than double the evening average of 1,126, and about twelve times the morning average of ~250. The pattern is monotonic: the later in the day, the more views. One plausible explanation is that late night is when I tend to post more impulsively about whatever’s dominating the news cycle, which also happens to be when more people are doom-scrolling. Another is that late night posts have more hours to accumulate views before I wake up and post something else that pushes them down. Either way, if I want views, apparently I should stay up later — which is not advice I needed the algorithm to give me.

Conclusion: Do This With Your Own Data

Everything in this post — pulling my data, discovering my categories, classifying 582 posts, and running the regressions — took a single afternoon. The data pull and classification itself ran in about 30 minutes; the rest was just analysis and writing. If you have a Threads account and a few API keys, you can run the same analysis on your own feed.

The most valuable step is explore() first. Don’t impose your categories from the top down. Run the exploration pass, look at what themes emerge with high frequency, and let your actual content tell you what it’s about. Your categories will be better for it, and you’ll probably learn something about yourself in the process that you wouldn’t have guessed going in.

From there, classify() gives you a labelled dataset you can take in any direction. A few starting points:

• Sentiment and tone. Instead of topic categories, define categories like “optimistic,” “cynical,” “ironic,” or “earnest.” You’ll get a mood profile of your posting history.
• Audience targeting. If you post for multiple audiences — say, researchers, practitioners, and general readers — define categories for each and see how your mix has shifted over time.
• Thread evolution. Classify posts by month and track how your topical distribution has changed. Are you posting more or less about AI than you were a year ago? The data will tell you.
• Quote extraction. Use extract() instead of classify() to pull structured fields out of free text — named entities, specific claims, URLs, anything you want to turn into a column.

Beyond social media, the underlying engine is cat-llm, which was designed for survey and qualitative data. If you’re a researcher sitting on thousands of open-ended survey responses, interview transcripts, or product reviews, the same pipeline applies. Define your codebook as a set of verbose category descriptions, run classify(), and get back a coded dataset in minutes rather than weeks. The package supports multi-model ensembles, chain-of-thought reasoning, and automatic inter-rater reliability metrics: all the things you’d want for academic coding workflows.

If you want to adapt it for your own platform or use case, cat-llm is open source and built to be forked. cat-vader is one fork; there’s no reason there couldn’t be a cat-reddit, a cat-bluesky, or a cat-transcripts for interview data. The core classification and exploration logic is platform-agnostic. All you need to wire up is a data ingestion layer for whatever source you’re working with.

If you build something interesting with it, I’d genuinely like to hear about it. Reach out at chrissoria@berkeley.edu.

One last thing: I’ll be re-running this analysis in six months to see whether I’ve made good on my vow to reduce my shit posting. The pipeline takes an afternoon. The habit change may take longer.

How I Did It, and How You Can Too?

Want to run this on your own data? Here’s the technical setup.

Getting Started

cat-vader is available on PyPI:

pip install cat-vader

You’ll also need a Threads access token. Generate one via the Meta for Developers portal (create an app, add the Threads product, and generate a long-lived user token), then add it to a .env file:

THREADS_ACCESS_TOKEN="your-token-here"
THREADS_USER_ID="your-numeric-user-id"

cat-vader will pick these up automatically when you call any function with sm_source="threads". Alternatively, you can pass your API key directly as a parameter in any function call — the .env file is just a convenience for avoiding repetition.

If you already have social media data — a CSV of posts from any platform, a scraped dataset, a platform export — you can skip the API setup entirely and pass your text directly to classify(), explore(), or extract() via the input_data parameter. The sm_source integration is a convenience layer on top of the same classification engine.

Pulling My Threads History

For the Threads API pull: cat-vader connects to your account and retrieves your full post history — every post you’ve made, along with engagement metrics — automatically, without any manual data export. You authenticate once via the Threads Graph API, store your credentials in a .env file, and cat-vader handles the rest.

Under the hood, the package paginates through your full post history (the API returns up to 100 posts per page), fetches engagement metrics for each post in a separate insights call, and returns everything as a single tidy DataFrame with one row per post. Columns include the post text, image URL (when an image was attached), media type, and metrics: likes, views, replies, reposts, quotes, and shares.

The key parameter is sm_source="threads", which can be passed to any of the main functions — classify(), extract(), or explore(). You can scope the pull to a specific time window using sm_months:

import catvader as cv

# Pull and classify the last 12 months of your personal Threads history
results = cv.classify(
    sm_source="threads",   # connect to your Threads account
    sm_months=12,          # fetch all posts from the past year
    categories=["Politics", "Technology & AI", "Economics", "Health & Science",
                "Education & Research", "Culture & Entertainment",
                "Social Issues", "Personal"],
    api_key="your-openai-api-key",
)

Here are my five most-liked posts from the dataset:

For my account, pulling my full history returned 850 posts going back to July 2023, about two and a half years. Of those, 176 were image posts, 5 were videos, and 582 had text content; the remainder were reposts or media-only posts.

One note on the metrics: the Threads Insights API takes a few hours to populate data for brand new posts, so very recent posts may show zeros. Older posts return accurate lifetime totals.

CatLLM now supports ensemble classification—running multiple models in parallel and combining their predictions through voting. This addresses a persistent concern in LLM-based classification: how do you know if a single model’s output is reliable?

The Problem with Single-Model Classification

When you classify survey responses with a single LLM, you’re trusting that model’s interpretation entirely. But different models have different training data, different biases, and different failure modes. A response that GPT-4o categorizes as “positive sentiment” might be labeled “neutral” by Claude, and “mixed” by Gemini. Which one is right?

For research applications where classification decisions feed into statistical analyses, this uncertainty matters. Ensemble methods offer a way to quantify and reduce it.

Three Approaches to Ensemble Classification

1. Cross-Provider Ensembles

You can combine models from different providers—OpenAI, Anthropic, Google, Mistral, and others—to get diverse perspectives on each classification:

import catllm as cat

results = cat.classify(
    input_data=df['responses'],
    categories=["Positive", "Negative", "Neutral"],
    models=[
        ("gpt-4o", "openai", "sk-..."),
        ("claude-sonnet-4-5-20250929", "anthropic", "sk-ant-..."),
        ("gemini-2.5-flash", "google", "AIza..."),
    ],
    consensus_threshold="majority"
)

Why this helps: Each provider’s models are trained on different data with different objectives. When three independently-developed models agree on a classification, that agreement carries more weight than any single model’s confidence score. When they disagree, you’ve identified responses that may require human review.

2. Self-Consistency with Temperature

You can also ensemble the same model against itself by running it multiple times with higher temperature (randomness). This samples from the model’s probability distribution rather than always taking the most likely output:

results = cat.classify(
    input_data=df['responses'],
    categories=["Category A", "Category B", "Category C"],
    models=[
        ("gpt-4o", "openai", "sk-..."),
        ("gpt-4o", "openai", "sk-..."),
        ("gpt-4o", "openai", "sk-..."),
    ],
    creativity=0.7,  # Higher temperature for varied outputs
    consensus_threshold="majority"
)

Why this helps: At temperature 0, a model always produces the same output for the same input. At higher temperatures, it samples from its full distribution of possible responses. If a classification is robust, the model should arrive at the same answer even when sampling differently. If it produces different answers across runs, that response is likely ambiguous or borderline.

This approach is cheaper than cross-provider ensembles (one API key, often lower per-token costs) while still providing a measure of classification stability.

3. Consensus Thresholds

CatLLM provides three voting rules for determining consensus:

# Majority: At least 50% of models must agree
consensus_threshold="majority"

# Two-thirds: At least 67% of models must agree
consensus_threshold="two-thirds"

# Unanimous: All models must agree
consensus_threshold="unanimous"

You can also specify custom numeric thresholds (e.g., consensus_threshold=0.75 for 75% agreement).

Majority is the least restrictive. With three models, two agreeing is sufficient. This maximizes the number of responses that receive a consensus classification.

Two-thirds requires stronger agreement. With three models, you still need two to agree (67%), but with six models, you’d need four. This reduces false positives at the cost of more responses falling below threshold.

Unanimous is the most restrictive. Every model must agree for a category to be marked present. This produces high-confidence classifications but may leave many responses without consensus, flagging them for human review.

Interpreting the Output

The results DataFrame includes columns for each model’s individual classification plus the consensus:

For the second response, GPT-4o and Gemini said “not positive” while Claude said “positive.” With majority voting, the consensus is 0 (not positive) because only 1/3 models agreed.

You can use the agreement patterns to:

• Identify systematic differences between models
• Flag ambiguous responses for manual review
• Report inter-model reliability alongside your results

Practical Considerations

Cost: Ensemble classification multiplies your API costs by the number of models. Three models means roughly 3x the cost. For large datasets, consider running ensembles on a sample first to calibrate, then using a single model for the full dataset.

Speed: Models are called in parallel, so wall-clock time doesn’t increase linearly. Three models typically take only slightly longer than one.

When to use ensembles: Ensembles are most valuable when classification decisions are consequential—when they feed into regression models, when you’re publishing findings, or when categories are subjective enough that reasonable people might disagree.

When a single model suffices: For exploratory analysis, prototyping, or cases where categories are unambiguous, a single model is faster and cheaper.

Try It

Ensemble classification is available in CatLLM 0.1.16+ and in the web app. In the web app, select “Model Comparison” to see each model’s output side-by-side, or “Ensemble” to get majority-vote consensus classifications.

Links

• Web App: https://huggingface.co/spaces/CatLLM/survey-classifier
• Python Package: https://pypi.org/project/cat-llm/
• GitHub: https://github.com/chrissoria/cat-llm

If you have questions or want to discuss ensemble methods for your research, reach out at ChrisSoria@Berkeley.edu.

I’ve been working on CatLLM, a Python package for classifying open-ended survey responses with LLMs. This week I converted it into a web app.

Try it here: https://huggingface.co/spaces/chrissoria/CatLLM

The Problem

If you’ve worked with open-ended survey data, you know the workflow: hundreds or thousands of free-text responses that need to be categorized before you can do any quantitative analysis. The traditional approach is manual coding—either doing it yourself or hiring RAs. It’s slow, expensive, and doesn’t scale.

What the App Does

The web app lets you classify survey responses without writing any code:

1. Upload your data — CSV, Excel, or PDF documents
2. Define categories — Either specify your own categories or let the model extract them from your data automatically
3. Run classification — The model assigns each response to one or more categories (multi-label classification)
4. Download results — Get a CSV with classifications plus a methodology write-up you can adapt for your paper

The same functionality is available in the Python package if you prefer working in code, but the web app removes the setup barrier for researchers who just want to try it out.

Free Models (For Now)

I’m covering the API costs temporarily so people can test it. The free tier currently includes:

• GPT-4o-mini (OpenAI)
• Claude 3 Haiku (Anthropic)
• Gemini 2.5 Flash (Google)
• Llama 3.3 70B (via Groq)
• DeepSeek V3.1
• Qwen3 235B
• Mistral Medium
• Grok 4 Fast

If you need more powerful models or have large-scale needs, you can bring your own API key.

Looking for Feedback

This is still early. I’d appreciate it if you:

• Break it — Find edge cases, report bugs, tell me what fails
• Suggest features — What would make this useful for your research?
• Collaborate — If you’re interested in working on a methods paper evaluating LLM classification for survey data, I’m open to it

You can reach me at ChrisSoria@Berkeley.edu or leave comments on the GitHub repo.

Links

• Web App: https://huggingface.co/spaces/chrissoria/CatLLM
• Python Package: https://pypi.org/project/cat-llm/
• GitHub: https://github.com/chrissoria/cat-llm

CatLLM now works with Huggingface. This means you can use open-weight and open-source models without needing the compute power to run them locally. Just get your Huggingface API key (I recommend the Pro subscription for heavy usage—it’s only $9/month) and you’ll have access to models like Qwen, DeepSeek, and Llama. This is useful for researchers who want to test open-weight models or take advantage of their lower cost for classification tasks.

Another benefit is access to thousands of user-trained models for specific tasks. For example:

• MedAlpaca-7B (Medical Domain) - medalpaca/medalpaca-7b - A 7-billion parameter LLM specifically fine-tuned for medical domain tasks, built on top of the LLaMA architecture. It’s designed to improve question-answering and medical dialogue capabilities.

• CodeLlama-7B (Code Generation & Understanding) - codellama/CodeLlama-7b-hf - Meta’s specialized code-focused LLM, available in 7B, 13B, 34B, and 70B parameter versions. It comes in three variants: base (general code), Python-specific, and Instruct (for code assistance).

• Aya-23-8B (Multilingual) - CohereLabs/aya-23-8B - Developed by Cohere Labs, Aya 23 is an instruction-tuned model supporting 23 languages including Arabic, Chinese, French, German, Hindi, Japanese, Korean, Spanish, and more.

You can also train and host your own models. Hugging Face provides several tools for fine-tuning:

• Transformers Library - Use the Trainer API for fine-tuning any model from the Hub
• PEFT (Parameter-Efficient Fine-Tuning) - Techniques like LoRA and QLoRA for efficient fine-tuning with less compute
• AutoTrain - A no-code/low-code solution for fine-tuning models directly on Hugging Face
• TRL (Transformer Reinforcement Learning) - For RLHF and preference tuning

You can upload and host your models in several ways:

• Web Interface - Go to huggingface.co/new, then use “Add File” → “Upload File”
• Python Libraries - Use model.push_to_hub("your-username/model-name") with Transformers or the huggingface_hub library
• Git - Since repos are Git-based, you can push directly via command line

Your model doesn’t need to be compatible with Transformers—any custom model works.

Here are a few examples of what you could do:

1. Region-specific language models - Fine-tune a model specifically for extracting information from Spanish-speaking respondents from a particular country, rather than Spanish generally. For example, a model trained on Dominican or Puerto Rican Spanish would better understand the distinct vocabulary, slang, and expressions that differ significantly from Mexican Spanish.

2. Specialized scoring models - Train a model specifically for detecting the quality of drawn shapes for cognitive impairment assessment. Instead of relying on general-purpose vision models, you could create one optimized for CERAD-style scoring tasks.

3. Domain-specific extraction models - Build a model designed to extract key details from long texts in your field—such as one trained to pull specific policy details from local city council meeting transcripts, or one that identifies funding amounts and grant recipients from foundation reports.

Getting Started

Using Huggingface with CatLLM is straightforward. Simply specify model_source="huggingface" and provide your Huggingface API key:

import catllm as cat

results = cat.multi_class(
    survey_input=df['responses'],
    categories=["Category 1", "Category 2", "Category 3"],
    api_key="your-huggingface-api-key",
    user_model="Qwen/Qwen3-VL-235B-A22B-Instruct:novita",
    model_source="huggingface",
    creativity=0,
    chain_of_thought=True
)

Here’s an example using CodeLlama to analyze code snippets for specific features:

import catllm as cat

# Analyze code snippets for security and quality features
code_analysis = cat.multi_class(
    survey_input=df['code_snippets'],
    categories=[
        "Contains SQL queries",
        "Has proper error handling",
        "Uses deprecated functions",
        "Contains hardcoded credentials",
        "Implements input validation"
    ],
    api_key="your-huggingface-api-key",
    user_model="codellama/CodeLlama-7b-Instruct-hf",
    model_source="huggingface",
    creativity=0,
    chain_of_thought=True
)

Here’s an example using Aya-23 to classify Spanish survey responses with categories written in Spanish:

import catllm as cat

# Classify Spanish survey responses about healthcare access
healthcare_analysis = cat.multi_class(
    survey_input=df['respuestas_encuesta'],
    categories=[
        "Este participante contestó que tiene acceso a seguro médico",
        "Este participante mencionó barreras financieras",
        "Este participante expresó dificultades con el idioma",
        "Este participante indicó satisfacción con su atención médica",
        "Este participante reportó largos tiempos de espera"
    ],
    api_key="your-huggingface-api-key",
    user_model="CohereLabs/aya-23-8B",
    model_source="huggingface",
    creativity=0,
    chain_of_thought=True
)

Here’s an example using MedAlpaca to classify medical interview notes:

import catllm as cat

# Classify patient interview notes for symptoms and conditions
medical_analysis = cat.multi_class(
    survey_input=df['patient_notes'],
    categories=[
        "Patient reports cardiovascular symptoms",
        "Patient mentions respiratory issues",
        "Patient describes chronic pain",
        "Patient indicates mental health concerns",
        "Patient reports medication side effects"
    ],
    api_key="your-huggingface-api-key",
    user_model="medalpaca/medalpaca-7b",
    model_source="huggingface",
    creativity=0,
    chain_of_thought=True
)

Important Limitation: MedAlpaca targets medical student-level knowledge and should never be used as a substitute for professional medical advice.

Full Provider Support

CatLLM now supports seven major providers:

• OpenAI (GPT-4o, GPT-5)
• Anthropic (Claude Sonnet 4, Claude 3.5)
• Google (Gemini 2.5 Flash/Pro)
• Huggingface (Qwen, Llama, DeepSeek, community models)
• xAI (Grok)
• Mistral (Mistral Large, Pixtral)
• Perplexity (Sonar models)

Get in Touch

If you have any questions about using Huggingface with CatLLM, or if you’d like guidance on how to fine-tune a model for your specific research needs to maximize consistency and quality of output, feel free to reach out. I’m happy to help researchers get the most out of these tools. You can contact me at ChrisSoria@Berkeley.edu.

Learn More

• View the documentation
• Install from PyPI: pip install cat-llm

I am honored to have been selected as a 2025 awardee of the Bashir Ahmed Graduate Fellowship at UC Berkeley’s Department of Demography.

The Bashir Ahmed Graduate Fellowship supports dissertation research for students in the Demography PhD Program and the Sociology & Demography PhD Program. This fellowship provides critical support as I continue my dissertation research at the intersection of social demography, epidemiology, and computational methods.

My research examines how social networks correlate with mortality outcomes at the county level, investigates how partisan social networks can worsen disease outcomes, and studies the impact of loneliness on aging and health. Beyond traditional demographic research, I also apply large language models to demographic studies, developing techniques for text classification and computational analysis of survey data.

What makes this recognition particularly meaningful to me is that this year the fellowship was not application-based—the faculty independently selected the recipient. I am deeply grateful to the Department of Demography and the Ahmed family for this recognition and support. This fellowship will enable me to further develop my research on population health dynamics and continue bridging computational methods with demographic inquiry.

You can learn more about the fellowship and past recipients on the UC Berkeley Demography website.

I’m excited to announce the release of a new Python package: llm-web-research. Part of the CatLLM ecosystem, this tool enables LLM-powered web research with a focus on accuracy over quantity—designed for researchers who need verified, high-quality data.

The Problem

When using LLMs for web research, a common issue is ambiguity. Searching for information about “John Smith” or “Springfield” can return incorrect results due to name conflicts and common entities. For research applications where false positives are costly, we need a more rigorous approach.

The Solution: Multi-Step Verification

The core innovation of llm-web-research is a 4-step verification pipeline that catches ambiguous queries before returning potentially incorrect answers:

1. Information Gathering — Initial web search to understand the entity and context
2. Ambiguity Detection — Explicit checks for name conflicts, common names, and contradictions
3. Skeptical Verification — Secondary search actively looking for contradicting information
4. Structured Output — JSON formatting with binary confidence scoring

The design philosophy is simple: no answer is better than a wrong answer.

Key Features

• Two research modes: precise_web_research() for maximum accuracy, web_research() for faster single-step searches
• Multi-provider support: Anthropic, Google Gemini, Perplexity
• Structured output: Returns pandas DataFrames with answers and source URLs
• Safety features: Incremental CSV saving for long-running searches, automatic “Information unclear” responses when uncertain

Installation and Usage

pip install llm-web-research

import llm_web_research as lwr

results = lwr.precise_web_research(
    search_question="founding year",
    search_input=["Apple", "Microsoft"],
    api_key="your-api-key",
    model_source="anthropic"
)

Use Cases

• Academic research requiring verified sources
• Fact-checking with high accuracy requirements
• Building high-quality datasets
• Automated due diligence tasks

Check out the GitHub repository for full documentation and examples, or install directly from PyPI.

Acknowledgments

This work was supported by the Bashir Ahmed Graduate Fellowship at UC Berkeley’s Department of Demography. I am grateful to the Ahmed family and the Department of Demography for their support of my research.