For the last few months assembling ResearchBuzz has been increasingly frustrating. AI news was crowding out most of the other categories and I felt like the newsletter itself was getting much too AI-heavy. Things finally came to a head when I found myself with a 6200-item RSS backlog and 50 articles in my newsletter queue, over half of which were AI-related. My workflow had become more of a workclog. It took some work to wrangle my feeds and separate out the AI topics, but I think I’ve got things moving again.
The first thing I did was use an Anthropic API call to filter out useless information from my RSS feed JSON. Once I got rid of news about academia appointments, traffic accidents, daily deals, local infrastructure, weather forecasts, and similar experiential boilerplate, I cut down my aggregated feed items a lot. By cutting broad categories in which I have no interest (sports, cryptocurrency, etc) I cut it down even more, eliminating between 80% or 90% of the original aggregate, giving me a manageable set of RSS feed items to review. (You can get more detail on what I did here.)
But it still seemed AI-heavy. I did some analysis on the filtered JSON and discovered it was about 25% AI-related stories. Argh. I decided to split off the AI-related items into the own JSON and handle those items separately. I redid the Anthropic API call to filter AI-related items into their own JSON as it went through the articles.
After that I learned a lesson. The lesson is don’t do it that way. Adding a second “by the way” task to a clearly-delinated “filter and bucket” job broke the prompt. Sports news leaked through. I got an article about a wrong-way collision and a deputy accidentally crashing his vehicle. My item set quality went from diamond to shiny tin. So I started over and instead created two sets of calls — the first to analyze filter the full json and the second to divide the filtered JSON into two JSONs, AI articles and non-AI articles. That worked very well. Now I have two sets of RSS items to review — one full of AI-related articles and one not.
To prevent ResearchBuzz from getting AI-heavy I am relegating AI-related articles I find into their own newsletter. You can find the latest issue here. Original-flavor ResearchBuzz will only cover AI-related resources if they’re AI-adjacent new resources (like a tracker for AI-related problems in education) or if it’s a topic I consider of exceptional importance (the OpenAI trial.) Everything else goes into the ResearchBuzz AI newsletter, which will have 15 items per issue and publish every time I aggregate 15 items or once per day, whichever comes first.
If you read ResearchBuzz you may know that individual items in the ResearchBuzz newsletter are indexed and thoroughly tagged at the ResearchBuzz Firehose. I set up the Firehose for two reasons: 1) I wanted to be able to go back through my own stuff with tags and categories, and 2) I wanted people with specific interests to be able to monitor those interests without having to read the entire newsletter. (If you’d like to learn more about how to use the Firehose for content discovery I have a big article here.)
Items from the new AI newsletter will also appear on the ResearchBuzz Firehose. I have added several new categories to acommodate AI-related news. Here they are with direct and RSS links:
AI In Education
Link: https://rbfirehose.com/category/ai-in-education/
RSS Feed: https://rbfirehose.com/category/ai-in-education/feed/
AI in Governance
Link: https://rbfirehose.com/category/ai-in-governance/
RSS Feed: https://rbfirehose.com/category/ai-in-governance/feed/
AI in Work
Link: https://rbfirehose.com/category/ai-in-work/
RSS Feed: https://rbfirehose.com/category/ai-in-work/feed/
AI Legislation
Link: https://rbfirehose.com/category/ai-legislation/
RSS Feed: https://rbfirehose.com/category/ai-legislation/feed/
AI Problems
Link: https://rbfirehose.com/category/ai-problems/
RSS Feed: https://rbfirehose.com/category/ai-problems/feed/
Anti-AI Activism
Link: https://rbfirehose.com/category/anti-ai-activism/
RSS Feed: https://rbfirehose.com/category/anti-ai-activism/feed/
Around the AI World
Link: https://rbfirehose.com/category/around-the-ai-world/
RSS Feed: https://rbfirehose.com/category/around-the-ai-world/feed/
Data Centers
Link: https://rbfirehose.com/category/data-centers/
RSS Feed: https://rbfirehose.com/category/data-centers/feed/
Please note that those categories are brand new and haven’t had time to build up yet, so don’t be surprised if you see only a few articles in each category. That’ll change soon.
The past week or so has been a whirlwind of just trying different things so I could make a managable workflow. There will probably be more tweaks and adjustments coming but I think I’ve got my base flow pretty settled. Stay tuned.
