If you read ResearchBuzz you may remember me mentioning recently that I was in the process of moving. My husband and I have sold our house and purchased a new one, which we will share with my mother and her husband. This is going to be a somewhat slow process because the house needs some renovation and upfitting before we can move in.

One of the things we’re doing is installing sump pumps. I have been doing some research on turning sump pump runoff into a water feature (like a backyard wildlife pond) but the search has been frustrating because no matter how carefully I style my queries, I keep tripping over pages about whether or not you can use sump pumps *in* a backyard pond.

I was staring in frustration at a particularly annoying page of search results when I thought to myself, “Man, I wish I had some way of marking which of these results are useful and which aren’t, and then have an AI analyze the different sets for language use and give me suggestions for how I can revise my search to get more useful stuff and less crap.” Then I thought, “Oh damn, that sounds like a good idea, I should make that.” So I did, and created a thing called QueryAnvil.

Hammering Out Searches With QueryAnvil

Here’s how QueryAnvil works: The user specifies in plain language what they’re trying to find (this is to assist the AI; experimentation deemed it necessary) and QueryAnvil queries Mojeek, grabs 20 results, and presents them to the user. The user can mark each result Good, Bad, or Skip.

Once the user has marked the results, they click “Analyze Selected Results.” QueryAnvil uses an API Ninjas tool to scrape the pages marked “Good” and “Bad,” while an OpenAI API call analyzes each set of results separately.

When I first started building QueryAnvil I wasn’t scraping the search results. Instead I was using the search result title and snippet for analysis. That sort of worked, and was small enough token-wise that I could use OpenAI’s GPT-4o mini LLM to generate brief reports. I wasn’t entirely happy with the results, though, so I added the page scraping.

Unfortunately that meant I needed a much bigger token limit to submit the scraped web pages to an OpenAI API call and GPT-4o wasn’t cutting it, so I switched to GPT-4.1. The good news is that the GPT-4.1 LLM has an enormous token window that has handled whatever the scraped pages could throw at it. The bad news is this LLM will not shut up. No matter how I craft my prompts I cannot get it to understand the concept of brief. So after the AI analysis I get a BIG ol’ report which starts like this:

I don’t see this as a long-term problem (more on that later) so I’m just accepting these reports for now and I generally end up reading them anyway. These reports are the kind of analysis you don’t get with the info-pellet style of AI searching. They show you how the search results are grouped thematically, what ideas are being addressed, and what words are being used.

When you get information this way you are learning about your topic. It is not directly via finding the perfect search result and exploring it, but rather via being exposed to search language and related ideas. Knowing this information will help you build both a better search and a better understanding. This is what you lose when using info-pellet, AI-forward search.

Okay, rant off. The report concludes with query words to add and avoid along with a couple of suggested searches.

I am still working on the prompt, but I’ve found the revised/suggested searches useful. As you might imagine, using this tool is pretty iterative as you explore what different keywords do with your search results. For that reason the program tracks the searches as you use them.

There’s also an export option to save the sessions as HTML, CSV, or JSON. Since I’m new to sump pumps and water features I know I’m going to forget some of this and want some way to retrace my steps.

To me, using AI as an analyzing assistant instead of the search driver is the only useful way to use it. The AI can analyze and summarize, but it’s up to me to digest the information and make the choices on what next step to take in the search journey. I am learning, but thanks to the AI’s abilities I can cover more ground.

This is a proof of concept however until I can use a local LLM instead of OpenAI. The GPT-5 LLM is overkill for my purposes and my searching is not worth the energy and water it would take. The idea of plugging in local AI is pretty exciting, though. Gotta keep my eyes out for some affordable hardware…