People are often surprised when I say I was working with "AI search" twenty years ago. It didn't have a chat window, and nobody called it AI. But the idea at its core, that a computer can find things by meaning rather than by matching words, is the same idea behind today's AI tools. Building the system around that engine is where my understanding of AI really began.
The project
In the mid-2000s I was the CTO of PatentCafe.com, and I was responsible for the technology behind ICO Suite (Intellectual Capital Office), its patent search and analytics service. I designed the databases and the overall architecture, wrote the import that loaded patent data from XML into SQL Server, built the export into the Engenium semantic database, and trained the semantic index. Pretty much every piece of technology in the system came through my hands.
The collection held more than 2 TB of data: over 3 million US granted patents and the US published applications, plus patent collections from around the world, including Europe, Great Britain, Canada and international PCT filings, and customers' own private portfolios. The search engine was Engenium's Semetric (later part of Kroll Ontrack), built on a technique called Latent Semantic Analysis (LSA), also known as Latent Semantic Indexing (LSI).
The problem with keywords
Patent searchers traditionally used Boolean keyword search. One of our training examples searched for:
semiconductor AND surface AND defect
That returned 21,872 hits, all with equal weight. Every one matched the words, and many of them had nothing to do with what the searcher actually wanted: methods of detecting surface defects on a semiconductor wafer during the manufacturing process. Keywords can't tell the difference between a document that uses the right words and a document about the right thing.
Searching by concept
LSA reads an entire full-text collection and builds a second database, a concept space, where every document and every idea is represented as a vector, a point in a space with many dimensions. Documents about similar things end up close together, even when they use different words.
A keyword search for "automobile" finds only "automobile." In the concept space, a query describing a two-seat automobile finds and ranks everything by how close it is to that concept:
- Coupe 100%
- Passenger vehicle 99%
- Convertible 95%
- Sedan 89%
- Motor vehicle 88%
- Car 87%
- Vehicle 86%
- Auto 84%
- Horseless carriage 72%
- Pickup truck 61%
- Train 47%
Nothing here requires a keyword match. The engine understands that "horseless carriage" and "sedan" describe the same kind of thing when they're used in that context, and that a train is related but further away.
Two ways to search
Heuristic Boolean ("SuperBoolean") kept the familiar keyword string but added a plain-English description of what you were really looking for. The same 21,872 hits came back, but now ranked by relevance, with the most useful patents first. The engine even knew that "gettering" was related to surface defects in wafers, though that word wasn't in the query.
Semantic search dropped keywords entirely. You pasted in a description of the invention, and the engine ranked the whole collection, such as all 3 million-plus US granted patents, by how close each one was to that concept. The first result was 100% relevant, and the first few pages held the patents that mattered. You could then narrow the results with a Boolean "search within" for precision.
Writing a good query
The training taught that a good semantic query should:
- describe one single, clear concept, because you can't "OR" two ideas together in a concept space
- be 200 to 500 words long, with enough context for the engine to understand the idea
- come from real descriptive text: a patent claim with its dependent claims, an invention disclosure, a competitor's product specification, or a plain-language description
The example in the deck was a patent claim that described both a method for producing spun yarn and the device that does it. As a query, it confused the engine. Split into two queries, one per concept, each worked well.
Beyond search: analytics
Because every result had a relevance score, the results could be analyzed as a whole. ICO Suite could chart who owned the top patents, cluster results by concept, show which patent classes a technology really lived in (often surprising ones), and plot the relevance curve across the first 20,000 results to see where the core technology ended. Its Patent Factor Index reports scored a patent's legal, commercial and technical strength.
Real uses included pasting a competitor's product description into a search of your own portfolio to find the patents most likely infringed, and pasting a patent's claims into a search of earlier-filed patents to look for prior art.
Why it still matters to me
Today's AI tools are built on the same foundation, at a vastly larger scale:
- Concept vectors became embeddings. Modern AI represents words, sentences and documents as vectors in exactly this way. "Vector search" and the retrieval behind many AI assistants are direct descendants of the concept space.
- A good query became a good prompt. One clear idea, enough context, real descriptive language: that is still the best advice I know for working with Claude or any other AI.
- Relevance is a score, not a certainty. The engine ranked by closeness, and the searcher still had to read the patents. AI works the same way. It is a powerful tool, and you still need to check the work.
When large language models arrived, they didn't feel like magic to me. I had already loaded millions of documents into a concept space, trained the index and watched how query wording changed the results. Modern AI felt like the next chapter of something I had built. That foundation is why I can help people understand AI without the hype, and why I'm comfortable working alongside it every day.
See what I'm working on now on my Projects page, or get in touch if you'd like a talk on how AI search works.