Heuristic Algorithm
Heuristic word used a lot in technical discussions. It's important to understand what it means really and how we can use it in our work.
Heuristic algorithms use heuristics to solve a problem. They use an educated guess to find a solution. The solution may not be optimal or exactly correct.
Heuristic comes from the same Greek word as Eureka, which means "I have found it."
The word doesn't mean "guessing." It means "finding a solution good enough for the problem."
Examples of Heuristic Algorithms
Most heuristic algorithms pick something largest, smallest, or most recent, based on the problem. This choice is what makes it a heuristic algorithm. It isn't sure to be optimal. Still, it's good enough for the problem.
- LRU Cache - It uses the heuristic that the least recently used item is the least likely to be used in the future.
- Search Ranking - Search engines use heuristics to rank search results based on various factors like relevance, popularity, etc.
- Context Selection - pick nearest tokens for the context using embeddings.
- Prompt Truncation - No logic. Just keep the most recent tokens and truncate the rest.
- Many more. Once you know what heuristic means, you see how often we use it at work without noticing.
To find a restaurant for dinner, we don't check every place in the city, with its menu and reviews.
We just see a place is crowded and guess it's good.