Heuristic Evaluators

Quote

A heuristic (from Ancient Greek εὑρίσκω (heurískō) 'method of discovery'), or heuristic technique (problem solving, mental shortcut, rule of thumb) is any approach to problem solving that employs a pragmatic method that is not fully optimized, perfected, or rationalized, but is nevertheless "good enough" as an approximation or attribute substitution ¹.

Heuristic Evaluators use predefined methods and criteria to assess the performance of your LLM Apps. These evaluators are based on established techniques such as BERTScore and Semantic Similarity measures, providing a structured approach to evaluation.

Supported Heuristic Evaluators

• BERTScore
• Semantic Similarity
• JSON Difference

Pricing Considerations for Heuristic Evaluators

Running heuristic evaluators is less expensive because they utilize text embedding models, such as text-embedding-3-smallfor evaluation. This contrasts with AI-Assisted Evaluators which use more resource-intensive language models like gpt-4o.

While Lynxius plans to reduce the pricing of heuristic evaluators in the near future, currently, all evaluators are charged equally to maintain a simple pricing structure.

---

1. ‘Heuristic’ (2024) Wikipedia. Available at: https://en.wikipedia.org/wiki/Heuristic (Accessed: May 23, 2024) ↩