Posted: September 2026
Contact: Tim Repke
The root of every systematic literature assessment is a comprehensive keyword search to filter the hundreds of millions of articles ever published to a more manageable set of references to be screened.
Constructing such search queries is a very specific process and it is hard to judge how some changes impact the overall result.
Typically, we use boolean queries such as these:
- ( "carbon border adjustment" OR "border carbon adjustment" OR CBAM ) AND (EU OR europ*)
- ( afforestation OR reforestation ) OR ( (tree or forest) NEAR/3 restoration )
- Most often, queries are more complex than that, for example:
https://github.com/destiny-evidence/impact-case-1/blob/main/ic1/query/revisions/query_20260408.py
In the ideal case, the research have a set of known-relevant studies and domain expertise to start coming up with keywords and keep expanding/improving the query to cover all known-relevant articles without being too inclusive. The goal is to find all relevant literature (making the query more generic, improving recall/sensitivity) while keeping the result set as small as possible (making the query more specific, improving precision or specificity).
We want to build tools to help researchers to make faster and better informed decisions on how to build a good query.
To do so, the tool should give feedback on which terms or groups of terms contribute how much to the overall search.
It should also provide suggestions on terms to add to the query.
In the future, it would also be nice to highlight not just how the overall counts change, but if adding a term adds a new "topical cluster" to the result set.
We have direct access to a large search index (hosted in elasticsearch and solr) and can thus directly access the index statistics for fast interactive workflows.
Some examples of ideas that are possible to implement right away based on the elasticsearch documentation:
Impact of terms:
- Get prevalence of individual terms in search index https://www.elastic.co/docs/reference/elasticsearch/rest-apis/term-vectors-examples#docs-termvectors-terms-filtering
Suggest terms:
- Recommendations from semantic embeddings
- Recommendations from generative AI
- Recommendations from search index, e.g. Which terms often come after a term of interest? For example "climate" -> "change" (100), "action" (80), ...
https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/search-as-you-type
https://www.elastic.co/docs/reference/aggregations/search-aggregations-bucket-terms-aggregation - Make wildcards explicit and get suggestions for options. For example: "flood*" -> flood, flooding, floods, flooded, ...
Near operator:
https://www.elastic.co/docs/reference/query-languages/query-dsl/query-dsl-intervals-query
Possible tasks
Some of the sub-tasks are listed here and they can be combined in any way based on the student's background and interest. This can be a very practical project to build a tool or an in-depth research project to develop more advanced suggestions, real-time semantic clustering (see solr for inspiration), or ways to display the "difference" (to be specified) between two queries and their result sets.
- MVP interface to host suggestion "modules"
- Visualisation of query (tree/graph structure?) with sub-query/term sizes
- Easy way to show example documents per sub-query
- Clustering view
- Research: how to compare two large query sets really fast (not just number of documents, but "semantic" overlaps/differences)
- ...
