David Bareham

The task of forecasting case importance has received far less attention in the legal domain compared to judgment prediction, but it is a task that is an essential step to capture within tools for assisting with the processing of cases submitted to a court. In this paper we propose a cornerstone framework for carrying out the task of forecasting case importance, using communicated cases, which are documents available prior to any decision being issued. The setting for our work is cases in the European Court of Human Rights, with a specific focus on Article 3, prohibition of torture. We set out proposals for a Retrieval-Augmented Generation (RAG) framework that makes use of Large Language Models augmented with Semantic Search, Knowledge Graph and Re-Ranking components and we evaluate the effectiveness of this framework and its components. Further experiments conducted evaluate the framework using both pre-trained and fine-tuned LLMs, as well as use of different prompting strategies. The basic experiments show a propensity for the LLMs to significantly overestimate the importance of cases, but when we augment the LLMs with the aforementioned components, we are able to gain uplifts in performance. Our framework and results provide a solid basis for determining the requirements for the development of successful automated tools to be used to assist with determining case importance.

We report on a first set of results from experiments undertaken to tackle the novel task of determining the optimal context window for extracting and contextualising citation instances within case law. The wider task of outcome prediction using AI tools cannot be undertaken without considering the role that citations play when new cases are being decided. This short paper aims to shine a light on the importance of this task and provide the foundation for developing AI tools to capture citations’ context by examining a corpus of legal cases taken from the European Court of Human Rights. Our results show that there is an identifiable “Goldilocks Zone” of scoped paragraph-level context windows that attention can be focused on for extracting citation instances.