David Bareham

In light of the increasing prominence of generative AI, this paper examines a number of critical challenges facing the stratified policy and governance frameworks shaping the adoption of this technology in the legal services sector. We argue that there are counterproductive tensions within the existing policy and governance framework, and that if we are to make the UK legal sector ‘ready, willing, and able’ to harness the potential of artificial intelligence, it is imperative that we set in place a new governance framework that is capable of setting, promoting, and supporting innovation across the legal sector as a whole. Furthermore, within this policy and governance framework we consider two critical factors that are relevant to state-of-the-art AI systems which present critical challenges to the development of legal AI: (1) data access issues in the domain of law and (2) knowledge, skills, and awareness of capabilities relating to the application of generative AI to legal problems.

The task of forecasting case importance has received far less attention in the legal domain compared to judgment prediction, but it is a task that is an essential step to capture within tools for assisting with the processing of cases submitted to a court. In this paper we propose a cornerstone framework for carrying out the task of forecasting case importance, using communicated cases, which are documents available prior to any decision being issued. The setting for our work is cases in the European Court of Human Rights, with a specific focus on Article 3, prohibition of torture. We set out proposals for a Retrieval-Augmented Generation (RAG) framework that makes use of Large Language Models augmented with Semantic Search, Knowledge Graph and Re-Ranking components and we evaluate the effectiveness of this framework and its components. Further experiments conducted evaluate the framework using both pre-trained and fine-tuned LLMs, as well as use of different prompting strategies. The basic experiments show a propensity for the LLMs to significantly overestimate the importance of cases, but when we augment the LLMs with the aforementioned components, we are able to gain uplifts in performance. Our framework and results provide a solid basis for determining the requirements for the development of successful automated tools to be used to assist with determining case importance.

We report on a first set of results from experiments undertaken to tackle the novel task of determining the optimal context window for extracting and contextualising citation instances within case law. The wider task of outcome prediction using AI tools cannot be undertaken without considering the role that citations play when new cases are being decided. This short paper aims to shine a light on the importance of this task and provide the foundation for developing AI tools to capture citations’ context by examining a corpus of legal cases taken from the European Court of Human Rights. Our results show that there is an identifiable “Goldilocks Zone” of scoped paragraph-level context windows that attention can be focused on for extracting citation instances.