ChatGPT ' Generative AI And The Law

• Published date: 05 June 2023
• Subject Matter: Litigation, Mediation & Arbitration, Technology, Arbitration & Dispute Resolution, Professional Negligence, New Technology
• Law Firm: Gatehouse Chambers
• Author: Ms Charlotte John

Readers will probably have heard by now of OpenAI's ChatGPT. Many of you will have experimented with it, or one of the other available generative artificial intelligence (AI) models such as Microsoft's Bing. If you have not yet utilised any of these new technologies, you will very likely have read about them. This article is based on my experience with this novel technology, chiefly ChatGPT, as at May 2023, and aims to provide some thoughts on the potential benefits and pitfalls for the legal profession in its use as well as examples of potential use cases.

ChatGPT first became publicly available as a free-to-use research preview in November 2022. Within just 5 days of its launch, it had attracted one million users, and in 2 months, this number had rocketed to 100 million. The AI's human-like conversation abilities and its capacity to generate novel content generated a flurry of excitement, with social media soon filled with ChatGPT-produced lyrics, sonnets, stories, and so on.

The model initially launched was GPT-3.5, which is still the model available for non-subscribers. In March 2023, OpenAI released GPT-4, presently accessible to subscribers to ChatGPT Plus at a cost of US$20 per month. GPT-4 marks a significant improvement over its predecessor. Any ChatGPT output in this article (as set out in text boxes) is generated by GPT-4 and has been produced specifically for this article.

The AI landscape is evolving rapidly. Since ChatGPT's initial launch, several competitor products have emerged, including Bing (which is based on OpenAI's GPT-4 with the added functionality of internet access), Google Bard and Anthropic Claude. Microsoft has also announced a partnership with OpenAI to integrate GPT-4 into its Office apps. Given the current pace of change, the AI landscape is likely to have evolved further by the time you read this article.

It seems likely that these developments will significantly impact the working lives for all white-collar workers, including in the practice of law. Bill Gates, in a blog post published on 21 March 2023, hailed AI development as:1

'as fundamental as the creation of the microprocessor, the personal computer, the Internet, and the mobile phone. It will change the way people work, learn, travel, get health care, and communicate with each other. Entire industries will reorient around it. Businesses will distinguish themselves by how well they use it.'

An introduction to generative AI and ChatGPT

Let's ask ChatGPT to provide you with a short introduction to ChatGPT and generative AI.

Prompt: 'Please provide a brief single paragraph introduction, written for an intelligent lay audience, to generative AI models such as ChatGPT and GPT-4. Please include brief details of the training process for such models and explain key technical terms.'

Generative AI models, like ChatGPT and GPT-4, are a type of artificial intelligence that can produce human-like text, transforming the landscape of natural language processing. These models are trained through a process called deep learning, which involves feeding massive amounts of text data into a neural network, which is a complex mathematical system inspired by the structure and function of the human brain. The neural network learns to recognize patterns and relationships in the data, allowing it to generate contextually appropriate responses to user inputs. A core concept in the architecture of these models is the 'transformer,' which excels at processing and understanding sequences of information, making it particularly well-suited for tasks involving language. By leveraging the power of transformers and the sheer scale of data, generative AI models such as ChatGPT and GPT-4 continue to push the boundaries of what is possible in the realm of human-AI interaction.

This, somewhat self-congratulatory explanation requires some unpacking. It is important to understand what ChatGPT is and what it is not.

ChatGPT was trained on vast amounts of data scraped from the internet, as well as repositories of books and articles. This dataset enables it to converse fluently on a vast array of subject matters.

However, ChatGPT is not a search engine. The cut-off date for the dataset it was trained on is September 2021. As I write this, various plugins and applications are being developed that enable OpenAI's GPT models to be connected to other sources of content (Microsoft's Bing being one such example). However, the GPT model available to non-subscribers through OpenAI's free-to-use web application is presently unable to access the internet. Consequently, it cannot look things up online (although it may sometimes respond as though it does).

Further, the resulting abilities should not be mistaken for the cognitive capabilities that characterise human intelligence. Without wishing to get too bogged down in terminology and contested definitions, artificial general intelligence (AGI) refers to the possibility of machines that possess human-like cognitive abilities and remains theoretical.2

You should therefore heed ChatGPT when it tells you that it is generating 'contextually appropriate responses to user inputs'. It is not truly thinking or reasoning in the way that humans do. This might be an oversimplification, but it may be helpful to think of ChatGPT as super-smart predictive text - a hyper-intelligent version of that which you have on your mobile phone. When you give it a prompt, what it is doing is generating a statistically plausible response to the prompt. That response is based on the data it has been fed from millions of books and webpages and so is often not merely plausible, but accurate as well. However, ChatGPT will also on occasion plug gaps with confabulation, including inventing facts (often referred to as 'hallucinations'). This tendency to produce plausible sounding but factually false or otherwise erroneous responses has been reduced in GPT-4, as compared with GPT-3.5, as a result of a more extensive training process. Nonetheless, it is still present.

While the AI has been trained on vast amounts of data and can produce impressive results, ChatGPT may also be described as being 'black box' technology, in that the inner workings of the AI model, particularly how it processes inputs and generates outputs, are not easily understandable or transparent.

ChatGPT and the legal profession

There appears to be a new article daily about the repercussions of ChatGPT and generative AI for the legal profession, varying wildly between gloomy prognostications about the end of the legal profession and over-hyped proselytising about its transformative potential.

Goldman Sachs in a recently published report posited that two-thirds of current occupations could be partially automated by AI, suggesting that the legal profession has particularly high exposure (second only to office and administrative support).3

It seems unlikely, so long as the legal profession continues to be regulated, that advances in AI will spell the end of the legal profession, as some commentators have suggested. It is reasonable to suppose that there will be job losses, however, particularly in support roles.

You may also have read that Sir Geoffrey Vos, Master of the Rolls, has even suggested that some judicial decision-making could be delegated to AI.4 Whilst generally being a tech enthusiast myself, the ethical implications of AI-assisted or AI-driven judicial decision-making strike me as matters requiring particularly anxious scrutiny, particularly given the 'black box' nature of this sort of technology and its propensity to replicate bias.

Thought needs to be given too as to the implications of AI for the teaching of law and training of future lawyers. I imagine that many educational institutions are giving careful consideration as to how to weed out ChatGPT plagiarism, as well as how to ensure that students learn to think independently and to make judicious use of these tools.

Exactly what this nascent technology heralds for the legal profession is a matter of considerable speculation. I will confine myself to three, less serious, short-term predictions:

1. By the end of the year, you will grow tired of seminars where speakers shoehorn in GPT-authored sonnets, limericks, or raps.
2. At least one hapless legal professional will find themselves notifying their insurer because they relied on erroneous GPT-generated content.
3. You will have used AI in some shape or form, whether via any of the aforementioned AI tools or integrated into the software or hardware you utilise.

On the longer-term view, whilst it would be easy to dismiss the excitement surrounding AI as mere hype, there is reason to be paying attention to what is happening in this field. The legal profession parses and produces enormous amounts of complex text, and it seems inevitable that generative AI will play a significant role in shaping its future.

Further support for that view can be drawn from the testing of GPT-3.5 and GPT-4 against the Universal Bar Exam (UBE), a standardised test used by many US jurisdictions to test the skills and knowledge required for legal practice, including of core subjects such as contract, tort, real property, evidence and procedure, and so on, prior to becoming licensed to practice law.

Whereas GPT-3 scored in the 10th percentile as compared to a human cohort, GPT-4 demonstrated a very significant improvement and scored in the 90th percentile.5 It is important to note that the UBE is not merely a multiple-choice test, which one might expect AI trained on internet data to be proficient at. The UBE also consists of problem-based essay questions, as well as competence in certain drafting tasks.

Interestingly, GPT-4 performed poorly on certain questions related to my own practice area. It failed to calculate the distribution of assets from a testamentary trust deemed to be invalid and also erred in its analysis of a real property question involving the designation of a future interest and...