Generative AI is starting to transform the legal industry, and the powerful new tools are yielding powerful new results. But getting good results from generative AI tools isn’t as simple as it might seem. Writing the right prompt, asking the right questions, will change the quality of the results that you receive.
Generative AI has the potential to superpower legal professional’s productivity and skills, but also has the risk of giving misleading or even false information. It is crucial to know how to create good prompts so that you can accelerate outcomes, and minimize the risks of poor results.
In this article, we will explore the crucial role of prompt engineering, the art of writing the right query. We’ll dig into the basics: what is prompt engineering and why it’s important, discuss a few approaches to prompt writing and share some best practices for you to employ when working with generative AI, specifically for legal tasks.
Prompt engineering is the art of crafting precise instructions to input into a generative AI model that will result in optimal answers. Just as with a regular conversation with another person, asking good questions leads to better answers. If you were to ask vague, meandering questions with no context to a stranger on the street, you might get a vague, unintelligible answer (or a lie). Similarly with generative AI - asking poor questions could lead to poor results.
Prompts - or questions, are the foundation of generative AI models. And learning how to write good prompts will allow you to harness the full potential of the AI models.
This is particularly important for legal professionals, where the risks of missing key case details, misrepresenting information, or getting made up (hallucinated) answers could be particularly risky.
As you work with generative AI, the prompts and questions that you ask, can be generally categorized into a few types.
Each prompt type has a distinct purpose, and knowing which type of task you are taking on can help guide you to the right ways to formulate your prompt.
There are many approaches towards “how” to prompt engineer. Each has its own strengths and weaknesses - but they are primarily related to broader scopes of projects: creating lengthy premade prompts and crafting multi-step AI engagements.
In this article we’ll only highlight a few, as they lay the foundation for more individual learnings to follow. Note that these approaches are most useful for crafting extensive prompting, not one-off requests.
Before we dive into specific tips and tricks, let’s review the anatomy of a prompt. Crafting an effective prompt requires attention to all of the key elements of a prompt. Think of each prompt as a short recipe with key ingredients. Failing to include an ingredient might not be felt for a small dish (say, you forget the salt in your baked good), but could also lead to some very unsavory results (say, forgetting the eggs for your cookies).
Here is what all good prompts are made of:
To enhance prompt engineering and optimize the use of generative AI in the legal field, consider the following best practices:
Bonus Tip: If conducting Legal Research, try to avoid basic generative AI prompts. Even with great prompt engineering, the risk that the AI hallucinates (makes up) a case is too high. Ask your AI providers if they have a dedicated Legal Research functionality that allows you to prompt the models to look through a curated dataset of legal precedent.
Now that we know about the basics of prompts, and the best practices surrounding them, let’s discuss some of what could go wrong for a legal professional using generative AI without good prompt engineering.
There are three main categories of problems that can arise with poor prompt engineering:
Understanding the risks of poor and non-thoughtful prompting will help create a better experience and better outputs for legal professionals enhancing their workflows. Although good prompting can help alleviate some of these risks, it is still important to check and verify the work that AI is doing, especially for key legal work.
Let’s cover a quick example for legal work, to illustrate what good vs poor prompting could look like.
Let’s consider the example of using a generative AI tool to identify inconsistencies across multiple depositions (or pieces of evidence). This is a particularly apt example, since it combines elements of analysis and creative thinking. We’ve found that analyzing a document and extracting key facts, although time consuming, is a relatively easy task to take on. But combing through two or more documents, extracting details from them, and doing a check to cross-reference each fact to find differences is an exponentially harder (and lengthy) task.
So how would you go about prompting your generative AI to identify inconsistencies between your depositions?
First let’s do it the “wrong” way.
“Identify the inconsistencies between these two documents.”
Unfortunately, this simple prompt might not yield the best results. Let’s review our prompt ingredients. Although this prompt is giving clear and unambiguous instructions, it is missing context, and doesn’t give any output guidelines.
This prompt is likely to yield incomplete results, such as missing key details from the case, or might make up inconsistencies that were not present in the documents.
So how could we improve this prompt?
First, we can break this task into smaller pieces. First, we identify all of the facts mentioned in the depositions. Then we ask the AI to parse through the case, with the facts identified as context, and pick out the inconsistencies. And of course, we don’t want to forget our context.
“I am working on an [employment law] case, where my client claims that [he was wrongfully terminated]. I have provided you with two depositions from our case. I am analyzing transcripts of depositions that recently occurred. I’d like you to help me identify any inconsistent testimony that may have occurred in these depositions.
Please read through any depositions I have provided you with thoroughly before identifying the inconsistent testimony. For each deposition, please list out all of the different facts and statements made by the witness.”
The first section of our prompt is context: what is the case we’re working on (for our example we mentioned just the basics of an employment law case, but you could enter all the pertinent details, including your state, your claims, etc). The second section is the clear, concise, and unambiguous instruction - telling the AI to perform one specific task. The second section is also making sure to balance getting clear information, in a targeted way, making sure the task doesn’t bite off more than it can chew (for example, identifying key facts AND analyzing them for inconsistencies all in one go.
Now we can move onto the second prompt that brings the task to a close.
“Now review the depositions again and identify any major inconsistencies in the deposition transcripts. Please only highlight inconsistencies that may indicate the witnesses may not be telling the truth.”
This prompt is also clear and concise. It gives parameters for the results, keeping the work focused and most meaningful. In this second prompt we won’t need to provide the context, since the model should be referencing our first message (or better yet - a specific “instructions” tab that includes case details and contexts).
You can also see that this sequence of prompts is following a few of the “approaches” to prompt engineering: it starts broader (list all the facts) and then narrows in on the task (identify inconsistencies) - see Least-to-Most Prompting, it limits the scope of the prompt to tackle the most impactful results (only identify major inconsistencies) - see Directional-Stimulus Prompting.
Prompt engineering is the cornerstone of using generative AI effectively, in and out of the legal industry. By understanding the significance of prompt engineering and adopting best practices, legal professionals can unleash the full potential of AI systems to analyze information, expand knowledge, and facilitate creative thinking.
Knowing about prompt engineering, understanding the different approaches and the core components of a prompt is a great first step in becoming a good prompter and a powerful AI user.
Remember, prompt engineering is a skill that improves with practice and experimentation. Where you can, ask for support from your generative AI providers to help you craft the best prompts. Stay curious, explore new approaches, and continue refining your prompts to unlock the true potential of generative AI in your legal work.