I've always been an AI sceptic. I'm not naive about it - there are great uses in medicine and science, but I've always been particularly sceptical about generative AI. You know, the kind that can generate text, images, and even video.
I've got around to trying a generative chatbot AI in my personal time, and I was quite surprised by the accuracy of some responses. Where there is a good body of work available on the internet (and this article will not cover some of the dubious and maybe-illegal practices carried out by the large AI companies for collecting training data) there seem to be good answers. I was riding the high of getting what was essentially a better search engine which valued my time.
At work, I was trying to get my head around one of the Land Registry's Practice Guides. For context, these are always changing and updated with new Land Registry practices, and in my experience, the Land Registry’s own track record for following their guidance is inconsistent. I was stumped on a particularly confusing piece of guidance which made little logical sense and appeared to exist purely to make an arbitrary extra requirement, in addition to the underlying property law.
In my personal time, I was also curious about how AI could affect law, so I decided to ask an AI chatbot. By now I was expecting a high degree of accuracy that had come from my other prompts, but this time was different - it was an area that I was very familiar with, and I was capable of sussing out bad reasoning.
The result seemed benign at first, the chatbot told me that I was missing something, and actually the logical point of view was correct, and pointed me to the Land Registry guidance. I clicked on the guidance, which led to a .gov.uk "Page not found". Hmmm, strange. I told the chatbot, which apologised, and pointed to a new source. Again "page not found". Further, the references were to incorrectly named practice guides, and paragraphs which did not exist. I went through more cycles of pointing out fake sources, being apologised to and told I was correct, while being presented with new fake sources. My subjective feeling went from a small amount of disappointment to a kind of embarrassed frustration that I was being gaslit by a computer.
I've never really been able to accuse computers of making "mistakes" before - you can usually find some kind of user or programmer error to blame. But here there was no user error to blame - it was bare gaslighting, emanating from the machine - here it was more than a computer carrying out an algorithm, it was telling me things. Sweet lies, but lies nonetheless.
The AI was giving me an answer I wanted to hear, but the correct answer was in the difficult to navigate practice guide. The more confusing answer was real and the simple answer was fake.
HM Land Registry Practice Guides are intended for conveyancers and can be obtuse and fraught with technical issues. They are updated as Land Registry practice improves. A mistake with registration can have serious effects. At the more serious end, there is a risk of non-registration which means a would-be proprietor may not get legal title. They may find it difficult or impossible to sell, lease or mortgage their property.
This may have significant financial consequences. And where there are significant consequences at stake - legal, financial, medical or family consequences, critically-thought, human-reasoned knowledge is without competition.
AI seems to be great for people who already know what they are doing, it's a high powered search engine with the feeling you got when Google replaced Ask Jeeves. But the attraction of AI to people thinking it will give them guidance or a plan in a technical area they know nothing about is a dangerous mirage.
AI is probably going to change the legal field, and practice a decade from now will look different. But as we make these changes, we should ask "How often will AI get things wrong?", when the large oligopolist corporations are pushing the narrative of "AI can get things right". I think the effect of AI on law will look more like Clippy v2 than an a full-on artificial lawyer, like how the office suite replaced the typewriter and how e-mail has largely replaced snail mail. Time will tell.
My takeaway is to make sure that you test the logic and go through the underlying sources. I've always been guilty of taking web search results at face value for topics I am not familiar with, any my laziness carried over to AI. My mistake was was not recognising that AI results don't give sources, they just guess sources.
NB. As at the time of writing, SIMO & Co Solicitors do not use generative AI as part of the delivery of our legal services.