What the public doesn’t get about case law
‘So I’ve found a few cases you might want to look at…’
I was catching up with a couple of law buddies last night - both very smart guys, both from corporate law. They hate, and have very successfully avoided, reading or caring about case law for essentially their entire legal careers, but sometimes they’ll ask me questions about the litigation-side of things. Yesterday we talked for a minute about what case law really is, and what people don’t get about it.
When I was in private practice, every so often I would have an eager client come to my office having done their own case law research.
Trigger big eye roll.
Perhaps unfairly, lawyers tend to find that kind of thing very annoying. Partially because lawyers really just want to be brought interesting cases and then completely left alone to exercise our ‘brilliance’ on them (most lawyers will tell you law is a great profession, except for the clients). But also because a big part of any client-serving profession is expectation management, and in law that can be a real challenge because of how complex the court system can be, and how difficult it can be for the average person to detach emotionally and look at things in the sort of logical way the courts demand.
Law, and ‘justice’, are very personal topics. Everyone feels like they know exactly what the just outcome would be in their situation, and can be really creative in finding sources to back up what they’re sure should happen (what we normally call ‘confirmation bias’). For people with legal cases, that often means heading over to a free case law database like Canlii and doing some searching until they find a case they think proves that they’re right and can’t possibly lose.
The problem is…
People fundamentally misunderstand the point of case law. Which isn’t their fault - most people aren’t lawyers and don’t interact with the legal system in any meaningful or consistent fashion. In short, here’s what people don’t get:
Reported case law is usually not representative of most legal outcomes, because the vast, vast, VAST majority of legal cases don’t get reported.
In fact, in many areas of law the entire reason a case gets reported is because it’s unusual or noteworthy in some way. See the problem?
Most people think all cases get reported. In criminal law, a case results in a withdrawal, a guilty plea + sentencing, or a trial - withdrawals can’t get reported, and way less than 1% (and probably way less than 0.1%) of sentencing or trial decisions get reported. In civil, I understand many more (and potentially all?) litigated cases get reported, but extremely few cases get litigated, the vast majority settling.
Combine that with the fact that reported case decisions don’t necessarily tell the whole story, and need to be read in a very specific way that requires some professional experience and training, and you have a recipe for an expectation management disaster.
Anyways, that was a fun discussion. I wish all of my former clients caught on as fast as my buddies last night did.
Regulating and litigating AI is a huge area that is going to explode, and even people in law don’t seem to see it coming
Generative AI’s moment
ChatGPT, generative AI and all of the sexy stuff in the news has hit an interesting point. Unlike blockchain technology, which is a very interesting idea and very useful in a few specific situations (thus far), generative AI is immediately useful to whole swatchs of average people across society and will only get more and more relevant as time goes on.
One dimension of this whole story that I spend a lot of time thinking about (that most people I’m sure would find worse than watching paint dry) is all of the regulation and liability. Once a lawyer… eh?
People really don’t understand AI. I mean, even people at the forefront of AI don’t fully understand AI. And the average person, frankly, doesn’t understand how a website or email or Microsoft Word works. And the people regulating this stuff don’t get it either, and are always at least 10 years behind where they should be to have any chance at effective regulation.
Whenever there’s a new technology capable of doing crazy things at scale that nobody understands, you’re guaranteed to get a bunch of early pioneers and pirates using it for all sorts of crazy uses that will make up the plot of some Oscar-nominated movie in 20 years. But before that? The lawsuits.
Aaahhh, the lawsuits
We all click “I accept” on cookie banners and terms/conditions all the time. And we’re not really thinking twice about it.
Privacy? Who the hell cares? Whatdya think, the CIA cares about my Candy Crush scores?
(do people still play candy crush? am I dating myself?)
The reality, though, as we slowly transition to the meta-verse (ie. live more and more of our lives online), this boring data-privacy stuff actually matters. But until we get fed click-baity, ultra-specific, scary examples, nobody cares.
Even when we get those examples, somehow people still don’t care! Remember kind of hearing about Cambridge Analytica and the whole election interference, Mark Zuckerberg testifying in front of Congress thing? Here’s one of many primers.
One of the guys at the center of that whole thing wrote this book called Mindf*ck, and if you want to understand why you should care about data and privacy and blablablabla…. you should read it. It’s a crazy read.
Long story short: it’s not that hard to collect seemingly innocent information about people at massive scale, and then use it to manipulate those people in weird, scary and super upsetting ways.
But people still don’t care. Even my wife, who is a very talented up-and-coming civil litigator, rolls her eyes into the back of her head whenever I talk about this stuff. (meanwhile, in a few years where it’s big business to sue/defend these types of suits, she’s gonna really care all of a sudden)
People think “who cares if people find out what netflix shows I watch.” Well, this lady cared when she sued over the potential for being outed as a result of insufficiently anonymized data Netflix released as part of an open contest to improve their algorithms. People think, “who cares if Target knows some basic information about me and looks at trends in what I buy.” Well, this alarmist article from a few years ago featured a story about a dad who wasn’t too thrilled that Target figured out his daughter was probably pregnant and started sending her targeted maternity ads.
There’s a lot of content online dedicated to making fun of anyone worried about data privacy ‘paranoid’, or explaining why it’s not nearly as scary as we might be led to believe. Check out this article, for example, which explains all about why the Target thing probably didn’t happen, or if it didn’t isn’t exactly what they said it was, or why even if it did we shouldn’t be that scared in general. Which appears to be written by a guy who’s a data scientist at Meta (‘Facebook’). Good, good. No conflicts of interest there.
My favourite thing online is when I look at reddit threads or comment sections where people start off: “Well I’m not a lawyer, but…” and then spout off a bunch of 100% certain legal predictions (with the kind of unearned confidence I wish I could bottle and chug before family get-togethers) and then tell you that it’s all fine and legal and stop freaking out.
(it’s especially adorable when you can tell they’ve really tried to word that opening part like a legally binding disclaimer)
Well, I am a lawyer. All of this stuff is absolutely stuff to think twice about, and is going to be what’s keeping the lights on at law firms for the next gazillion years as the influence of AI and tech spreads, and people start understanding how this stuff actually works.
Come to think of it, this may be a very nice career back-up plan. You know what, forget about all that privacy stuff. It’s probably fine.
The grind
Finished creating that highlight-extractor for my wife, which is essentially a very basic script right now that outputs an excel file with all of the highlights and basic info about the transcript, which is all she wanted and exactly the format she wanted it in. That’s fun. Let’s see how she likes it.
It’s interesting the choices you can make now when putting together even basic NLP scripts, in terms of just short-cutting with GPT.
For example, the first few pages of a discovery transcript tell you the name of the proceeding, the date of the examination, the witness(s) getting examined, etc. Historically, I would have had to go looking for some sort of already half-baked ML-powered solution and spent time adapting it for my use case, or sat down and done some serious reg-exing given enough examples she could provide. That would take time, and usually be pretty breakable.
Now, I can just extract all the text and send it in an API call to GPT 3.5-turbo with some basic instructions on what information I need extracted and what form I’d like it returned in. In fairness this isn’t always quite as easy as it sounds - you still need to do some prompt engineering to make sure you’re reliably extracting what you need, returning the format you want, and can deal with null or edge cases. And it costs a fraction of a penny and an API call, rather than free and local compute. But still, it’s nice to have the option!
I’m now going to return to hacking away at lawbrarian. It’s time to explore langchain’s agents and tools. The fact that there’s a ‘human’ tool, and that you can code custom tools, is blowing my mind. Just take my money.
Seriously though - people (ahem my wife) do not seem capable of understanding just how crazy the chains you can create with this thing are, at least until you put a working example in front of them. But the implications of the ‘human tool’ in particular are mind-blowing - where your AI script can now do a bunch of chained together complex tasks, empowered by custom tools you’ve given it ideally suited to the job, and then ask you for help/guidance where it gets stuck, that is some scary stuff.
I’m gonna try to teach it to construct boolean searches and do legal research on Canlii. That will be a heck of a lot more scalable than sitting down and essentially writing my own set of textbooks, so let’s try that first.