As dire warnings about artificial intelligence dominate the headlines, how are law firms responding to the AI safety debate?
As big tech continues its incursion into legal, legal AI vendors and law firms are launching their own proprietary models. This is unsurprising given the unease around generative AI (GenAI) frontier models after public statements from big tech stakeholders (and one former employee) about the dangers of GenAI. These included proposals that frontier models should pause AI development to allow regulators and governments to consider something like the tech equivalent of the nuclear non-proliferation treaty.
Safety first
Treating AI like a nuclear weapon feels like a resurgence of p(doom), an individual’s estimate of the likelihood that AI will cause human extinction or a permanent loss of human control. It is expressed either as a number between 0 and 1, or a percentage, and is subjective rather than conditional. Originally, p(doom) was a conversation topic among AI security technologists. However, it was revived globally when former Anthropic researcher Jacob Coxon posted on X that he had resigned because: ‘The people building AI earnestly believe that it could kill us all by the end of the decade.’
'We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain'
Dario Amodei, Anthropic
Anthropic researcher Evan Hubinger responded with his own p(doom) projection: ‘We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.’ He added: ‘To be clear… I think the risk from present models is low. What I am worried about is superintelligence arising from recursive self-improvement.’ Recursive self-improvement is when an AI agent can autonomously build the next generation of AI agents, with no humans in control.
The conversation was posted shortly after an announcement from Anthropic explaining how Claude is helping the company develop the next, more intelligent version of itself, although it is not (yet?) working autonomously.
Anthropic CEO Dario Amodei responded with a blog about the need to manage the pace of AI development. ‘We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,’ he wrote. OpenAI’s Sam Altman agreed, but there were dissenters, notably Nvidia’s Jensen Huang and president Donald Trump. In a 26 August article, Bill Gates said that governments and organisations were not sufficiently prepared for the AI era. This escalated into a broader debate about the need for national and international regulation to address AI safety concerns.
Big tech and big money
On 25 August, Google followed Microsoft and Anthropic to launch a legal GenAI offering. Gemini Enterprise for Legal offers purpose-built skills, partner agents and model context protocol (MCP) integration with legal AI platforms. And on 17 September, OpenAI launched Astra for Law, ‘powered by GPT-6 Astra, with tools, settings and context to support the expertise and judgement of lawyers and legal technology firms’. It also includes 26 partner plugins – including Thomson Reuters, Harvey, Legora and iManage – and 47 community plugins for legal work in ChatGPT.
Frontier AI companies are continuing to raise investment at unrealistically high valuations. OpenAI went some way towards recognising this when Altman announced that there would be no IPO this year. And legal AI companies are mirroring frontier models by raising funds strategically – if the bubble bursts, the cash they raise now will help them survive.
On 9 September, Harvey announced a $550m funding round at a $15.5bn valuation. Legora is reportedly in discussions to raise funds at a valuation of at least $10bn, nearly double its April valuation of £5.6bn.
Legal AI vendors’ response to the AI safety debate has been to launch legal-specific large language models not linked to US frontier AI companies. Both Harvey’s Tenet and Thomson Reuters’ Thomson are post-trained AI models for legal, based on Chinese open-weight models. (An ‘open-weight model’ means an AI model’s core components having been publicly released for anyone to download.)
As more US law firms invest in proprietary systems to establish organisational AI sovereignty, Latham & Watkins has taken this a step further, becoming the first law firm to announce publicly that it is investing in its own hardware and fine-tuning open-weight models. Latham purchased several Nvidia GPU servers, enabling it to run customised Nvidia Nemotron 3 open-weight AI models within the firm’s infrastructure. Chief information officer Rene Mendoza told the Financial Times: ‘Sometimes we may have information that is so sensitive, client information that we really want to protect, we don’t want to put it to any cloud vendor.’ The firm wanted flexibility in light of ‘consumption costs coming’ for AI usage. ‘We are not hitching our wagon to any one company,’ he added.
The latest agentic buzzword, ‘multiplayer AI’ – agentic AI deployed in a collaborative environment – could also describe the current legal AI landscape.
Royal rumble
Last week, King Charles hosted a private AI safety summit to consider whether there should be a ‘shared set of principles’ for the future application of AI. The King warned AI leaders of the ‘existential dangers’ of the technology falling into the wrong hands and being used in potentially catastrophic ways. ‘Surely we need sufficient means of control before it’s all too late?’ he said. Delegates included AI minister Kanishka Narayan, Nvidia’s Jensen Huang, representatives from OpenAI and Anthropic, and Paolo Benanti, who advises the Pope on AI. This meeting of state and church leaders with AI frontier companies acknowledged the power and danger of AI. It proposed what sounds like the equivalent of the nuclear non-proliferation treaty, except for the absence of major global players that are certainly key to shaping the future of AI. Next week, the Labour party conference will vote on proposals to create a more powerful AI regulator for the UK.
Multiplayer AI
Multiplayer AI is another potential differentiator, and it is more cost-effective than purchasing servers and customising models. Ethan Mollick, Wharton associate professor researching the impact of AI on work and education and author of AI best-seller Co-Intelligence, wrote on LinkedIn: ‘Organisations can develop their own approaches here, building upon the models and harnesses currently available and combining them with their own organisational capabilities and employee talent. Figuring it out before other firms is a potentially very large advantage.’
In his Legal Tech Trends newsletter, consultant Peter Duffy of Titans highlights Wordsmith Intake as an example of multiplayer AI in legal because ‘the agents do the actual work’ rather than helping lawyers work faster. Wordsmith AI, an agentic platform that supports in-house legal teams, recently extended its Series B funding to $84m. Its legal intake app goes beyond triage to start working on matters. According to its website: ‘Agents review documents, apply playbooks, gather context and prepare responses as soon as work arrives. Lawyers step in when judgement is required.’
Agentic insurance?
Last month I considered the implications of agentic AI for professional indemnity insurance. A recent Gazette feature highlighted a survey by Everywhen Insurance which found that 41% of organisations identified AI mistakes as their biggest risk (tinyurl.com/2k393mc8). Currently, most AI mistakes discovered in legal relate to litigation and people not checking AI output. However, as agentic AI for legal gains traction, it increases the potential for the AI itself to make a mistake. This could raise liability issues later.
Native AI firm Crosby is buying insurance for its AI agents. What are the insurance implications when AI works semi-autonomously? When it is supervised by lawyers, rather than assisting them? Jon Cook, co-founder and director of Quality PI, explains: ‘At present, the SRA minimum terms professional indemnity insurance policy which law firms have to purchase is silent on the question of AI use. It is therefore assumed to be a part of private legal practice. Errors flowing from it are therefore catered for under a firm’s PI insurance. It’s not been properly tested yet though.
‘There is an insurer who provides AI coverage now. It does so on the basis that the firm would pay small and medium-sized losses while the insurer would act as a backstop for catastrophe provision. The benefit of a firm using this insurer is the quality of its “paper” or solvency rating, because it is effectively adding to the firm’s credibility for delivering a “guaranteed” service if anything went wrong.’
The SRA’s Misuse of AI warning notice published on 17 August reiterates that solicitors and law firms that use AI remain responsible for maintaining professional standards and client confidentiality.
Progress report
Legal is taking a more measured approach to AI adoption. There is a clear shift from initial evangelism to achieving an acceptable level of proficiency across the firm or legal department.
Titans’ research report, The AI Fluency Framework, commissioned by Harvey, is based on a series of interviews with Harvey customers. It identifies five levels of AI fluency within two broad categories: individual productivity and institutional capability. The tipping point is when collective AI proficiency becomes an institutional asset. Another transition, ‘the builder shift from consuming to creating’ tends to be more challenging. The report includes tools and tips for moving everyone to AI proficiency, clearing the path for the builder shift, and identifying rituals to sustain and grow sufficient institutional capability to thrive in the AI era.
No comments yet