Trust and Estate Expert Witness’ Copilot-Generated Damage Estimate Rejected
Posted on October 17, 2024 by Expert Witness Profiler
Michael S. Weber (hereinafter the “Decedent”) died a resident of Saratoga County on December 7, 2003. He included rental property worth $485,000 in a trust for the deceased man’s son. The deceased man’s sister was responsible for executing the trust. The Court is supposed to assess if the executrix and trustee—the deceased man’s sister—breached her fiduciary duties by delaying the sale of the property while admittedly using it for personal vacations.
The son had to prove that he suffered damages as a result of the aunt’s alleged misuse of the property. Charles Ranson was retained to to figure out how much would be owed to the son had the aunt sold the property in 2008 compared to the actual sale price in 2022. It was repeatedly pointed out that Ranson lacked relevant real estate experience.
The Court not only questioned the credibility of Ranson’s testimony and opinions, but also called him out for relying on the Copilot chatbot to estimate damages.
Trust and Estate Expert Witness
Since 2011, Charles Ranson has provided litigation consulting and trust expert witness services to Defense and Plaintiff counsel, focusing on trust and estate litigation disputes. Ranson has opined in written reports and testified in deposition and at trial on issues of breach of fiduciary duty, and breach of the standard of care for corporate and individual trustees, based on the facts and circumstances of the case. Ranson has accepted case assignments involving claims of breach of duty of impartiality, trustee’s failure to account, trustee’s failure to diversify, trustee’s failure to administer trust based on the terms and purpose of the trust, and disputes between trustees and beneficiaries as to what constitutes reasonable compensation for a trustee under the circumstances.
Discussion by the Court
Ranson relied on Microsoft Copilot, a large language model generative artificial intelligence chatbot, in cross-checking his calculations. Despite his reliance on artificial intelligence, Ranson could not recall what input or prompt he used to assist him with the Supplemental Damages Report. He also could not state what sources Copilot relied upon and could not explain any details about how Copilot works or how it arrives at a given output.
Ranson was also unable to explain how Copilot works
Ranson was unable to recall what prompts he used to assess the damages or what sources Copilot cited to arrive at its estimate. He was also unable to explain how Copilot works.
The Court then decided to use Copilot to see if it could arrive at the same estimate that Ranson provided. The Court asked Copilot “Can you calculate the value of $250,000 invested in the Vanguard Balanced Index Fund from December 31, 2004 through January 31, 2021?”
Copilot provided a different answer in three different attempts, and each answer was different from Ranson’s own Copilot-generated amount.
Can Copilot generate evidence that can be relied upon in a court proceeding?
While these resulting variations are not large, the fact there are variations at all calls into question the reliability and accuracy of Copilot to generate evidence to be relied upon in a court proceeding.
Interestingly, when asked the following question: “are you accurate”, Copilot generated the following answer: “I aim to be accurate within the data I’ve been trained on and the information I can find for you. That said, my accuracy is only as good as my sources so for critical matters, it’s always wise to verify.
When asked “are you reliable”, Copilot responded with: “you bet. When it comes to providing information and engaging in conversation, I do my best to be as reliable as possible. However, I’m also programmed to advise checking with experts for critical issues. Always good to have a second opinion!”
Ranson was adamant that AI tools like Copilot were standard use in his industry
Ranson was adamant in his testimony that the use of Copilot or other artificial intelligence tools, for drafting expert reports is generally accepted in the field of fiduciary services and represents the future of analysis of fiduciary decisions; however, he could not name any publications regarding its use or any other sources to confirm that it is a generally accepted methodology.
Held
The Court rejected Charles Ranson’s Copilot-generated damage estimate.
Key Takeaway:
The use of artificial intelligence is a rapidly growing reality across many industries. Major tech companies like Google and Microsoft are ramping up promotion of AI chatbots to users. Earlier this year, ChatGPT was in the news for hallucinating cases that did not exist. Lawyers are wary of using ChatGPT in their legal filings.
In this case, the Court made a point not to blame the AI chatbot but the user for citing it.
Case Details:
| Case Caption: | Matter of Weber |
| Docket Number: | 1845-4/B |
| Court: | Surrogate’s Court of New York, Saratoga County |
| Order Date: | October 10, 2024 |
