A University of Waterloo researcher leading an open-source project for courts says early real-world tests produced insufficient consensus among algorithms.
The Story
Maura Grossman, co-director of a Canadian research team working on an open-source tool to help courts distinguish real evidence from AI-generated or altered material, says it has been difficult to achieve laboratory-level performance in real-world situations.
Grossman said the project, which is nearing the halfway point of a two-year effort, combines state-of-the-art algorithms that work together as a committee to vote on whether a piece of evidence was generated by AI.
She said that in two real-life trials, the system’s algorithms were not able to reach enough agreement to be reliably helpful.
Grossman described the problem as “a wicked technical problem” and said the team is continuing to build the tool over the next year.
She added that the next phase will involve input from law enforcement and policymakers, and that the team is moving toward a more modest view of what one tool can accomplish.
“We’re still hopeful, we’re still continuing to build, but I think we’ve gotten more modest in terms of thinking of one tool being the answer,” Grossman said. “As opposed to this is a problem that’s going to need some help from the law, from regulation, from policymakers, from educating people to become better at knowing how to decipher what’s real and what’s not.”
Grossman said some of the difficulty comes from how evidence can change after it is created—for example, images shared on social media are typically compressed and may lose metadata that could otherwise help assess authenticity.
“That’s part of it, and same thing if I email it to you and you email it to six other people, the data just changes through all this transfer,” she said.
Grossman said the project is not intended to replace human judgment in court, emphasizing that any tool would still require people to explain why they believe evidence is authentic or not.
“You’re going to need somebody to be able to testify why they think what they think and all of that,” she said.
The research initiative is one of two funded Canadian AI Safety Institute initiatives through the Canadian Institute for Advanced Research’s Canadian AI Safety Institute program, launched by the federal government in 2024 as part of its AI safety strategy.
Grossman said the team’s advisory board includes judges and organizations supporting self-represented litigants, and that the broader group includes technologists and legal scholars based in universities in Ontario and British Columbia.
← More stories