Ottawa startup’s AI platform outperformed other models on two memory benchmarks.
Ottawa-based AI startup Backboard.io announced today that its tech outperformed other leading AI companies on two “memory management” benchmarks in independent evaluations.
The company is seeking to solve what it calls “AI amnesia,” which is the bottleneck of limited portable memory in generative AI tools. Founded by Assent co-founder Rob Imbeault last April, the startup claims it lets users pack up their interaction and plug the “memory” of their chats into thousands of different large language models (LLMs)—including free, open-source models—so they’re not locked into a single vendor. If you were using ChatGPT to plan a workout routine, for example, Backboard’s API would let you switch over to Perplexity and still remember your last exercises.
“The future of AI isn’t about clever tricks or bolt-ons, it’s about building systems that can be trusted over time.”
Rob Imbeault, Backboard.io
Backboard announced Wednesday that it has achieved accuracy of 90.1 percent on the LoCoMo benchmark test, which measures the accuracy at which a model will recall information across a long-term chat conversation. Separately, it claimed to achieve 93.4 percent accuracy on the LongMemEval benchmark, a more complex test that looks at memory abilities like information extraction, multi-session reasoning, and reasoning across time. These are all key functions of using AI tools in business, the company says.
“The future of AI isn’t about clever tricks or bolt-ons, it’s about building systems that can be trusted over time,” Imbeault said in a statement.
In an interview with BetaKit on Wednesday, co-founder Jonathan Murray claimed that New Math Data, a Texas-based engineering firm, conducted the LongMemEval benchmark test to evaluate whether it wanted to use Backboard’s tech, then reached out when they found that it had surpassed the highest publicly reported result. BetaKit has reached out to New Math Data to confirm this.
Being able to accurately pull up context from past projects without worrying that the LLM is hallucinating is a key business problem, Murray argued.
“Think about that over the course of years,” Murray said. “If you have a long-context horizon that’s scoring in the 60s or 70s [for accuracy], that’s not good.”
Backboard’s tech also operates “off-model,” meaning that its storage of data and memory isn’t tied to one single LLM. A Backboard user can ask questions of different chatbots but have the memory stored across each, like one conversation.
RELATED: Backboard.io looks to solve AI amnesia with “substantial” pre-seed round
Chatbots’ ability to access long-term memories are limited by their computation power and how much context they can process at once. They’re also prone to hallucinations (when AI systems make things up), which has been a thorn in the side of enterprises trying to widely adopt AI. A recent PwC Canada report found that 72 percent of Canadian organizations name responsible AI a top priority, yet 36 percent have no dedicated governance function to manage risks.
The benchmark claim comes a month after Backboard announced it had raised an undisclosed but “substantial” pre-seed funding round led by Mistral Venture Partners, with participation from N49P, DevCap, Garage Capital, and two undisclosed angel investors.
Backboard’s team of six people will be 10 by the end of the month. It’s also hiring around eight interns to build and help out with core product features, as it hopes to inspire the next generation of startup founders.
“We’re really in a great position for the next wave of startups to be built on Backboard,” Murray said.
The team also hopes to release an open-source marketplace of products built on the platform later this year.
Feature image courtesy Unsplash. Photo by Mohammad Rahmani.
