Can we trust an AI chatbot when it comes to business? The airline Air Canada saw fit to offer a chatbot service to answer customer questions on its website. And in November 2022, one of its customers wanted to know the terms and conditions allowing him to obtain a reduction for a trip.
By questioning the chatbot, it assured him that it was possible to obtain this reduction retroactively after purchasing a place at the normal rate and subsequently making a request by email to the company, within 90 days. The customer therefore paid the sum of 1,639 Canadian dollars for a round trip between Vancouver and Toronto, hoping to subsequently be reimbursed approximately 760 Canadian dollars by applying the reduction.
Unfortunately for him, the company refused to give him this refund, explaining that the information given by the chatbot was incorrect. The case therefore continued in court, the customer having sued Air Canada in order to recover the amount he was claiming from the company.
Who is the most reliable?
As reported by Ars Technica, Canadian judges ruled in his favor last week, ordering the airline to pay him the sum of $812 in compensation.
The airline was trying to make the argument that the information communicated by its chatbot program was not as reliable as the information displayed on the company’s website. And that consequently the customer should have referred to the page indicated by the chatbot rather than taking the advice provided by it at face value.
Liar chatbot
The Canadian judge did not accept this interpretation. He explains that “Air Canada did not take sufficient care to ensure that the information provided by its chatbot was accurate.”
Likewise, the judge considers that the company does not explain how the information provided on its web page devoted to reductions was by nature more credible than that offered by its chatbot.
Air Canada will therefore have to reimburse the amount requested by the dissatisfied customer. As Ars Technica notes, the airline seems to have since stopped offering the services of its chatbot on the site.
AI yes, but at what cost?
And there is no doubt that the decision will push the designers of these tools to pay careful attention to the answers they offer to customers: many chatbots increasingly use large language model (LLM) technologies, tools that are likely to make errors in their responses.
In an article from March 2023, the CIO of Air Canada explained that he had initiated several experiments with voice chatbots based on artificial intelligence technologies in order to lighten the workload of the company’s employees and contractors in matters of customer support.
The launch of these tools was then planned for summer 2023.