Picture: LVMH.
Shoemakers are always the worst shod, as the saying goes. This may also apply to data and artificial intelligence experts. In any case, this is what Fanjuan Shi suggested during the last edition of the Retail & e-commerce show, organized by the Hub Institute.
The Head of Data Strategy and Data Value Office of the luxury group LVMH is frequently involved in assessing the quality of Data & AI projects. Ironically, the teams of these projects often struggle to prove the success of their initiatives with data.
Projects rejected for lack of attention to proof of value
On the occasion of a recent review of 34 projects in which he participated, Fanjuan Shi was able to observe that “for a third of the projects, the team has difficulty answering the question: ‘how do you demonstrate, with the right KPIs, the value generated? »
This shortcoming penalizes the prioritization of Data & AI projects during their review by experts, but even more so during presentations to Comex.
“He doesn’t understand the technicalities, nor does he have to grasp all the details. What the Comex wants to know is the ROI. However, the members of the project are finding it difficult to meet expectations on this point”, underlines the specialist of the multinational.
In order to remedy this, Fanjuan Shi therefore recommends the implementation of test and measurement solutions. These have two advantages. First, they provide visibility into the measurement of value.
But, in addition, they contribute directly to trust and ultimately to project sponsorship. The first stage of “test & measurement” involves selecting “the right indicators”. And the expert warns against changes in KPIs during the project, as is common.
Accurate and stable KPIs are key
The starting point for the creation of a new data product is in principle the improvement of a specific element. Therefore, the indicator(s) to measure the target optimization should be stable.
Another component is the adoption of a “relevant and consistent” method for benchmarking. This means opting for a controlled and statistically robust methodology. Last key elements: the choice of test audiences and the duration of these tests (“neither too long nor too short”).
“Choosing consumers from the North Pole to sell fridges is probably not the right approach. Audiences must be representative and homogeneous in order to effectively assess the use case. »
To these obvious characteristics must also be added particular attention with regard to biases. Fanjuan Shi therefore calls for eliminating or minimizing the biases introduced into the methodology. Several parameters must be taken into account.
For example, this will ensure that the incremental value generated by a feature is not linked to a random effect. “That means having enough sample volumes to prevent such bias. »
How to convince a Comex? KPIs and business language
Within LVMH, Fanjuan Shi designed a decision tree based on business questions. These aim to define the most appropriate test methodology, which should be systematically reproduced for similar use cases.
“The systematization of the methodology makes it possible to make benchmarks between periods and experiences. It is also a means of reaching more reliable conclusions,” he promotes.
Using its matrix, LVMH therefore recommends the preferred methodologies and provides its teams with the recommended sample sizes to achieve statistically robust results.
“Comex have a common language: ROI and business KPIs. To gain their trust, several elements are necessary. A precise and stable KPI. A good test methodology. Finally, it is to use visual dashboards and business language to convince them of the value increment of the Data project,” concludes Fanjuan Shi.