The technological revolution of the Ai that we are currently experiencing brings new cases of use every day. Players are being observed to gain ever more competitive advantages by leveraging their data across the Ai. As a result, more and more companies are embarking on projects in the field of wheelchair science. This usually begins with the recruitment or internal promotion of a data scientist who must evaluate the data of the company. However, it can be seen that 50% of data science projects fail even before they bring value to the company.
The success of an Ai project does not depend simply on data being brought into contact with a data scientist. It requires the establishment of a methodology and an organization.
A real business problem
Data science is growing in maturity. More and more formations are being created around the Ai. They teach cases of typical Ai usage and the techniques to be applied in such situations. This standardisation creates the risk of proposing projects not adapted to the specific characteristics of the company. For example, a project school case would be to create a churn detection model (customer that leaves to compete). A priori, such a model would be of interest to the company. What respectable company doesn’t want to keep its current customers? However, churn may not be one of the company’s major problems at the time. There is then a good chance that this model will not be used or even deployed.
As the field of data science matures, it has developed its own standard methods. Thus, a certain situation will generate a certain type of model. However, these standard techniques are not always aligned with business needs. Thus, starting from a customer relationship history, a data scientist will be able to propose a model of churn reduction (of customers going to the competition). However, churn may not be a limiting point in the company at all. There is therefore a gap between the need for the company and standard data science projects.
Before embarking on a data science project, it is essential to ask the right questions. What are the limitations or blocking points of my activity? What do I really need to solve my limitations? For this first point, we finally fall back to the obvious aspects of project management. Before creating a product, you need to understand the customer need.
A metric aligned with needs
By definition, an artificial intelligence model seeks, through data, to minimize the gap between its prediction and the true value of the variable it seeks to predict. There are several ways of estimating this difference and this leads to different models. There are also metrics that do not count towards the creation of the model, but estimate the overall performance of the model. In addition, there are standards for using metrics according to the type of problem to be solved. These are the metrics used by data scientists to justify model performance.
Unfortunately, these standard metrics are not always understandable and relevant to the profession. An example is a fraud detection system developed in collaboration with customers. In this case, the model seeks to separate companies that have actually frauded from those that have not. Internally, we use a metric (the Roc) that measures the level of separation between fraudulent and safe businesses. This metric, however, makes no sense to the client. Instead, we have proposed a metric that estimates, based on the level of risk accepted, the number of prospects that will be marked as risky and the rate of actually fraudulent businesses in this set. With this, the client can estimate the level of risk they are willing to accept, based on their internal constraints (case processing capabilities, loss of revenue).
In identifying the client need, it is essential to define a business metric, which will allow the end user to estimate the quality of the model and its impact on its activities.
A performance validated by experts
Once we have identified a real problem to be solved, we will have to ensure that the way in which it is solved is consistent with the knowledge of the experts. A trusted link must then be created between the solution and the user. Thus, the user and / or domain experts must be involved in the development and validation of the system, and we will ensure that the system reproduces the standard heuristics of the domain. It also tests the new heuristics the model has found. This last point is particularly detailed in another of our articles on the explanability of Ai systems.
When the Ellisphere failure score was created and put into production in 2018, there were numerous studies to confirm the relevance of the score. Predictions were submitted to financial analysts to compare their business analyses with the model and, at the same time, to compare the arguments. This allowed us to check traditional heuristics such as: companies that do not make profits are risky, or that companies are particularly at risk between the second and fifth years of life and then less risky afterwards.
A suitable technical solution
Most data scientist or blog training courses focus on algorithms, but unfortunately only rarely mention the deployment phase. How can a statistical model move from the personal computer of the data scientist to an environment where end users can interact? Sources that talk about deployment mainly propose to deploy a web API: a web interface, which queries like a website, which returns the result of the model if the input data is transmitted to it.
The solution of a web API makes a hypothesis about the company: that the company has an appropriate IT infrastructure to transmit data and process the response. A small SME does not necessarily have such an infrastructure. Many processes can be done on Excel and the concepts of database and information system can be very far from the everyday reality of the company. For an Ai project to be successful, consideration must be given from the outset to how the end-user will interact with the solution, so that the end-user can actually benefit.
What to conclude?
The valuation of a company’s data is a major current issue. However, many companies have not yet fully integrated the use of data into their processes, for example through Business Intelligence. Setting up Ai projects is then a complicated subject and many projects fail to bring value. A few simple rules, however, limit the risk of failure. They mainly refocus the project on need and context, and push the data science part back into implementation details. Finally, even if the Ai today allows unexpected applications yesterday, it must be kept in mind that it is a tool to serve a concrete problem.
