Check for predictive models; how to make time for the fun stuff as a data scientist

15 May 2019

Article written by Jurriaan Nagelkerke, Principal Consultant

How to make time for the fun stuff as a data scientist

As a data scientist we excel in developing new models. It is where we get much of our satisfaction from work. Creating value for the company by using data with the right techniques for preparation, analyzing steps and machine learning is what we do. And we like to devote as much time as possible to creating this value. The downside of our trade is that all production models and data products which we have created, also need to be maintained. So the more you create, the more you have to maintain. Which means you get to spend less time on the ‘fun’ stuff. Which happens to be the field where a data scientist can have the most value for the organization…

Check for your predictive models

Usually, prediction models can be used for a reasonable amount of time. Months, and sometimes even years. That is, as long as the model in question is in line with the business goals and everything goes as planned in the scoring. But how do you know if such is still the case? For this reason, we have created a check for production models. By performing a number of checks you can easily find out if everything still goes as planned.
A good start: the online check

So how does this check work? Our check for predictive models starts with an online check. First, we ask 10 questions to determine if any changes within or outside of your organization might pose a threat to the usability of your predictive model. After 5 minutes, you will know where you stand in this regard.
Risks in four fields

We take a look at risks in four different fields: Organization, Data, World and Time. The scores for each domain add up to one final result. If the results from this initial Online Check indicate an increased risk, it will be followed by a thorough scan to do extra research on this particular risk. This can be done by a list of additional questions that are part of the (self) scan. We will send you these questions by e-mail.

Expert session and follow-up

The results from this scan (questionnaire) and any follow-up are to be discussed during an expert session. This gives you a clear view on existing and possible risks with regard to using your predictive model. You will know where to take action and how you can continue using the prediction model with minimal resources and maximum effect. We will also discuss tips and experiences from other organizations to show how a prediction model can be used without risk for a longer period of time.

Quality assured by online check for predictive models

Our check for predictive models helps companies to keep an eye on the quality for those business practices that use predictive modelling. These are part of the ongoing professionalization in the field of data science and enable companies to operate based on data. Predictive models that are being developed are increasingly becoming a crucial part for business processes. The ‘upkeep’ of these models is becoming more and more of a challenge for organizations.

Investment in quality and continuity

At first, the check brings more work for the data scientist. Only the data scientist is qualified for filling out the self scan(s). Furthermore, his participation is essential for the expert session where the results will be presented. But in the end the check will help the data scientist. By checking the predictive model, both the quality of the results and the continuity of the business processes is secured. This gives the data scientist more room for developing new analysis and other data products, also because managing previous successes will take up less time than before.

Expert session with bespoke advise and tips

An expert session provides insight into possible risks of existing predictive models. You will receive hands-on tips on correct documentation and effective monitoring. Another part of the expert session are issues based on experiences in various fields. This enables the data scientist to manage the different models with a minimum amount of effort. He can even decide to outsource this management knowing that ‘all is well’ and that he will automatically receive notice when this is no longer the case.

Useful to you? Test it yourself

Do you think the Online Check can help you to ensure the quality and management of one of your predictive models? Then do the online check yourself!

Do the online check (only available in Dutch) >