Scrum en Data & Analytics; een match made in heaven?

4 October 2017

Article written by Jeanine Schoonemann, Principal Consultant

Scrum is trendy – if you want to keep up with other companies, you can’t be without it… Or can you? In our assignments for clients, we have now built up a great deal of experience in “agile” working and we are regularly involved in heated debates about the added value of scrum for analytics. In this article we want to talk you through our experiences with scrum and our vision for how to use it effectively in the field of Data & Analytics.


What is scrum?

If you aren’t already familiar with scrum, you can see the basic principles of it in the image below and quickly remind yourself of the definitions used in this article. If you would like more information on the methodology behind scrum, you can read about it in this article by our colleague Anja Meerding.


Experiences with scrum

Given that more and more companies have started using scrum, several Cmotions consultants have acquired experience in order to start using the scrum method. How effective actually is using scrum? And how did the consultants find scrum in the field of Data & Analytics? This article is the result of a session with eight consultants who each worked in one or more companies that use scrum.

One of the major advantages of scrum is that employees from various disciplines have to collaborate in a single scrum team, which results in short lines of communication. This gives analysts access to essential knowledge about what “the business” genuinely needs and how data, which they analyse on a daily basis, is generated in the processes. Collaborating with colleagues from other disciplines is therefore highly informative and often results in increased mutual understanding. Working on the basis of scrum principles calls for intensive collaboration which often creates a good atmosphere and makes a positive contribution to team spirit. The retrospective – a moment of evaluation after each sprint – also plays a part in this. A good scrum master will ensure there is an open and honest atmosphere so that every team member feels able to give their opinion. If something still goes wrong, in the team or elsewhere, escalation is straightforward. In the daily stand-up you can flag any things that are perhaps not going well or things that are reliant on other teams, and so on. The scrum master, in conjunction with the product owner, must make sure the problems are resolved as quickly as possible.

Another advantage of scrum is the visibility of the work both within the team and beyond. The daily stand-up means that all the members of the scrum team are up-to-speed on each other’s work, and therefore each team member has a good idea of what’s going on in the team. In addition to that, the demo at the end of each sprint also reinforces visibility within the organisation. The demo forces analysts to turn their results into insights for the audience. This is also certainly an advantage for the Data & Analytics department. The field of Data & Analytics is often seen as a black box and using the scrum makes it more transparent for the business to see what is going on in the field of Data & Analytics.

Beyond good collaboration and visibility, scrum also enforces a way of working whereby priorities are clearly set and coordinated. The product owner knows what is of importance to all stakeholders and knows their motives and priorities. On this basis, the product owner can make and prioritise the product backlog for the scrum team. This brings the most important work tasks up to the top of this backlog and puts them first in the sprint planning to be taken forward to the next sprint. Consequently, the priorities are clear for the entire scrum team. The focus is therefore clearly on whatever is most important at that time.

In principle, using scrum should create more flexibility, and on many levels it certainly does. This is because priorities can change depending on the sprint planning and therefore other work tasks can be planned. This can also be a drawback for the field of Data & Analytics. Some analysis tasks (such as building a prediction model or exploratory analyses) are difficult to encapsulate in a single user story. In practice, we often see this type of user stories split up into multiple different user stories to solve this. As a result, each separate user story no longer delivers a specific product. If the scrum team’s priorities change for a given sprint, this can result in there being a long period of time between the split user stories, which is something that can have a negative impact on the quality of the analysis.

As in any other field, in Data & Analytics it is often incredibly tricky to estimate in advance how much work and how complex a “new” analysis will be. On the face of it, that’s not a problem… Or so you might think. In practice, however, we see this more innovative work receiving a lot of points and this, when combined with a lower priority, often means that these user stories keep on being pushed forward and there is not much scope left for the data analyst to be creative. In our experience, a data analyst has a lot to gain from having scope to be creative and not purely having a reactive role in the organisation. Scope to be creative broadens your outlook, helps you to discover new techniques, which can often lead to major innovations in the work tasks you are doing. In a world where virtually everything becomes out-of-date as soon as it gains any recognition, it is simply indispensable to have scope to be creative and innovative!

Reliance on other teams and team members
Although the collaboration within a scrum team is one of the biggest benefits of this methodology, in practice we unfortunately still often encounter two problems:

  1. The scrum members are not full-time in the scrum team;
  2. Not all departments you need to work with use scrum.

We often find that this makes collaboration within the team much harder: it can even mean that two team members rarely or never see each other because they are with the team on different days. This also, obviously, makes the bonds within the team much weaker as not everyone can work on the user stories with the same focus. As well as being annoying for the scrum team, this also often puts great pressure on the relevant member of staff as there are multiple departments wanting something from them and it is often hard for them to make a clear separation.

When it comes to the collaboration between different teams, the fact that not everybody uses scrum mainly causes problems in terms of the timelines in which the work tasks are carried out. This can lead to unnecessary delays and therefore irritation.

Scrum and the new way of working
Some companies that use scrum ask all team members to always be present. This means they have a daily stand-up and team members are able to collaborate properly. We find that scrum can also be applied in combination with the new way of working. But isn’t it terribly old-fashioned to be forcing staff to be physically present at their place of work every day? These days, there are all sorts of modern solutions to make the scrum digital. There is software like Jira and Projectplace which can act as a scrum board, you can have discussions on Skype and use a smart board to make it all the more simple and accessible. Of course, it is still important to make clear arrangements about this so that team members can also see each other in the flesh and work together. This is usually done by arranging fixed days of the week when the team members should be physically present.


How scrum can represent added value in Data & Analytics

Many organisations consider scrum to be indispensable for a whole range of different processes. But is that also true for the field of Data & Analytics? As we see it at Cmotions, scrum can also represent added value in this field, but that using it also requires a few minor adjustments:

  • Create (preferably for each sprint) scope to be creative and innovative and make sure it is retained in each stand-up and sprint meeting;
  • If analyses absolutely have to be broken up, make sure everything is properly prioritised so the work tasks can be handled within a reasonable length of time of each other;
  • Don’t underestimate the importance of full-time input in a scrum team and a company that “fully” does scrum;
  • Combine scrum with the new way of working by also using scrum digitally.

Conclusion: Should you apply scrum in Data & Analytics? Go for it! But keep an eye on the points mentioned above.

Are you thinking about starting to use scrum too? Would you like to know more about scrum? Would you like to let us know how well scrum is working in your organisation?

If so, please get in touch!


Do you want to know more about this subject? Please contact Jeanine Schoonemann using the details below

Jeanine Schoonemann, Principal Consultant

+31 6 55 89 75 12

Latest news

How do I visualise neighborhood data on a map in R? Polygon plotting explained

7 June 2019

As a data analyst you want to provide clear cut insights for your end users, enabling... read more

Python & R vs. SPSS & SAS – comparison renewed after two years

18 February 2019

Two years ago we published an article in which we compared the four most used programmes... read more

Modelplot[R/py]: graphs to expose the business value of predictive models

8 October 2018

It will be a familiar challenge to many data scientists and data analysts: you have built... read more

Subscribe to our newsletter

Never miss anything in the field of advanced analytics, data science and its application within organizations!