The challenges of building a data platform

On April 2021 Pieter de Jong, communication lead of the SCOREwater project conducted a short interview with Bas Vanmeulebrouk, lead developer of the SCOREwater platform at Civity. Civity being the project partner with extensive knowledge about date management frameworks and urban platforms. 

The goal of this interview is to highlight some of the challenges surrounding the developments of data platform. To give you as a reader a glimpse behind the scenes of the development of the SCOREwater platform. Without further ado we will jump right into the conversation. 

Pieter | There is a broad diversity in the needs of the project partners. How to ensure a data platform that serves all? 

Bas | We need to make sure that the platform is flexible enough to serve the needs of all partners, a platform that can cope with heterogeneous data – meaning data with high variability of data types and formats. At the start of the project not every city had the same amount clarity on which data was needed to reach their goals. For example, for the Barcelona case it was already clear at the start of the project which sensors were needed, what information could be gathered with it and which possible analysis they could do. However, for the Amersfoort case the selection of what to measure and what to do with the measurements was part of the project. During the first phase of the project efforts have been made to get to the same amount of detail for all the three cities represented within the project, so it would fit in the SCOREwater platform. 

P | Was building the flexibility into the platform difficult? 

B | The challenge is not so much a technical one when designing a flexible platform, but more how to present all the data. Some data on the platform will be open data, other data needs to be paid for. Some partners of the SCOREwater project have commercial objectives, others don’t. The challenge is to build a platform that can accommodate both approaches.  

P | Could you explain in more detail how this would work? 

B | The supplier of the data should be able to choose what licence they want to attach to their data. You cannot just say “all the data on the SCOREwater platform is available under open-source licence xyz”, it is up to the supplier of the data.  

P | So no one size fits all approach… 

B| Exactly, there will be different revenue models for the suppliers of data. For example, MeetJeStad (community science collective) will share all the data without limits, so you won’t need an API key limiting the amount of (free) requests someone can make. While in the case of closed data you need to secure the data with an API key.  

P | I have learned so far that the platform needs to be able to accommodate for open and closed data and needs to be able to support payments. What would be another challenge that the platform should accommodate for? 

B| There is a big heterogeneity in data. …

>> continue reading <<

This is a preview of the full article published in the SCOREwater booklet dealing with the second year of the project. Among other things you will find: an update on the Amersfoort, Barcelona and Göteborg case study, links to interesting webinars and (scientific) publications, and many more! You can request the full publication here: https://bit.ly/Year2Update-internal