Understanding the Requirements of Data Spaces in the Energy Sector

Abstract

Data Management (DM) is crucial for maintaining the transparency, integrity, and reproducibility of research findings by systematically organizing, storing, preserving, and sharing data throughout the lifecycle of research projects in various domains. This is particularly critical in data-intensive sectors like the energy sector. This sector faces unique challenges due to the complex nature of its data, ranging from sensor readings to policy assessments. DM is important not only for effective data han- dling, maintenance, and accessibility, but it also significantly enhances the reliability and trustworthiness of scientific research. By ensuring data is findable, accessible, interoperable, and reusable (FAIR), DM supports the credibility of outcomes and enhances data sharing practices, facilitating innovation and applied research in this rapidly evolving field. In this thesis, we explored DM within the energy sector by identifying its require- ments, assessing current practices, and understanding the perspectives of profession- als in the field. Our research methodology began with a systematic literature review to collect foundational knowledge on the field’s challenges and requirements. This was followed by a survey that focused mainly on the top 10 most mentioned DM requirements to understand the current state of DM in the energy sector. We dis- covered a strong emphasis on data quality for analytical purposes and the need for systems that are scalable and capable of integrating diverse data sources. Interest- ingly, while real-time data processing was not seen as a high priority by the majority of survey respondents, those with in-depth DM expertise highlighted its importance, indicating different perceptions based on DM knowledge. Additionally, our survey showed a preference for simulation tools over graphical visualization and highlighted a significant gap in familiarity with the FAIR principles among professionals, which pointed to limited external data sharing practices. To address one of these identified needs, we introduced the ckanext-gitimport extension as a proof of concept. This ex- tension is designed to simplify the collection of metadata from external repositories. In summary, our work contributes to the understanding of DM in the energy sector by highlighting its current state, challenges, and areas for improvement. Through a combination of literature review, survey analysis, and the development of the exten- sion, we lay the groundwork for future advancements in DM practices, essential for enabling data sharing in the energy sector.