Background to the project
The changing energy landscape requires rigorous analysis to support robust investment and policy decisions. Power systems are complex, hence researchers and analysts often rely on large numerical computer models for a variety of purposes, ranging from price projections to policy advice and system planning. Such models include unit commitment, dispatch, and generation expansion models.
These models require a large amount of input data, such as information about existing power stations, interconnector capacity, yearly electricity consumption, and ancillary service requirements, but also (hourly) time series of load, wind and solar power generation, and heat demand. Fortunately, most of these data are publicly available, from sources such as transmission system operators, regulators, or industry associations.
However, data collection is tedious. The bits and pieces of data are sometimes hard to find, often poorly documented, and almost always tedious to process: files are provided in different formats; downloading requires repetitive manual clicking; data structures between different sources are incompatible; daylight savings time and leap years are treated differently; URLs change frequently; and older data are updated without informing users (and sometimes deleted altogether).
Double work is inefficient. Currently dozens, if not hundreds, of modelling teams in Europe spend significant resources gathering and processing data, all doing essentially the same thing. Highly skilled people waste a lot of time gathering data, time that would be better spent doing actual research. We, the project team members, are power system modellers and have gone through this process ourselves, a sometimes quite frustrating experience. Double-work is a waste of resources. Providing aggregated and aligned data at a central place is a public good that had been short in supply.
Moreover, the licenses and conditions under which data can be used are often unclear. Regularly, data owners exclude commercial use of their data, putting energy companies and consulting firms in a situation of legal uncertainty.
Double work, poor documentation and legal uncertainty is in an unsatisfactory state of affairs. This is why we set up the project Open Power System Data.