IT concept and glossary
This page provides background information to the Open Power System Data platform IT framework. We also provide a user-oriented step-by-step guide if you want to use our platform.
Our “IT philosophy”
We have seen many IT projects come and go because of lacking user support. We want an IT enviroment that is simple, easy to maintain, decentralized, and scalable. This is why we are using a lightweight, decentralized structure rather than a central database server. Rather than re-inventing the wheel with custom-developed or proprietary solutions, we build on existing open-source software solutions (Jupyter Notebooks and Python), collaborative software development tools (Git and GitHub), and established industry standards for open data (Data Packages).
Data Packages
We follow a standard defined by Open Knowledge and compile all information in Data Packages, that is: one or more CSV files for data, a JSON file for metadata, and a README file. Data Packages allow easy integration into existing IT frameworks: an Excel macro, a Python module, and many more interfaces are readily available.
Data: CSV files (and more)
Data is provided in CSV files. This allows for easy and fast download, as no database is required. CSV files are human-readable and can be used by almost any software. Note that we follow the international standard of actually using commas as separators (not semicolons, as common in Germany). In addition, Excel (XLSX) and SQLite files are provided.
Scripts and documentation: Jupyter Notebooks
The scripts that we use to automatically download, process, and aggregate data are written in Python. We use Jupyter (IPython) Notebooks as a single file for coding and documentation. Jupyter Notebook is an application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text. The Notebooks created in this project are available under the open-source MIT license on GitHub.
Metadata: JSON files
All metadata is published in the JSON format and follows the Data Package, Tabular Data Package and JSON Table Schema specifications by Open Knowledge International.
Version control
Old versions of data and scripts are available. All previous versions of Data Packages are available at a stable URL on our data platform. The Jupyter Notebook scripts are versioned through GitHub.
Questions? Suggestions?
If you have questions or suggestions, please contact Ingmar Schlecht.