Open Geospatial Consortium

Submission Date: 2024-09-23

Approval Date: 2024-10-07

Internal reference number of this OGC® document: 24-002r2

Category: OGC® Community Standard Work Item Justification

Authors:   openEO Project Steering Committee

openEO Community Standard

Copyright notice

Copyright © 2024 Open Geospatial Consortium

To obtain additional rights of use, visit http://www.opengeospatial.org/legal/

1. Introduction

This document provides a justification to the OGC Technical Committee (TC) for consideration of openEO as a Community Standard. This justification, along with the submitted candidate Community Standard, will form the basis for TC review and vote to approve the start of a Work Item as the first step in the Community Standard process for this Standard.

The submitters agree to abide by the TC Policies and Procedures and OGC Intellectual Property Rights Policy (http://www.opengeospatial.org/ogc/policies) during the processing of this submission.

Once approved, the Community Standard Work Item defined by this document is valid for six (6) months.

2. Overview of proposed submission

openEO specifies an open application programming interface (API) for connecting applications and other client software to big Earth observation cloud back-ends in a simple and unified way.

The openEO specification aims at increasing the interoperability of big EO data processing of satellite imagery in the cloud. Implementations of openEO can be used to add an interoperability layer on top of existing services. Its development has been driven by the need to overcome the challenges associated with different tools, APIs, and data formats in geospatial technology. openEO has been developed from the bottom up, with each version of the specification supported by implementations.

The primary use case for specifying openEO was to simplify and unify the data processing using a common API and a specification for a set of pre-defined processes. As such, users can still work in their favored programming language without worrying about data organization and pre-processing. Users can avoid vendor lock-in as the generated process descriptions can be executed at multiple provider endpoints, making it easier to compare and reproduce processing results between different providers.

The specification consists of two parts:

  • The openEO API (version 1.2.0), an HTTP API description specified as an OpenAPI document; and

  • The openEO Processes (version 1.2.0), a set of requirements for separately versioned pre-defined processes conforming to the openEO API specification.

The openEO specification is openly licensed, governed and maintained. The specification is released under the Apache 2.0 license and is publicly available via GitHub. The project is governed by a Project Steering Committee: https://openeo.org/psc/

3. Motivation of supporting OGC Members for this submission

3.1. European Space Agency (ESA)

openEO provides valuable standardization and abstraction of EO processing and analysis chains across different federated compute environments. The latter point is especially important in the fragmented European cloud landscape. The openEO specification has been adopted by various organizations, partners or stakeholders of ESA (e.g. EC JRC, EURAC, INPE, various academic labs) and companies (e.g. VITO, Sinergise, EODC, EOX, eGEOS, mundialis, Google Earth Engine). ESA has also invested in the development of the operational openEO platform service (https://openeo.cloud/) based on the openEO specification. Thanks to this, the Copernicus Data Space Ecosystem (CDSE), successor to the Copernicus DIAS initiative, jointly implemented and managed by ESA and the European Commission, has selected the openEO specification as a core API to enable the community to process EO data and collaborate on workflows and analytics. ESA is also investing on the OGC Geodatacube API standardization process of which openEO is considered as one of the key standards. Finally at ESA various efforts to operationalize EO science and application results are building on openEO due to its FAIR compliant properties, which is why the evolution of the EO Exploitation Platform Common Architecture will also support openEO. Making openEO an OGC Community standard is a key step to foster more widespread adoption in industry and public domain and allow to extend its current reach beyond Europe.

ESA’s EO Exploitation Platform Common Architecture (EOEPCA) Project has previously focused on the OGC API Processes to provide a standards-based interoperable approach to user-defined processing via OGC EO Application Packages. The successor EOEPCA+ Project embraces openEO as a complementary approach that has gained traction in the community. EOEPCA+ intends to develop an approach in which OGC API Processes, OGC EO Application Packages and openEO can be used in combination in user-defined workflows. This is relevant to both the OGC GeoDatacube API and the OGC API Processes Part 3 (Workflows) standardization activities. EOEPCA+ is committed to adoption of standards-based approaches, and hence encourages establishing openEO as an OGC Community standard.

3.2. Natural Resources Canada (NRCan)

NRCan in collaboration with other Canadian federal government departments is currently exploring establishment of an EO exploitation platform to serve Canadian needs. It is expected that EOEPCA and the upcoming EOEPCA+ will be core elements of the architecture for this envisioned platform. Given the importance of openEO in the context of EOEPCA, and its ability to work together with other OGC standards-based approaches, it will be a necessary part of Canada’s future EO exploitation platform. NRCan is also assisting with development of the OGC Geodatacube API, which, as noted by ESA, openEO is a core element of. It is anticipated that the OGC Geodatacube API will be key for enabling interoperability between various geodatacubes in Canada and internationally, making openEO an important part of realizing this vision. Given the importance of openEO within key international activities that aim to greatly increase geospatial interoperability, NRCan feels that it is important for openEO to be recognized as an OGC Community Standard.

3.3. University of Münster (UoM)

The Institute for Geoinformatics has a 30 year history of research in interoperability for spatial data. It took the initiative for the initial 2017 H2020 project that started openEO. The reason for this was that at that time, cloud platforms for processing Earth observation data were not interoperable, which led to platform lock-in and made it hard to compare results for the same process obtained from two different platforms. Many such platforms were based on closed source software. Difficulty to compare cloud platforms conflicts with principles of open science and hinders reproducibility and research validation. The openEO API resolved this problem by providing a single, uniform API to a variety of back-end cloud platforms. In addition, it made it easier to develop clients in different data science languages or environments, which in turn made it easier for end-users to switch languages, or to compare developments carried out in different data science languages. The currently existing and evolving ecosystem of clients and back-ends using the openEO specification encourages the development of new analysis methods, and has led to an increase in communications between and among researchers and/or research software engineers that previously had very little interaction.

4. Relationship to other OGC Standards

The openEO API builds on top of other standards and specifications and reuses other standards whenever possible. openEO’s basis is HTTP with JSON as request and response encoding. The API specification reuses building blocks from the OGC API – Common Standard. openEO adheres to the STAC API specification, which is based on the OGC API – Features Standard. openEO allows the exposure of a wide variety of visualization and download services using existing standards/specifications including OGC WMS, OGC WMTS, OGC WFS, OGC WCS, OGC API – Tiles, OGC API – Maps, OGC API – Features, and XYZ. For Authentication the openEO API uses the commonly used standards HTTP Basic (RFC 7617) and OpenID Connect.

5. Alignment with OGC Standards Baseline

The openEO API aligns with the OGC APIs Standard baseline (especially OGC API – Common) and STAC. The openEO API standard fits well within the OGC API roadmap.

OGC API – Processes and the openEO API are schematically similar with regards to processes and batch jobs and complement each other for these parts. The specification caters for different user audiences: openEO API comes out of the box with a wider scope of features, while OGC API - Processes only covers processing and additional features require the integration of other OGC API specifications. openEO allows for a more user-centric and less technical approach to big EO cloud processing, backed by an extensive set of pre-defined processes and its ecosystem. OGC API - Processes is a highly flexible API for executing any kind of processes on any kind of data with the caveats that get implied by this flexibility. On the other hand, openEO makes a different trade-off prioritizing interoperability between different implementations over flexibility by prescribing data models and process schemas. Users compose their algorithms in an abstract way as graphs using the openEO processes and leave the concrete realization of the process graph to the server implementation. OGC API – Processes is more focused on executing concrete implementations of an algorithm, e.g. provided as application packages, without implying a specific data model. Nevertheless, the openEO Processes could be used as a streamlined set of processes for OGC API – Processes. OGC API – Processes is currently missing pre-defined processes for increased interoperability between implementations. Additionally, work is ongoing to specify openEO user-defined processes as an encoding for OGC API – Processes - Part 3.

As requested by the reviewers, a detailed comparison of the openEO API and the OGC API - Processes has been compiled: https://github.com/Open-EO/openeo-api/blob/ogcapi-processes/crosswalks/ogcapi-processes.md If deemed useful, this could be the basis for further alignment in future versions of the specifications.

The openEO API is also considered as one building block for the emerging OGC GeoDataCube API Standard.

This specification enhances OGC operations and community involvement by providing a user-centric and less technical approach to big EO cloud processing, backed by an extensive set of pre-defined processes and a its surrounding ecosystem.

6. Evidence of implementation

The openEO specification is backed by a considerably large ecosystem of open source servers, clients and tools. The following is a limited choice of the available implementations of the openEO specification. There are at least 10 additional implementations of the specification, which can be found through the links below.

All the implementations listed below are developed as open source software, and can be found, along with further implementations, on https://github.com/open-EO. Some implementation are also available in other GitHub organizations, e.g. https://github.com/IBM/tensorlakehouse-openeo-driver

Although many of the implementations are maintained close to the specification, they are indepenant projects and would not be part of the community standard.

Note: The specification is a set of building blocks that server and client implementations can pick from, which are called "profiles". Profiles are similar to conformance classes in OGC standards. Such profiles are available for both the API and the processes:

Due to the extensiveness of API and processes, server implementation are often incomplete as none of the services needs the full set of functionality of the API or processes. The profiles will be used to report completeness of the implementations below.

6.1. openeo-python-client

Implementation name: openEO Python Client

Date of most recent version: 2024-07-26 (v0.31.0)

Implementation description: Python client for openEO, provides a very pythonic interface to the openEO API and processes.

Is implementation complete?

  • ✓ Yes

  • ❏ No

6.2. openeo-js-client

Implementation name: openEO JavaScript Client

Date of most recent version: 2024-07-11 (v2.6.0)

Implementation description: openEO client for JavaScript, NodeJS, and Typescript

Is implementation complete?

  • ✓ Yes

  • ❏ No

6.3. openeo-r-client

Implementation name: openEO R Client

Date of most recent version: 2024-02-25 (v1.3.1)

Implementation description: Provides an R client interface to the openEO API and processes.

Is implementation complete?

  • ✓ Yes

  • ❏ No

6.4. openeo-web-editor

Implementation name: openEO Web Editor

Date of most recent version: 2024-07-11 (v0.13.0)

Implementation description: A user-friendly web-based interface for the openEO API.

Is implementation complete?

  • ✓ Yes

  • ❏ No

6.5. openeo-geopyspark-driver

Implementation name: openEO Geotrellis backend

Date of most recent version: 2024-07-31 (0.39.0)

Implementation description: A backend implementation based on Geotrellis and Apache Spark. The web application is developed in Python, while most of the raster processing engine is based on Scala. It is 100% open source, and focuses on providing large scale processing capabilities in the cloud. It is used in production environments for Terrascope, openEO platform and the Copernicus Dataspace Ecosystem.

Is implementation complete?

  • ❏ Yes

  • ✓ No

If not, what portions of the proposed Community standard are implemented?

The implementation currently supports these API profiles (L1-4): L1, L1A, L1B, L2, L3, L3-UDF. It also includes selected funtionalities from the L4 profile.

6.6. openeo-earthengine-driver

Implementation name: openEO Google Earth Engine backend

Date of most recent version: 2024-08-27 (rolling release)

Implementation description: An openEO API compliant implementation of Google Earth Engine (GEE). It is built on top of the GEE JavaScript API and supports a subset of the functionality of GEE through the openEO interface. It offers a datacube abstraction on top of GEE that can be used with the openEO client and allows free access to the GEE offering.

Is implementation complete?

  • ❏ Yes

  • ✓ No

If not, what portions of the proposed Community standard are implemented?

The implementation currently supports these API profiles (L1-4): L1, L1A, L1B, L1C, L2, L3-FS, L3-SWS. It also includes selected funtionalities from the L3 and L4 profiles.

7. Information on adoption

The European Space Agency (ESA) is adopting openEO as one of two options for future implementations of interoperable EO processing workflows and services. This adoption is happening in the frame of the projects EOEPCA+, ESA APEx, and Earth Code.

The Earth observation community is increasingly using openEO to describe EO processing workflows, and relying on the available backends to generate results. The following is a non-exhaustive list of publicly documented cases:

While it is hard to obtain public numbers on overall uptake by users, some of the backends already report 300 active users on a monthly basis. This is already a substantial number considering that these are mainly researchers and developers working in Earth Observation processing. A further increase is expected as we see that more backend implementations reach a maturity level that make them competitive with other well-known proprietary offerings for EO processing.

The openEO client libraries are downloaded by a broad audience of users, for example:

  • Python: ~5000 in June/July 2024

  • R: ~1000 in June/July 2024

7.1. Deployments

The following is a list of publicly available deployments of the openEO specification.

Server deployments:

Client deployments:

8. Public availability

Is the proposed Community standard currently publicly available?

  • ✓ Yes

  • ❏ No

It’s made available under Apache 2.0 license at the following locations:

9. Supporting OGC Members

  • University of Münster - Institute for Geoinformatics

  • Eurac Research

  • VITO (Flemish Institute for Technological Research)

  • GeoConnections - Natural Resources Canada

  • EUMETSAT

  • European Space Agency (ESA)

  • EOX IT Services GmbH

  • Telespazio VEGA UK Ltd

  • Planet Labs PBC

  • German Aerospace Center - DLR

  • Matthias Mohr - Softwareentwicklung

10. Intellectual Property Rights

Will the contributor retain intellectual property rights?

  • ✓ Yes - The specification is open source, released under Apache 2.0 license

  • ❏ No

If yes, the contributor will be required to work with OGC staff to properly attribute the submitter’s intellectual property rights.

If no, the contributor will assign intellectual property rights to the OGC.