Resource Catalog

Description & purpose: This Notebook is designed to showcase the initial functionality of the Earth Observation Data Hub. It provides a snapshot of the Hub, the pyeodh API client and the various datasets as of September 2024. The Notebook “user” would like to understand more about the satellite data available for their test areas. This user is also interested in obtaining a series of smaller images and ultimately creating a data cube. The Notebook series (of 3) is designed in such a way that it can be run on the EODH AppHub (Notebook Service) or from a local environment.

Author(s): Alastair Graham, Dusan Figala, Phil Kershaw

Date created: 2024-09-05

Date last modified: 2024-09-18

Licence: This notebook is licensed under Creative Commons Attribution-ShareAlike 4.0 International. The code is released using the BSD-2-Clause license.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Links: * Oxidian: https://www.oxidian.com/ * CEDA: https://www.ceda.ac.uk/ * EO Data Hub: https://eodatahub.org.uk/

# Run this cell if pyeodh is not installed, or needs updating
!pip install --upgrade pyeodh

# Import the Pythoin API Client
import pyeodh

First we need to create an instance of the Client, which is our entrypoint to EODH APIs. From there we can start to search the collections held within the platform.

# Connect to the Hub
client = pyeodh.Client().get_catalog_service()

# Print a list of the collections held in the Resource Catalogue (their id and description).
# As the Resource Catalogue fills and development continues, the number of collections and the richness of their descriptions will increase
for collect in client.get_collections():
    print(f"- {collect.id}: {collect.description}")

- cmip6: CMIP6
- cmip6: CMIP6
- cordex: CORDEX
- cordex: CORDEX
- ukcp: UKCP
- ukcp: UKCP
- defra-airbus: A collection of Airbus data for the DEFRA use case.
- defra-planet: A collection of Planet data for the DEFRA use case.
- airbus_sar_data: The German TerraSAR-X / TanDEM-X satellite formation and the Spanish PAZ satellite (managed by Hisdesat Servicios Estratégicos S.A.) are being operated in the same orbit tube and feature identical ground swaths and imaging modes - allowing Airbus and Hisdesat to establish a unique commercial Radar Constellation. The satellites carry a high frequency X-band Synthetic Aperture Radar (SAR) sensor in order to acquire datasets ranging from very high-resolution imagery to wide area coverage.
- eocis-sst-cdrv3: EOCIS Sea-Surface Temperatures V3

The attributes of a catalogue are mapped to a series of properties. For instance, in the following cell we print the id, title and description for the supported-datasets catalogue.

ceda_cat = client.get_catalog("supported-datasets/ceda-stac-fastapi")
print("id: ", ceda_cat.id)
print("title: ", ceda_cat.title)
print("description: ", ceda_cat.description)

id:  ceda-stac-fastapi
title:  Supported Datasets
description:  Catalogue containing supported datasets

The Hub API endpoints are wrapped in methods inside pyeodh and are structured into classes, following the same logic as the underlying APIs. This means that, for xample, to fetch a collection item we first need to get the collection from the resource catalogue. The following cell provedes a code example to do this.

# GET /stac-fastapi/collections/{collection_id}/items/{item_id}
cmip6 = client.get_catalog("supported-datasets/ceda-stac-fastapi").get_collection('cmip6')
cmip6

<pyeodh.resource_catalog.Collection at 0x749550dbc890>

Some API responses are paginated (e.g. collection items), and you can simply iterate over them.

# GET /stac-fastapi/collections/cmip6/items
items = cmip6.get_items()

# Warning: this will take a long time for large catalogues such as cmip6
for item in items:
    print(item.id)

Attempting to create a collection with id that already exists will result in 409 error code. To see the example in action delete the test collection first by running the cell below.

Delete a colletion

rc.get_collection("test1").delete()

Create new collection example

test1 = rc.create_collection(id="test1", title="Test", description="Test collection")
print(test1.description)

Update a collection

test1.update(description="Different description")
print(test1.description)

Create an item

testitem1 = test1.create_item(id="test1.testitem1")
print(f"Created {testitem1.id} in collection {testitem1.collection}")

Update an item

testitem1.update(properties={"foo": "bar"})
print(testitem1.properties)

Delete an item

testitem1.delete()

Find out more about the Resource Catalog

print(f"Livecheck: PING-{rc.ping()}")
print("\nAPI conforms to:", *rc.get_conformance(), sep="\n")

Search the Catalog

for result in rc.search(collections=['cmip6']):
    print(result.id)