AIforEarthChallenge2024

Challenge Overview

Large Earth foundation models have the potential to revolutionize how we use satellite data to address environmental and social challenges. We launched the AI for Earth 2024 Challenge to encourage the development and application of large Earth models for impact. We invite engineers, scientists, researchers, and enthusiasts to join us!

Introduction

Large Earth foundation models can transform interaction with Earth observation (EO) data by easily ingesting and processing large satellite datasets for use on diverse tasks. They efficiently process vast satellite datasets for diverse tasks, reducing costs and unlocking valuable Earth insights. While the field is growing, limited budgets hinder scaling up model training and testing. The hefty demands of model development and shared global challenges we face underscore the importance of working collaboratively to deploy these models to solve pressing social, environmental, and climate issues. Clay and the #AIForEarthChallenge2024 co-organizers designed this challenge to stimulate development and application of large Earth foundation models towards a wide range of impactful uses.

Open science approaches necessitate collective metrics to make progress towards. Although there are many proposed standards for benchmarks, we lack a common benchmark to measure real impact. This challenge is kickstarting our collective effort to build a common benchmark based on the Common Task Framework principles, enabling researchers and organizations to deploy and test models on environmental tasks. It represents a commitment toward collaborating in the open to scale solutions for climate, nature, and people (if you’re interested in being involved in defining benchmarks, Clay has launched a collaborative benchmarking working group, kicked off by this prize). Ultimately, concentrating efforts and publicly available results will drive innovation and provide open information about model performance, helping users select the best models for their needs. This challenge, co-organized by the Clark University Center for Geospatial Analytics, Clay, Development Seed, Ode, and the Taylor Geospatial Institute, aims to unlock new insights about the Earth, develop powerful tools to better understand and protect our planet, and create a benchmark to reward AI satellite models with operational excellence. It is supported through Clay's Social Impact Partnership with Amazon Web Services.

The Challenge

Foundation models hold promise to be effective across various downstream tasks to help solve pressing challenges for people and the planet, but they must be built with these applications in mind.

We have chosen seven tasks intended to highlight the operational value of foundation models and the type of signal and function needed, as proposed here.

Type of signal anchor: Spatial, Spectral, and Temporal Type of functions: Object Detection, Change Detection, Classification, Regression, Generative

1. Landcover classification [Spectral, Classification]: Classify an AoI on yearly land cover into the following 9 classes: No Data, Water, Trees, Flooded vegetation, Crops, Built area, Bare ground, Snow/ice, Clouds, Rangeland. The scoring and time reference will be Impact Observatory 10m landcover.
2. Aquaculture detection [Spatial, Classification via Similarity Search]: Identify the presence of aquaculture facilities on a given year at 10m resolution. Sourcing and time reference is the open databases of aquaculture from Canada (Nova Scotia), the United States, and Norway. This will be assessed with a simulation of human-in-the-loop learning, where 1) first 10 similarity search results are returned 2) positive and negative results are labeled, and 3) search is performed once again, using these results for the evaluation metric.
3. Carbon above ground stock [Spectral, Regression]: Estimate the above-ground carbon stock, with HGB at 300m resolution estimates as scoring and time reference.
4. Crop yield estimation [Temporal, Regression]: Estimate crop yields at 1km2 resolution of maize and wheat using this open dataset of China as scoring and time reference. There should be two output columns for the Predictions column for this task: wheat_predict (wheat yield predictions), maize_predict (maize yield predictions).
5. Disaster response floods [Spectral, Change detection]: Map the extent of a flood, with NRT as scoring and time reference at 10m resolution. [Note this dataset license is NC-SA]
6. Disaster fire severity [Temporal, Change detection ]: Map the severity of a fire, with NTBS as scoring and time reference at 30m resolution.
7. Cloud Gap Imputation [Temporal, Generative]: Fill in cloudy pixels in a time series (3 scenes) of Sentinel-2 imagery. The time difference between the scenes varies from a couple of days, to a couple of months. The scoring and time reference will be this HLS multi-temporal cloud gap imputation dataset.

Your predictions for each challenge will be tested on either a subset of the reference data or an analogous dataset over a different region.

Participants are required to develop one Apache-licensed Jupyter Notebook in Python that can perform the following tasks with the following constraints:

• Only one foundation model can be used across all tasks. We do not provide specific foundation model requirements, except that one model must be used as a base across all the tasks.
• For foundation model training and inference: Only fully open data (including commercial use) that is also openly available (available on public access on AWS, PC or Google, e.g., Sentinel, Landsat) may be used.
• You can finetune, use LoRA, use embeddings or any other technique as you see fit to augment the foundation model’s performance. For task #5, you may use the scoring dataset for fine tuning, but not to train the foundation model itself, as it is not open for commercial use.
• Participants are allowed to use other datasets that would've been available at the time of the prediction, but SHOULD include embeddings/representations from a foundation model as well.
• Participants must include evaluation code in their notebook, provided in this template submission notebook, for each of the tasks

Please review this example pseudocode submission notebook to clarify what is expected in terms of submission content and format and this template submission notebook to access evaluation code that you must incorporate into your submission. If it’s helpful, this repository also provides basic code solutions that you can build off of for your submission.