Skip to content

Instantly share code, notes, and snippets.

@darkblue-b

darkblue-b/EEP_notes_jan26.md

Last active January 25, 2026 04:19
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save darkblue-b/c5a128a445e6f815e615e5c089f2a757 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save darkblue-b/c5a128a445e6f815e615e5c089f2a757 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
EEP_notes_jan26.md

Notes on

EARTH EMBEDDINGS AS PRODUCTS: TAXONOMY, ECOSYSTEM, AND STANDARDIZED ACCESS

Heng Fang† ∗  Adam J Stewart‡ ∗  Isaac Corley§ *  Xiao Xiang Zhu *  Hossein Azizpour†
† KTH Royal Institute of Technology, Stockholm, Sweden
‡ Chair of Data Science in Earth Observation, Technical University of Munich, Munich, Germany
§ Wherobots, San Antonio, USA

arXiv:2601.13134v1 19 Jan 2026 LINK

Introduction

• A comprehensive survey that organizes existing geospatial embedding products into a structured taxonomy and provides a detailed metadata atlas (resolution, license, etc.). • Unified Integration: implements standardized data loaders for these embeddings in TorchGeo

An overview landscape is proposed : a) Analysis Frameworks & Tools; b) Embeddings data artifacts; c) charting downstream application value, specifically mapping tasks and retrieval tasks. Embeddings are differentiated as either location-typed, patch-typed or pixel-typed. Details of existing products are shown.

🌍 1. Foundation Models for Earth Observation (EO)

These are the leading projects that aim to build general-purpose models capable of representing Earth from satellite imagery and other geospatial modalities.

🔧 Projects:

  • Clay Foundation Model – Hugging Face 2024
    A multimodal foundation model for Earth using diverse data sources.

  • Major TOM – AFrancis IGARSS 2024
    Expandable datasets and models for global EO coverage.

  • Earth Index Embeddings – Earth Genome, 2025
    A large-scale embedding system built from Earth observation data.

  • Copernicus-Embed – Zhu et al., AI4Copernicus Project
    Foundation model leveraging Copernicus Sentinel data.

  • Presto Embeddings – NASA Harvest
    Embedding framework for satellite time series and land use analysis.

  • Tessera Embeddings – GeoTessera Docs REPO pixel-based Temporal spectral embeddings for Earth representation.

  • Google Satellite Embedding (AlphaEarth) – Google Earth Engine
    An early-stage embedding model using Google's global satellite data.

  • OlmoEarth – AllenAI, 2025
    Latent image modeling approach for multimodal Earth observation.

📚 Key Papers:

  • XXZhu 2025 LINK – “On the Foundations of Earth Foundation Models”
  • CFBrown 2025 LINK – “AlphaEarth Foundations”
  • KKlemmer 2023 LINK – “SatCLIP: Global Location Embeddings with Satellite Imagery”

📦 2. Datasets

Large-scale, open-access datasets play a central role in training and evaluating Earth foundation models.

🗃 Datasets:

  • EuroSAT – Zenodo
    Land use classification dataset using Sentinel-2 satellite data.

  • EuroCrops – PMC 10495462
    Crop type mapping dataset for Europe.

  • National Land Cover Database (NLCD) – Photogrammetric Engineering & Remote Sensing 2001 LINK
    USA land cover classes

  • SSL4EO-S12 – IEEE Geoscience and Remote Sensing 2023 LINK
    Multimodal, multitemporal dataset for self-supervised learning.

  • Copernicus-Pretrain – IEEE Geoscience and Remote Sensing 2023 an extension of the SSL4EO-S12 dataset to all major Sentinel missions (S1-S5P)

🧠 3. Models & Methods

These include both classical and cutting-edge machine learning approaches used in building Earth foundation models.

⚙ Core Methods:

  • SatCLIP – AAAI 2025 etc LINK Vision-language model for global location representations.

  • MMEarth – EU/CV 2024 Multimodal pretext tasks for geospatial representation learning.

  • ResNet – KHe IEEE/CV 2016 Baseline CNN architecture widely used in EO.

  • ConvNeXt V2 – Woo et al., IEEE/CVF 2023
    Efficient ConvNet architecture using masked autoencoders (MAE).

  • DINO, DINOv2, DINOv3 – INRIA 2021–2023, META LINK Vision transformers with self-supervised learning capabilities.

  • MAE (Masked Autoencoders) – IEEE/CVF 2021 LINK
    Self-supervised learning for vision transformers.

🧬 Distillation & Advanced Approaches:

  • Distillation methods – Transfer knowledge from large models.
  • Neural plasticity-inspired models – [ZXiong, arXiv 2024]
    Inspired by biological learning mechanisms.
  • Multi-label guided soft contrastive learning – [YWang, IEEE TGRS, 2024]
  • Barlow Twins – [Zbontar et al., arXiv 2021]
    Method for learning representations without contrastive loss.
  • Continual Barlow Twins – [IEEE JSTARS, 2023]
    Extends Barlow Twins to continual learning in EO segmentation.

🛠 4. Tools & Benchmarks

These are software systems and frameworks that support development, evaluation, or deployment of EO AI models.

🧰 Tools:

🏆 Challenges:

  • Embed2Scale Challenge – CVPR CAlbrecht 2025
    Large-scale Earth vision challenge focused on scale-aware embeddings.

  • TerraMind Blue-Sky Challenge – [JJakubik, arXiv 2025]
    Generative modeling for Earth observation.

🧭 5. Key Themes & Trends

  1. Foundation Models: TorchGeo now includes data loaders designed for search/retrieval (Clay, Major TOM, Earth Index), and for dense prediction tasks like land cover mapping (Copernicus, Presto, Tessera, Google). TorchGeo allows us to enable fair, side-by-side benchmarking of different embedding models on the same downstream tasks, forming the basis for future experiments. Projects are encouraged to strengthen and improve explainability.

1.1 Major TOM Notes Major TOM embeddings are not (yet) really product-oriented and are aimed with a similar purpose to the MT Core datasets - to make it easier to experiment and benchmark model outputs (hence, unlike TESSERA and AEF which came a few months after, MT embeddings do not have consistent or aggregated temporal scope). We haven't had enough time to finish off the preprint, but my current plan is to provide a simple MT Embedding benchmark at this year's EGU and integrate that into the arxiv pre-print. --Miko

1.2 Earth Index / Earth Genome Use the Earth Index application (earthindex.ai) for non-technical users to use the embeddings we published on source.coop. Users of the web app (non-technical journalists, indigenous communities/allies, NGOs) have been our main focus. Users of the source.coop embeddings have generally been more technical folks interested in exploring/innovating in what's possible --BenStrong

1.3 Clay Clay and Presto offer documented tutorials on generating new embeddings with their models. In CLAY, the encoder receives unmasked patches, latitude-longitude data, and timestep information. Notably, the last 2 embeddings from the encoder specifically represent the latitude-longitude and timestep embeddings.

  1. Self-Supervised Learning (SSL):

  2. Multimodal Integration:

  3. Open Data & Tools: Open-source projects (e.g., TorchGeo, Copernicus-Embed) and public datasets (EuroSAT, EuroCrops) are crucial for reproducibility and democratization of EO AI. Projects are encouraged to increase Input Data Diversity, and to adopt cloud-native data formats for geospatial data.

  4. Benchmarking: Projects are encouraged to standardize in benchmarking. Benchmarks including NeuCo-Bench and Embed2Scale.

📌 Research Directions

  • Unified Earth Foundation Models:
  • Interpretability in EO AI: Exploring how these embeddings can be interpreted by domain experts.
  • Ethics and Bias: Investigating fairness and bias in global EO models trained on unevenly distributed data.
  • Edge Deployment: Making these large foundation models deployable on resource-constrained platforms (e.g., for field use).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment