OlmoEarth Embeddings using Google Earth Engine data (V2)

[ivanzvonkov]

V2 of #414

Purpose

This PR adds the code necessary to produce a Togo cropland map using OlmoEarth embeddings and Google Earth Engine

Contents

generate_embeddings.ipynb
One-stop shop Colab notebook for generating embeddings with OlmoEarth and Google Earth Engine. Inference can be done directly in the notebook or by deploying a Docker container.

gee_create_OlmoEarth_cropland.js
Google Earth Engine script for creating a Togo cropland using OlmoEarth embeddings in Cloud Storage.

gee_OlmoEarth_cropland_eval.js
Google Earth Engine script for evaluating the OlmoEarth cropland map and comparing with other cropland maps in Togo.

deploy.sh
Script to build and deploy OlmoEarth embedding generation docker container to Google Cloud.
The docker container can then be used inside Google Cloud Run jobs.

Dockerfile
Builds docker container for running job.py

job.py
Python script for running inference and generating embeddings using OlmoEarth and two Google Cloud Storage buckets. Expects Google Earth Engine data to already be exported using Colab notebook.

---

Note

Add Cloud Run job + GEE scripts and a Colab notebook to generate OlmoEarth embeddings from GEE data and create/evaluate a Togo cropland map.

• Cloud Run tooling (scripts/tools/cloud_run):
  • Dockerfile to build GPU-enabled image, set GCLOUD_PROJECT/IN_BUCKET/OUT_BUCKET, and preload OLMOEARTH_V1_NANO.
  • deploy.sh to auth, create Artifact Registry, build, and push the image.
  • job.py to read GEE-exported GeoTIFF tiles from GCS, normalize, batch-infer embeddings with OlmoEarth, write GeoTIFF, and upload to output bucket.
  • README.md with deployment and job run instructions.
• GEE scripts (scripts/tools/gee):
  • gee_create_OlmoEarth_cropland.js to mosaic embeddings, train RF, classify, and export cropland predictions asset.
  • gee_OlmoEarth_cropland_eval.js to load predictions and baselines and report accuracy/F1; README.md overview.
• Notebook (scripts/tools/generate_embeddings.ipynb):
  • End-to-end Colab workflow to export GEE data, run OlmoEarth inference, and upload embeddings; references Cloud Run as an alternative.

^{Written by Cursor Bugbot for commit a8a4f19. This will update automatically on new commits. Configure here.}

[gabrieltseng]

Thanks for this @ivanzvonkov ,

As I understand it, these allow me to:

1. Generate embeddings in GEE. I can either do this (a) via cloud_run or (b) via the notebook.
2. Use the two gee*.js scripts to make and analyze the cropland maps.

If this is the case could you:

• Add a readme to cloud_run explaining the steps to generate the embeddings using these scripts
• Add a readme (or a line in the notebook) to explain its relation to the files in cloud_run
• Add a readme to scripts/tools describing what all the new files do

There's also a failing CI check but I am happy to address this once you have added documentation.

[ivanzvonkov]

@gabrieltseng Added the readme's and addressed the linting issues!

[ivanzvonkov]

@gabrieltseng resolved the band order issue in the code, but it exists in the last Togo runs.

[gabrieltseng]

A few more small documentation updates, but otherwise LGTM. thanks Ivan!