# Earth-Copilot **Repository Path**: mirrors_microsoft/Earth-Copilot ## Basic Information - **Project Name**: Earth-Copilot - **Description**: An AI powered geospatial application that allows you to explore and visualize Earth science data using natural language. - **Primary Language**: Unknown - **License**: MIT - **Default Branch**: main - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2025-09-17 - **Last Updated**: 2025-10-02 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/microsoft/earth-copilot) # 🌍 Welcome to Earth Copilot! **An AI-powered geospatial application that allows you to explore and visualize vast Earth science data using natural language.** ## 🎯 Tutorial Overview Use this tutorial to build your own Earth Copilot in Azure and interact with geospatial data without writing code. Earth Copilot will allow you to: - Interact with geospatial data using natural language queries - Generate and execute SpatioTemporal Asset Catalog (STAC) API queries - Explore 126+ Microsoft Planetary Computer (MPC) STAC collections - Search private, public, or hybrid datasets - Render diverse STAC collections on Azure maps - Integrate Earth Copilot into your own research workflows in Azure ### πŸ“– About Earth Copilot 2.0 Inspired by the NASA and Microsoft Earth Copilot 1.0 prototype, this version features a production-ready architecture built on modern AI agent frameworks, creating an intelligent system that understands scientific questions, automatically finds the right datasets, and delivers interactive visualizationsβ€”all through natural conversation. Whether you're a scientist or student, Earth Copilot helps you spend less time finding the right data and more time unlocking insights. πŸŽ₯ **Watch Satya Nadella introduce NASA Earth Copilot at Microsoft Ignite 2024**: [View Here](https://www.linkedin.com/posts/microsoft_msignite-activity-7265061510635241472-CAYx/?utm_source=share&utm_medium=member_desktop) ## ✨ What Earth Copilot Does ![Earth Copilot Interface](documentation/images/EC.png) **1) Queries that require map visualizations** - "Show me satellite imagery of Dubai urban expansion in 2020" - "Show me radar imagery of Houston Texas during Hurricane Harvey August 2017" - "Show me HLS Landsat imagery for Washington State forests from 2024" - "Show me burned area mapping for Montana wildfire regions 2023" - "Show me land surface temperature for Kansas wheat fields during summer 2024" Data Flow: User Query β†’ React UI β†’ Azure Function App β†’ Semantic Kernel β†’ Microsoft Planetary Computer STAC APIs β†’ Results β†’ Map Visualization and Response **2) Contextual queries looking for information** - "How was NYC impacted by Hurricane Sandy" - "What was the impact of Hurricane Florence 2018 in the North Carolina Outer Banks in 2018" - "How did vegetation recover after flooding in Missouri River valley 2023" - "What are the long-term climate trends affecting Pacific Northwest forests" - "Explain the correlation between El NiΓ±o events and wildfire patterns in California" Data Flow: User Query β†’ React UI β†’ Azure Function App β†’ Semantic Kernel & LLM β†’ Results β†’ Chat Response **3) Queries grounded in private storage data through Retrieval Augmented Generation (RAG)** - "Analyze our proprietary STAC collection for mineral exploration sites in Nevada" - "Compare our private agricultural monitoring data with public MODIS vegetation indices" - "Search our internal disaster response imagery catalog for similar flood patterns" - "Query our custom satellite constellation data for urban heat island analysis" - "Find relevant datasets in our private STAC catalog for infrastructure monitoring projects" Data Flow: React UI β†’ Data Catalog Selection β†’ User Chat Query β†’ NASA VEDA AI Search Index & LLM β†’ Chat Response ### Search Examples ![alt text](documentation/images/grand_canyon_elevation.png) ![alt text](documentation/images/nyc_sandy_impact.png) ![alt text](documentation/images/dubai_landsat.png) ![alt text](documentation/images/thermal_LA.png) ![alt text](documentation/images/data_catalog.png) ## Architecture ![documentation/images/architecture.png](documentation/images/architecture.png) ### Core Services **βš›οΈ React UI (`earth-copilot/react-ui/`)** - **Main entry point UI:** Unified query search for all requests - **Chat Interface**: GeoCopilot chat panel with conversation history leveraging Azure AI and semantic understanding - **Map Visualization**: Azure Maps integration with satellite data overlays - **Data Catalog Pane**: Connection to private, public and hybrid data sources **πŸ”§ Router Function App (`earth-copilot/router-function-app/`)** - **Semantic Kernel Translator**: Agentic query classification and entity extraction that translates natural language to customized STAC parameter queries - **Location Resolver**: Multi-strategy geocoding for map rendering of diverse data catalog types (Azure Maps, Nominatim, AI) - **Temporal Resolver**: Resolves query datetimes to correct search format - **Collection Mapper**: Intelligent satellite data collection selection that routes search to most appropriate STAC data catalog depending on query intent - **STAC Integration**: Function app API connectivity to Microsoft Planetary Computer for executing translated queries **πŸ—„οΈ Core Infrastructure (`earth-copilot/core/`)** - **Configuration Management**: Environment variables and settings - **Logging System**: Structured logging for debugging and monitoring - **Error Handling**: Robust error recovery and feedback ## πŸ“ Project Structure ``` EC/ β”œβ”€β”€ πŸ“ earth-copilot/ # Main application directory β”‚ β”œβ”€β”€ πŸ“ react-ui/ # React frontend application β”‚ β”‚ β”œβ”€β”€ πŸ“ src/ β”‚ β”‚ β”‚ β”œβ”€β”€ πŸ“ components/ # React components β”‚ β”‚ β”‚ β”‚ β”œβ”€β”€ Chat.tsx # Main chat interface β”‚ β”‚ β”‚ β”‚ β”œβ”€β”€ MapView.tsx # Azure Maps integration β”‚ β”‚ β”‚ β”‚ β”œβ”€β”€ DatasetDropdown.tsx # Data source selection β”‚ β”‚ β”‚ β”‚ └── ... β”‚ β”‚ β”‚ β”œβ”€β”€ πŸ“ services/ # API integration services β”‚ β”‚ β”‚ β”‚ β”œβ”€β”€ api.ts # Backend API calls β”‚ β”‚ β”‚ β”‚ └── vedaSearchService.ts # VEDA search integration β”‚ β”‚ β”‚ └── πŸ“ ui/ # UI layout components β”‚ β”‚ β”œβ”€β”€ .env.example # Frontend environment template β”‚ β”‚ β”œβ”€β”€ package.json # Node.js dependencies β”‚ β”‚ └── vite.config.ts # Vite build configuration β”‚ β”œβ”€β”€ πŸ“ router-function-app/ # Azure Functions backend β”‚ β”‚ β”œβ”€β”€ function_app.py # Main API endpoints β”‚ β”‚ β”œβ”€β”€ semantic_translator.py # NL to STAC translation β”‚ β”‚ β”œβ”€β”€ location_resolver.py # Geocoding services β”‚ β”‚ β”œβ”€β”€ collection_profiles.py # Data collection mappings β”‚ β”‚ β”œβ”€β”€ local.settings.json.example # Function app config template β”‚ β”‚ β”œβ”€β”€ requirements.txt # Python dependencies β”‚ β”‚ └── host.json # Azure Functions configuration β”‚ β”œβ”€β”€ πŸ“ core/ # Shared utilities β”‚ β”‚ β”œβ”€β”€ config.py # Configuration management β”‚ β”‚ └── logging.py # Logging utilities β”‚ β”œβ”€β”€ πŸ“ infra/ # Infrastructure as Code β”‚ β”‚ β”œβ”€β”€ main.bicep # Main Bicep template β”‚ β”‚ └── πŸ“ app/ # Application-specific resources β”‚ β”œβ”€β”€ πŸ“ ai-search/ # Azure AI Search setup β”‚ β”‚ └── πŸ“ scripts/ # Search index creation β”‚ └── azure.yaml # Azure Developer CLI config β”œβ”€β”€ πŸ“ documentation/ # Project documentation β”‚ β”œβ”€β”€ πŸ“ images/ # Screenshots and diagrams β”‚ β”œβ”€β”€ endpoint_architecture.md # API documentation β”‚ β”œβ”€β”€ semantic_translator_logic.md # Translation logic docs β”‚ └── ... β”œβ”€β”€ πŸ“ tests/ # Test suites β”‚ β”œβ”€β”€ πŸ“ e2e/ # End-to-end tests β”‚ β”œβ”€β”€ πŸ“ integration/ # Integration tests β”‚ β”œβ”€β”€ πŸ“ unit/ # Unit tests β”‚ └── πŸ“ debug/ # Debug utilities β”œβ”€β”€ πŸ“ scripts/ # Utility scripts β”‚ └── πŸ“ stac_availability/ # STAC data exploration β”œβ”€β”€ πŸ“ tools/ # Development tools β”œβ”€β”€ .env.example # Root environment template β”œβ”€β”€ README.md # This file β”œβ”€β”€ AZURE_SETUP_GUIDE.md # Azure setup instructions β”œβ”€β”€ STARTUP_GUIDE.md # Quick start guide β”œβ”€β”€ setup-all-services.sh # Automated setup script └── requirements.txt # Root Python dependencies ``` ## πŸ”§ Environment Set Up ### Prerequisites To develop and deploy this application, recommended technical background includes: - **Azure Cloud Services** - Creating and configuring Azure AI Foundry, Azure Maps, Azure Functions, and Azure AI Search - **Python Development** - Backend API development, Azure Functions, and Python package management - **React/TypeScript** - Frontend development, Vite build system, and modern JavaScript frameworks - **AI/ML Concepts** - Understanding of LLMs, Semantic Kernel, and natural language processing - **Geospatial Data** - STAC standards and satellite imagery concepts - **Infrastructure as Code** - Bicep templates and Azure resource deployment - **Environment Configuration** - Managing API keys, environment variables, and service connections ### Configure Agent Mode You can deploy this application using Agent mode in Visual Studio Code, your preferred IDE or using Code Spaces directly: [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/microsoft/earth-copilot) ![](documentation/images/vsc_agentmode.png) ### Create your Azure Services Create all required Azure services for complete architecture. **Follow the detailed guide: [AZURE_SETUP_GUIDE.md](AZURE_SETUP_GUIDE.md)** This guide covers step-by-step creation of required services: **Required Services:** - **Azure AI Foundry** - Core AI query processing and natural language understanding with GPT-5 deployment - **Azure Maps** - Geographic coordinate resolution and location services - **Azure Function App** - Serverless compute for backend API endpoints - **Azure AI Search** - Vector search and document indexing for enhanced query capabilities **Optional Services:** - **Azure Application Insights** - Performance monitoring and diagnostics - **Azure Key Vault** - Secure storage of API keys and secrets - **Azure Static Web Apps** - Frontend hosting for production deployment **Data Sources:** - **Microsoft Planetary Computer STAC APIs** - Global satellite data catalog with standardized geospatial data access - **NASA VEDA** - Hosts Earth science datasets from NASA missions and research programs An example of what earth copilot resource group and services could look like: ![/workspaces/Earth-Copilot/documentation/images/resource_group.png](documentation/images/resource_group.png) An example of what Foundry model deployment could look like: ![alt text](documentation/images/foundry.png) ### Connect your environment variables The application requires **multiple environment files** for different components: 1. **Root Environment**: Create `.env` file in project root containing Azure service credentials - Template: See `.env.example` for required variables - Purpose: Backend API services (Azure OpenAI, Azure Maps, etc.) 2. **React UI Environment**: Create `.env` file in `earth-copilot/react-ui/` - Template: See `earth-copilot/react-ui/.env.example` - Purpose: Frontend variables (all prefixed with `VITE_`) 3. **Function App Settings**: Configure `earth-copilot/router-function-app/local.settings.json` - Template: See `local.settings.json.example` in the same directory - Purpose: Azure Functions runtime configuration **Setup**: Copy each `.example` file, remove the `.example` extension, and fill in your Azure service credentials. ## πŸš€ Getting Started ### **Complete Setup Required** Earth Copilot requires Azure services and proper configuration to function. Follow these guides in order: 1. **πŸ”§ Azure Services Setup**: [AZURE_SETUP_GUIDE.md](AZURE_SETUP_GUIDE.md) - Create Azure AI Foundry (GPT-5 deployment) - Set up Azure Maps and AI Search - Configure environment variables 2. **πŸš€ Application Startup**: [STARTUP_GUIDE.md](STARTUP_GUIDE.md) - Install dependencies and run the application - Multiple startup options (automated, VS Code, manual) - Testing and troubleshooting 3. **βš™οΈ Technical Reference**: [SYSTEM_REQUIREMENTS.md](SYSTEM_REQUIREMENTS.md) - Detailed system requirements and dependencies - Version compatibility matrix - Advanced troubleshooting ### ⚠️ Important Notes - **Semantic Kernel 1.36.2** required - the setup script installs the correct version - **Imports**: Run `python verify-requirements.py` to check for compatibility ### **Quick Commands (After Setup)** ```bash # First-time setup (after Azure services are created) ./setup-all-services.sh # Start all services (automated - recommended) ./run-all-services.sh # Manual development (two terminals required) # Terminal 1: cd earth-copilot/router-function-app && func host start # Terminal 2: cd earth-copilot/react-ui && npm run dev # Access at: http://localhost:5173 ``` --- # πŸ“Š Earth Copilot Data Catalogs & Features Reference Below is a summary of collections and visualization capabilities available in Earth Copilot based on the latest assessment of MPC data catalog availability. ## πŸ“ˆ **Data Catalog Availability Rates** **πŸ”οΈ Elevation & Terrain - Excellent Availability** Primary collections: cop-dem-glo-30, cop-dem-glo-90 | Best for: topography, watersheds, slope analysis **πŸ”₯ Fire Detection - High Availability** Primary collections: modis-14A1-061, modis-14A2-061, modis-64A1-061 | Best for: wildfire tracking, burn assessment **🌱 Vegetation/Agriculture - High Availability** Primary collections: modis-13Q1-061, modis-11A1-061, modis-15A2H-061 | Best for: crop health, forest monitoring **🌑️ Temperature/Thermal - High Availability** Primary collections: modis-11A1-061, goes-cmi | Best for: heat analysis, thermal stress **🌟 Snow & Ice - High Availability** Primary collections: modis-10A1-061, viirs-snow-cover | Best for: snow coverage, seasonal analysis **πŸ™οΈ Urban/Infrastructure - Good Availability** Primary collections: naip, sentinel-2-l2a | Best for: city planning, development tracking **πŸ“‘ SAR/Radar - Good Availability** Primary collections: sentinel-1-grd, sentinel-1-rtc | Best for: flood mapping, all-weather monitoring **πŸ›°οΈ Optical Satellite - Good Availability** Primary collections: sentinel-2-l2a, landsat-c2-l2, hls2-l30 | Best for: urban development, coastal monitoring **☁️ Climate & Weather - Variable Availability** Primary collections: era5-pds, daymet-daily-na | Best for: historical weather, climate analysis **🌊 Marine & Ocean - Variable Availability** Primary collections: goes-cmi, modis-sst | Best for: ocean temperature, water quality ## πŸ“„ License MIT License - see [LICENSE.txt](LICENSE.txt) for details. ## Trademarks This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow [Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general). Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies. --- ## 🀝 Acknowledgments Earth Copilot 2.0 was advised by Juan Carlos Lopez and inspired by NASA's Earth Copilot 1.0 developed by Minh Nguyen. A big thank you to our collaborators: - **Microsoft Planetary Computer** - **NASA** - **Microsoft Team**: Juan Carlos Lopez, Jocelynn Hartwig & Minh Nguyen *Built for the Earth science community with ❀️ and AI*