| | --- |
| | license: mit |
| | tags: |
| | - reinforcement-learning |
| | - multi-agent |
| | - time-series |
| | - diffusion-model |
| | - energy-management |
| | - smart-grid |
| | --- |
| | # SolarSys: Scalable Hierarchical Coordination for Distributed Solar Energy |
| |
|
| | SolarSys is a novel **Hierarchical Multi-Agent Reinforcement Learning (HRL)** system designed to manage energy storage and peer-to-peer (P2P) trading across large communities of solar-equipped residences. This repository contains the full source code for the SolarSys system, including the trained policies, the custom Gym environment, and the hierarchical diffusion model used for data generation. |
| |
|
| | --- |
| |
|
| |
|
| | ## System Architecture |
| |
|
| | The core of SolarSys is a two-level decision hierarchy: |
| |
|
| | 1. **Low-Level (Intra-Cluster):** Individual households use a **MAPPO** agent to make instantaneous decisions (charge, discharge, local P2P trade, grid trade) based on local meter readings and price signals. |
| | 2. **High-Level (Inter-Cluster):** Cluster Managers use a **Mean-Field** policy to coordinate bulk energy transfers between clusters, ensuring the overall system remains balanced against grid constraints. |
| |
|
| |
|
| |  |
| |
|
| | --- |
| |
|
| | ## Data Generation Framework |
| |
|
| | To enable large-scale simulation with realistic temporal dynamics, SolarSys includes a **Hierarchical Diffusion Model** for generating synthetic, long-duration energy profiles that maintain both long-term (seasonal/monthly) and short-term (daily/hourly) characteristics. |
| |
|
| | * **Model:** Hierarchical Diffusion U-Net |
| | * **Input:** Location based housing dataset(eg. Ausgrid dataset, newyork dataset) |
| | * **Output:** High-resolution time series for Grid Usage and Solar Generation (kWh). |
| |  |
| |
|
| | To ensure all necessary libraries are available, install dependencies using the provided `requirements.txt` file: |
| |
|
| | ```bash |
| | pip install -r requirements.txt |
| | --- |
| | |
| | ## Repository Structure |
| | |
| | The project is organized into core modules and data folders. |
| | |
| | ```tree |
| | SolarSys_Hugging_face/ |
| | ├── assets/ |
| | ├── requirements.txt # Project Dependencies |
| | ├── Dataset/ # Energy Data (Raw, Processed, and Synthetic) |
| | │ ├── Data/ |
| | │ │ ├── Ausgrid_processed_data.zip # Processed data (Australia) |
| | │ │ ├── Ausgrid_raw_data/ |
| | │ │ └── Pecan_street_processed_data.zip # Processed data (US) |
| | │ └── Generated_Data.zip # Example synthetic data (2000 agents) |
| | ├── Data_generation_tool_kit/ # Hierarchical Diffusion Model Code |
| | │ ├── Hier_diffusion_energy/ |
| | │ │ ├── hierarchial_diffusion_model.py # H-Diffusion U-Net Architecture |
| | │ │ └── global_scaler.gz # Global normalization scaler, please delete and rerun if had problem |
| | │ ├── dataloader.py # |
| | │ ├── train.py # Diffusion model training script |
| | │ └── generate.py # Script to generate long-term sequences |
| | ├── Models/ # Trained RL Policies for trial, please retrain your model for best results |
| | │ ├── 100agents_10size/ |
| | │ │ └── models/ |
| | │ │ ├── inter_ep10000.pth # High-Level (Inter-Cluster) policy |
| | │ │ └── low_clusterX_ep10000.pth # Low-Level (Intra-Cluster) policies |
| | │ └── (Additional scaling folders) # e.g., 5agents_5size/, 1000agents_20size/, etc. |
| | ├── SolarSys/ # SolarSys Implementation (MAPPO + MeanField) |
| | │ ├── Environment/ |
| | │ │ ├── cluster_env_wrapper.py # Vectorized environment wrapper |
| | │ │ └── solar_sys_environment.py # Core Gym environment definition |
| | │ ├── mappo/ # MAPPO policy definition (Intra-Cluster) |
| | │ │ └── trainer/ |
| | │ │ └── mappo.py # MAPPO algorithm |
| | │ ├── meanfield/ # MeanField policy definition (Inter-Cluster) |
| | │ │ └── trainer/ |
| | │ │ └── meanfield.py # MeanField algorithm core |
| | │ ├── cluster.py # Inter-Cluster Coordinator logic |
| | │ ├── training_freezing.py # Main SolarSys HRL training script |
| | │ └── cluster_evaluation.py # Evaluation script for SolarSys |
| | └── Other_algorithms/ # Baselines |
| | ├── HC_MAPPO/ # Hierarchical Cluster MAPPO |
| | │ ├── Environment/ |
| | │ ├── HC_MAPPO_train.py # Training script for HC-MAPPO |
| | │ └── HC_MAPPO_evaluation.py # Evaluation script for HC-MAPPO |
| | └── Flat_System/ # Flat Baselines |
| | ├── maddpg/ # Multi-Agent DDPG |
| | │ ├── maddpg_train.py |
| | │ └── maddpg_evaluation.py |
| | ├── mappo/ # Flat MAPPO |
| | │ ├── mappo_train.py |
| | │ └── mappo_evaluation.py |
| | ├── meanfield/ # Flat MeanField Actor-Critic (MFAC) |
| | │ ├── meanfield_train.py |
| | │ └── meanfield_evaluation.py |
| | ├── PG/ # Policy Gradient (PG) |
| | │ ├── pg_train.py |
| | │ └── pg_evaluation.py |
| | └── solar_sys_environment.py # Copy of the flat environment for baselines |
