### 1. Create New Submission Directory Create a new folder in the `new_submission` top directory: ```bash geobench_leaderboard/ └── new_submission/ ├── results_and_parameters.csv ├── additional_info.json ``` ### 2. Add Results and Parameters Details Add a CSV file (`results_and_parameters.csv`) with the columns below. Please note that if terratorch-iterate is used for experiments, this table may be created automatically upon completion of an experiment. Please see https://github.com/The-AI-Alliance/GEO-Bench-Leaderboard for more: - `backbone`: backbone used for experiment, (e.g. Prithvi-EO-V2 600M) - `dataset`: some or all of the GEO-bench datasets. Please see Info page to learn more. - `Metric`: the type of metric used for evaluation. Depending on the dataset, this may be one of the following: `Overall_Accuracy`, `Multilabel_F1_Score`, `Multiclass_Jaccard_Index` - `experiment_name`: if terratorch-iterate used, this will the experiment_name used in mlflow. Otherwise, a unique name may be used for all results relating to a single backbone - `batch_size_selection`: denotes whether the batch size was fixed during hyperparameter optimization. May be `fixed` or `optimized` - `early_stop_patience`: early stopping patience using for trainer - `n_trials`: number of trials used for hyperparameter optimization - `Seed`: random seed used for repeated experiment. 10 random seeds must be used for each - `batch_size`: batch size used for repeated experiments for each backbone/dataset combination. - `weight_decay`: weight decay experiments for each backbone/dataset combination. - `lr`: learning rate used for repeated experiments for each backbone/dataset combination. Obtained from hyperparameter optimization (HPO) - `test metric`: metric obtained from running backbone on the dataset during repeated experiment. Please see Info page to learn more. ### 3. Add Additional Information Create a JSON file (`additional_info.json`) with information about your submission and any new models that will be included. The JSON file MUST have the same file name and contain the same keys as the `examples/additional_info.json` file. ### 4. Submit PR 1. Fork the repository 2. Add your results following the structure above and in the PR comments add more details about your submission 3. Create a pull request to main