EDEN: multimodal synthetic dataset of Enclosed garDEN scenes
Hoang-An Le, Partha Das, Thomas Mensink, Sezer Karaoglu, Theo Gevers
University of Amsterdam
Reading time ~6 minutes
A garden enclosed, my sister, my bride, a garden enclosed, a fountain sealed!
Your branches are a grove of pomegranates, with all choicest fruits
(Songs of Songs, 4:12-13)
Abstract
Multimodal large-scale datasets for outdoor scenes are mostly designed for urban driving problems. The scenes are highly structured and semantically different from scenarios seen in nature-centered scenes such as gardens or parks. To promote machine learning methods for nature-oriented applications, such as agriculture and gardening, we propose the multimodal synthetic dataset for Enclosed garDEN scenes (EDEN). The dataset features more than 300K images captured from more than 100 garden models. Each image is annotated with various low/high-level vision modalities, including semantic segmentation, depth, surface normals, intrinsic colors, and optical flow. Experimental results on the state-of-the-art methods for semantic segmentation and monocular depth prediction, two important tasks in computer vision, show positive impact of pre-training deep networks on our dataset for unstructured natural scenes.


Data
Data | Size |
---|---|
RGB | 333G |
Segmentation | 13G |
Depth | 241G |
Surface Normal | 130G |
Camera poses | 197M |
Light information | 131M |
Segmentation
- The instance labels
*_Inst.png
are encoded in 32-bit integer png images, using the following function
instance_label = semantic_label * 100 + instance_number
- Semantic labels follow the full TrimBot2020 annotation set, while the 3DRMS challenge and experiments done in the paper follow the reduced label set.
Label full | Label reduced | Name | Label full | Name |
---|---|---|---|---|
0 | 0 | void | ||
2 | 1 | grass | ||
1 | 2 | ground | 3 | dirt |
4 | gravel | |||
5 | mulch | |||
6 | pebble | |||
7 | woodchip | |||
8 | 3 | pavement | ||
10 | 4 | hedge | ||
20 | 5 | topiary | ||
30 | 6 | rose | 31 | rose stem |
32 | rose branch | |||
33 | rose leaf | |||
34 | rose bud | |||
35 | rose flower | |||
100 | 7 | obstacle | 103 | fence |
104 | step | |||
105 | flowerpot | |||
106 | stone | |||
102 | 8 | tree | ||
220 | 9 | background | 223 | sky |
- To convert from full annotation set to the reduced set, the python mapping code is provided below
full2reduced = {2: 1, # grass
1: 2, 3: 2, 4: 2, 5: 2, 6:2, 7:2, # ground
8: 3, #
10:4, # hedge
20: 5, # topiary
30: 6, 31: 6, 32: 6, 33: 6, 34: 6, 35: 6, # flower
100: 7, 103: 7, 104: 7, 105: 7, 106: 7, # obstacle
102: 8, # tree
220: 9, 223: 9, # background
0: 0 # unknown
}
Compared to 3DRMS data
A certain scenes (see below) within EDEN have been re-rendered from those of 3DRMS, some with name changes. Hence the images basically stay the same, yet with some small differences due to different random seeds (such as leaf, grass particles, etc.).
EDEN | 3DRMS |
---|---|
0001 | 0001 |
0026 | 0026 |
0033 | 0033 |
0039 | 0039 |
0042 | 0043 |
0053 | 0052 |
0068 | 0069 |
0127 | 0128 |
0160 | 0160 |
0192 | 0192 |
0223 | 0224 |
0288 | 0288 |
Paper
Presentation
Citation
If you find the material useful please consider citing our work
@inproceedings{le21wacv,
author = {Le, Hoang{-}An and Das, Partha and Mensink, Thomas and Karaoglu, Sezer and Gevers, Theo},
title = {{EDEN: Multimodal Synthetic Dataset of Enclosed garDEN Scenes}},
booktitle = {Proceedings of the IEEE/CVF Winter Conference of Applications on Computer Vision (WACV)},
year = {2021},
}