HydReSGeo Dataset¶

The HydReSGeo dataset is published in [HydReSGeo]. In the following, the file structure is described and the files in rs/masks/ are summarized.

File structure¶

├── gpr
│   ├── field_A
│   │   ├── plot_A_2017-08-15T11:03:21+02.00.sgy
│   │   └── ...
│   └── ...
├── hyd
│   ├── TDR.csv
│   ├── bromide.csv
│   ├── coresamples.csv
│   ├── irrigation_protocol.txt
│   ├── read_in_hydro_data.ipynb
│   ├── sensor_pos.txt
│   └── tensio.csv
├── rs
│   ├── fieldspec.csv
│   ├── hyp
│   │   ├── 20170815_hyp_meas1
│   │   │   ├── Auto000.cub
│   │   │   ├── Auto000.cue
│   │   │   ├── Auto000.hdr
│   │   │   ├── Auto000.jpg
│   │   │   ├── Auto000_PAN.tiff
│   │   │   ├── Auto000_highres.hdr
│   │   │   └── ...
│   │   └── ...
│   ├── lwir
│   │   ├── ir_export_20170815_P0000004_000_12-23-22.csv
│   │   └── ...
│   └── masks
│       ├── hyp_masks.csv
│       ├── ignore_hyp_datapoints.csv
│       ├── ignore_hyp_fields.csv
│       ├── ignore_hyp_measurements.csv
│       ├── meta_IR.txt
│       ├── positions_IR.csv
│       └── positions_hyp_lowres.csv
├── rs_masked
└── site

File descriptions¶

The file descriptions for the geophysical files (gpr), the hydrological files (hyd), the remote sensing files (rs/fieldspec.csv, rs/hyp, and rs/lwir), and the site files (site) are described in [HydReSGeo].

Overall, we divide the hyperspectral data into folders, which include images (= files = datapoints), which consist of different zones (= measurement fields). The fields/zones in each hyperspectral and LWIR image are named as A1-D2 or zone1-zone8 as follows:

zone_dict = {
        "A1": "zone1",
        "A2": "zone2",
        "B1": "zone3",
        "B2": "zone4",
        "C1": "zone5",
        "C2": "zone6",
        "D1": "zone7",
        "D2": "zone8"}

Over the three measurement days of the HydReSGeo dataset, the sensor positions and angles of the hyperspectral camera and LWIR camera change. This change is taken into account by including time-dependend masks in rs/masks/, which are described in the following.

`hyp_masks.csv`¶

This file includes information about four wodden bars which are included in the measurement area and should be masked. The columns are:

measurement: Measurement folder name in the format YYYYmmDD_meas[1-9].
start_row, end_row, start_col, end_col: Start and end rows and columns for the mask.
bar[1-4]_p[1-2]_[x,y], bar[1-4]_height: Information about the geometry of the wodden bar. This is used in ProcessEnviFile.

`ignore_hyp_datapoints.csv`¶

This file includes information about which hyperspectral images (datapoints) need to be excluded for various reasons. The columns are:

measurement: Measurement folder name in the format YYYYmmDD_meas[1-9].
filenumber: Number of the hyperspectral image in the respective measurement folder.

`ignore_hyp_fields.csv`¶

This file includes information about the zones/fields to be ignored in each hyperspectral image due to several reasons: a GPR measurement within that field at the same time, the irrigation platform, or a person walking through the image. The columns are:

measurement: Measurement folder name in the format YYYYmmDD_meas[1-9].
filenumber: Number of the hyperspectral image in the respective measurement folder.
zone: Zone/field which needs to be ignored within the respective file. For the HydReSGeo dataset, eight zones are defined. They are numerated either as A1, A2, B1, B2, C1, C2, D1, and D4, or as zone1 to zone8 for technical reasons.

`ignore_hyp_measurements.csv`¶

This file includes information about which measurement folders to be ignored. The column is:

measurement: Measurement folder name in the format YYYYmmDD_meas[1-9].

`meta_IR.csv`¶

This file is not important for this repository (for now) and can be ignored.

`positions_hyp_lowres.csv`¶

This file includes information about the eight different measurement zones of the HydReSGeo dataset as well as the spectralon (= white reference) with respect to the hyperspectral images. The columns are:

measurement: Measurement folder name in the format YYYYmmDD_meas[1-9].
spec_row_start, spec_row_end, spec_col_start, and spec_col_end: Start and end rows and columns for the spectralon.
zone[1-8]_[row/column]_[start/end]: Start and end rows and columns for the eight measurement zones.

`positions_IR.csv`¶

This file includes information about the eight different measurement zones of the HydReSGeo dataset with respect to the LWIR data. The columns are:

measurement: Measurement folder name in the format YYYYmmDD_meas[1-9].
zone[1-8]_[row/column]_[start/end]: Start and end rows and columns for the eight measurement zones.

Opening the CSV files¶

The CSV files can be opened in python3 with pandas:

import pandas as pd

df = pd.read_csv("hyp_masks.csv", sep="\s+")

References¶

HydReSGeo(1,2): S. Keller, F. M. Riese, N. Allroggen, and C. Jackisch, “HydReSGeo: Field experiment dataset of surface-subsurface infiltration dynamics acquired by hydrological, remote sensing, and geophysical measurement techniques,” GFZ Data Services, 2020. DOI:10.5880/fidgeo.2020.015

HydReSGeo Dataset¶

File structure¶

File descriptions¶

hyp_masks.csv¶

ignore_hyp_datapoints.csv¶

ignore_hyp_fields.csv¶

ignore_hyp_measurements.csv¶

meta_IR.csv¶

positions_hyp_lowres.csv¶

positions_IR.csv¶