Cloning, combining, and aggregating datasets

Prev Next

In this document

You will learn how to clone your dataset, combine two datasets to create a new one, and how to aggregate it to a different temporal resolution.

Cloning the dataset

You can clone any dataset using the Clone dataset tool accessible from the main menu via Tools/Clone Dataset or use the clone functionality included in the Database manager.

  1. Open the Clone dataset tool.

  2. Select date/time range if you want to clip the cloned dataset.

  3. Enter the new dataset name and select the database to store it in. The name must be unique for the given project.

  4. Clone the dataset using the “Clone” button.

The dataset with the changed name will be cloned and saved into the selected database.

Combining datasets

If you need to combine two datasets, e.g., you have new monthly data and want to add it to an existing dataset, you can use the Combine tool. You can access it from the main menu via Tools/Combine.

Key points to remember:

  • By combining the dataset you are adding data from secondary dataset to the primary dataset, creating a new one from the combination.

  • The primary dataset takes priority when both datasets contain values for the same time stamp. Values from the primary dataset will be used in the new dataset.

Open the Combine tool and follow these steps to combine datasets:

  1. Select the datasets you want to combine in the primary and secondary dataset dropdowns. Secondary dataset’s data will be added to the primary one. If any time stamps overlap, data from the primary datasets will be used.

  2. If required, you can clip the new dataset and set new from-to dates to create a custom time range.

  3. Add the name of the new dataset and select the database where the dataset will be stored.

  4. Additional options. Examine the individual combinations in the visualization under the image.

    • Combine records: Decide whether you want to skip timestamps missing in the primary dataset or if you wish to combine all records.

    • Combine columns: Decide whether you want to skip columns missing in the primary dataset or if you wish to combine all columns.

    • Combine columns with the same name: Determine by the selection what will be done in the case when both datasets contain columns with the same name.

Execute the merge using the “Combine” button at the bottom-right. Combining flags in the datasets follows the same principles as other data values.

Data visualization showing primary and secondary datasets with missing values and merging options.

Cases when combining datasets is not allowed

Combining datasets is not allowed in the following cases:  

  • The database of the primary, secondary, or combined dataset is not available

  • One of the datasets is empty

In case the option to merge columns is selected, the following scenarios are not allowed:

  • Merging columns of the same name but different parameter types

  • Merging GTI columns of the same name but different GTI configurations

  • Merging columns of the same name but with values in different units

Aggregating datasets

You can aggregate any dataset to a lower temporal resolution using the Aggregate tool. You can access it from the main menu via Tools/Aggregate.

  1. Select the dataset you want to aggregate. Its original temporal resolution is shown under the selector.

  2. Choose the desired temporal resolution. You can edit the aggregation rules using the Edit button next to the target temporal resolution selector to the right.

  3. Select the aggregation period. The default period is set to the full duration of the dataset's timeline.

  4. Enter the output dataset’s name and select the database to store it.

Execute the aggregation using the “Aggregate” button at the bottom-right.

Editing the aggregation rules

You can edit several aggregation parameters, such as how to handle the aggregation of hourly or daily values, whether to aggregate the data as averages or sums, or if the flags will be added or not.

  1. Determine the aggregation method per parameter and timestamp duration.

  2. Select if you want flags to be added to the dataset for every parameter separately and state flag numbers to be added to the aggregated dataset.

  3. View the full list of flags included for the selected parameter.  

  4. Keep the time zone adjustment selected. If disabled, some data might get shifted and added to the surrounding days due to the aggregation principles.

Click “OK” to apply the changes and to close the aggregation configuration window.