8000 first cut at aggregation logic by jhamman · Pull Request #35 · NCAR/GARD · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

first cut at aggregation logic #35

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8000
Merged
merged 6 commits into from
Dec 8, 2016
Merged

first cut at aggregation logic #35

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
ededba4
first cut at aggregation logic
Dec 7, 2016
36c6575
Merge remote-tracking branch 'origin/feature/interactive_option' into…
Dec 8, 2016
a391c8d
exit for invalid agg functions (for now)
Dec 8, 2016
83707c1
no normalization when pass_through is true
Dec 8, 2016
707d0a7
per variable agg_method
Dec 8, 2016
30d4536
update config docs
Dec 8, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 32 additions & 29 deletions docs/config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,17 +51,18 @@ Notes:
| preloaded | string | Yes | n/a | filepath of the preloaded training parameters dataset |
| interpolation_method | integer | No | 1 | nearest neighbor= 1, bilenear =2 |
| normalization_method | integer | No | 0 | no normalization = 0, mean/stddev from: training data = 1 |
| time_indices | integer | Yes | -1 | |
| time_indices | integer | Yes | -1 | list of timesteps in file to aggregate over (GEFS only) |
| agg_method | integer | No | 0 | per variable aggregation method when aggregating over time_indices: mean = 0, minimum = 1, maximum = 2, sum = 3 (GEFS only) |
| nvars | integer | Yes | -1 | number of variables to be used in training |
| data_type | string | Yes | n/a | |
| lat_name | string | Yes | n/a | |
| lon_name | string | Yes | n/a | |
| time_name | string | Yes | n/a | |
| nfiles | integer | Yes | -1 | |
| data_type | string | Yes | n/a | dataset type: GEFS or GCM |
| lat_name | string | Yes | n/a | netCDF variable name for latitude |
| lon_name | string | Yes | n/a | netCDF variable name for longitude |
| time_name | string | Yes | n/a | netCDF variable name for time |
| nfiles | integer | Yes | -1 | number of files in each file list |
| input_transformations | integer | No | 0 | no transform = 0, quantile mapping = 1, log transform = 2, cube root = 3, fifth root = 4 |
| var_names | string | Yes | n/a | variables names to use in training |
| file_list | string | Yes | n/a | |
| selected_time | integer | Yes | -1 | |
| file_list | string | Yes | n/a | path to file containing a list of training files |
| selected_time | integer | No | -1 | if set, only this time step will be read from each input file (GEFS only) |
| calendar | string | Yes | n/a | |
| calendar_start_year | integer | No | 1900 | |
| timezone_offset | real | No | 0 | hours to offset time variable (e.g. `time_name`) to account for timezone. |
Expand All @@ -71,21 +72,23 @@ Notes:

| Name | Type | Required? | Default | Description |
|-----------------------|---------|-----------|---------|------------------------------------------------------------------------------------------|
| name | string | Yes | n/a | |
| preloaded | string | Yes | n/a | |
| name | string | Yes | n/a | Name of the prediction parameters dataset |
| preloaded | string | Yes | n/a | filepath of the preloaded prediction parameters dataset |
| interpolation_method | integer | No | 1 | nearest neighbor= 1, bilenear =2 |
| normalization_method | integer | No | 0 | no normalization = 0, mean/stddev from: prediction data = 1, training data = 2 |
| nvars | integer | Yes | -1 | |
| data_type | string | Yes | n/a | |
| lat_name | string | Yes | n/a | |
| lon_name | string | Yes | n/a | |
| time_name | string | Yes | n/a | |
| nfiles | integer | Yes | -1 | |
| time_indices | integer | Yes | -1 | list of timesteps in file to aggregate over (GEFS only) |
| agg_method | integer | No | 0 | per variable aggregation method when aggregating over time_indices: mean = 0, minimum = 1, maximum = 2, sum = 3 (GEFS only) |
| nvars | integer | Yes | -1 | number of prediction parameters to use in downscaling |
| data_type | string | Yes | n/a | dataset type: GEFS or GCM |
| lat_name | string | Yes | n/a | netCDF variable name for latitude |
| lon_name | string | Yes | n/a | netCDF variable name for longitude |
| time_name | string | Yes | n/a | netCDF variable name for time |
| nfiles | integer | Yes | -1 | number of files in each file list |
| transformations | integer | No | 0 | no transform = 0, quantile mapping = 1, log transform = 2, cube root = 3, fifth root = 4 |
| input_transformations | integer | No | 0 | no transform = 0, quantile mapping = 1, log transform = 2, cube root = 3, fifth root = 4 |
| var_names | string | Yes | n/a | |
| file_list | string | Yes | n/a | |
| selected_time | integer | Yes | -1 | |
| var_names | string | Yes | n/a | variables names to use in prediction |
| file_list | string | Yes | n/a | path to file containing a list of prediction filepaths |
| selected_time | integer | No | -1 | if set, only this time step will be read from each input file (GEFS only) |
| calendar | string | Yes | n/a | |
| calendar_start_year | integer | No | 1900 | |
| timezone_offset | real | No | 0 | hours to offset time variable (e.g. `time_name`) to account for timezone. |
Expand All @@ -94,16 +97,16 @@ Notes:

| Name | Type | Required? | Default | Description |
|-----------------------|---------|-----------|---------------|------------------------------------------------------------------------------------------|
| name | string | Yes | n/a | |
| preloaded | string | Yes | n/a | |
| nvars | integer | Yes | -1 | |
| nfiles | integer | Yes | -1 | |
| data_type | string | Yes | n/a | |
| lat_name | string | Yes | n/a | |
| lon_name | string | Yes | n/a | |
| time_name | string | Yes | n/a | |
| name | string | Yes | n/a | Name of the observation parameters dataset |
| preloaded | string | Yes | n/a | filepath of the preloaded observation parameters dataset |
| nvars | integer | Yes | -1 | number of observation variables to downscale (currently must be 1) |
| nfiles | integer | Yes | -1 | number of files in each file list |
| data_type | string | Yes | n/a | dataset type, typically "obs" |
| lat_name | string | Yes | n/a | netCDF variable name for latitude |
| lon_name | string | Yes | n/a | netCDF variable name for longitude |
| time_name | string | Yes | n/a | netCDF variable name for time |
| input_transformations | integer | No | 0 | no transform = 0, quantile mapping = 1, log transform = 2, cube root = 3, fifth root = 4 |
| var_names | string | Yes | n/a | |
| file_list | string | Yes | n/a | |
| var_names | string | Yes | n/a | variables names to use in obs dataset |
| file_list | string | Yes | n/a | path to file containing a list of obs filepaths |
| calendar | string | Yes | n/a | |
| calendar_start_year | integer | No | 1900 | 10000 |
19 changes: 15 additions & 4 deletions src/config/configuration.f90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,10 +172,12 @@ function read_training_options(filename, debug) result(training_options)
integer :: name_unit, i

! namelist variables to be read
integer :: nfiles, nvars, calendar_start_year, selected_time, interpolation_method, normalization_method
integer :: nfiles, nvars, calendar_start_year, selected_time
integer :: interpolation_method, normalization_method
double precision :: timezone_offset
integer, dimension(MAX_NUMBER_TIMES) :: time_indices
integer, dimension(MAX_NUMBER_VARS) :: selected_level
integer, dimension(MAX_NUMBER_VARS) :: agg_method
character(len=MAXSTRINGLENGTH) :: name, data_type, calendar
character(len=MAXVARLENGTH) :: lat_name, lon_name, time_name
character(len=MAXFILELENGTH) :: preloaded
Expand All @@ -188,7 +190,8 @@ function read_training_options(filename, debug) result(training_options)
lat_name, lon_name, time_name, &
file_list, var_names, &
calendar, calendar_start_year, &
selected_time, time_indices, &
selected_time, agg_method, &
time_indices, &
interpolation_method, preloaded, &
selected_level, input_transformations, &
timezone_offset, normalization_method
Expand All @@ -206,6 +209,7 @@ function read_training_options(filename, debug) result(training_options)
calendar = ""
calendar_start_year = 1900
selected_time = -1
agg_method = kAGG_TYPE_AVG
time_indices = -1
interpolation_method = kNEAREST
input_transformations = kNO_TRANSFORM
Expand All @@ -227,6 +231,7 @@ function read_training_options(filename, debug) result(training_options)
allocate(training_options%file_names(nfiles,nvars))
allocate(training_options%var_names(nvars))
allocate(training_options%selected_level(nvars))
allocate(training_options%agg_method(nvars))
allocate(training_options%input_Xforms(nvars))

! finally, store the data into the config structure
Expand All @@ -248,6 +253,7 @@ function read_training_options(filename, debug) result(training_options)
training_options%calendar_start_year = calendar_start_year
training_options%time_file = 1
training_options%selected_time = selected_time
training_options%agg_method = agg_method
call copy_array_i(time_indices, training_options%time_indices)
training_options%selected_level = selected_level(1:nvars)
training_options%data_type = read_data_type(data_type)
Expand Down Expand Up @@ -332,10 +338,12 @@ function read_prediction_options(filename, debug) result(prediction_options)
integer :: name_unit, i, j

! namelist variables to be read
integer :: nfiles, nvars, calendar_start_year, selected_time, interpolation_method, normalization_method
integer :: nfiles, nvars, calendar_start_year, selected_time
integer :: interpolation_method, normalization_method
double precision :: timezone_offset
integer, dimension(MAX_NUMBER_TIMES) :: time_indices
integer, dimension(MAX_NUMBER_VARS) :: selected_level
integer, dimension(MAX_NUMBER_VARS) :: agg_method
character(len=MAXSTRINGLENGTH) :: name, data_type, calendar
character(len=MAXVARLENGTH) :: lat_name, lon_name, time_name
character(len=MAXFILELENGTH) :: preloaded
Expand All @@ -348,7 +356,7 @@ function read_prediction_options(filename, debug) result(prediction_options)
lat_name, lon_name, time_name, &
file_list, var_names, &
calendar, calendar_start_year, &
selected_time, &
selected_time, agg_method, &
input_transformations, transformations,&
interpolation_method, preloaded, &
selected_level, time_indices, &
Expand All @@ -367,6 +375,7 @@ function read_prediction_options(filename, debug) result(prediction_options)
calendar = ""
calendar_start_year = 1900
selected_time = -1
agg_method = kAGG_TYPE_AVG
transformations = kNO_TRANSFORM
input_transformations = kNO_TRANSFORM
interpolation_method = kNEAREST
Expand All @@ -389,6 +398,7 @@ function read_prediction_options(filename, debug) result(prediction_options)
allocate(prediction_options%var_names(nvars))
allocate(prediction_options%selected_level(nvars))
allocate(prediction_options%transformations(nvars))
allocate(prediction_options%agg_method(nvars))
allocate(prediction_options%input_Xforms(nvars))

! finally, store the data into the config structure
Expand All @@ -409,6 +419,7 @@ function read_prediction_options(filename, debug) result(prediction_options)
prediction_options%calendar = calendar
prediction_options%calendar_start_year = calendar_start_year
prediction_options%selected_time = selected_time
prediction_options%agg_method = agg_method
prediction_options%selected_level = selected_level(1:nvars)
call copy_array_i(time_indices, prediction_options%time_indices)
prediction_options%time_file = 1
Expand Down
Loading
0