Comparing changes

* Modify ApplyEMOS to avoid the potential duplication of cell methods, if probability forecasts are input with an existing cell method e.g. to describe a period diagnostic. * Remove whitespace.

* Use np.interp instead of scipy interp1d * Revert "Use np.interp instead of scipy interp1d" This reverts commit 3ea6cd6. * Fix docstring so that the extended documentation appears. * add more detail to the docs * fix spacing * change word --------- Co-authored-by: Belinda Trotta <btrotta-bom@users.noreply.github.com>

* Modify duration subdivision plugin to correctly recalculate the forecast periods. Modify the netcdf save code to explicitly exclude the inclusion of any duplicate cell methods. * Belt and braces test to demonstrate that slight variations in cell method properties means they would not be removed as duplicates. * Update improver/utilities/save.py Co-authored-by: gavinevans <gavin.evans@metoffice.gov.uk> --------- Co-authored-by: gavinevans <gavin.evans@metoffice.gov.uk>

@brhooper

* Recreated checksums * Added the requested functionality to convert a forecast to climatological anomaly, and appropriate tests. **Main Script** The structure of the new CalculateClimateAnomalies class follows that agreed with @brhooper. - Initialisation - Verification of inputs - Matching standard names - Matching units - Matching grids - Matching time coordinates - Calculation of anomalies: both unstandardised and standardised - Creation of output cube with appropriate metadata * Removed whitespace from files. * Resolved test coverage issues Codecov highlighted. * Resolved remaining test coverage issues. * Resolved formatting issues. * Partial commit. Switching branches to work on another ticket. Resolved prior requests from Ben Hooper following review. Only now require implementation of further metadata adjustments. * Implemented points from previous review, including compliance with CF conventions for anomalies. Metadata added for outputed cube: - Standard names suffixed with "anomaly" if mean cube provided or "standard anomaly" if variance cube also provided - Units set to None if variance cube used and copied from the provided diagnostic cube if no variance cube provided - "Reference epoch" scalar coordinate with (1) timebound from mean cube for anomaly cube, and (2) validity time from diagnostic cube - Cell method of "reference epoch: anomaly" * Resolved failure of pre-commmit checks due to comparing to 'None' using '==' and not 'is' * Resolved warning from pre-commit checks that 'expected_attributes' variable assigned but never used * Added new pytest tests for site data and replaced a test using 'pytest.mark.xfail' with 'pytest.raises(AssertionError)' to ensure the test fails for the correct reason. * Added unit test for raising an error when the time bounds of mean cube and variance cube do not match. * Resolved accidental reuse of a test name * Added missing unit test highlighted by codecov * Ensured all redundant comments removed and added a new test for when the user incorrectly sets 'standardised_anomaly' to false when instantiating and yet provides a variance cube * Increased readability of unit tests and resolved new test failures previously hidden behind faulty fixture setup * Adding the missing test file * Adjusted unit tests following review * Implemented unit test changes suggested in the latest first review * Standardised check that a reference epoch coordinate is present, which was different for one check * Fixed double hashes for comments and simplified testing of reference epoch metadata with Ben's function from last review * Removed references to the diagnostic_cube in the _add_reference_epoch_metadata function as this was mistakenly added * Resolved reformatting requests in previous review * Aimed to fix checksums issue and remove whitespace * Reverted mistaken changes to test_mathematical_operations.py * Changed all instances of 'standard anomaly' with 'standardized anomaly' to reflect requested cf-convention changes in previous review + fixed incorrect type annotations * Removed whitespace

* Threshold interpolation plugin and cli * changes made after PR comments * add unit tests for threshold interpolation plugin * plugin changes and completed iris unit tests * Converted unit tests from iris test to pytest. * acceptance test * added acceptance tests and tidied up * formatting plugin * rewritten as a class * commit for rebase * commit to run suite * first review changes complete * pre-commit checks done * Update improver_tests/utilities/test_threshold_interpolation.py Co-authored-by: bayliffe <benjamin.ayliffe@metoffice.gov.uk> * wip * second review changes * pre-commit tests done * kgos recreated, unit test added and small changes * test fixed and file renamed * check sums sorted * checksums updated --------- Co-authored-by: Katharine Hurst <katharine.hurst@metoffice.gov.uk> Co-authored-by: bayliffe <benjamin.ayliffe@metoffice.gov.uk>

* Start of a precip duration plugin. * Precipitation duration plugin fundamentally working. * Add generic time collapse function that sets the correct time and forecast period coordinate points as per the improver metadata. * Precipitation classification plugin in a working and tidied state. Still needs unit tests, a CLI, and associated acceptance tests. * Add a CLI for precipitation duration. * Add unit tests for collapse_time function. * Improve legibility of collapse_time unit tests. * Starting to add unit tests. * Tests and fixes. * More unit tests and tweaks to plugin. * Test for collapse_time function demonstrating behaviour when a cube with a single values time coordinate is passed in. * Add unit test covering multi-realization data. * Style fixes. * Add acceptance tests. * Additional acceptance test. * Update checksums. * Style fixes. * Modify one parameterised test and remove basic test. * Modify time collapse function to use enforce_time_point_standard utility from metadata/utilities. * Modify CLI to accept comma separated list of inputs to allow for multiple thresholds. Ensure string types, which may be supplied by the CLI are handled by the plugin. A new acceptance test and unit tests for multiple thresholds. * Remove var_names from threshold coordinates in final output. * Make probs into bool type before multiplying to reduce memory usage. May need to make add a test that inputs are bool otherwise this is a horrible potential bug for someone. * Add model id attribute option to the precipitation duration CLI and plugin. * Percentile generation matches numpy method. Still the output looks very different. * Use int type for hit count * Promote any scalar threshold dimensions to ensure that the indexing is working across the expected dimensions. * Lots of comments. * Remove bool type setting as this actually realized the data twice, once as float32 and once as bool. * Reorder loop and realize data to reduce number of lazy selects and deselects which were dominating the processing time. * More comments and start of longer documentation. Added mandatory percentile argument. Fixed up acceptance tests. * Add check for realization coordinate on inputs. * Simplify loop over possible values * Update checksums. * Extended documentation for the precipitation duration percentile generation method. * Improvements to doc strings and extended documentation. * Unit tests rewritten. * Style checks applied. * Add diagnostic name and update checksums for updated KGO. * Move precipitation_type directory to precipitation. Many files affected. * Code changes and acceptance test data addition. * Unit tests back to working but need restucturing and extension for spot data. * Unit tests extended to cover new methods. * Style fixes. * Metadata changes. * Add unit tests for processing of spot data. * Style fixes. * Remove threshold var_names. * Updated checksums for precipitation_duration acceptance tests with new metadata. * Fixed up time coordinate being set incorrectly. Typos addressed. Documentation moved to align with new directory structure. * Update image paths in documentation. * Int type change and doc-string improvements.

* Fix bug which meant skip_ecc_bounds option was not passed through the SpotManipulation plugin in to the ResamplePercentiles plugin. Add test for bug fix. * Improved doc-strings

* add VirtualTemperature to PROCESSING_MODULES * added to CONTRIBUTING.md --------- Co-authored-by: Katherine Tomkins <katherine.tomkins@cazldf00003N.spice.sc.metoffice.gov.uk> Co-authored-by: Katherine Tomkins <katherine.tomkins@cazldf00004M.spice.sc.metoffice.gov.uk>

…#2099) * Threshold plugin modified to be able to collapse multiple dimensions. In theory this could probably be generalised, but in this case the intended use case is time and one of realization or percentile. Work still required to fix up time coordinate if it is collapsed. * Add handling of the time coordinate when collapsing time to generate probabilities in the threshold plugin. * Remove print statements from conftest. * update CLI argument type for coord_collapse. * Ensure forecast period is updated suitably when collapsing time coordinate. Additional test coverage. * Style fixes. * Review doc-string and type hinting changes. * Add methods to apply a cell method when we collapse coordinates within the threshold plugin. * Update CLI doc-string. Add an explicit test for collapsing a scalar coordinate. Add a catch in the plugin that raised an exception if attempting to collapse a coordinate that is not present, allowing for the removal of a later try/except clause.

…ur (#2105) * add VirtualTemperature to PROCESSING_MODULES * added to CONTRIBUTING.md * formatting * widen search to any pressure cube in HumidityMixingRatio * updated comment * updated code comments * debug comments for virtual temp run * removed debug! * enforce units for humidity_mixing_ratio in virtual_temperature * further units * enforce hmr units before calculation * hmr-temp units before calc * vt only * add units to virtual temp cube * addresses review comment * updated PR with latest findings * virt temp unit debug * DEBUG 2 * debug 3 * debug 4 * debug 5 * removed debug * debug 6 * removed debug * debug 7 * debug 8 * debug 9 * removed debug * add units manually * ensure virtual temperature calculation via copy * remove copy in favour of scheduler processes * calculation on cube not data * adresses review comment about status flag * adresses review comment about status flag * addressed review comment * addresses review comments * better use of whitespace * possibly more than 2 cubes handled * weird typo --------- Co-authored-by: Katherine Tomkins <katherine.tomkins@cazldf00003N.spice.sc.metoffice.gov.uk> Co-authored-by: Katherine Tomkins <katherine.tomkins@cazldf00004M.spice.sc.metoffice.gov.uk>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Comparing changes

Open a pull request

Commits on Feb 14, 2025

Commits on Feb 17, 2025

Commits on Feb 18, 2025

Commits on Feb 21, 2025

Commits on Mar 4, 2025

Commits on Mar 7, 2025

Commits on Mar 11, 2025

Commits on Mar 19, 2025

Commits on Mar 25, 2025

This comparison is taking too long to generate.

Uh oh!