8000 Fix Numeric filters by RodneyRichardson · Pull Request #571 · dotliquid/dotliquid · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Fix Numeric filters #571

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 96 commits into
base: develop
Choose a base branch
from

Conversation

RodneyRichardson
Copy link
Contributor
@RodneyRichardson RodneyRichardson commented Feb 20, 2025

Handles numeric parameters consistently across standard filters

  • Fixes filters that control the range of numbers: Abs, AtMost, AtLeast, Round, Ceil, Floor
  • Fixes numeric operation filters: Plus, Minus, DividedBy, Modulo, Times
  • Adds Sum filter.

Uses the following logic:

  • Performs conversion from string representation of numbers (including Thousands separators)
  • Non-numeric parameters are treated as zero (null, non-numeric strings, etc).
  • The return type matches the input type, when possible, or the next size type to handle overflow.

Closes #570 and #483.

Copy link
codecov bot commented Feb 20, 2025

Codecov Report

Attention: Patch coverage is 99.32203% with 2 lines in your changes missing coverage. Please review.

Project coverage is 92.18%. Comparing base (37c86a3) to head (6102c5a).

Files with missing lines Patch % Lines
src/DotLiquid/LegacyFilters.cs 98.18% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #571      +/-   ##
===========================================
+ Coverage    91.81%   92.18%   +0.37%     
===========================================
  Files           72       73       +1     
  Lines         2784     2930     +146     
  Branches       660      713      +53     
===========================================
+ Hits          2556     2701     +145     
- Misses         165      167       +2     
+ Partials        63       62       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@microalps
Copy link
Contributor
microalps commented 8000 Feb 21, 2025

Is this bug exclusive to Abs method? I noted several methods with the same issue in #566

@RodneyRichardson
Copy link
Contributor Author

Is this bug exclusive to Abs method? I noted several methods with the same issue in #566

It also shows up in at_least and at_most, but they don't return the expected behaviour when fixed! I think I will fix this bug on them, and add it to this PR, and add tests for the current incorrect behaviour. We can then fix that when we re-integrate your golden tests and resolve the issues one at a time. Not sure how you want to approach changing the behaviour when current behaviour doesn't match the Ruby/shopify behaviour.

@microalps
Copy link
Contributor
microalps commented Feb 21, 2025

Is this bug exclusive to Abs method? I noted several methods with the same issue in #566

It also shows up in at_least and at_most, but they don't return the expected behaviour when fixed! I think I will fix this bug on them, and add it to this PR, and add tests for the current incorrect behaviour. We can then fix that when we re-integrate your golden tests and resolve the issues one at a time. Not sure how you want to approach changing the behaviour when current behaviour doesn't match the Ruby/shopify behaviour.

I think the behavior there is that null/"abc" as the input is treated the same as 0. Hence null | at_least "5" = 5 and null | at_least "-5" = 0

Also see my latest push for Golden tests, there is an entire set of tests marked failing for "Non-integer arguments for numeric methods". I think the minimum is Abs/Ceil/Floor as they have the same exact syntax and fix.

@microalps
Copy link
Contributor

Comments in #571 (comment) were not addressed.

@RodneyRichardson
Copy link
Contributor Author

Comments in #571 (comment) were not addressed.

Ah, I missed the abs/ceil/floor comment. I'll sort that too.

@RodneyRichardson
Copy link
Contributor Author

Fixed for Ceil and Floor. I've also tweaked Abs to be consistent in implementation (e.g. return decimal, not double).

@RodneyRichardson
Copy link
Contributor Author
RodneyRichardson commented Feb 25, 2025

Round also seems to fit into this, but is a bit more complicated.

General behaviour seems to be

  • if input is invalid, then return 0
  • if places is invalid, then it rounds to zero decimal places.
  • if places is a valid number, do floor(places) then clamp it in the range [0, 16]

Ruby seems to limit places to 16 decimal places, so I've done this too: https://replit.com/@rrjames/Liquid-Tests#main.rb

Note: The Ruby implementation for a negative value for places returns 0, but this feels inconsistent with the other invalid places, so I've implemented it differently and I round to zero decimal places.

@microalps, @daviburg : Thoughts?

@RodneyRichardson RodneyRichardson changed the title Fix NullReferenceException in StandardFilters.Abs Fix Numeric Limit filters Feb 25, 2025
@daviburg
Copy link
Member

Is this bug exclusive to Abs method? I noted several methods with the same issue in #566

It also shows up in at_least and at_most, but they don't return the expected behaviour when fixed! I think I will fix this bug on them, and add it to this PR, and add tests for the current incorrect behaviour. We can then fix that when we re-integrate your golden tests and resolve the issues one at a time. Not sure how you want to approach changing the behaviour when current behaviour doesn't match the Ruby/shopify behaviour.

Fixing mathematics in DotLiquid was one of my earliest contribution to the code base, and got me in trouble because 'bug fixing' meant breaking consumers of the library who took a dependency on the current output. Hence the later introduction of compatibility versioning in the project. IMHO, this is one such case that needs to go behind a version flag.

@daviburg
Copy link
Member
8000

Round also seems to fit into this, but is a bit more complicated.

General behaviour seems to be

  • if input is invalid, then return 0
  • if places is invalid, then it rounds to zero decimal places.
  • if places is a valid number, do floor(places) then clamp it in the range [0, 16]

Ruby seems to limit places to 16 decimal places, so I've done this too: https://replit.com/@rrjames/Liquid-Tests#main.rb

Note: The Ruby implementation for a negative value for places returns 0, but this feels inconsistent with the other invalid places, so I've implemented it differently and I round to zero decimal places.

@microalps, @daviburg : Thoughts?

We should not artificially limit ourselves to fewer decimal places than our underlying implementation language supports. Otherwise, Liquid would be driven by the lowest common denominator amongst all implementations.

@RodneyRichardson RodneyRichardson changed the title Fix Numeric Limit filters Fix Numeric filters Apr 8, 2025
@RodneyRichardson
Copy link
Contributor Author

Ready for another review. I've addressed most of your comments - simplified some functions, and fixed a few small edge case bugs. I've left the class name on calls to static functions, as per David's request, and I haven't been able to reduce DoMathsOperation further.

@daviburg daviburg requested a review from Copilot June 3, 2025 01:05
Copy link
@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request enhances the handling of numeric filters by improving consistency in coercion, conversion, and promotion of numeric types in various filters (such as Abs, AtLeast, AtMost, Round, Ceil, Floor, Plus, Minus, DividedBy, Modulo, Times) and by adding a new Sum filter. Key changes include centralized numeric conversion via NumericConverter, updates to the ExpressionUtility type promotion logic, and version attribute updates in filter methods along with modifications to improve error handling and type precision.

Reviewed Changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/DotLiquid/Util/NumericConverter.cs Implements numeric type checking and conversion with new promotion rules.
src/DotLiquid/Util/ExpressionUtility.cs Refactors binary result type calculation and conversion casting.
src/DotLiquid/StandardFilters.cs Updates filter implementations to use NumericConverter methods and adjusts filter versioning.
src/DotLiquid/LegacyFilters.cs Aligns legacy filter behaviors with new versioning and numeric handling.
(Other test files and project files) Updates tests, project file references and documentation to support the numeric changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

abs throws an exception on nil variable
3 participants
0