Raised `Invalid request parameters` errors using `application/x-www-form-urlencoded` as content type #2326

kemuridama · 2025-05-07T11:22:13Z

We updated rack to v2.2.14 to correspond a security issue of CVE-2025-46727. And then we occured ActionController::BadRequest: Invalid request parameters: total number of query parameters (xxxx) exceeds limit (4096) errors.

Our service uses application/x-www-form-urlencoded as a content type in POST requests. According to test cases in rack/test/spec_request.rb, I think that rack will parse a request body of application/x-www-form-urlencoded as params. So rack raises that error by limiting a length of params is over.

Is this behavior intentional?

The text was updated successfully, but these errors were encountered:

ioquatix · 2025-05-07T12:25:57Z

Yes, are you expecting more than 4000 parameters?

kemuridama · 2025-05-07T13:05:44Z

Thank you for your answer.

Yes. I thought rack parses a POST request body of application/x-www-form-urlencoded as same as other content types. So I was just wondering this behavior.

jeremyevans · 2025-05-07T13:38:53Z

If you are using more than 4096 parameters, you need to configure a higher limit. the easiest way to do so is by setting the RACK_QUERY_PARSER_PARAMS_LIMIT environment variable to a higher value before loading Rack.

I apologize that this negatively affected your application, but in order to avoid denial of service, we had to pick a limit, and your application is apparently over the limit.

mrsimo · 2025-05-07T16:42:26Z

We updated today and started getting these errors as well. We didn't expect a change like this to come in a patch version bump.

I can sort of understand where this is coming from, but I feel like this is something that would usually happen at a higher proxy level, no? We have configured an explicit client_max_body_size in nginx already, is there an additional concern in Rack? Particularly for the error on number of parameters (as opposed to the bytesize), we're getting errors when the body isn't even 500Kb.

jeremyevans · 2025-05-07T17:39:55Z

Unfortunately, there isn't a way to fix the denial of service issue other than placing a limit on the number of parameters. On 64-bit platforms (all modern platforms really) Ruby objects are 40 bytes each. You need one for the key and one for the value, so that is 80 bytes per parameter. The parameter value can be empty. With 4 characters per parameter (3 for key and 1 for the =, with the value being empty), that allows for a 20x increase in process memory per byte in request body. So a 500KB body can be expanded to over 10MB of objects, per request, and the majority of Ruby users are using webservers that support serving concurrent requests. Rack's defaults allow for requests to create up to 80MB for parameters per request (since the default bytesize is limit 4MB), which I'm sure many if not most users would consider too high already.

As mentioned above, we had to pick a limit, and your application is apparently over the limit. I apologize that this negatively affected your application, but in order to avoid denial of service, this change was necessary. You should use RACK_QUERY_PARSER_PARAMS_LIMIT to configure a higher params limit for your application.

jeremyevans · 2025-05-07T17:45:58Z

Forget to account for the & to separate parameters, so it's only a maximum 16x increase in process memory per request body byte. So potentially 64MB process memory required per request with Rack's default limits. Remaining points are the same.

jeremyevans · 2025-05-07T18:42:56Z

My two previous comments implied that you could actually cause that much memory usage after the patch, but that isn't accurate. The reason we added the limit on the number of parameters is to prevent that much memory usage. Please take my 2 previous comments as what would be possible with just a bytesize limit and no limit on the number of parameters.

mrsimo · 2025-05-08T08:02:54Z

Thanks a lot for the explanations @jeremyevans, makes a lot more sense now, I can see how this can be used by malicious actors. We'll give it some thought and see what we can do!

zelaznik · 2025-05-08T18:40:15Z

For our app, I noticed that adding this piece of middleware to the start of our chain fixed the failure. Our endpoint which was failing was a SOAP endpoint, a POST request that included a 500kb payload in the body. I don't know rack well enough to know if this is going to cause issues elsewhere.

class SetContentTypeFirst
  def initialize(app)
    @app = app
  end

  def call(env)
    if env['CONTENT_TYPE'].nil? && env['HTTP_CONTENT_TYPE'].present?
      env['CONTENT_TYPE'] = env['HTTP_CONTENT_TYPE']
    end

    @app.call(env)
  end
end

jeremyevans · 2025-05-08T18:43:25Z

If your server/middleware is putting HTTP_CONTENT_TYPE into the env hash, it is not Rack compliant and needs to be fixed: https://github.com/rack/rack/blob/main/SPEC.rdoc#label-HTTP_+Headers

zelaznik · 2025-05-08T19:00:36Z

Thanks for the quick response! I'll have to figure out what's going on on my end.

J-Swift · 2025-05-08T20:33:48Z

This broke our CI suite as well, even though we are fronted by nginx to handle max body size. I'm having issues with getting it to play nicely with dotenv however, and I'm not sure if the problem is me or not. But setting RACK_QUERY_PARSER_BYTESIZE_LIMIT doesnt seem to be happening at the right spot if I try to use .env files, even though passing manually when running the suite (RACK_QUERY_PARSER_BYTESIZE_LIMIT=... bundle exec rspec ...) works fine.

jeremyevans · 2025-05-08T21:31:49Z

If you would like to set the limits inside the app:

Rack::Utils.default_query_parser = Rack::QueryParser.make_default(32, 
  bytesize_limit: 4194304, params_limit: 4096)

elliotcm · 2025-05-09T13:42:53Z

This also seems to break mailgun email ingestion with attachments over 4MB.

J-Swift · 2025-05-09T13:49:09Z

That worked, thanks @jeremyevans. One thing I realized when setting that up is that we are actually on rack 2.x (because of rails 6). So its possible that was contributing to my initial issue.

Also, for anyone else on rack 2, the make_default signature is slightly different:

Rack::Utils.default_query_parser = Rack::QueryParser.make_default(
  Rack::Utils.key_space_limit,
  Rack::Utils.param_depth_limit,
  bytesize_limit: ...
)

jeremyevans · 2025-05-09T13:51:13Z

This also seems to break mailgun email ingestion with attachments over 4MB.

Then mailgun needs to increase the bytesize limit (shown earlier in thread), assuming they want to accept larger attachments.

elliotcm · 2025-05-09T13:54:37Z

This also seems to break mailgun email ingestion with attachments over 4MB.

Then mailgun needs to increase the bytesize limit (shown earlier in thread), assuming they want to accept larger attachments.

Sorry no by that I meant when Mailgun delivers the emails to our Rails apps and the attachment is over 4MB then it's now being rejected by this Rack change, so everyone using Mailgun to receive emails of that size (and presumably likewise with other similar providers) is going to have to make this change.

jclusso · 2025-05-12T22:08:21Z

I think the changelog should've noted this. Seems a lot of people have had it break their applications. Fortunately we caught it in test.

ioquatix · 2025-05-13T00:42:57Z

Should we consider increasing the default bytesize limit? e.g. maybe 10MiB?

J-Swift · 2025-05-13T01:35:59Z

I think the main thing is that many people are already protected by eg nginx, and so its a bit annoying to "double configure" the settings here. I solved our usecase above, but might have appreciated if there was just a "disable" option.

jclusso · 2025-05-13T02:01:10Z

I kind of agree with @J-Swift, this feels like a setting that we can disable. Maybe it should even default to off and only be enabled by users that are using Rack directly and not behind nginx.

JDrizzy · 2025-05-19T00:10:27Z

Following on from setting a limit inside the app, would the following middleware that changes these limits based on paths configured be a good idea? e.g.

# lib/middleware/custom_path_query_parser.rb
# 
class CustomPathQueryParser
  def initialize(app, limited_paths = {})
    @app = app
    @limited_paths = limited_paths
  end

  def call(env)
    path = env['PATH_INFO']
    limit = @limited_paths[path]
    Rack::Utils.default_query_parser = Rack::QueryParser.make_default(Rack::Utils.param_depth_limit)
    return @app.call(env) unless limit
    Rack::Utils.default_query_parser = Rack::QueryParser.make_default(
      Rack::Utils.param_depth_limit,
      bytesize_limit: limit
    )
    @app.call(env)
  end
end

# config/application.rb
#
require_relative '../lib/middleware/custom_path_query_parser'

module Appy
  class Application < Rails::Application

    ...

   # No idea where to place it at the moment 🤷‍♂️
    config.middleware.insert_before 1, CustomPathQueryParser, {
      '/rails/action_mailbox/mailer/inbound_emails' => 104_857_600 # ~100MB
    }

Ideally we only want to increase this limit for a specific paths, and not globally. The above code would allow us to set a custom limit for the inbound_emails endpoint, while keeping the default limit for all other endpoints.

jeremyevans · 2025-05-19T00:35:59Z

The middleware isn't thread safe. It's also inefficient (it would be faster to create appropriate query parsers up front, instead of creating one per request).

I assume a better approach would be to override the equivalent of Rack::Request#query_parser for the action mailbox controller to use a query parser with a higher limit, assuming it is calling that method and not Rack::Utils.default_query_parser directly.

ioquatix · 2025-05-19T02:22:32Z

ISTR I've proposed this before but I can't find the exact reference. Anyway, I don't think it would be a bad idea to provide local per-request overrides in env, e.g. env["rack.query_parser"], e.g.:

class CustomPathQueryParser
  def initialize(app, query_parsers = {})
    @app = app
    @query_parsers = query_parsers
  end

  def call(env)
    path = env['PATH_INFO']
    env["rack.query_parser"] = @query_parsers[path]
    
    @app.call(env)
  end
end

Ideally we do this consistently for all similar globals, IOW, rack request handling can be fully defined by the contents of env.

sircpl · 2025-05-20T00:10:22Z

These limits are being triggered for us in unexpected ways. Users attempting to upload files have hit limits on the number of parameters when attempting to upload a file. This was traced back to the fact that rack treats POST requests with a missing Content-Type header as form data and thus eligible for parameter parsing (test case).

In this scenario it really doesn't make sense for us to increase the number of parameters since the body is not intended to be parsed. We'll probably resort to setting a default Content-Type header on appropriate paths.

jeremyevans · 2025-05-20T01:22:02Z

These limits are being triggered for us in unexpected ways. Users attempting to upload files have hit limits on the number of parameters when attempting to upload a file. This was traced back to the fact that rack treats POST requests with a missing Content-Type header as form data and thus eligible for parameter parsing (test case).

In this scenario it really doesn't make sense for us to increase the number of parameters since the body is not intended to be parsed. We'll probably resort to setting a default Content-Type header on appropriate paths.

This should only be a problem if code somewhere is asking rack to parse body params for the request (calling Rack::Request#POST or Rack::Request#params). Doing so for every request is inefficient and in many cases unnecessary. Parameter parsing generally should only be done if the information is actually needed. From a brief look at the popular Ruby web frameworks:

Sinatra parses query string params and body params before dispatching to routes, even for static files it serves
Rails appears to parse query string params before dispatching to routes. I'm not sure whether it parses body params before dispatching to routes (if it does, I couldn't find where).
Grape and Roda don't appear to parse either the query string params or the body params unless it is explicitly requested.

Might be useful to override Rack::Request#POST to print the call stack, then submit the request without content-type, and see what code path is hitting it.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Raised `Invalid request parameters` errors using `application/x-www-form-urlencoded` as content type #2326

Raised `Invalid request parameters` errors using `application/x-www-form-urlencoded` as content type #2326

Raised Invalid request parameters errors using application/x-www-form-urlencoded as content type #2326

Raised Invalid request parameters errors using application/x-www-form-urlencoded as content type #2326

Comments

Raised `Invalid request parameters` errors using `application/x-www-form-urlencoded` as content type #2326

Raised `Invalid request parameters` errors using `application/x-www-form-urlencoded` as content type #2326