Large Observation results in High Load #585
Comments
|
We're aware of this and are looking into potential solutions. Large tables put the Jinja2 formatting into overdrive. The issue is that these huge tables add significant processing time, surpassing the timeout period configured for nginx. You can increase the timeout to get the report, but you can also create a table so large that no reasonable timeout period will work. On the one hand, the tables we've seen that cause this issue are so large they may be better off contained in an auxiliary document, like a spreadsheet. That's one potential solution for the time being. We will be looking into the table processing to see if it can be made more efficient, but it seems unlikely that we can shave enough time off for these tables. Another possible solution is a significant rework of how report generation works. Today, many reports generate in under a minute, so you can request the report and receive it back in the response without ever approaching the timeout. With the addition of Jinja2 formatting and templating in custom fields, it's possible to assemble a report that will take much longer to generate. We may need to move to a model used by some other products with large reports—i.e., you request the report, the generation is queued in the background, and the report document is placed somewhere for you to fetch once it's finished generating. |
Describe the bug
Trying to generate a report containing an observation with a large number of tables results in 100% load.
To Reproduce
Steps to reproduce the behavior:
Count Document Selection
Words 13308 0
Characters (no spaces) 88919 0
Characters 94707 0
3. Try to generate the report
5. System hangs, generation results in 50x timeout
A clear and concise description of what you expected to happen.
GW should generate the report as normal or refuse to generate with a clear indication of the problem.
Server Specs:
The text was updated successfully, but these errors were encountered: