Handling of 500 profiles is slow #3810
Comments
|
So the issue is that when a sync is not executed then the TFTP directory and the Cobbler database are diverging. While this is technically not an issue it may lead to unexpected behaviour by the user as this means that an edit is not instantaneous. I think to properly implement this, the concept of transactions should be introduced and then after the full transaction, a single save & sync is executed for all items. This however is a drastic change to the design of the API. Since I am currently developing the new CLI Golang-based and we need to ensure version stability for Golang even the implicitly mentioned low-effort solution of exposing the parameters directly would mean countless new API endpoints (even though the old ones would just be wrapper calls). Of course, we could hide the fact that we added the two new parameters in the Golang client but that is implementation-wise the same effort. |
|
I think the transactions can be added without much changes to the current API: This is already partially implemented via unsaved_items. This should be extended to copy, move and delete operations. I can work on it. Would you accept such patch? |
|
@nadvornik Sure. This is a good idea. When looking at the CI you can ignore the Performance testsuite and integration tests as our dedicated GH runners are not setup yet. I will review (and most probably merge) #3721 which will introduce quite some changes, so I suggest waiting to start implementing this patch after that is done. If you think it is possible (and you have the time) I would love if this could be implemented on an |
Is your feature request related to a problem?
As an xmlrpc API user
I want to create/modify large number of profiles (500)
so that I can setup and maintain large installation environment.
Provide a detailed description of the proposed feature
I am testing Cobbler with SUMA with this PR: uyuni-project/uyuni#9134
Creating/modifying of large number of profiles is slow because Cobbler calls sync when each profile is saved. Sync rewrites complete configuration, so it leads to quadratic time. With 500 profiles it takes more than 1 hour, then it ends up with API token timeout.
The same problem is with deleting profiles or deleting a distribution with linked profiles.
The goal is to support up to 500 proxy servers and the particular proxy will be specified in kernel options in corresponding profile.
Alternatives you've considered
I haven't found any way to fix it with current xmlrpc API.
Additional information
The internal function Collection.add already has the option "with_sync". Exposing this option in "save" method in xmlrpc API would fix this issue.
Profiler output:
The text was updated successfully, but these errors were encountered: