[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Page MenuHomePhabricator

Migrate wahrani from Toolforge GridEngine to Toolforge Kubernetes
Closed, ResolvedPublic

Description

Kindly migrate your tool(https://grid-deprecation.toolforge.org/t/wahrani) from Toolforge GridEngine to Toolforge Kubernetes.

Toolforge GridEngine is getting deprecated.
See: https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/

Please note that a volunteer may perform this migration if this has not been done after some time.
If you have already migrated this tool, kindly mark this as resolved.

If you would rather shut down this tool, kindly do so and mark this as resolved.

Useful Resources:
Migrating Jobs from GridEngine to Kubernetes
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework#Grid_Engine_migration
Migrating Web Services from GridEngine to Kubernetes
https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation#Move_a_grid_engine_webservice
Python
https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation#Rebuild_virtualenv_for_python_users

Event Timeline

My apologies if this ticket comes as a surprise to you. In order to ensure WMCS can provide a stable, secure and supported platform, it’s important we migrate away from GridEngine. I want to assure you that while it is WMCS’s intention to shutdown GridEngine as outlined in the blog post https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/, a shutdown date for GridEngine has not yet been set. The goal of the migration is to migrate as many tools as possible onto kubernetes and ensure as smooth a transition as possible for everyone. Once the majority of tools have migrated, discussion on a shutdown date is more appropriate. See T314664: [infra] Decommission the Grid Engine infrastructure.

As noted in https://techblog.wikimedia.org/2022/03/16/toolforge-gridengine-debian-10-buster-migration/ some use cases are already supported by kubernetes and should be migrated. If your tool can migrate, please do plan a migration. Reach out if you need help or find you are blocked by missing features. Most of all, WMCS is here to support you.

However, it’s possible your tool needs a mixed runtime environment or some other features that aren't yet present in https://techblog.wikimedia.org/2022/03/18/toolforge-jobs-framework/. We’d love to hear of this or any other blocking issues so we can work with you once a migration path is ready. Thanks for your hard work as volunteers and help in this migration!

Sorry for this late answer.

I already migrated my jobs from GridEngine (jsub) to Kubernetes (toolforge-jobs). My "crontab.grid_stopped" file is empty with no scheduled task.

is there any change to do in order to unlock my tool ?

@wahrani your Kubernetes jobs are still active, as you can verify with the command toolforge jobs list. Only the GridEngine part of Toolforge was disabled for your tool, while your Kubernetes jobs should be working fine. Is any part of your tool not working as expected?

@fnegri yes Kubernetes jobs are active but the tool is disabled from december 20th (TOOL_DISABLED file is present on the tool root directory).

When I attempt to make some modification, I get the following message : "User has been blocked".

@wahrani sorry for the delay, which command did you run when you received the message "User has been blocked"? Only grid-related commands (jsub, jstart, etc.) should be blocked. If you see that message for other commands, it could be a bug, but I need more details to find out.

@fnegri thank you for your reply. At this time my job is working very well. Why "TOOL_DISABLED file" is always present on the tool root directory ?

fnegri claimed this task.

@wahrani that file is only to track the tools where the Grid Engine functionalities are disabled. We should probably have called it "GRID_DISABLED" instead of "TOOL_DISABLED", but you can ignore it. We will remove all those files next month after the Grid migration is completed, until that time it's useful for us to find which tools are still actively using the Grid.