Feature to run extra-steps #2020

treezio · 2024-04-02T11:10:06Z

treezio
Apr 2, 2024

I don't know if this is totally out of the updatecli scope, but I'm facing an scenario in our organization that might fit with updatecli purpose.

Currently updatecli is automatically running as a GitHub Actions workflow taking care of upgrading our system dependencies for Ruby projects, which means bumping:

Ruby version for Docker base image
bumping .ruby-version file for rbenv local env config
bumping ruby version for the project in Gemfile

Updating Gemfile requires re-generating Gemfile.lock in order to keep dependencies consistency. This is currently being manually assessed, which means checking out the updatecli branch locally, running bundle update --ruby command and commiting changes.

I was thinking that maybe it could be possible that somehow updatecli runs this required extra steps so all changes required for an update to be completed are automated on the apply process.

I could do this as extra steps in a github actions workflow but as the PR is already created by updatecli before manipulating the "extra-files" it might be a bit complex to implement.

I hope this makes sense somehow,
Thank you!

dduportal · 2024-04-02T11:31:26Z

dduportal
Apr 2, 2024
Maintainer

Quick question : is it possible to run this « extra step » as part of your shell target?
(Moving the command to a script for instance)

long term, it means a native ruby provider in updatecli would help, to take care of both files (a bit like golang)

1 reply

treezio Apr 2, 2024
Author

Yes, I think that might fit my use case. Currently taking a look at @olblak proposal 👇🏻

olblak · 2024-04-02T11:31:56Z

olblak
Apr 2, 2024
Maintainer

Hi @treezio,

It's definitely possible to use the shell plugin, I have many examples using that approach.

For example the npm autodiscovery plugin fallbacks to running npm command if it detects a package-lock.json

name: Bump "exec-bin" package version
sources:
    npm:
        name: Get "exec-bin" package version
        kind: npm
        spec:
            name: exec-bin
            versionfilter:
                kind: semver
                pattern: ^1.0.0
targets:
    package-lock.json:
        name: Bump "exec-bin" package version to {{ source "npm" }}
        kind: shell
        spec:
            changedif:
                kind: file/checksum
                spec:
                    files:
                        - package-lock.json
                        - package.json
            command: npm install --package-lock-only --dry-run=$DRY_RUN exec-bin@{{ source "npm" }}
            environments:
                - name: PATH
            workdir: .
        disablesourceinput: true

The changedif is trick here which allow to decide if the target was changed are not by monitoring the files package-lock.json and package.json. They are other changedif kind if needed.
Shame on me I never documented that part

1 reply

treezio Apr 2, 2024
Author

This looks pretty good, not sure if there is a similar approach for bundler, I'll take a look and try to implement something similar!

Otherwise I'd need Gemfile to be updated with the new ruby version before running the bundle check command, which takes place in the targets runtime.

Is it possible to use the output file from an updated target in another target?

something like:

-> update Gemfile with $source_ruby_version
-> update Gemfile.lock running bundle update --ruby over the Gemfile updated in the previous target

olblak · 2024-04-02T12:48:42Z

olblak
Apr 2, 2024
Maintainer

Otherwise as @dduportal suggested another option would be to run an embedded script like on
https://github.com/olblak/updatecli-policy-docusaurus/blob/main/policy/updatecli.d/docusaurus.yaml

But IMHO it would be nice to better understand your usecase because maybe we could add a ruby integration.

Like a new ruby plugin that integrate with https://guides.rubygems.org/rubygems-org-api/ and a new ruby autodiscovery plugin that suggest update for each entries from a Gemfile

1 reply

treezio Apr 2, 2024
Author

Well, let me know if I can help somehow, I'm happy to answer any questions :)

Although as of today we delegate most of Gemfile dependencies management to dependabot, updatecli takes care of updating ruby versions, which requires updating Gemfile and Gemfile.lock

Snippet from a Gemfile:

source "https://rubygems.org"
ruby "3.2.2"
gem "rails", "~> 6.1.7.6"

snippet from the related Gemfile.lock

RUBY VERSION
   ruby 3.2.2p53

Gemfile.lock is managed by bundler meaning it should not be manually updated, that's why I want to implement this "Extra Step"

olblak · 2024-04-02T17:03:30Z

olblak
Apr 2, 2024
Maintainer

I just check and apparently some documentation is missing so a few things that may help you writing the right pipeline.

Using dependson, you can specify that one target must be executed after another one.
When combined with dependsonchange the second target will only be executed if the first one is in a "changed" state

Example

updatecli/e2e/updatecli.d/success.d/command.yaml

Line 90 in ceea46e

6:

And finally you can have conditional targets using conditionsid with yet another example on https://github.com/updatecli/updatecli/blob/main/e2e/updatecli.d/success.d/conditionalTarget.yaml

1 reply

treezio Apr 2, 2024
Author

Thank you for this information, this comes really handy 🙏🏻

treezio · 2024-04-02T17:07:07Z

treezio
Apr 2, 2024
Author

I'll need to rethink the approach for this scenario.
There is a dependency with the ruby version installed when running bundler actions regarding ruby version updates in the Gemfile.

This means that in case i want to update for instance from 3.2.2 to 3.2.3, I would:

Update ruby version in Gemfile lock
Install/Set Ruby version 3.2.3 locally
Run bundle ruby --update to update ruby version in Gemfile.lock

The tricky thing here is that I'd need to install the ruby version to be updated (step 2) so I don't get this error when running bundle ruby --update:

bundle update --ruby
Your Ruby version is 3.2.2, but your Gemfile specified 3.2.3

This means that I'd need to know the ruby version to be updated beforehand in order to create the environment in the CI System, and that's what I haven't been able to figure out how to do it.

GH Actions workflow using the 3.2.2 to 3.2.3 example:

Setup Ruby 3.2.3 version Environment -> Which I need to figure out how to discover this value before running updatecli.
Run updatecli

Bump Gemfile
Run bundle update --ruby (using 3.2.3 runtime version)

Another option could be doing this in different Pull Requests, but I don't like that solution.

Thank you again for all your support guys, you are awesome!

2 replies

olblak Apr 4, 2024
Maintainer

This means that I'd need to know the ruby version to be updated beforehand in order to create the environment in the CI System, and that's what I haven't been able to figure out how to do it.

I faced similar issues on some project, where the update policy also needed to update version used by CI.
For example this policy: ghcr.io/updatecli/policies/golang/version:0.1.0 bump the Golang version and the one specificied in GitHub action workflow. You can run updatecli manifest show ghcr.io/updatecli/policies/golang/version:0.1.0 to see the content of the policy.

Also to retrieve the latest ruby version, there are different ways to retrieve it but I am aware of this manifest on the jenkins project

https://github.com/jenkins-infra/jenkins.io/blob/326a039811b58f85b0415fb836127e8a397e9a8f/updatecli/updatecli.d/ruby.yaml#L43

Because you can't use the githubrelease plugin because version published there don't follow semantic versioning and the ruby project maintain multiple release line at the same time so nothing guarantee you that the "latest" one is the one you are looking for.

Imho it would make sense to have a specific Updatecli ruby plugin to retrieve the latest version, similarly to Golang or Jenkins so we can also retrieve changelogs

olblak Apr 4, 2024
Maintainer

Also today I discovered this yaml file that contain Ruby versions https://raw.githubusercontent.com/ruby/www.ruby-lang.org/master/_data/downloads.yml

So you could use something like https://gist.github.com/olblak/ecd251a3d13b0db4652e2ca5eecf7cf9 to retrieve the stable version for the previous release line

treezio · 2025-02-18T10:58:35Z

treezio
Feb 18, 2025
Author

Hi !
Never came back to this, but we came up with a solution.
Although we do not rely in updatecli for this "Extra step", still acts as the main triggerer for the update to be executed!

You can find some details about it here !
https://medium.com/sequra-tech/automating-ruby-version-upgrades-f71d19e26aeb

6 replies

treezio Feb 18, 2025
Author

Oh nice ! That totally makes sense to me ! I guess the key point is the dependsonchanges: true config

I'd need to dig deeper about this though, as I've tried to run it locally with no luck in the output result:

gemfileLock
-----------

**Dry Run enabled**

The shell 🐚 command "/bin/sh /var/folders/bx/z5d9rykj6wz90q88f94rvc640000gp/T/updatecli/bin/dac4c33c1402d1267a8c489cd4c5fb0710054bd1f81a14176e51d37fc7526204.sh" exited on error (exit code 1) with the following output:
----
You must use Bundler 2 or greater with this lockfile.
----

command stderr output was:
----
/System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems.rb:283:in `find_spec_for_exe': Could not find 'bundler' (2.4.19) required by your /private/var/folders/bx/z5d9rykj6wz90q88f94rvc640000gp/T/updatecli/github/repo/Gemfile.lock. (Gem::GemNotFoundException)
To update to the latest version installed on your system, run `bundle update --bundler`.

I'll take a look whenever I have some time. Might be related to asdf bins management

Many thanks for sharing this !

olblak Feb 18, 2025
Maintainer

The dependsonchange means the target is only executed if the resource it depends on changed.

The challenge with the shell plugin is that you introduce a strong dependency on the tool you need in the target

treezio Feb 18, 2025
Author

true, I'll try to take a look !
Thanks !

olblak Feb 18, 2025
Maintainer

Also you may need to allow some environment variable.

    spec:
      command: |
        bundle config --local deployment false 
        bundle update --ruby
      environments:
        - name: "PATH"

By default the shell plugin doesn't have access to your default environment.
We documented it on https://www.updatecli.io/docs/plugins/resource/shell/#_environments

treezio Feb 18, 2025
Author

Oh, nice hint, did not know about that 👌🏻

updatecli

Feature to run extra-steps #2020

Uh oh!

Uh oh!

treezio Apr 2, 2024

Replies: 6 comments · 12 replies

Uh oh!

dduportal Apr 2, 2024 Maintainer

Uh oh!

treezio Apr 2, 2024 Author

Uh oh!

olblak Apr 2, 2024 Maintainer

Uh oh!

Uh oh!

treezio Apr 2, 2024 Author

Uh oh!

olblak Apr 2, 2024 Maintainer

Uh oh!

treezio Apr 2, 2024 Author

Uh oh!

olblak Apr 2, 2024 Maintainer

Uh oh!

treezio Apr 2, 2024 Author

Uh oh!

Uh oh!

treezio Apr 2, 2024 Author

Uh oh!

olblak Apr 4, 2024 Maintainer

Uh oh!

olblak Apr 4, 2024 Maintainer

Uh oh!

treezio Feb 18, 2025 Author

Uh oh!

Uh oh!

treezio Feb 18, 2025 Author

Uh oh!

olblak Feb 18, 2025 Maintainer

Uh oh!

treezio Feb 18, 2025 Author

Uh oh!

olblak Feb 18, 2025 Maintainer

Uh oh!

treezio Feb 18, 2025 Author

treezio
Apr 2, 2024

Replies: 6 comments 12 replies

dduportal
Apr 2, 2024
Maintainer

treezio Apr 2, 2024
Author

olblak
Apr 2, 2024
Maintainer

treezio Apr 2, 2024
Author

olblak
Apr 2, 2024
Maintainer

treezio Apr 2, 2024
Author

olblak
Apr 2, 2024
Maintainer

treezio Apr 2, 2024
Author

treezio
Apr 2, 2024
Author

olblak Apr 4, 2024
Maintainer

olblak Apr 4, 2024
Maintainer

treezio
Feb 18, 2025
Author

treezio Feb 18, 2025
Author

olblak Feb 18, 2025
Maintainer

treezio Feb 18, 2025
Author

olblak Feb 18, 2025
Maintainer

treezio Feb 18, 2025
Author