8000 A series of post-refactoring fixes by heroin-moose · Pull Request #2685 · cobbler/cobbler · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

A series of post-refactoring fixes #2685

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jun 26, 2021
Merged

Conversation

heroin-moose
Copy link
Contributor
@heroin-moose heroin-moose commented Jun 23, 2021

Highlights:

  • Remove the last of get_boot_loaders() invocations
  • Fix BaudRates usage
  • Use Archs and RepoArchs enums instead of str
  • Do not check if a file exists before removing
  • Fix bootloader cleanup during system removal
  • Fix dhcpd template for iPXE
  • Fix incorrect PXELINUX filename
  • Use distro.remote_boot_* instead of distro.remote_grub_*
  • Unbreak iPXE

While the most of the changes are trivial, there are some pieces I'm kinda
worried about. Please take a carefull look at the patches that remove code
branches related to x86 (not x86_64) and replace global state (like
repo.arch) with local.

This function no longer exists and is superseded by boot_loaders
@Property.
For format strings we require BaudRates member values (int), not the
member itself.
@heroin-moose
Copy link
Contributor Author
heroin-moose commented Jun 23, 2021

Huh... I believe the tests fail because the failed code was not executed prior to this PR. I'll look into that too.

@nodeg nodeg added the main Not a release but referring to the Git main branch label Jun 24, 2021
@nodeg nodeg added this to the v3.3.0 milestone Jun 24, 2021
@SchoolGuy SchoolGuy requested review from SchoolGuy and nodeg June 24, 2021 08:49
@SchoolGuy SchoolGuy self-assigned this Jun 24, 2021
Copy link
Member
@SchoolGuy SchoolGuy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my eyes this is LGTM. The tests should pass before we merge this though...

@heroin-moose
Copy link
Contributor Author
heroin-moose commented Jun 24, 2021

Huh, looks like os.path.exists() dereferences symlinks. That's why it does not remove the stale link. Why the link is pointing to a non-existent file is another question though.

@heroin-moose
Copy link
Contributor Author

Okay, I have a fix now. However, it is kind of masking the problem, e.g. it's now possible to cleanup up from a "broken state" (we have a symlink pointing to a non-existent file). It's not clear to me how we end up with this broken state. For some reason even if my system only has one bootloader (pxe), write_all_system_files() tries to create grub configuration anyway, because grub_path variable contains a path instead of None. The problem is that this path does not actually exist. I suspect that there is some missing condition in get_config_filename().

@heroin-moose heroin-moose force-pushed the fixes branch 3 times, most recently from 817a14e to 5b24194 Compare June 24, 2021 10:41
@heroin-moose
Copy link
Contributor Author

Hmmm. I see. Sometimes arch is str, sometimes is Archs.

@heroin-moose
Copy link
Contributor Author
heroin-moose commented Jun 24, 2021

Yeah... What happens is that write_pxe_file() bails out without creating the configuration file for Grub because of this check:

        if boot_loaders is None or format not in boot_loaders:
            return None

format is "grub" and "boot_loaders" does not contain anything (weird, why, it should contain "pxe"). Anyway, even if boot_loaders would contain "pxe", there symlink creation should be conditional -- if write_pxe_file() creates a file, then we should create a symlink. If it does not -- we should not.

@heroin-moose
Copy link
Contributor Author

Okay, works, I will port write_pxe_file() to using Archs and will come with another bunch of patches.

@SchoolGuy
Copy link
Member

@heroin-moose Would you consider this PR as mergable now?

Copy link
Member
@SchoolGuy SchoolGuy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still LGTM in my eyes. Tests are passing. Let's wait for @nodeg just to be sure!

@heroin-moose
Copy link
Contributor Author
heroin-moose commented Jun 24, 2021

I think I shall do the proper fix. Give me a couple of hours.

@heroin-moose heroin-moose force-pushed the fixes branch 3 times, most recently from a185beb to 1eba6e9 Compare June 24, 2021 20:43
Fix the last remaining parts of the code where plain strings are used
instead of the respective enum members. Also remove code branches that
refer to x86 arch that is no longer exist.
Apart from TOCTOU races this has another implication: if a symlink is
pointing to a non-existent file, os.path.exists() will return False,
hence the link will never be removed. In the tftpgen case this will lead
us to a deadlock if we have inconsistent Grub configuration: it's not
possible to create a new link before removing the old one, and we cannot
remove the old one because the existing link is broken.

Both utils.rmfile() and utils.mkdir() can handle ENOENT and EEXIST
respectively.
Currently the cleanup code does not remove any configuration files in
TFTPD directory, because both Grub and PXELINUX filenames are wrong.
Also there is no code to remove the link from system_link/<name> to
system/<mac>. This commit fixes the issue.

I know very little about yaboot, but the code in tftpgen.py generates
some links and whatnot that obviously should be removed too. I've left a
FIXME, hopefully someone will tend to that soon.
Currently it's not possible to enable iPXE without specifying DHCP
Filename Override option for a system because of the weird condition in
the dhcpd template. To make things worse, this option does not even
affect the actual iPXE configuration, it only affects PXELINUX.

Digging into Git history reveals commit
cf76e29 that introduced this condition,
to my understanding, to prevent sending DHCP Boot File Name option to
petitboot. However, petitboot simply ignores this option, so let's make
UX for iPXE users more pleasant.
After the multilevel menu patch get_config_filename() became smart in
order to deal with multiple bootloaders per system, that is now a thing.
This has unfortunate side-effect: simply calling system.get_config_filename()
may yeild unexpected results.

Consider a system that supports the following list of bootloaders:

  ["pxe", "ipxe", "grub"]

Without the expclicit "loader" parameter, system.get_config_filename()
will always return a filename in Grub format (11:22:33:44:55:66), not
PXELINUX format (01-11-22-33-44-55-66), as Grub is the dominant option
in the get_config_filename() lookup logic.

This commit fixes the most straighforward case:

  $ cobbler system edit --name "test" --boot-loaders "pxe grub"
  $ find /var/lib/tftpboot -name 11:22:33:44:55:66
  /var/lib/tftpboot/grub/system/11:22:33:44:55:66  <--- RIGHT
  /var/lib/tftpboot/pxelinux.cfg/11:22:33:44:55:66 <--- WRONG
The latter is gone, these are leftovers.
@codecov
Copy link
codecov bot commented Jun 24, 2021

Codecov Report

Merging #2685 (521c005) into master (998e3f5) will increase coverage by 0.22%.
The diff coverage is 18.64%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2685      +/-   ##
==========================================
+ Coverage   39.22%   39.45%   +0.22%     
==========================================
  Files          86       86              
  Lines       13363    13361       -2     
==========================================
+ Hits         5242     5271      +29     
+ Misses       8121     8090      -31     
Impacted Files Coverage Δ
cobbler/actions/sync.py 38.86% <0.00%> (-0.32%) ⬇️
cobbler/remote.py 0.00% <0.00%> (ø)
cobbler/services.py 0.00% <0.00%> (ø)
cobbler/actions/reposync.py 7.37% <10.00%> (+0.26%) ⬆️
cobbler/modules/managers/isc.py 10.59% <12.50%> (+0.38%) ⬆️
cobbler/tftpgen.py 25.85% <29.03%> (+3.00%) ⬆️
cobbler/items/system.py 74.85% <0.00%> (+0.59%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 998e3f5...521c005. Read the comment docs.

@SchoolGuy
Copy link
Member

@heroin-moose Is this now everything except the Inheritance?

Copy link
Member
@SchoolGuy SchoolGuy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am once again confirming that this PR is fine. :)

@heroin-moose
Copy link
Contributor Author
heroin-moose commented Jun 25, 2021

@SchoolGuy, this is the first batch :) I have more (like removing enable_ipxe), but I think it would be better to commit them separately.

@heroin-moose
Copy link
Contributor Author

Fixed one more iPXE bug.

Currently CobblerSvc does not implement settings() method. This breaks
iPXE: attempts to fetch an iPXE script from the Cobbler API result in
the following AttributeError:

  File "/var/www/cobbler/svc/services.py", line 89, in application
    cw = CobblerSvc(server="http://127.0.0.1:%s" % remote_port)
  File "/usr/lib/python3.6/site-packages/cobbler/services.py", line 50, in __init__
    self.dlmgr = download_manager.DownloadManager(self.collection_mgr)
  File "/usr/lib/python3.6/site-packages/cobbler/download_manager.py", line 35, in __init__
    self.settings = collection_mgr.settings()
  File "/usr/lib/python3.6/site-packages/cobbler/cobbler_collections/manager.py", line 98, in settings
    return self.api.settings()
  AttributeError: 'CobblerSvc' object has no attribute 'settings

This commit provides the missing method.
@SchoolGuy
Copy link
Member

I have now extensively checked this and will merge this. Automated tests are sadly not existing (that is why this PR is even needed). Since the contributor has tested this in his own infrastructure I will trust him with this.

@SchoolGuy SchoolGuy merged commit 7ee54a1 into cobbler:master Jun 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
main Not a release but referring to the Git main branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0