cobbler get-loaders fails on Centos 7 new install #2660

AlbertBraden · 2021-06-07T20:44:38Z

Describe the bug

cobbler get-loaders fails with 404 error on Centos 7 new install

Steps to reproduce

Install Centos 7 from https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2
Install cobbler and its requirements:

yum install -y epel-release
yum upgrade -y
yum install -y net-tools bind-utils pykickstart syslinux fence-agents rsync debmirror dhcp wget
yum install -y python2-pip httpd cobbler cobbler-web xinetd vim

Configure and start cobbler
Run get-loaders:

[root@area51-os-cobbler-01 ~]# cobbler get-loaders
task started: 2021-06-07_202830_get_loaders
task started (id=Download Bootloader Content, time=Mon Jun  7 20:28:30 2021)
path /var/lib/cobbler/loaders/README already exists, not overwriting existing content, use --force if you wish to update
downloading https://cobbler.github.io/loaders/COPYING.elilo to /var/lib/cobbler/loaders/COPYING.elilo
Exception occured: <class 'urlgrabber.grabber.URLGrabError'>
Exception value: [Errno 14] HTTPS Error 404 - Not Found
Exception Info:
  File "/usr/lib/python2.7/site-packages/cobbler/remote.py", line 82, in run
    rc = self._run(self)
   File "/usr/lib/python2.7/site-packages/cobbler/remote.py", line 176, in runner
    return self.remote.api.dlcontent(self.options.get("force",False), self.logger)
   File "/usr/lib/python2.7/site-packages/cobbler/api.py", line 735, in dlcontent
    return grabber.run(force)
   File "/usr/lib/python2.7/site-packages/cobbler/action_dlcontent.py", line 73, in run
    urlgrabber.grabber.urlgrab(src, filename=dst, proxies=proxies)
   File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 773, in urlgrab
    return default_grabber.urlgrab(url, filename, **kwargs)
   File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 1220, in urlgrab
    return _run_callback(opts.failfunc, opts)
   File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 1071, in _run_callback
    return cb(obj)
   File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 1065, in _do_raise
    raise obj.exception

!!! TASK FAILED !!!

Expected behavior

get-loaders completes successfully

Cobbler version

[root@area51-os-cobbler-01 ~]# cobbler version
Cobbler 2.8.5
  source: ?, ?
  build time: Tue Oct 15 01:59:43 2019

Operating system

CentOS Linux release 7.9.2009 (Core)

Cobbler log

Mon Jun  7 20:42:33 2021 - DEBUG | REMOTE CLI Authorized; user(?)
Mon Jun  7 20:42:33 2021 - INFO | REMOTE start_task(Download Bootloader Content); event_id(2021-06-07_204233_get_loaders); user(?)
Mon Jun  7 20:42:34 2021 - INFO | dlcontent

Screenshots

Additional information

It looks like https://cobbler.github.io/loaders/COPYING.elilo doesn't exist.

The text was updated successfully, but these errors were encountered:

AlbertBraden · 2021-06-07T20:45:12Z

This seems pretty new; I installed cobbler on centos 7 a few weeks ago and did not encounter it.

nodeg · 2021-06-08T06:59:17Z

We removed the get-loaders support with #2572 but this should not affect you since you are still on version 2.8.5. But together with this the bootloaders from the Cobbler website were removed, too. See cobbler/cobbler.github.io@3d9d3c3
@SchoolGuy Any suggestions on how we should proceed here since version 2.x is EOL?

SchoolGuy · 2021-06-08T13:59:16Z

@AlbertBraden The bootloaders of our website should have never been used in the first place. They were not updated for years and since I cannot guarantee for the content of it, I have taken them down. As said in other discussions and issues, I am expecting people to get them for themselves from the distros for numerous reasons. I am very sorry but updating, signing, as well es publishing the bootloaders is a task for more then a whole team, thus we cannot take care of this, even if it makes your life more complicated.

AlbertBraden · 2021-06-08T14:08:32Z

Is there a document that explains how to manually update the bootloaders?

SchoolGuy · 2021-06-08T15:23:15Z

@AlbertBraden They should be by the distro you want to install at least. And then you can use the normal workflows of Cobbler.

AlbertBraden · 2021-06-08T21:38:01Z

After googling around it appeared that I needed syslinux.efi so I did this:

mkdir -p /usr/local/src/loaders/
cd /usr/local/src/loaders/
wget https://mirrors.edge.kernel.org/pub/linux/utils/boot/syslinux/syslinux-6.03.zip
unzip syslinux-6.03.zip
cp efi64/com32/elflink/ldlinux/ldlinux.e64 /var/lib/tftpboot/
cp efi64/efi/syslinux.efi /var/lib/tftpboot/

Then I edited dhcp.template and changed the filename to syslinux.efi, and ran cobbler sync, but the pxe-boot is hanging:

What am I doing wrong?

dansou901 · 2021-06-08T23:44:07Z

If I remember correctly version 6.0.3 has a bug. For efi, you should be using grub.

AlbertBraden · 2021-06-09T14:03:50Z

I searched on my Cobbler host and found /var/www/cobbler/ks_mirror/centos7-minimal-1908-x86_64/EFI/BOOT/grubx64.efi so I copied that into /var/lib/tftpboot/ along with grub.cfg, and changed the filename in dhcp.template, and that gives me a nice menu:

But it fails when I select an option:

What am I missing?

SchoolGuy · 2021-06-09T14:12:40Z

@AlbertBraden Exactly what it tells you: It can't find the kernel where it expectes the kernel to be. That can either be due to a misconfiguration, a missing cobbler sync or due to a timing problem when your environment is to large and Cobbler has not yet completed the sync it is currently doing.

AlbertBraden · 2021-06-09T14:32:00Z

Until recently, I would install cobbler and run get-loaders and it would just work. Now that running get-loaders has been replaced with a complex and difficult process that is new to Cobbler users, can we get a document to help us download and install the correct loader and menu?

SchoolGuy · 2021-06-09T15:01:31Z

@AlbertBraden Yes you may. Actually the (easy) process is already documented: https://cobbler.readthedocs.io/en/latest/quickstart-guide.html#importing-your-first-distribution

The more complex/advanced one is distributed over the whole docs but I am working on improving the docs...

SchoolGuy · 2021-06-09T15:03:21Z

Keep in mind that this is for the master branch and it might not work always.

For the 2.8.5 release you may have a look here: https://cobbler.readthedocs.io/en/release28/3_general/cobbler%20direct%20command.html#import

AlbertBraden · 2021-06-10T03:16:37Z

I read those two documents and I see the discussion about importing, but importing the distribution is a separate step from setting up the loader and menu, and I'm not seeing the loader/menu steps in there. I ended up spending most of the day experimenting with grub.cfg, and I ended up with this:

[root@area51-os-cobbler-01]# cat /var/lib/tftpboot/grub.cfg

set default="0"

function load_video {
  insmod efi_gop
  insmod efi_uga
  insmod video_bochs
  insmod video_cirrus
  insmod all_video
}

load_video
set gfxpayload=keep
insmod gzio
insmod part_gpt
insmod ext2

set timeout=10

search --no-floppy --set=root -l 'CentOS 7 x86_64'

menuentry 'Install CentOS 7' --class fedora --class gnu-linux --class gnu --class os {
        linuxefi /images/centos7-minimal-1908-x86_64/vmlinuz ksdevice=bootif lang=  kssendmac text  ks=http://192.168.100.2/cblr/svc/op/ks/profile/centos7-minimal-1908-x86_64
        initrdefi /images/centos7-minimal-1908-x86_64/initrd.img
}

This works for pxe-booting, but I am seeing a bunch of errors in /var/log/messages on the cobbler server. Do I still have something misconfigured?

Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23167]: Client 192.168.100.14 finished grubx64.efi
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23168]: RRQ from 192.168.100.16 filename /grub.cfg-01-e4-43-4b-c0-ed-10
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23168]: Client 192.168.100.16 File not found /grub.cfg-01-e4-43-4b-c0-ed-10
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23169]: RRQ from 192.168.100.16 filename /grub.cfg-C0A86410
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23169]: Client 192.168.100.16 File not found /grub.cfg-C0A86410
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23170]: RRQ from 192.168.100.16 filename /grub.cfg-C0A8641
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23170]: Client 192.168.100.16 File not found /grub.cfg-C0A8641
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23171]: RRQ from 192.168.100.16 filename /grub.cfg-C0A864
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23171]: Client 192.168.100.16 File not found /grub.cfg-C0A864
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23172]: RRQ from 192.168.100.16 filename /grub.cfg-C0A86
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23172]: Client 192.168.100.16 File not found /grub.cfg-C0A86
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23173]: RRQ from 192.168.100.16 filename /grub.cfg-C0A8
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23173]: Client 192.168.100.16 File not found /grub.cfg-C0A8
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23174]: RRQ from 192.168.100.16 filename /grub.cfg-C0A
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23174]: Client 192.168.100.16 File not found /grub.cfg-C0A
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23175]: RRQ from 192.168.100.16 filename /grub.cfg-C0
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23175]: Client 192.168.100.16 File not found /grub.cfg-C0
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23176]: RRQ from 192.168.100.16 filename /grub.cfg-C
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23176]: Client 192.168.100.16 File not found /grub.cfg-C
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23177]: RRQ from 192.168.100.16 filename /grub.cfg
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23177]: Client 192.168.100.16 finished /grub.cfg
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23178]: RRQ from 192.168.100.16 filename /EFI/BOOT/x86_64-efi/command.lst
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23178]: Client 192.168.100.16 File not found /EFI/BOOT/x86_64-efi/command.lst
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23179]: RRQ from 192.168.100.16 filename /EFI/BOOT/x86_64-efi/fs.lst
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23179]: Client 192.168.100.16 File not found /EFI/BOOT/x86_64-efi/fs.lst
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23180]: RRQ from 192.168.100.16 filename /EFI/BOOT/x86_64-efi/crypto.lst
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23180]: Client 192.168.100.16 File not found /EFI/BOOT/x86_64-efi/crypto.lst
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23181]: RRQ from 192.168.100.16 filename /EFI/BOOT/x86_64-efi/terminal.lst
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23181]: Client 192.168.100.16 File not found /EFI/BOOT/x86_64-efi/terminal.lst
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23182]: RRQ from 192.168.100.16 filename /grub.cfg
Jun 10 02:40:03 area51-os-cobbler-01 in.tftpd[23182]: Client 192.168.100.16 finished /grub.cfg

SchoolGuy · 2021-06-11T06:54:58Z

@AlbertBraden Yes these files are normally there if Cobbler is the one generating the TFTP directory and the grub.cfg for you. If you want to UEFI Boot your machines take a look at the mkgrub.sh script. Documentation for it can be found here: https://cobbler.readthedocs.io/en/latest/scripts.html#mkgrub-sh

The script is not deployed with Cobbler 2.8.5 but works perfectly well with it since it is independent of Cobbler. Just grab it from the master branch. Remember that your distro needs to support the noarch packages for Grub, so you can install the ppcle64 modules onto an x86_64 machine. Normally that should be the case since you need them not only for Cobbler but also to boot VMs with other archs on your machine.

SchoolGuy · 2021-06-11T06:56:01Z

@dansou901 @hbokh Please help me out here if you have a bit of time. Maybe I am seeing this from the wrong standpoint, you have more experience as users as I have...

hbokh · 2021-06-11T10:44:24Z

Interesting! This is going to bite us too in time, on our now +1 year old Cobbler 3.1.2 on Debian Buster (not CentOS7).

FWIW, we have these loaders in /var/lib/cobbler/loaders/:

$ tree /var/lib/cobbler/loaders/
/var/lib/cobbler/loaders/
├── COPYING.syslinux
├── COPYING.yaboot
├── grub
│   ├── grub.0
│   ├── grub.cfg
│   ├── grubx64.efi
│   └── x86_64-efi -> /usr/lib/grub/x86_64-efi
├── grub-x86_64.efi
├── grub-x86.efi
├── menu.c32
├── pxelinux.0
├── README
└── yaboot

2 directories, 11 files

On my even older Cobbler 3.1.2 at $HOME, running cobbler get-loaders gives this output:

# cobbler get-loaders
task started: 2021-06-11_123903_get_loaders
task started (id=Download Bootloader Content, time=Fri Jun 11 12:39:03 2021)
running python triggers from /var/lib/cobbler/triggers/task/get_loaders/pre/*
running shell triggers from /var/lib/cobbler/triggers/task/get_loaders/pre/*
shell triggers finished successfully
path /var/lib/cobbler/loaders/README already exists, not overwriting existing content, use --force if you wish to update
path /var/lib/cobbler/loaders/COPYING.yaboot already exists, not overwriting existing content, use --force if you wish to update
path /var/lib/cobbler/loaders/COPYING.syslinux already exists, not overwriting existing content, use --force if you wish to update
path /var/lib/cobbler/loaders/yaboot already exists, not overwriting existing content, use --force if you wish to update
path /var/lib/cobbler/loaders/pxelinux.0 already exists, not overwriting existing content, use --force if you wish to update
path /var/lib/cobbler/loaders/menu.c32 already exists, not overwriting existing content, use --force if you wish to update
path /var/lib/cobbler/loaders/grub-x86.efi already exists, not overwriting existing content, use --force if you wish to update
path /var/lib/cobbler/loaders/grub-x86_64.efi already exists, not overwriting existing content, use --force if you wish to update
*** TASK COMPLETE ***

This might come in handy for @AlbertBraden to at least get some loaders in the right place.

In the mean time I'm having a hard time setting up Cobbler 2.8.5 on CentOS7 in a VM again.
If I find anything useful I'll get back here, but since this version of Cobbler is pretty dated and EOL I'm not going to run for it TBH.

AlbertBraden · 2021-06-11T12:53:07Z

This is a completely terrible experience for cobbler operators, and the timing is particularly bad as we are scrambling to build Centos 7 dev environments to test the migration to Centos 8, and then from Centos 8 to something else before the end of the year. If you could just make 2.8.5 work for a few more months (without the operator spending many hours hacking loaders and menus), that would be a huge service to the operators who are using your product.

hbokh · 2021-06-11T15:57:50Z

cobbler check on CentOS7 and Cobbler 2.8.5 gives this informative output:

Some network boot-loaders are missing from /var/lib/cobbler/loaders, you may run 'cobbler get-loaders' to download them, or, if you only want to handle x86/x86_64 netbooting, you may ensure that you have installed a recent version of the syslinux package installed and can ignore this message entirely. Files in this directory, should you want to support all architectures, should include pxelinux.0, menu.c32, elilo.efi, and yaboot. The 'cobbler get-loaders' command is the easiest way to resolve these requirements.

I wonder if anyone still needs yaboot and / or elilo.efi.
So far I have a gut feeling the cobbler get-loaders is not the cause anymore of the problem here, but (U)EFI / Grub booting.

AlbertBraden · 2021-06-11T16:11:06Z

Yes, UEFI is the issue, but UEFI works "out of the box" after running get-loaders. Without get-loaders it is a mess.

hbokh · 2021-06-11T18:33:15Z

I created an "archive" with the old loader files. I hope this can clean up your mess a bit.

https://github.com/hbokh/cobbler-loaders

SchoolGuy · 2021-07-05T10:59:10Z

@opoplawski Could you create a patch which removes get-loaders from any version available in the RH Ecosystem please?

opoplawski · 2021-12-10T04:20:46Z

Done in 3.2.2-5 (Fedora, EPEL8, copr orion/cobbler) and 2.8.5-2 (EPEL7).

SchoolGuy · 2023-03-04T21:23:57Z

This will not be fixed and the reasons are outlined in detail here. No activity for a while. Closing issue.

AlbertBraden added the Bug Report Reporting a bug label Jun 7, 2021

nodeg added the 2.8.5 Release 2.8.5 label Jun 8, 2021

nodeg added the Deprecated Deprecated feature which will be removed in future versions. label Jun 8, 2021

SchoolGuy mentioned this issue Jul 12, 2021

No menu was displayed after pxe boot (3.2.0 on el8) #2708

Open

SchoolGuy mentioned this issue Aug 13, 2021

cobbler get-loaders --force failing with Exception value: 'str' object has no attribute 'get' #2587

Closed

SchoolGuy mentioned this issue Jan 17, 2022

Loaders borked on non-latest Cobbler #2885

Closed

SchoolGuy added this to Cobbler Server Jul 3, 2022

SchoolGuy moved this to Todo in Cobbler Server Jul 3, 2022

SchoolGuy closed this as not planned Won't fix, can't repro, duplicate, stale Mar 4, 2023

github-project-automation bot moved this from Todo to Done in Cobbler Server Mar 4, 2023

SchoolGuy added this to the v2.8.x milestone Mar 4, 2023

xuanyuanaosheng mentioned this issue Oct 17, 2024

基于Cobbler实现物理机的自动安装以及初始化 xuanyuanaosheng/xuanyuanaosheng.github.io#21

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

cobbler get-loaders fails on Centos 7 new install #2660

cobbler get-loaders fails on Centos 7 new install #2660

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cobbler get-loaders fails on Centos 7 new install #2660

cobbler get-loaders fails on Centos 7 new install #2660

Comments

Uh oh!

Describe the bug

Steps to reproduce

Expected behavior

Cobbler version

Operating system

Cobbler log

Screenshots

Additional information

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!