Bug 666435

### See also: 
     https://fedorahosted.org/cobbler/ticket/572

Created attachment 471221 [details]
Patch Proposed

Howdy team, 

I wrote a patch for honor the environment variables. 

Follow the test procedure: 

  a) upgrade package
[root@server ~]# rpm -Uvh cobbler-2.0.7-5.el5sat.TEST.00387446.3.noarch.rpm 
Preparing...                ########################################### [100%]
   1:cobbler                ########################################### [100%]
Stopping cobbler daemon: [  OK  ]
Starting cobbler daemon: SERVING!
[  OK  ]
[root@server ~]# 
  
  b) showing env var

[root@server ~]# env | grep http
http_proxy=http://proxy-server.example.com:3128

  c) no files in directory 

[root@server ~]# tree /var/lib/cobbler/loaders/
/var/lib/cobbler/loaders/

0 directories, 0 files

  d) close output on iptables to ensure traffic uses proxy

[root@server ~]# iptables -A OUTPUT -p tcp --dport 80 -o eth0 -j LOG
[root@server ~]# iptables -A OUTPUT -p tcp --dport 443 -o eth0 -j LOG
[root@server ~]# iptables -A OUTPUT -p tcp --dport 80 -o eth0 -j REJECT
[root@server ~]# iptables -A OUTPUT -p tcp --dport 443 -o eth0 -j REJECT
[root@server ~]# iptables -L -n -v 
Chain INPUT (policy ACCEPT 123M packets, 81G bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 105M packets, 50G bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 LOG        tcp  --  *      eth0    0.0.0.0/0            0.0.0.0/0           tcp dpt:80 LOG flags 0 level 4 
   87  5205 LOG        tcp  --  *      eth0    0.0.0.0/0            0.0.0.0/0           tcp dpt:443 LOG flags 0 level 4 
    0     0 REJECT     tcp  --  *      eth0    0.0.0.0/0            0.0.0.0/0           tcp dpt:80 reject-with icmp-port-unreachable 
   17  2264 REJECT     tcp  --  *      eth0    0.0.0.0/0            0.0.0.0/0           tcp dpt:443 reject-with icmp-port-unreachable 


   e) get loaders

[root@server ~]# cobbler get-loaders
task started: 2010-12-30_180723_get_loaders
task started (id=Download Bootloader Content, time=Thu Dec 30 18:07:23 2010)
downloading http://mdehaan.fedorapeople.org/loaders/README to /var/lib/cobbler/loaders/README
downloading http://mdehaan.fedorapeople.org/loaders/COPYING.elilo to /var/lib/cobbler/loaders/COPYING.elilo
downloading http://mdehaan.fedorapeople.org/loaders/COPYING.yaboot to /var/lib/cobbler/loaders/COPYING.yaboot
downloading http://mdehaan.fedorapeople.org/loaders/COPYING.syslinux to /var/lib/cobbler/loaders/COPYING.syslinux
downloading http://mdehaan.fedorapeople.org/loaders/elilo-3.8-ia64.efi to /var/lib/cobbler/loaders/elilo-ia64.efi
downloading http://mdehaan.fedorapeople.org/loaders/yaboot-1.3.14-12 to /var/lib/cobbler/loaders/yaboot
downloading http://mdehaan.fedorapeople.org/loaders/pxelinux.0-3.61 to /var/lib/cobbler/loaders/pxelinux.0
downloading http://mdehaan.fedorapeople.org/loaders/menu.c32-3.61 to /var/lib/cobbler/loaders/menu.c32
*** TASK COMPLETE ***
 
   f) files downloaded
 
[root@server ~]# tree -upgt /var/lib/cobbler/loaders/
/var/lib/cobbler/loaders/
|-- [-rw-r--r-- root     root    ]  yaboot-1.3.14-12
|-- [-rw-r--r-- root     root    ]  pxelinux.0-3.61
|-- [-rw-r--r-- root     root    ]  menu.c32-3.61
|-- [-rw-r--r-- root     root    ]  COPYING.syslinux
|-- [-rw-r--r-- root     root    ]  README
|-- [-rw-r--r-- root     root    ]  elilo-3.8-ia64.efi
|-- [-rw-r--r-- root     root    ]  COPYING.yaboot
`-- [-rw-r--r-- root     root    ]  COPYING.elilo

0 directories, 8 files
[root@server ~]# 


Proxy Output: 
  $ tail -f /var/log/squid/access.log

1293760629.586     61 10.10.56.126 TCP_MISS/200 1532 GET http://mdehaan.fedorapeople.org/loaders/README - DIRECT/128.197.185.45 text/plain
1293760629.645     53 10.10.56.126 TCP_MISS/200 1115 GET http://mdehaan.fedorapeople.org/loaders/COPYING.elilo - DIRECT/128.197.185.45 text/plain
1293760629.710     53 10.10.56.126 TCP_MISS/200 1110 GET http://mdehaan.fedorapeople.org/loaders/COPYING.yaboot - DIRECT/128.197.185.45 text/plain
1293760629.829    113 10.10.56.126 TCP_MISS/200 18494 GET http://mdehaan.fedorapeople.org/loaders/COPYING.syslinux - DIRECT/128.197.185.45 text/plain
1293760630.133    302 10.10.56.126 TCP_MISS/200 356982 GET http://mdehaan.fedorapeople.org/loaders/elilo-3.8-ia64.efi - DIRECT/128.197.185.45 text/plain
1293760630.414    278 10.10.56.126 TCP_MISS/200 198681 GET http://mdehaan.fedorapeople.org/loaders/yaboot-1.3.14-12 - DIRECT/128.197.185.45 text/plain
1293760630.557    139 10.10.56.126 TCP_MISS/200 15203 GET http://mdehaan.fedorapeople.org/loaders/pxelinux.0-3.61 - DIRECT/128.197.185.45 text/plain
1293760630.753    192 10.10.56.126 TCP_MISS/200 39675 GET http://mdehaan.fedorapeople.org/loaders/menu.c32-3.61 - DIRECT/128.197.185.45 text/plain

Created attachment 473400 [details]
Patch for /etc/init.d/cobblerd

Hello, 

I made some changes on our spec file, which includes an option to read the /etc/sysconfig/cobblerd to set the http_proxy variable for read. This way guarantee that variables will be loaded when cobblerd  service will be load.  

$ cat /etc/sysconfig/cobblerd 
export HTTP_PROXY="http://proxy_server:3128"

$ service cobblerd restart

$ cobbler get-loaders 

Please, add this on the next cobblerd package for Red Hat. 

Best, 
Marcelo Moreira de Mello

Hello, 

Just to share that patch were committed in upstream. 

$> git show 0218888f9e72a6ffb358003f19686b040d7fd02a
commit 0218888f9e72a6ffb358003f19686b040d7fd02a
Author: Scott Henson <shenson>
Date:   Wed Jan 12 12:08:22 2011 -0500

    Allow dlcontent to use proxy environment variables

diff --git a/cobbler/action_dlcontent.py b/cobbler/action_dlcontent.py
index b3adae0..d73f114 100644
--- a/cobbler/action_dlcontent.py
+++ b/cobbler/action_dlcontent.py
@@ -62,13 +62,33 @@ class ContentDownloader:
           ( "%s/grub-0.97-x86_64.efi" % content_server, "%s/grub-x86_64.efi" % dest),
        )
 
-       self.logger.info("downloading content required to netboot all arches")
-       for f in files:
-          src = f[0]
-          dst = f[1]
-          if os.path.exists(dst) and not force:
-             self.logger.info("path %s already exists, not overwriting existing content, use --force if you wish to update" % dst)
-             continue
+       proxies = {} 
+       for var in os.environ.keys():
+          if (var.lower() == "http_proxy"):
+             proxies['http'] = os.environ[var]
+
+          if (var.lower() == "ftp_proxy"):
+             proxies['ftp'] = os.environ[var]
+
+       if (len(proxies) > 0):
+          for f in files:
+             src = f[0]
+             dst = f[1]
+             if os.path.exists(dst) and not force:
+                self.logger.info("path %s already exists, not overwriting existing content, use --force if you wish to update" % dst)
+                continue
+             os.chdir(dest)
+             self.logger.info("downloading %s to %s" % (src,dst))
+             urlgrabber.grabber.urlgrab(src, proxies=proxies)  
+
+       else:
+          self.logger.info("downloading content required to netboot all arches")
+          for f in files:
+             src = f[0]
+             dst = f[1]
+             if os.path.exists(dst) and not force:
+                self.logger.info("path %s already exists, not overwriting existing content, use --force if you wish to update" % dst)
+                continue
           self.logger.info("downloading %s to %s" % (src,dst))
           urlgrabber.urlgrab(src,dst)

Hello, 

Since the patch already were committed in upstream, for futures cobbler packages shipped by Red Hat honour the fix, we probably will just need to add in our SPEC the https://bugzilla.redhat.com/attachment.cgi?id=473400 and a new NOTE in our documentation that proxy is supported by doing a configuration in /etc/sysconfig/cobblerd. 

Best Regards, 
Marcelo Moreira de Mello

Lana - within changes made for bug 492961 - which was being used to track corrections to errors in the cobbler documentation, it seems we added a section for 'cobbler buildiso' usage by mistake (which was a proposed addition). 

This has caused this bug - bug 666435 & bug 667924 to be reported against Satellite 5.4.0. 

As noted within 506485 we have not spent time for development or QE in testing this functionality to flesh it out, as also shown by these two new bug reports. 

I am converting this into a documentation bug for Satellite 5.4.1 release cycle. Can we please use it to track the removal of cobbler buildiso from our reference guide. 

I'm made this comment public to allow GSS and others to easily find this. I've also changed the subject of the bug to reflect the change. 

Regards,
Clifford

This could not be a documentation bug as this feature exists and is supported in Satellite 5.3.

As a matter of fact we have customers using the feature and not supporting will mean breaking their current build scenarios.

That feature works very well and comes straight for Spacewalk.

It seems that few things were moved around between two versions of Spacewalk which occured between Satellite 5.3 to 5.4.

The work around I found to fix it in Satellite 5.4 is:

Used /var/lib/cobbler/menu.c32 from RHN Satellite 5.3 and copied into RHN Satellite 5.4 as /var/lib/cobbler/loaders/menu.c32

After that Cobbler build iso works:

# cobbler buildiso
task started: 2010-11-02_182138_buildiso
task started (id=Build Iso, time=Tue Nov 2 18:21:38 2010)
using/creating tempdir: /buildiso
building tree for isolinux
copying miscellaneous files
copying kernels and initrds for profiles
generating a isolinux.cfg
generating profile list
done writing config
running: mkisofs -o /root/generated.iso -r -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -V Cobbler\ Install -R -J -T /buildiso
received:
ISO build complete
You may wish to delete: /buildiso
The output file is: /root/generated.iso
*** TASK COMPLETE *** 

Regards

Gilles

What happened to "we support what we ship" ?

I've just hit the exact same buildiso issue at a major customer & used Gilles' fix to be able to correct this after a Satellite upgrade, the customer has been using 'cobbler buildiso' heavily since 5.3 because of network restrictions due to security, it is my opinion that we can't just remove functions that we've shipped previously.

Thanks Gilles.
Aurelien (GPS Consultant - Sydney, AU)

To be exact and to detail further:
- compared to Gilles, I didn't have any /var/lib/cobbler/menu.c32
- tried 'cobbler get-loaders' but as the system is behind a proxy -> bug
- copied /usr/lib/syslinux/menu.c32 to /var/lib/cobbler/loaders/menu.c32 -> ok

'cobbler buildiso' now works fine.
Aurelien (GPS Consultant - Sydney, AU)

It's more than just a doco mistake. Because of the impact on customers

The functionality was in 5.3 and documented and still is in 5.4 and documented.
Now if one cannot build servers with PXE Boot than we have to provide a way to build ISOs.

Regards,

Gilles

In the Kickstart chapter in the new User Guide, I have included the following procedure:

<procedure id="proc-Getting_Started_Guide-Kickstarting_from_Bare_Metal_-Cobbler_Boot_ISO">
<title>Cobbler Boot ISO</title>
	 <para>
		The Cobbler boot ISO is a disk image that can be built on a &SAT; server and burned to a CD or DVD. It can then be used to boot a bare metal system.
	</para>
<step>
	<para>
		To create a boot ISO, log in to the &SAT; server as root and run <command>cobbler buildiso</command>. The ISO will contain all kernel and initrd images stored in the &SAT;, along with all associated kernel argument settings.
	</para>
	 <para>
		Kickstart files will be sourced remotely, which means that changes to the kickstart templates can be made without having to re-burn the disk. However, if you create a new kickstart profile and want to use it with the Cobbler boot ISO, you will need to create a new disk.
	</para>
</step>
<step>
	<para>
		Insert the Cobbler boot ISO disk. A menu of available kickstarts will be displayed. Select the appropriate kickstart and the system will start to install automatically. <remark>A screenshot here would be good. LKB</remark>
	</para>
	 <note>
	<title>Note</title>
	 <para>
		Any time you add a kickstart within Satellite, you will need to recreate the ISO and re-burn it to an optical disk.
	</para>
	</note>
</step>
</procedure>

For the purposes of fixing this bug, I have commented the procedure out. If it is decided later on to have this procedure visible, please raise a new bug.

LKB

Taking this bug for verification.

This book has now been dropped to translation (RT#75265).
No further updates can be accepted. Please raise a new bug for any changes.
LKB

5.4.1 Satellite books are now available on docs.redhat.com. Please raise a new bug for any issues.

LKB

Reopening, as Cliff found some more instances.

Found and fixed several instances in 5.3 and 5.4 Deployment Guides.

LKB

Requested respin in RT#114048

LKB