Bug 719083 - yum client operations fails, if the mirrorlist url is not FQDN
Summary: yum client operations fails, if the mirrorlist url is not FQDN
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Red Hat Update Infrastructure for Cloud Providers
Classification: Red Hat
Component: Documentation
Version: 2.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Lana Brindley
QA Contact: wes hayutin
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-07-05 17:52 UTC by Kedar Bidarkar
Modified: 2016-02-18 06:14 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-07-20 22:26:25 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Kedar Bidarkar 2011-07-05 17:52:20 UTC
Description of problem:

[root@ip-10-46-81-181 yum.repos.d]# yum install zsh 
Loaded plugins: rhui-lb, security
Could not retrieve mirrorlist https://ip-10-90-217-118/pulp/mirror/content/dist/rhel/rhui/server-6/releases/6Server/x86_64/os error was
14: PYCURL ERROR 51 - ""
Error: Cannot retrieve repository metadata (repomd.xml) for repository: rhui-rhel-server-6-releases. Please verify its path and try again


Version-Release number of selected component (if applicable):
pulp - 201 and rhui - 2.0.34

How reproducible:
client config rpms, use only the hostname, provided during the CDS registration.


Steps to Reproduce:
1.
2.
3.
  
Actual results:

In rh-cloud.repo, the mirrorlist url is

mirrorlist=https://ip-10-90-217-118/pulp/mirror/content/dist/rhel/rhui/server-6/releases/$releasever/$basearch/os

Expected results:

mirrorlist=https://ip-10-90-217-118.ec2.internal/pulp/mirror/content/dist/rhel/rhui/server-6/releases/$releasever/$basearch/os



Additional info:

Comment 1 Kedar Bidarkar 2011-07-05 17:59:03 UTC
yum operations are successful only when the following things are done.

1) Disable the rhui-lb plugin in rhui-lb.conf file.
2) Using the baseurl instead of the mirrorlist in rh-cloud.repo file.

baseurl=https://ip-10-90-217-118.ec2.internal/pulp/repos/content/dist/rhel/rhui/server-6/releases/$releasever/$basearch/os


[root@ip-10-46-81-181 yum.repos.d]# cat /etc/yum/pluginconf.d/rhui-lb.conf
[main]

enabled=0
cds_list_file=/etc/yum.repos.d/rhui-load-balancers
[root@ip-10-46-81-181 yum.repos.d]# cat rh-cloud.repo 
[rhui-rhel-server-6-releases]
name=Red Hat Enterprise Linux Server 6 Releases (RPMs)
#mirrorlist=https://ip-10-90-217-118/pulp/mirror/content/dist/rhel/rhui/server-6/releases/$releasever/$basearch/os
baseurl=https://ip-10-90-217-118.ec2.internal/pulp/repos/content/dist/rhel/rhui/server-6/releases/$releasever/$basearch/os
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
sslverify=1
sslcacert=/etc/pki/entitlement/ca.crt
sslclientcert=/etc/pki/entitlement/product/content.crt
sslclientkey=/etc/pki/entitlement/key.pem


YUM operations are successful only due to above things.

[root@ip-10-46-81-181 yum.repos.d]# yum clean all
Loaded plugins: security
Cleaning up Everything
[root@ip-10-46-81-181 yum.repos.d]# yum install zsh 
Loaded plugins: security
Could not retrieve mirrorlist https://ip-10-90-217-118/pulp/mirror/content/dist/rhel/rhui/server-6/releases/6Server/x86_64/os error was
14: PYCURL ERROR 51 - ""
Error: Cannot retrieve repository metadata (repomd.xml) for repository: rhui-rhel-server-6-releases. Please verify its path and try again
[root@ip-10-46-81-181 yum.repos.d]# vi rh-cloud.repo 
[root@ip-10-46-81-181 yum.repos.d]# yum install zsh 
Loaded plugins: security
rhui-rhel-server-6-releases                              | 3.7 kB     00:00     
rhui-rhel-server-6-releases/primary_db                   | 2.9 MB     00:00     
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package zsh.x86_64 0:4.3.10-4.1.el6 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package Arch       Version               Repository                       Size
================================================================================
Installing:
 zsh     x86_64     4.3.10-4.1.el6        rhui-rhel-server-6-releases     2.1 M

Transaction Summary
================================================================================
Install       1 Package(s)
Upgrade       0 Package(s)

Total download size: 2.1 M
Installed size: 4.8 M
Is this ok [y/N]: y
Downloading Packages:
zsh-4.3.10-4.1.el6.x86_64.rpm                            | 2.1 MB     00:00     
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
  Installing     : zsh-4.3.10-4.1.el6.x86_64                                                                                                                                 1/1 

Installed:
  zsh.x86_64 0:4.3.10-4.1.el6                                                                                                                                                    

Complete!

Comment 2 Kedar Bidarkar 2011-07-05 18:05:13 UTC
Another few things to take in to account and note.

1) for local vm's, (hostname provided by local RedHat dhcp server )

hostname command output's the following:

[root@dhcp201-188 ~]# hostname
dhcp201-188.englab.pnq.redhat.com

2) for amazon instances

hostname command output's the following:

[root@ip-10-122-106-61 noarch]# hostname
ip-10-122-106-61




[root@ip-10-122-106-61 noarch]# nslookup ip-10-122-106-61
Server:		172.16.0.23
Address:	172.16.0.23#53

Non-authoritative answer:
Name:	ip-10-122-106-61.ec2.internal
Address: 10.122.106.61

Comment 3 Kedar Bidarkar 2011-07-05 18:21:31 UTC
Because the local vm's are being used extensively than the amazon instances for testing, we didn't face this issue till now, unless tried on amazon instances.


For local vm's we provide, FQDN during CDS registration.
(as hostname command output here is <hostname>.<domainname> )

For Amazon instances we provide, just hostname during CDS registration.
(as hostname command output here is <hostname> )


I feel the RHUI is being inconsistent in different environments, this needs to be addressed.

Also, would like to know that, is it mandatory that all the nodes, RHUA and CDS have to be under one domain or it can be under different domains.

For example : 

RHUA belongs to compute-1.internal domain: domU-12-31-39-13-C8-D1.compute-1.internal

and 

CDS belonging to ec2.internal domain:  ip-10-88-218-7.ec2.internal

Is the above setup supposed to work as, I face CDS sync failure with this kind of setup.

Comment 4 Kedar Bidarkar 2011-07-05 18:24:16 UTC
NOTE : Added appropriate domain names to search for under /etc/resolv.conf

Comment 5 Jay Dobies 2011-07-06 20:56:34 UTC
Ok, I suspect what this is going to turn into is a documentation bug because this probably needs to be explained.

There are two main places that FQDN comes into play:
- The unique identifier for a CDS on the message bus the RHUA uses to talk to them
- Any time an SSL connection is used, which means when CDS instances sync from the RHUA and when clients download content

They are different in their reasons and usage, so I'll explain them separately:

= CDS Hostnames =
When a CDS attaches to the message bus the RHUA uses to communicate with it, it uses its hostname (the result of calling `hostname` on the CDS itself) as the unique identifier on the bus.

When registering a CDS to the RHUI, that same identifier must be used so that the RHUA knows how to contact the CDS. Technically speaking, the RHUA will never need to resolve that name to an IP, it simply needs to use it as the destination on the message bus.

What I'm thinking is that we make sure the user guide mentions the need for the hostname set on the CDS to be the same as what's specified in the register call. I think we have a note on that, but I'm not 100% sure.

= SSL =
While the CDS hostname usage is a Pulp requirement, the rest of the issues are innate to SSL itself. Here's a quick run down.

- A client makes a request to example.com.
- example.com sends its SSL certificate back to the client. That certificate has a "Common Name" as part of its metadata (more commonly referred to as its CN).
- The client gets the certificate and then needs to make a decision.

When I say client, I mean the programmatic client (yum, wget, firefox, etc), not a user per se. I mention that because this is a case of the computer doing what it was told to do, not what the user may have intended.

The client was told to go to example.com. However, in the malicious world of the internet, it's possible that someone has poisoned the DNS cache (among other ways) and the actual server that was accessed isn't example.com.

So if the user told the client to go to example.com, but the server presented a certificate identifying itself as foo.com, the client assumes that something bad is going on and will likely throw an error.

-- Tangent: This is what "sslverify=1" means in a yum repo definition. This error will typically read something about "peer certificate verify failed" or something else not very helpful like that.

Again, this is all basic SSL stuff, nothing weird or fancy we did in RHUI. So it's not something we can really control, nor would we want to change it if we could since it's an established security standard.

That's also why unless you're running an actual DNS server, you have to remember to update /etc/hosts in your environments so that the FQDN -- as expected by the CN of the server's SSL certificate -- resolves to the IP for that server. That's largely an annoyance of testing environments since most production environments will set these instances up such that the FQDNs are configured.

Comment 6 Jay Dobies 2011-07-06 20:58:03 UTC
"Also, would like to know that, is it mandatory that all the nodes, RHUA and CDS
have to be under one domain or it can be under different domains."

I didn't explicitly comment on this, but the answer should hopefully be derivable from what I wrote. In short, the domains are completely arbitrary so long as they are:
- Configured correctly as the hostname of the CDS instance
- Resolvable on both the CDS (to connect back to the RHUA over SSL) and the client (to connect to _all_ CDS instances over SSL)

Comment 7 Jay Dobies 2011-07-06 21:01:54 UTC
I added Lana on the CC list and moved it over to the Install Guide component. Based on the contents of comment #5, I'll leave it to her to decide if and where this should be mentioned in the docs.

I left the bug assigned to me for now. If we decide to munge this into the docs we can reassign it to Lana to track it as a docs bug.

Comment 8 Lana Brindley 2011-07-17 22:51:12 UTC
Taking.

LKB

Comment 9 Lana Brindley 2011-07-19 03:35:26 UTC
We do go to some pains in the RHUI Installer chapter (throughout the SSL section) to mention where FQDN is required, and give examples. I'm not sure we need to explain exactly why this is so. After reading all the comments in this bug, I'm tempted to close it as WORKSFORME at this stage, but I'm happy to go into further discussion about the matter if there's a particular point that needs to be called out in the text that I've missed.

LKB


Note You need to log in before you can comment on or make changes to this bug.