Description of problem: EC2 has introduced a new AMI copy feature that allows for copying of AMIs between regions: "We are excited to announce the immediate availability of a new feature: Amazon Machine Image (AMI) Copy. AMI Copy enables you to copy your AMIs across AWS regions, thus making it easier for you to leverage multiple AWS regions and accelerate your geographical expansion and help increase application performance and availability. To use AMI Copy, simply select the AMI to be copied from within the AWS Management Console, choose the destination region, and start the copy. AMI Copy can also be accessed via the EC2 Command Line Interface or EC2 API as described in the EC2 User Guide. Once the copy is complete, the new AMI can be used to launch new EC2 instances in the destination region." If you boot an instance and then rebundle it to a new AMI, that new AMI is broken if you copy it to a new region, because it has the old region written in the yum repo config.
commit 20b326119a0d9b7721f2603301ca3af7b6fb470e Author: Wes Hayutin <whayutin> Date: Thu Apr 18 09:21:53 2013 -0400 bugzilla 921116 partial fix for ami copy issue, sets the region dynamically. Need to update the names of repos in .repo file
commit 8a725be1a8e965a9e0fbf2a628f38f9bbf5222e1 Author: wes hayutin <whayutin> Date: Fri Apr 19 12:17:36 2013 -0400 bugzilla 921116, had to add a fix to dynamically set the load balancer diff --git a/rel-eng/rhui-2.0/amazon/client/usr/lib/yum-plugins/amazon-id.py b/rel-eng/rhui-2.0/amazon/client/usr/lib/yum-plug index ecb6d37..8180e5d 100644 --- a/rel-eng/rhui-2.0/amazon/client/usr/lib/yum-plugins/amazon-id.py +++ b/rel-eng/rhui-2.0/amazon/client/usr/lib/yum-plugins/amazon-id.py @@ -15,7 +15,7 @@ import base64 import urllib2 - +import json from urllib2 import ProxyHandler from yum.plugins import TYPE_CORE @@ -63,11 +63,12 @@ def init_hook(conduit): # bz#921116, copy ami to another region # Setting repo.mirrorlist causes traceback, setting repo.url instead for repo in rhui_repos: + original = repo.mirrorlist region_new = json.loads(_load_id())["region"] - start = repo.mirrorlist.find(".") + 1 - end = repo.mirrorlist.find(".", start) - region_old = repo.mirrorlist[start:end] - repo.url = repo.mirrorlist.replace(region_old, region_new) + start = original.find(".") + 1 + end = original.find(".", start) + region_old = original[start:end] + repo.mirrorlist = original.replace(region_old, region_new).encode('ascii') def _load_id(): ''' diff --git a/rel-eng/rhui-2.0/amazon/client/usr/lib/yum-plugins/rhui-lb.py b/rel-eng/rhui-2.0/amazon/client/usr/lib/yum-plugin index 14834bd..ab3ad00 100644 --- a/rel-eng/rhui-2.0/amazon/client/usr/lib/yum-plugins/rhui-lb.py +++ b/rel-eng/rhui-2.0/amazon/client/usr/lib/yum-plugins/rhui-lb.py @@ -12,6 +12,8 @@ # http://www.gnu.org/licenses/old-licenses/gpl-2.0.txt. import httplib +import json +import urllib2 import urlparse from urlgrabber import grabber @@ -21,6 +23,8 @@ from yum.plugins import TYPE_CORE, PluginYumExit requires_api_version = '2.5' plugin_type = (TYPE_CORE,) +ID_DOC_URL = "http://169.254.169.254/latest/dynamic/instance-identity/document" + def postreposetup_hook(conduit): """ @@ -58,6 +62,18 @@ def postreposetup_hook(conduit): up_lb = None balancers_tried = 0 for lb in cds_balancers: + + # Dynamically set the cds load balancer to the appropriate region + # bz#921116, copy ami to another region + original = lb + region_new = json.loads(_load_id())["region"] + start = original.find(".") + 1 + end = original.find(".", start) + region_old = original[start:end] + new = original.replace(region_old, region_new).encode('ascii') + lb = new + +
To change the repo names.. a customer could do the following.. [root@ip-10-197-31-139 ~]# yum search rh-amazon-rhui-client Loaded plugins: amazon-id, rhui-lb, security rhui-ap-southeast-2-client-config-server-6 | 2.6 kB 00:00 rhui-ap-southeast-2-client-config-server-6/primary_db | 2.5 kB 00:00 rhui-ap-southeast-2-rhel-server-releases | 3.7 kB 00:00 rhui-ap-southeast-2-rhel-server-releases/primary_db | 20 MB 00:04 rhui-ap-southeast-2-rhel-server-releases-optional | 3.5 kB 00:00 rhui-ap-southeast-2-rhel-server-releases-optional/primary_db | 2.4 MB 00:02 ============================================= N/S Matched: rh-amazon-rhui-client ============================================= rh-amazon-rhui-client.noarch : Yum repository and entitlement certificate configuration Name and summary matches only, use "search all" for everything. [root@ip-10-197-31-139 ~]# yumdownloader rh-amazon-rhui-client.noarch Loaded plugins: amazon-id, rhui-lb rh-amazon-rhui-client-2.2.79-1.el6_3.noarch.rpm | 41 kB 00:00 [root@ip-10-197-31-139 ~]# rpm -e rh-amazon-rhui-client --nodeps warning: /etc/yum/pluginconf.d/rhui-lb.conf saved as /etc/yum/pluginconf.d/rhui-lb.conf.rpmsave warning: /etc/yum.repos.d/rhui-load-balancers.conf saved as /etc/yum.repos.d/rhui-load-balancers.conf.rpmsave warning: /etc/yum.repos.d/redhat-rhui.repo saved as /etc/yum.repos.d/redhat-rhui.repo.rpmsave warning: /etc/yum.repos.d/redhat-rhui-client-config.repo saved as /etc/yum.repos.d/redhat-rhui-client-config.repo.rpmsave [root@ip-10-197-31-139 ~]# rpm -Uvh rh-amazon-rhui-client-2.2.79-1.el6_3.noarch.rpm Preparing... ########################################### [100%] 1:rh-amazon-rhui-client ########################################### [100%] [INFO:choose_repo] choose_repo:36 2013-04-19 12:30:20,241: Zone [us-west-1c] [INFO:choose_repo] choose_repo:47 2013-04-19 12:30:20,241: Setting region in redhat-rhui.repo [INFO:choose_repo] choose_repo:49 2013-04-19 12:30:20,241: Executing [sed -i 's/REGION/us-west-1/' /etc/yum.repos.d/redhat-rhui.repo] [INFO:choose_repo] choose_repo:53 2013-04-19 12:30:20,263: Enabling binary repos in redhat-rhui.repo [INFO:choose_repo] choose_repo:72 2013-04-19 12:30:20,265: Enabling load balancer plugin [INFO:choose_repo] choose_repo:74 2013-04-19 12:30:20,265: Executing [sed -i 's/enabled=0/enabled=1/' /etc/yum/pluginconf.d/rhui-lb.conf] [INFO:choose_repo] choose_repo:78 2013-04-19 12:30:20,273: Setting region in load balancer config [INFO:choose_repo] choose_repo:80 2013-04-19 12:30:20,274: Executing [sed -i 's/REGION/us-west-1/' /etc/yum.repos.d/rhui-load-balancers.conf] [INFO:choose_repo] choose_repo:84 2013-04-19 12:30:20,282: Enabling client config repo [INFO:choose_repo] choose_repo:87 2013-04-19 12:30:20,283: Executing [sed -i 's/enabled=0/enabled=1/' /etc/yum.repos.d/redhat-rhui-client-config.repo] [INFO:choose_repo] choose_repo:91 2013-04-19 12:30:20,290: Setting region in redhat-rhui-client-config.repo [INFO:choose_repo] choose_repo:93 2013-04-19 12:30:20,291: Executing [sed -i 's/REGION/us-west-1/' /etc/yum.repos.d/redhat-rhui-client-config.repo] [root@ip-10-197-31-139 ~]# yum repolist Loaded plugins: amazon-id, rhui-lb, security rhui-us-west-1-client-config-server-6 | 2.6 kB 00:00 rhui-us-west-1-client-config-server-6/primary_db | 2.5 kB 00:00 rhui-us-west-1-rhel-server-releases | 3.7 kB 00:00 rhui-us-west-1-rhel-server-releases/primary_db | 20 MB 00:00 rhui-us-west-1-rhel-server-releases-optional | 3.5 kB 00:00 rhui-us-west-1-rhel-server-releases-optional/primary_db | 2.4 MB 00:00 repo id repo name status rhui-us-west-1-client-config-server-6 Red Hat Update Infrastructure 2.0 Client Configuration Server 6 1 rhui-us-west-1-rhel-server-releases Red Hat Enterprise Linux Server 6 (RPMs) 10,365 rhui-us-west-1-rhel-server-releases-optional Red Hat Enterprise Linux Server 6 Optional (RPMs) 6,074 repolist: 16,440 [root@ip-10-197-31-139 ~]#
set of changes for this issue commit b3244ab25430787b2162c6045e5a78e4c7269888 Author: Wes Hayutin <whayutin> Date: Mon Apr 22 09:04:39 2013 -0400 move region change into postreposetup_hook commit a45ce378c3daafa74c7bfc83491e2d9f7b7e9f3c Author: wes hayutin <whayutin> Date: Sat Apr 20 13:28:33 2013 -0400 bz#921116, added explanation in each repo file for 'REGION' appearing in the repo name and url commit 12cd4f2727c6ac471db9ddb9abb0d62f8e98e2b6 Author: wes hayutin <whayutin> Date: Sat Apr 20 13:26:34 2013 -0400 add level 5 debug logging to dynamic region setting commit 0800f18969a9b9ef6bbdb5f59cbe2179960dfb76 Author: wes hayutin <whayutin> Date: Sat Apr 20 12:36:44 2013 -0400 bz#921116, do not set the region name in the repo files, leave it as REGION so it is consistent across regions. commit 371a6009a62c7cab4266e5b3d3a06dd923a39ee1 Author: wes hayutin <whayutin> Date: Sat Apr 20 12:35:41 2013 -0400 only change the region if it doesn't match the region in the id document commit 8a725be1a8e965a9e0fbf2a628f38f9bbf5222e1 Author: wes hayutin <whayutin> Date: Fri Apr 19 12:17:36 2013 -0400 bugzilla 921116, had to add a fix to dynamically set the load balancer commit 20b326119a0d9b7721f2603301ca3af7b6fb470e Author: Wes Hayutin <whayutin> Date: Thu Apr 18 09:21:53 2013 -0400 bugzilla 921116 partial fix for ami copy issue, sets the region dynamically. Need to update the names of repos in .repo files
Hey, applying the patch on RHEL 5.6 ami-2eb1825a I've got quite an unforeseen stack; rhui plugins can't be loaded... [mkovacik@localhost rhui-client]$ ssh -i ~/.pem/mkovacik-eu-west-1.pem root.compute.amazonaws.com [root@ip-10-59-142-183 ~]# rpm -Uvh /tmp/rh-amazon-rhui-client-2.2.81-1.el5.noarch.rpm Preparing... ########################################### [100%] 1:rh-amazon-rhui-client warning: /etc/yum/pluginconf.d/rhui-lb.conf saved as /etc/yum/pluginconf.d/rhui-lb.conf.rpmsave ########################################### [100%] [INFO:choose_repo] choose_repo:36 2013-04-24 09:56:06,835: Zone [eu-west-1c] [INFO:choose_repo] choose_repo:57 2013-04-24 09:56:06,836: Enabling binary repos in redhat-rhui.repo [INFO:choose_repo] choose_repo:76 2013-04-24 09:56:06,836: Enabling load balancer plugin [INFO:choose_repo] choose_repo:78 2013-04-24 09:56:06,837: Executing [sed -i 's/enabled=0/enabled=1/' /etc/yum/pluginconf.d/rhui-lb.conf] [INFO:choose_repo] choose_repo:82 2013-04-24 09:56:06,843: Setting region in load balancer config [INFO:choose_repo] choose_repo:84 2013-04-24 09:56:06,843: Executing [sed -i 's/REGION/eu-west-1/' /etc/yum.repos.d/rhui-load-balancers.conf] [INFO:choose_repo] choose_repo:88 2013-04-24 09:56:06,848: Enabling client config repo [INFO:choose_repo] choose_repo:91 2013-04-24 09:56:06,849: Executing [sed -i 's/enabled=0/enabled=1/' /etc/yum.repos.d/redhat-rhui-client-config.repo] warning: /etc/yum.repos.d/redhat-eu-west.repo saved as /etc/yum.repos.d/redhat-eu-west.repo.rpmsave warning: /etc/yum.repos.d/redhat-eu-west-client-config.repo saved as /etc/yum.repos.d/redhat-eu-west-client-config.repo.rpmsave [root@ip-10-59-142-183 ~]# yum clean all ; yum -d 5 repolist -v Plugin "amazon-id" can't be imported Plugin "rhui-lb" can't be imported Loaded plugins: fastestmirror, security Cleaning up Everything Plugin "amazon-id" can't be imported Loading "fastestmirror" plugin Not loading "rhnplugin" plugin, as it is disabled Plugin "rhui-lb" can't be imported Loading "security" plugin Config time: 0.170 Yum Version: 3.2.22 Determining fastest mirrors Could not retrieve mirrorlist https://rhui2-cds01.REGION.aws.ce.redhat.com/pulp/mirror/rhui-client-config/rhel/server/5/x86_64/os error was [Errno 4] IOError: <urlopen error (-2, 'Name or service not known')> Error: Cannot find a valid baseurl for repo: rhui-REGION-client-config-server-5 [root@ip-10-59-142-183 ~]# cat /etc/yum.repos.d/rhui-load-balancers.conf rhui2-cds01.eu-west-1.aws.ce.redhat.com rhui2-cds02.eu-west-1.aws.ce.redhat.com
Created attachment 739757 [details] Screen log, RHEL 5.5, rh-amazon-rhui-client, build 83-1 Works OK for RHEL 5.5
Created attachment 739920 [details] Screen log, RHEL 5.5, us-east-1 i.e. oldest possible, round-trip, rh-amazon-rhui-client, build 83-1 Works OK
Created attachment 744645 [details] jbeap-ami-c9d600a0.log
Created attachment 744646 [details] mrg-ami-fcdc4195.log
Created attachment 744647 [details] vsa-ami-c9d600a0.log
Created attachment 744695 [details] jbews-ami-c61ebdaf.log
We believe this is fixed. If you are still seeing issues, please feel free to re-open.