Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1449418 - UnicodeEncodeError error on CapsuleGenerateAndSync task when provided custom CA has non-unicode characters
Summary: UnicodeEncodeError error on CapsuleGenerateAndSync task when provided custom ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Pulp
Version: 6.2.8
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: Unspecified
Assignee: Tomer Brisker
QA Contact: jcallaha
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-05-09 22:32 UTC by Paul Dudley
Modified: 2022-03-13 14:16 UTC (History)
24 users (show)

Fixed In Version: tfm-rubygem-katello-3.4.4,pulp-2.13.4.12-1,katello-installer-base-3.4.5.33-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-08-22 20:07:08 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 21345 0 None None None 2017-10-16 14:02:42 UTC
Pulp Redmine 2960 0 Normal CLOSED - CURRENTRELEASE UnicodeEncodeError in case of a non-ASCII character in comments provided with SSL cert/key/CA 2018-01-06 20:31:54 UTC
Pulp Redmine 3253 0 Normal CLOSED - CURRENTRELEASE UnicodeDecodeError in case of a non-ASCII character in comments provided with SSL cert/key/CA 2018-09-18 18:32:56 UTC
Red Hat Knowledge Base (Solution) 3391131 0 None None None 2018-03-23 17:53:41 UTC

Description Paul Dudley 2017-05-09 22:32:37 UTC 
CapsuleGenerateAndSync task errors with:

[E] PLP0000: 'ascii' codec can't encode character u'\xe9' in position 76773: ordinal not in range(128) (Katello::Errors::PulpError)

At the moment this is affecting the following repository, along with seneravl others:
- Red_Hat_Virtualization-Red_Hat_Enterprise_Virtualization_Manager_3_6_RPMs_x86_64

The repository was successfully synced on the Satellite several hours earlier.
Comment 1 Paul Dudley 2017-05-09 22:33:00 UTC 
The full traceback:

      File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
        R = retval = fun(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in __call__
        return super(Task, self).__call__(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in __call__
        return super(PulpTask, self).__call__(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
        return self.run(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 762, in sync
        sync_report = sync_repo(transfer_repo, conduit, call_config)
      File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 658, in wrap_f
        return f(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 80, in sync_repo
        self._current_sync = sync.RepoSync(repo, sync_conduit, call_config)
      File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 86, in __init__
        self.nectar_config = nectar_utils.importer_config_to_nectar_config(config.flatten())
      File "/usr/lib/python2.7/site-packages/pulp/plugins/util/nectar_config.py", line 97, in importer_config_to_nectar_config
        download_config = DownloaderConfig(**download_config_kwargs)
      File "/usr/lib/python2.7/site-packages/nectar/config.py", line 136, in __init__
        self._process_ssl_settings()
      File "/usr/lib/python2.7/site-packages/nectar/config.py", line 175, in _process_ssl_settings
        os.write(data_arg_os_handle, data_arg_value)
    UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 76773: ordinal not in range(128)
Comment 2 Michael Hrivnak 2017-05-10 15:16:34 UTC 
Ah, so it is failing to parse the importer's config! Specifically it seems to be finding a non-ascii character in an SSL certificate. Is that reasonable? It strikes me as unexpected.

Can you access the importer config and attach either the entire config here, or just the SSL certs?
Comment 4 Michael Hrivnak 2017-05-10 16:40:26 UTC 
Yes, the whole config would be helpful. Thanks!
Comment 5 Michael Hrivnak 2017-05-10 16:41:21 UTC 
Justin, what do you think about this? How would such characters end up in the cert?
Comment 9 Rob Mokkink 2017-07-20 08:40:45 UTC 
We implemented the following workaround:

Put the files "/usr/share/pki/ca-trust-source/ca-bundle.*" and  "/usr/share/pki/ca-trust-legacy/ca-bundle.*" in "/etc/pki/ca-trust/source/blacklist/". Remove the files "/etc/pki/ca-trust/source/anchors/katello-server*" and run command "update-ca-trust". Then runthe satellite installer with the following options:

   satellite-installer --scenario satellite \
    --certs-server-cert  /etc/pki/tls/certs/host.cert \
    --certs-server-cert-req  /etc/pki/tls/private_new/host.csr \
    --certs-server-key  /etc/pki/tls/private/host.key \
    --certs-server-ca-cert /etc/pki/tls/certs/ca-bundle.crt \
    --certs-update-server --certs-update-server-ca


Our own rootca's do not contain any characters like "é" or "ö".
Comment 10 Satellite Program 2017-07-24 14:06:34 UTC 
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/20307 has been resolved.
Comment 12 Satellite Program 2017-08-03 22:08:04 UTC 
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/20307 has been resolved.
Comment 13 Michael Hrivnak 2017-08-07 18:07:20 UTC 
Tanya and Patrick, could you make an upstream bug to track this? It seems to be failing to write the cert contents out to disk.
Comment 14 pulp-infra@redhat.com 2017-08-07 19:01:37 UTC 
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.
Comment 15 pulp-infra@redhat.com 2017-08-07 19:01:42 UTC 
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.
Comment 16 David Davis 2017-08-11 15:35:00 UTC 
Removing the pulp redmine issue as its status is independent of this BZ.
Comment 19 pulp-infra@redhat.com 2017-08-29 13:01:18 UTC 
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.
Comment 20 pulp-infra@redhat.com 2017-08-29 13:01:23 UTC 
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.
Comment 21 Tanya Tereshchenko 2017-08-29 14:21:07 UTC 
Move to NEW, since Pulp upstream bug is not fixed yet.
Comment 23 pulp-infra@redhat.com 2017-09-01 15:31:20 UTC 
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.
Comment 24 pulp-infra@redhat.com 2017-09-01 21:31:26 UTC 
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.
Comment 25 Chris Roberts 2017-09-06 19:38:21 UTC 
Waiting for this to be merged:

https://github.com/pulp/nectar/pull/61
Comment 26 pulp-infra@redhat.com 2017-09-06 21:01:53 UTC 
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.
Comment 27 pulp-infra@redhat.com 2017-09-08 14:01:24 UTC 
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.
Comment 29 Satellite Program 2017-09-21 18:06:33 UTC 
Upstream bug assigned to chrobert
Comment 30 Satellite Program 2017-09-21 20:06:30 UTC 
Upstream bug assigned to chrobert
Comment 31 Satellite Program 2017-09-21 22:06:29 UTC 
Upstream bug assigned to chrobert
Comment 32 pulp-infra@redhat.com 2017-09-22 03:31:43 UTC 
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.
Comment 33 pulp-infra@redhat.com 2017-09-22 03:31:48 UTC 
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.
Comment 37 pulp-infra@redhat.com 2017-10-04 17:34:02 UTC 
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.
Comment 38 pulp-infra@redhat.com 2017-10-04 18:03:10 UTC 
Requesting needsinfo from upstream developer mansari because the 'FailedQA' flag is set.
Comment 39 Tanya Tereshchenko 2017-10-04 18:30:26 UTC 
Upstream bug is fixed and merged, moving to POST
Comment 40 pulp-infra@redhat.com 2017-10-04 18:32:11 UTC 
All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST.
Comment 45 pulp-infra@redhat.com 2017-10-19 14:02:18 UTC 
The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug.
Comment 46 pulp-infra@redhat.com 2017-10-27 00:02:07 UTC 
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.
Comment 48 Satellite Program 2017-11-16 09:07:38 UTC 
Upstream bug assigned to tbrisker
Comment 49 pulp-infra@redhat.com 2017-11-16 09:48:35 UTC 
Requesting needsinfo from upstream developer mansari because the 'FailedQA' flag is set.
Comment 50 Satellite Program 2017-11-16 11:08:29 UTC 
Upstream bug assigned to tbrisker
Comment 51 pulp-infra@redhat.com 2017-11-28 22:39:28 UTC 
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.
Comment 52 Ammar Ansari 2017-11-28 22:54:17 UTC 
Waiting for this to be merged:

https://github.com/pulp/pulp/pull/3227
Comment 53 pulp-infra@redhat.com 2017-11-29 01:03:03 UTC 
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.
Comment 54 pulp-infra@redhat.com 2018-01-06 20:31:56 UTC 
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.
Comment 55 pulp-infra@redhat.com 2018-01-06 21:01:44 UTC 
Requesting needsinfo from upstream developer mansari because the 'FailedQA' flag is set.
Comment 56 pulp-infra@redhat.com 2018-01-08 14:31:32 UTC 
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.
Comment 57 pulp-infra@redhat.com 2018-01-08 14:31:37 UTC 
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.
Comment 58 pulp-infra@redhat.com 2018-01-09 12:01:34 UTC 
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.
Comment 59 Ammar Ansari 2018-01-19 00:28:05 UTC 
Waiting for a final review by a core dev on this:

https://github.com/pulp/pulp/pull/3227
Comment 60 pulp-infra@redhat.com 2018-01-19 00:31:59 UTC 
Requesting needsinfo from upstream developer mansari because the 'FailedQA' flag is set.
Comment 61 pulp-infra@redhat.com 2018-03-13 19:01:51 UTC 
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.
Comment 62 pulp-infra@redhat.com 2018-03-13 22:31:56 UTC 
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.
Comment 63 pulp-infra@redhat.com 2018-03-13 23:31:56 UTC 
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.
Comment 65 pulp-infra@redhat.com 2018-05-04 07:02:58 UTC 
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.
Comment 76 jcallaha 2018-08-15 21:02:40 UTC 
Verified in Satellite 6.3.3 Snap 3

I used https://github.com/iNecas/ownca.git to create custom certs for my satellite and capsule. In those, I used characters from the Greek alphabet.

The newly update katello-certs-check didn't immediately reject the certs due to the presence of non-ascii characters.

Then applying those certs to the satellite and capsule also worked.

Finally, the Capsule was able to successfully sync all content from the Satellite.
Comment 80 errata-xmlrpc 2018-08-22 20:07:08 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2550
Comment 81 pulp-infra@redhat.com 2018-09-18 18:32:57 UTC 
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.
Note You need to log in before you can comment on or make changes to this bug.