Bug 1449418 - UnicodeEncodeError error on CapsuleGenerateAndSync task when provided custom CA has non-unicode characters
Summary: UnicodeEncodeError error on CapsuleGenerateAndSync task when provided custom ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Pulp
Version: 6.2.8
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: Unspecified
Assignee: Tomer Brisker
QA Contact: jcallaha
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-05-09 22:32 UTC by Paul Dudley
Modified: 2021-04-06 17:50 UTC (History)
24 users (show)

Fixed In Version: tfm-rubygem-katello-3.4.4,pulp-2.13.4.12-1,katello-installer-base-3.4.5.33-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-08-22 20:07:08 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 21345 0 None None None 2017-10-16 14:02:42 UTC
Pulp Redmine 2960 0 Normal CLOSED - CURRENTRELEASE UnicodeEncodeError in case of a non-ASCII character in comments provided with SSL cert/key/CA 2018-01-06 20:31:54 UTC
Pulp Redmine 3253 0 Normal CLOSED - CURRENTRELEASE UnicodeDecodeError in case of a non-ASCII character in comments provided with SSL cert/key/CA 2018-09-18 18:32:56 UTC
Red Hat Knowledge Base (Solution) 3391131 0 None None None 2018-03-23 17:53:41 UTC

Description Paul Dudley 2017-05-09 22:32:37 UTC
CapsuleGenerateAndSync task errors with:

[E] PLP0000: 'ascii' codec can't encode character u'\xe9' in position 76773: ordinal not in range(128) (Katello::Errors::PulpError)

At the moment this is affecting the following repository, along with seneravl others:
- Red_Hat_Virtualization-Red_Hat_Enterprise_Virtualization_Manager_3_6_RPMs_x86_64

The repository was successfully synced on the Satellite several hours earlier.

Comment 1 Paul Dudley 2017-05-09 22:33:00 UTC
The full traceback:

      File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
        R = retval = fun(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in __call__
        return super(Task, self).__call__(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in __call__
        return super(PulpTask, self).__call__(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
        return self.run(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 762, in sync
        sync_report = sync_repo(transfer_repo, conduit, call_config)
      File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 658, in wrap_f
        return f(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 80, in sync_repo
        self._current_sync = sync.RepoSync(repo, sync_conduit, call_config)
      File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 86, in __init__
        self.nectar_config = nectar_utils.importer_config_to_nectar_config(config.flatten())
      File "/usr/lib/python2.7/site-packages/pulp/plugins/util/nectar_config.py", line 97, in importer_config_to_nectar_config
        download_config = DownloaderConfig(**download_config_kwargs)
      File "/usr/lib/python2.7/site-packages/nectar/config.py", line 136, in __init__
        self._process_ssl_settings()
      File "/usr/lib/python2.7/site-packages/nectar/config.py", line 175, in _process_ssl_settings
        os.write(data_arg_os_handle, data_arg_value)
    UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 76773: ordinal not in range(128)

Comment 2 Michael Hrivnak 2017-05-10 15:16:34 UTC
Ah, so it is failing to parse the importer's config! Specifically it seems to be finding a non-ascii character in an SSL certificate. Is that reasonable? It strikes me as unexpected.

Can you access the importer config and attach either the entire config here, or just the SSL certs?

Comment 4 Michael Hrivnak 2017-05-10 16:40:26 UTC
Yes, the whole config would be helpful. Thanks!

Comment 5 Michael Hrivnak 2017-05-10 16:41:21 UTC
Justin, what do you think about this? How would such characters end up in the cert?

Comment 9 Rob Mokkink 2017-07-20 08:40:45 UTC
We implemented the following workaround:

Put the files "/usr/share/pki/ca-trust-source/ca-bundle.*" and  "/usr/share/pki/ca-trust-legacy/ca-bundle.*" in "/etc/pki/ca-trust/source/blacklist/". Remove the files "/etc/pki/ca-trust/source/anchors/katello-server*" and run command "update-ca-trust". Then runthe satellite installer with the following options:

   satellite-installer --scenario satellite \
    --certs-server-cert  /etc/pki/tls/certs/host.cert \
    --certs-server-cert-req  /etc/pki/tls/private_new/host.csr \
    --certs-server-key  /etc/pki/tls/private/host.key \
    --certs-server-ca-cert /etc/pki/tls/certs/ca-bundle.crt \
    --certs-update-server --certs-update-server-ca


Our own rootca's do not contain any characters like "é" or "ö".

Comment 10 pm-sat@redhat.com 2017-07-24 14:06:34 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/20307 has been resolved.

Comment 12 pm-sat@redhat.com 2017-08-03 22:08:04 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/20307 has been resolved.

Comment 13 Michael Hrivnak 2017-08-07 18:07:20 UTC
Tanya and Patrick, could you make an upstream bug to track this? It seems to be failing to write the cert contents out to disk.

Comment 14 pulp-infra@redhat.com 2017-08-07 19:01:37 UTC
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.

Comment 15 pulp-infra@redhat.com 2017-08-07 19:01:42 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 16 David Davis 2017-08-11 15:35:00 UTC
Removing the pulp redmine issue as its status is independent of this BZ.

Comment 19 pulp-infra@redhat.com 2017-08-29 13:01:18 UTC
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.

Comment 20 pulp-infra@redhat.com 2017-08-29 13:01:23 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 21 Tanya Tereshchenko 2017-08-29 14:21:07 UTC
Move to NEW, since Pulp upstream bug is not fixed yet.

Comment 23 pulp-infra@redhat.com 2017-09-01 15:31:20 UTC
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

Comment 24 pulp-infra@redhat.com 2017-09-01 21:31:26 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 25 Chris Roberts 2017-09-06 19:38:21 UTC
Waiting for this to be merged:

https://github.com/pulp/nectar/pull/61

Comment 26 pulp-infra@redhat.com 2017-09-06 21:01:53 UTC
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

Comment 27 pulp-infra@redhat.com 2017-09-08 14:01:24 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 29 pm-sat@redhat.com 2017-09-21 18:06:33 UTC
Upstream bug assigned to chrobert@redhat.com

Comment 30 pm-sat@redhat.com 2017-09-21 20:06:30 UTC
Upstream bug assigned to chrobert@redhat.com

Comment 31 pm-sat@redhat.com 2017-09-21 22:06:29 UTC
Upstream bug assigned to chrobert@redhat.com

Comment 32 pulp-infra@redhat.com 2017-09-22 03:31:43 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 33 pulp-infra@redhat.com 2017-09-22 03:31:48 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 37 pulp-infra@redhat.com 2017-10-04 17:34:02 UTC
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 38 pulp-infra@redhat.com 2017-10-04 18:03:10 UTC
Requesting needsinfo from upstream developer mansari@redhat.com because the 'FailedQA' flag is set.

Comment 39 Tanya Tereshchenko 2017-10-04 18:30:26 UTC
Upstream bug is fixed and merged, moving to POST

Comment 40 pulp-infra@redhat.com 2017-10-04 18:32:11 UTC
All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST.

Comment 45 pulp-infra@redhat.com 2017-10-19 14:02:18 UTC
The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug.

Comment 46 pulp-infra@redhat.com 2017-10-27 00:02:07 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 48 pm-sat@redhat.com 2017-11-16 09:07:38 UTC
Upstream bug assigned to tbrisker@redhat.com

Comment 49 pulp-infra@redhat.com 2017-11-16 09:48:35 UTC
Requesting needsinfo from upstream developer mansari@redhat.com because the 'FailedQA' flag is set.

Comment 50 pm-sat@redhat.com 2017-11-16 11:08:29 UTC
Upstream bug assigned to tbrisker@redhat.com

Comment 51 pulp-infra@redhat.com 2017-11-28 22:39:28 UTC
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

Comment 52 Ammar Ansari 2017-11-28 22:54:17 UTC
Waiting for this to be merged:

https://github.com/pulp/pulp/pull/3227

Comment 53 pulp-infra@redhat.com 2017-11-29 01:03:03 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 54 pulp-infra@redhat.com 2018-01-06 20:31:56 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 55 pulp-infra@redhat.com 2018-01-06 21:01:44 UTC
Requesting needsinfo from upstream developer mansari@redhat.com because the 'FailedQA' flag is set.

Comment 56 pulp-infra@redhat.com 2018-01-08 14:31:32 UTC
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

Comment 57 pulp-infra@redhat.com 2018-01-08 14:31:37 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 58 pulp-infra@redhat.com 2018-01-09 12:01:34 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 59 Ammar Ansari 2018-01-19 00:28:05 UTC
Waiting for a final review by a core dev on this:

https://github.com/pulp/pulp/pull/3227

Comment 60 pulp-infra@redhat.com 2018-01-19 00:31:59 UTC
Requesting needsinfo from upstream developer mansari@redhat.com because the 'FailedQA' flag is set.

Comment 61 pulp-infra@redhat.com 2018-03-13 19:01:51 UTC
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 62 pulp-infra@redhat.com 2018-03-13 22:31:56 UTC
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

Comment 63 pulp-infra@redhat.com 2018-03-13 23:31:56 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 65 pulp-infra@redhat.com 2018-05-04 07:02:58 UTC
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 76 jcallaha 2018-08-15 21:02:40 UTC
Verified in Satellite 6.3.3 Snap 3

I used https://github.com/iNecas/ownca.git to create custom certs for my satellite and capsule. In those, I used characters from the Greek alphabet.

The newly update katello-certs-check didn't immediately reject the certs due to the presence of non-ascii characters.

Then applying those certs to the satellite and capsule also worked.

Finally, the Capsule was able to successfully sync all content from the Satellite.

Comment 80 errata-xmlrpc 2018-08-22 20:07:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2550

Comment 81 pulp-infra@redhat.com 2018-09-18 18:32:57 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.


Note You need to log in before you can comment on or make changes to this bug.