Red Hat Bugzilla – Bug 924008
Unknown binary attributes can cause migration to fail
Last modified: 2015-03-05 05:09:08 EST
This bug is created as a clone of upstream ticket: https://fedorahosted.org/freeipa/ticket/3469 From an old upstream thread https://www.redhat.com/archives/freeipa-users/2012-January/msg00197.html When we query the remote LDAP server those are are processed into an internal IPA entry, thus passes through our schema parser and converted into the appropriate data type. The default type is unicode. If the incoming attribute is binary we fail completely. The blacklist options don't help because those are processed after we have the entries. Simo had a suggestion where we query only for and migrate certain attributes.
Fixed upstream in scope of ticket 2660 (commit ids are in the ticket): https://fedorahosted.org/freeipa/ticket/2660 Moving to POST.
Steps to verify are in the tkt - https://fedorahosted.org/freeipa/ticket/3469
Pasting the reproduction steps in the Bugzilla: Rob's reproduction steps: 1) Install IPA box A 2) Add new schema, defining an octetstring attributetype and an objectclass with it as MAY 3) Add a user with --addattr objectclass=<new> 4) Stuff in a binary value for it using ldapmodify. I used a Certificate. 5) Install IPA on box B 6) Try to migrate from A Jan Cholasta's reproduction steps: I have also tested a scenario where a directory string attribute of a user contains invalid UTF-8 value, which I think matches the ticket description better. In this case, migration to IPA without my patches fails miserably with an internal error. Migration to IPA with my patches (plus a small patch to use my new code in LDAPClient.add_entry as well) fails with an invalid syntax error, but only for the single user (overall it succeeds), which is IMHO far better result.
Rob, can you add steps to reproduce this as a downstream customer migrating from a directory server to IPA. Migrating from IPA to IPA is not a valid migration path. One would set up a replica ;-) I tried migration with a binary jpegPhoto and the user and jpegPhoto attribute migrated fine from 389-ds-base.
IPA->IPA migration is perfectly reasonable. Lots of folks have done it for many reasons. I suggested it only because it is trivial to set up. jpegPhoto is already a binary value so we don't interpret the contents. You need to pick a directoryString attribute and stuff binary or UTF-8 into it.
IPA -> IPA migration is very popular because people start with POC and then want to move to production by reinstalling and moving data rather than from starting from scratch or calling current POC deployment a production.
With 389-ds-base migration, I added a binary jpeg to the attribute carLicense (need to turn off syntax checking in order to do this) Migration failed with internal server error modifying entry "cn=Philomena Hazen, ou=People, dc=example,dc=com" :: [ PASS ] :: Adding attribute with binary value to a user (Expected 0, got 0) :: [ 14:50:13 ] :: EXECUTING: echo Secret123 | ipa migrate-ds --user-container="ou=People,dc=example,dc=com" --group-container="ou=groups,dc=example,dc=com" ldap://hp-sl230sgen8-01.testrelm.test:389 ipa: ERROR: an internal error has occurred /var/log/httpd/error_log [Tue Feb 18 14:50:14.137375 2014] [:error] [pid 13173] ipa: ERROR: unable to convert the attribute "carLicense" value " big longlonglong ..... value of attribute in log ......... like [Tue Feb 18 14:50:14.137400 2014] [:error] [pid 13173] [Tue Feb 18 14:50:14.137402 2014] [:error] [pid 13173] \b\x06\x06\t\r\t [Tue Feb 18 14:50:14.137403 2014] [:error] [pid 13173] \v\v\x0c\x0c\x0c\x07\t\r\x0e\r\x0c\x0e\v\x0c\x0c\v\xff\xdb [Tue Feb 18 14:50:14.137405 2014] [:error] [pid 13173] \v\xff\xc4 [Tue Feb 18 14:50:14.137407 2014] [:error] [pid 13173] \x16\x17\x18\x19\x1a%&'()*456789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz\x83\x84\x85\x86\x87\x88\x89\x8a\x92\x93\x94\x95\x96\x97\x98\x99\x9a\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xff\xc4 [Tue Feb 18 14:50:14.137409 2014] [:error] [pid 13173] \v\xff\xc4 [Tue Feb 18 14:50:14.137411 2014] [:error] [pid 13173] \x16$4\xe1%\xf1\x17\x18\x19\x1a&'()*56789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x92\x93\x94\x95\x96\x97\x98\x99\x9a\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xff\xda [Tue Feb 18 14:50:14.137413 2014] [:error] [pid 13173] \xfc\x19\xd6>=\xfcA\xb3\xf0\xe7\x83\xe2\xcc\xf7\x07u\xc4\xe4~\xee\xd6!\xc3;zc\xf5<W\xe9\x9f\xc1\xdf\x81~\x1d\xfd\x99\xfc\x12\xba?\x82c\xdd+(k\xab\xa9\x7f\xd6]I\x8e\xac{\x0fA\xd8W\x96\xfe\xc7\x7f\x0e4\xcf\xd9\xa3\xe1lSx\x85bO\x11\xebq\xad\xcd\xeb\xb6\x03\xc2\xa4eb\xcf\xa0\x07\x9frk\xb5\xf1'\xc7}6t\x1b\xee\xad\xd8s\x95\xdc2k\xe6q\xd8\xb7\x8e\x93\xa7\x07\xee/\xc7\xcc\xfdc\x86\xb0Tr\xcaJ\xa4\xf5\xab/\xc1v_\xa9\xdb/\x88\xe3\x13\xba;>\xec\x02\xb8\xe8O\xbdT\xbe\xf1\x14\x93_"\x02\xa3a\xc9$\xe4\x0f\xc6\xbc\xefL\xf8\xb5i\xa8\xdeyQH\xa7\xe5\xdc\xad\x9e\x0f5\xbfi\xaa\xb5\xf8\x06\xdbg\x078\x1c\xe6\xbc\xf9\xd0QG\xdfa\xa7\xce\xaevwZ\xaf\xda\xb0\xf2\x16v\xce\x0f^jx. [Tue Feb 18 14:50:14.137417 2014] [:error] [pid 13173] \xf9\xb1\xfc\xd8\x18\xed\\7\xecK\xab\xda|G\xf8mer\xa5\x1a\xf2\xcdE\xbd\xc2\x0e\xa1\xd4`\x12=\b\xaf\xa5\xf4\r\x11c@J No users or groups were migrated
Honza, AIUI previously this would blow up the entire migration process and your changes are such that it only prevents the affect user(s) from being migrated. Is this correct? And I guess if so, then what Jenny found is expected behavior.
It does blow up ... internal server error and no users or groups are migrated - there are other users and groups that were not migrated.
My changes prevent internal errors for attributes that are not touched anywhere in the code. For attributes that *are* touched somewhere in the code it will still blow up if there's invalid data, which is the case with carLicense above. I'm not sure where exactly it is touched, but if it wasn't, the framework would not try to decode it and would not blow up.
Anyway, the changes were done in <https://fedorahosted.org/freeipa/ticket/3521>, not <https://fedorahosted.org/freeipa/ticket/2660>, and will be released with 3.4, so the crash is expected on RHEL7.
The upstream ticket associated with this is 3468 ... neither of the tickets mentioned and the fixed in version is ipa-3.2.1-1.el7.
Ok, let us move with this bug forward. Jan, do you have any working reproduction scenario showing what we actually fixed in 7.0? Given that migration crashes even with carlicense which is not specifically touched anywhere in the code, it seems to me that this issue is indeed not fixed properly 7.0. If we do not have a clear reproduction scenario, I'd propose to remove this bug from RHEL-7.0 and target 7.1, where the patches from 3521 would land, as Jan mentioned in Comment 14.
Jenny: See comment 1. Martin: This is *not* fixed in 7.0, there is no reproduction scenario. In 7.1, the invalid carLicense scenario should work. (I have checked that it works with current development version of IPA 3.4.)
That's what I thought. Jenny, are you OK with me moving this bug to 7.1?
Hey Martin, Yes I am okay with moving it to RHEL 7.1. It should not block RHEL 7. Thanks Jenny
I would like to finish automating regression test for this, but I need to know what the expected behavior is .. [1] no crash, migration completes successfully [2] User with the binary data is not migrated? message in error_log explaining why [3] Output from migration states that this user was not migrated and to check the error_log for details [4] All other users and groups are migrated successfully Does this sound reasonable?
Moving to 7.1. I also linked this bug to FreeIPA ticket 3521 as it seems more appropriate. This ticket was already pushed, so moving to POST. Honza, please help Jenny with her question in Comment 20.
This is the result of migrate-ds when migrating from one IPA server with users test1 and test2, where test2 has invalid data in carLicense, to another IPA server: ----------- migrate-ds: ----------- Migrated: user: test1 Failed user: admin: Constraint violation: invalid password syntax - passwords with storage scheme are not allowed test2: carLicense: value #0 invalid per syntax: Invalid syntax. Failed group: admins: This entry already exists. Check GID of the existing group. Use --group-overwrite-gid option to overwrite the GID editors: This entry already exists. Check GID of the existing group. Use --group-overwrite-gid option to overwrite the GID ipausers: This entry already exists trust admins: This entry already exists ---------- Passwords have been migrated in pre-hashed format. IPA is unable to generate Kerberos keys unless provided with clear text passwords. All migrated users need to login at https://your.domain/ipa/migration/ before they can use their Kerberos accounts.
Looks good. Moving to POST as the code is already checked in upstream.
Verified. IPA Version: ----------- ipa-server-4.1.0-13.el7 Snip from automation log: ------------------------- :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: [ LOG ] :: ipa-migrate-bugzilla-011: bz924008 Unknown binary attributes can cause migration to fail :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: [ LOG ] :: Turn off syntax checking on the directory server in order to add binary value to a string defined attribute :: [ PASS ] :: Shutdown the directory server on the client (Expected 0, got 0) :: [ PASS ] :: Edit the dse.ldif file (Expected 0, got 0) :: [ PASS ] :: Start the directory server on the client (Expected 0, got 0) :: [ LOG ] :: EXECUTING: /usr/bin/ldapsearch -x -h qe-blade-04.testrelm.test -p 389 -D "cn=Directory Manager" -w xxxxxxxx -b "cn=config" | grep nsslapd-syntaxcheck :: [ PASS ] :: Adding attribute with binary value to a user (Expected 0, got 0) :: [ LOG ] :: EXECUTING: echo xxxxxxxx | ipa migrate-ds --user-container="ou=People,dc=example,dc=com" --group-container="ou=groups,dc=example,dc=com" --with-compat ldap://qe-blade-04.testrelm.test:389 :: [ PASS ] :: Migrate users and groups (Expected 0, got 0) :: [ PASS ] :: Verifying puser1 was migrated (Expected 0, got 0) :: [ PASS ] :: Verifying 'puser2' was migrated (Expected 0, got 0) :: [ PASS ] :: Verifying 'philomena_hazen' was NOT migrated (Expected 2, got 2) :: [ PASS ] :: Verifying group 'group1' was migrated (Expected 0, got 0) :: [ PASS ] :: Verifying group 'group2' was migrated (Expected 0, got 0) :: [ LOG ] :: Cleaning up migrated users :: [ LOG ] :: Turn syntax checking back on for the directory server :: [ PASS ] :: Shutdown the directory server on the client (Expected 0, got 0) :: [ PASS ] :: Edit the dse.ldif file (Expected 0, got 0) :: [ PASS ] :: Start the directory server on the client (Expected 0, got 0) :: [ PASS ] :: Remove attribute with binary value from user (Expected 0, got 0) :: [ LOG ] :: EXECUTING: /usr/bin/ldapsearch -x -h qe-blade-04.testrelm.test -p 389 -D "cn=Directory Manager" -w xxxxxxxx -b "cn=config" | grep nsslapd-syntaxcheck :: [ LOG ] :: Duration: 29s :: [ LOG ] :: Assertions: 14 good, 0 bad :: [ PASS ] :: RESULT: ipa-migrate-bugzilla-011: bz924008 Unknown binary attributes can cause migration to fail ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-0442.html