Bug 1351737

Summary: glibc: strcasecmp failure on ppc64le
Product: [Fedora] Fedora Reporter: Rafael Fonseca <rdossant>
Component: glibcAssignee: Florian Weimer <fweimer>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: abokovoy, arjun.is, codonell, dan, dj, fweimer, jakub, j, law, mfabian, nalin, npmccallum, pfrankli, rharwood, siddhesh, skumari
Target Milestone: ---   
Target Release: ---   
Hardware: ppc64le   
OS: Unspecified   
Whiteboard:
Fixed In Version: glibc-2.23.90-26.fc25 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-07 10:18:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1071880, 1051573    
Attachments:
Description Flags
krb5 debugging printfs
none
krb5 debugging printfs
none
Output from debugging none

Description Rafael Fonseca 2016-06-30 17:03:53 UTC
Description of problem: kerberos fails to build because of the following error in the tests:

LD_LIBRARY_PATH=`echo -L../../../lib | sed -e "s/-L//g" -e "s/ /:/g"`; export LD_LIBRARY_PATH;  KRB5_CONFIG=../../../config-files/krb5.conf LC_ALL=C  ./t_etypes
Unexpected result while parsing: aes des3-cbc-sha1-kd
Expected: 18 17 16
Result: 18 17
allow_weak_crypto was false


Version-Release number of selected component (if applicable): 1.14.1-8


How reproducible: always on ppc-koji


Additional info: adding "mod_list(ENCTYPE_DES3_CBC_SHA1, sel, weak, &list);" in aes part of krb5int_parse_enctype_list() in init_ctx.c makes the test pass.

Comment 1 Dan Horák 2016-06-30 17:14:26 UTC
link to build in ppc koji - http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3501464

From what I can see it used exactly the same buildroot (same NVRs for packages in the buildroot) as the build in primary koji.

Comment 2 Dan Horák 2016-06-30 17:28:02 UTC
And because krb5-1.14.1-7.fc25 now also fails in the current rawhide buildroot, we should be able to find the differences in the buildroot.

(rawhide, krb5-1.14.1-7.fc25.src.rpm) http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3501356
(f24, krb5-1.14.1-8.fc25.src.rpm) http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3501244
(rawhide, krb5-1.14.1-8.fc25.src.rpm) http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3501128

Comment 3 Robbie Harwood 2016-06-30 17:46:53 UTC
Only difference between f24 and f25/rawhide is the package split, but that came in with -7 in f25/rawhide and built fine at the time.  So something's changed under me, I think...

Package split also built fine on rhel-7's ppc, ppc64, and ppc64le builders.

Comment 4 Rafael Fonseca 2016-06-30 18:40:32 UTC
The problem seems to happen when converting a string to encryption type. For some reason, strcasecmp is not returning the right result.

Check alias 'des3-cbc-sha1-kd' against 'des3-cbc-sha1-kd': 0(cmp) 49(case_cmp)

This is the printf I added in enctype_util.c, just before the checking for the alias. The (cmp) result is returned by strcmp while (case_cmp) comes from strcasecmp.

Comment 5 Sinny Kumari 2016-07-01 12:58:22 UTC
I confirm that this build issue is related to changes added in glibc-2.23.90-24.fc25 package update.

Have done local rebuild of krb5-1.14.1-8.fc25 with glibc versions
glibc-2.23.90-22.fc25, glibc-2.23.90-23.fc25 and glibc-2.23.90-24.fc25 locally with no any additional changes made. krb5 builds successfully with glibc-2.23.90-22.fc25 and glibc-2.23.90-23.fc25 but fails with glibc-2.23.90-24.fc25 .

We suspect that it might be related to ppc64(le) specific changes made with commit https://sourceware.org/git/?p=glibc.git;a=commit;h=c8376f3e07602aaef9cb843bb73cb5f2b860634a in glibc.

Comment 6 Rafael Fonseca 2016-07-04 10:41:29 UTC
Switching component to glibc.

Comment 7 Florian Weimer 2016-07-04 10:46:49 UTC
(In reply to Rafael Fonseca from comment #4)
> The problem seems to happen when converting a string to encryption type. For
> some reason, strcasecmp is not returning the right result.
> 
> Check alias 'des3-cbc-sha1-kd' against 'des3-cbc-sha1-kd': 0(cmp)
> 49(case_cmp)

Is this under the C locale, or some other locale?

Could you please specify the architecture where you saw this?  Fedora no longer supports powerpc (32-bit).

Comment 8 Florian Weimer 2016-07-04 10:48:41 UTC
Okay, based comment 1, it happens at least on ppc64le.

Comment 9 Sinny Kumari 2016-07-04 10:59:30 UTC
(In reply to Florian Weimer from comment #7)
> (In reply to Rafael Fonseca from comment #4)
> > The problem seems to happen when converting a string to encryption type. For
> > some reason, strcasecmp is not returning the right result.
> > 
> > Check alias 'des3-cbc-sha1-kd' against 'des3-cbc-sha1-kd': 0(cmp)
> > 49(case_cmp)
> 
> Is this under the C locale, or some other locale?
> 
> Could you please specify the architecture where you saw this?  Fedora no

It occurs on both pcc64 and ppc64le.
Scratch build for latest krb5 is at http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3501128

Comment 10 Sinny Kumari 2016-07-04 11:11:43 UTC
Did ppc64(le) scratch build of glibc glibc-2.23.90-25.fc25 with reverted commit https://sourceware.org/git/?p=glibc.git;a=commit;h=c8376f3e07602aaef9cb843bb73cb5f2b860634a. Scratch build can be found at http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3509465 . 

Updating local test machine (ppc64 guest) with new glibc scratch build glibc-2.23.90-26 leads to krb5 builds successfully.

Comment 11 Florian Weimer 2016-07-04 11:43:29 UTC
I'm trying to use this test program to reproduce the issue without krb5.  So far, no luck.

#include <string.h>
#include <stdlib.h>
#include <stdbool.h>
#include <malloc.h>
#include <locale.h>

int
main (void)
{
  if (setlocale (LC_ALL, "") == NULL)
    {
      printf ("error: setlocale: %m\n");
      return 1;
    }

  enum { max_align = 128 };
  const char *str = "des3-cbc-sha1-kd";
  size_t blob_size = max_align + strlen (str) + 1;
  char *left = memalign (max_align, blob_size);
  char *right = memalign (max_align, blob_size);
  if (left == NULL || right == NULL)
    {
      printf ("error: out of memory\n");
      return 1;
    }

  bool errors = false;
  for (int left_align = 0; left_align < max_align; ++left_align)
    {
      memset (left, 'L', blob_size);
      strcpy (left + left_align, str);
      for (int right_align = 0; right_align < max_align; ++right_align)
        {
          memset (right, 'R', blob_size);
          strcpy (right + right_align, str);
          if (strcasecmp (left + left_align,
                          right + right_align) != 0)
            {
              printf ("error: mismatch; left_align = %d, right_align = %d\n",
                      left_align, right_align);
              errors = true;
            }
        }
    }

  free (right);
  free (left);
  return errors;
}

Comment 12 Florian Weimer 2016-07-04 11:51:21 UTC
(In reply to Rafael Fonseca from comment #4)
> The problem seems to happen when converting a string to encryption type. For
> some reason, strcasecmp is not returning the right result.
> 
> Check alias 'des3-cbc-sha1-kd' against 'des3-cbc-sha1-kd': 0(cmp)
> 49(case_cmp)
> 
> This is the printf I added in enctype_util.c, just before the checking for
> the alias. The (cmp) result is returned by strcmp while (case_cmp) comes
> from strcasecmp.

Could you please post the patch that you used for debugging?  Thanks.

Comment 13 Rafael Fonseca 2016-07-04 12:29:37 UTC
Created attachment 1176050 [details]
krb5 debugging printfs

These are the modifications I made to krb5.

Comment 14 Rafael Fonseca 2016-07-04 12:34:15 UTC
Created attachment 1176051 [details]
krb5 debugging printfs

This is the right version.

Comment 15 Rafael Fonseca 2016-07-04 12:37:09 UTC
Created attachment 1176053 [details]
Output from debugging

This is the output when using the modifications above.

Comment 16 Florian Weimer 2016-07-04 14:13:14 UTC
I reported this upstream.  It does not appear to be an alignment issue.

Comment 17 Florian Weimer 2016-07-07 10:18:16 UTC
This bug only ever affected rawhide, where it has been fixed.