Description of problem: Recent build have started to fail in koschei. The issue doesn't seem to be on FreeIPA side, since the build only fails for ppc64 and ppc64le. The build fails during ipa_cldap_tests. https://koji.fedoraproject.org/koji/taskinfo?taskID=19597252 How reproducible: always Build output: make[4]: Entering directory '/builddir/build/BUILD/freeipa-4.4.4/daemons/ipa-slapi-plugins/ipa-cldap' make[5]: Entering directory '/builddir/build/BUILD/freeipa-4.4.4/daemons/ipa-slapi-plugins/ipa-cldap' ../../test-driver: line 107: 9527 Segmentation fault (core dumped) "$@" > $log_file 2>&1 FAIL: ipa_cldap_tests ================================================================== ipa-server 4.4.4: ipa-slapi-plugins/ipa-cldap/test-suite.log ================================================================== # TOTAL: 1 # PASS: 0 # SKIP: 0 # XFAIL: 0 # FAIL: 1 # XPASS: 0 # ERROR: 0 .. contents:: :depth: 2 FAIL: ipa_cldap_tests ===================== FAIL ipa_cldap_tests (exit status: 139) ============================================================================ Testsuite summary for ipa-server 4.4.4 ============================================================================ # TOTAL: 1 # PASS: 0 # SKIP: 0 # XFAIL: 0 # FAIL: 1 # XPASS: 0 # ERROR: 0 ============================================================================
can we get core dumps ?
an expanded backtrace: <mock-chroot> sh-4.4# cd /builddir/build/BUILD/freeipa-4.4.4/daemons/ipa-slapi-plugins/ipa-cldap <mock-chroot> sh-4.4# libtool --mode=execute gdb ipa_cldap_tests GNU gdb (GDB) Fedora 7.99.90.20170420-12.fc27 Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "ppc64le-redhat-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from ipa_cldap_tests...done. (gdb) bt full No stack. (gdb) run Starting program: /builddir/build/BUILD/freeipa-4.4.4/daemons/ipa-slapi-plugins/ipa-cldap/ipa_cldap_tests Program received signal SIGSEGV, Segmentation fault. 0x0000000000000000 in ?? () (gdb) bt full #0 0x0000000000000000 in ?? () No symbol table info available. #1 0x00003fffb72c8640 in ?? () No symbol table info available. #2 0x00003fffb72ca544 in ?? () No symbol table info available. #3 0x00003fffb7fba268 in resolve_ifunc (sym_map=0x3fffb7ff95d0, map=<optimized out>, value=70367522366656) at ../sysdeps/powerpc/powerpc64/dl-machine.h:666 No locals. #4 elf_machine_rela (skip_ifunc=<optimized out>, reloc_addr_arg=0x3fffb731ecd0, version=<optimized out>, sym=<optimized out>, reloc=0x3fffb72c1a78, map=0x3fffb7ff95d0) at ../sysdeps/powerpc/powerpc64/dl-machine.h:708 refsym = 0x3fffb72b4a78 value = 70367522366656 reloc_addr = 0x3fffb731ecd0 r_type = 21 sym_map = 0x3fffb7ff95d0 #5 elf_dynamic_do_Rela (skip_ifunc=<optimized out>, lazy=<optimized out>, nrelative=<optimized out>, relsize=<optimized out>, reladdr=<optimized out>, map=<optimized out>) at do-rel.h:137 ndx = <optimized out> version = 0x3fffb72bf3cc symtab = 0x3fffb72b1778 relative = <optimized out> r = 0x3fffb72c1a78 #6 _dl_relocate_object (scope=0x3fffb7ff9948, reloc_mode=<optimized out>, consider_profiling=<optimized out>) at dl-reloc.c:259 ranges = {{start = 70367522323288, size = 18072, nrelative = 51, lazy = 0}, {start = 0, size = 0, nrelative = 0, lazy = 0}} textrels = <optimized out> errstring = 0x0 lazy = <optimized out> skip_ifunc = <optimized out> #7 0x00003fffb7fa588c in dl_main (phdr=<optimized out>, phnum=<optimized out>, user_entry=<optimized out>, auxv=<optimized out>) at rtld.c:2047 l = 0x3fffb7ff95d0 lnp = <optimized out> consider_profiling = 0 start = 349196431098357 stop = <optimized out> i = 17 ph = <optimized out> mode = <optimized out> main_map = <optimized out> file_size = 70368744174952 file = <optimized out> has_interp = <optimized out> i = <optimized out> prelinked = <optimized out> rtld_is_main = <optimized out> start = <optimized out> stop = <optimized out> diff = <optimized out> tcbp = <optimized out> __PRETTY_FUNCTION__ = "dl_main" ---Type <return> to continue, or q <return> to quit--- first_preload = <optimized out> r = <optimized out> rtld_phdr = <optimized out> cnt = <optimized out> count_modids = <optimized out> preloads = <optimized out> npreloads = <optimized out> preload_file = "/etc/ld.so.preload" rtld_multiple_ref = true was_tls_init_tp_called = <optimized out> #8 0x00003fffb7fd18b4 in _dl_sysdep_start (start_argptr=<optimized out>, dl_main=0x3fffb7fa27d0 <dl_main>) at ../elf/dl-sysdep.c:253 phdr = <optimized out> phnum = <optimized out> user_entry = 536875828 av = <optimized out> #9 0x00003fffb7fa1de8 in _dl_start_final (arg=0x3ffffffff550, info=0x3fffffffefb0) at rtld.c:303 start_addr = <optimized out> rtld_total_time = 0 #10 0x00003fffb7fa74b4 in _dl_start (arg=0x3ffffffff550) at rtld.c:411 entry = <optimized out> info = {l = {l_addr = 70367535824896, l_name = 0x0, l_ld = 0x3fffb7fefc10, l_next = 0x0, l_prev = 0x0, l_real = 0x0, l_ns = 0, l_libname = 0x0, l_info = {0x0, 0x0, 0x3fffb7fefc90, 0x3fffb7fefc80, 0x3fffb7fefc20, 0x3fffb7fefc40, 0x3fffb7fefc50, 0x3fffb7fefce0, 0x3fffb7fefcf0, 0x3fffb7fefd00, 0x3fffb7fefc60, 0x3fffb7fefc70, 0x0, 0x0, 0x3fffb7fefc10, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3fffb7fefca0, 0x0, 0x0, 0x3fffb7fefcb0, 0x3fffb7fefd30, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3fffb7fefcc0, 0x0, 0x0, 0x3fffb7fefcd0, 0x0, 0x0, 0x3fffb7fefd20, 0x3fffb7fefd10, 0x3fffb7fefd40, 0x0, 0x3fffb7fefd60, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3fffb7fefd50, 0x0 <repeats 25 times>, 0x3fffb7fefc30}, l_phdr = 0x0, l_entry = 0, l_phnum = 0, l_ldnum = 0, l_searchlist = {r_list = 0x0, r_nlist = 0}, l_symbolic_searchlist = {r_list = 0x0, r_nlist = 0}, l_loader = 0x0, l_versions = 0x0, l_nversions = 0, l_nbuckets = 0, l_gnu_bitmask_idxbits = 0, l_gnu_shift = 0, l_gnu_bitmask = 0x0, {l_gnu_buckets = 0x0, l_chain = 0x0}, { l_gnu_chain_zero = 0x0, l_buckets = 0x0}, l_direct_opencount = 0, l_type = lt_executable, l_relocated = 1, l_init_called = 0, l_global = 0, l_reserved = 0, l_phdr_allocated = 0, l_soname_added = 0, l_faked = 0, l_need_tls_init = 0, l_auditing = 0, l_audit_any_plt = 0, l_removed = 0, l_contiguous = 0, l_symbolic_in_local_scope = 0, l_free_initfini = 0, l_rpath_dirs = {dirs = 0x0, malloced = 0}, l_reloc_result = 0x0, l_versyms = 0x0, l_origin = 0x0, l_map_start = 0, l_map_end = 0, l_text_end = 0, l_scope_mem = {0x0, 0x0, 0x0, 0x0}, l_scope_max = 0, l_scope = 0x0, l_local_scope = {0x0, 0x0}, l_file_id = {dev = 0, ino = 0}, l_runpath_dirs = {dirs = 0x0, malloced = 0}, l_initfini = 0x0, l_reldeps = 0x0, l_reldepsmax = 0, l_used = 0, l_feature_1 = 0, l_flags_1 = 0, l_flags = 0, l_idx = 0, l_mach = {<No data fields>}, l_lookup_cache = { sym = 0x0, type_class = 0, value = 0x0, ret = 0x0}, l_tls_initimage = 0x0, l_tls_initimage_size = 0, l_tls_blocksize = 0, l_tls_align = 0, l_tls_firstbyte_offset = 0, l_tls_offset = -1, l_tls_modid = 0, l_tls_dtor_count = 0, l_relro_addr = 0, l_relro_size = 0, l_serial = 0, l_audit = 0x3ffffffff428}, start_time = 349196429014622} #11 0x00003fffb7fa1578 in _start () from /lib64/ld64.so.2 _dl_rtld_libname2 = {name = 0x0, next = 0x0, dont_free = 0} any_debug = 0 start_time = 349196429014622 audit_list = 0x0 _dl_rtld_libname = {name = 0x20000238 "/lib64/ld64.so.2", next = 0x3fffb7ff11c8 <newname>, dont_free = 0} tls_init_tp_called = true preloadlist = 0x0 load_time = 1954588 library_path = 0x0 version_info = 0 relocate_time = 0 _dl_skip_args = 0 __GI__dl_starting_up = 1 _rtld_global = {_dl_ns = {{_ns_loaded = 0x3fffb7ff1300, _ns_nloaded = 58, _ns_main_searchlist = 0x3fffb7ff15d8, _ns_global_scope_alloc = 0, _ns_unique_sym_table = {lock = {mutex = { ---Type <return> to continue, or q <return> to quit--- __data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 1, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 16 times>, "\001", '\000' <repeats 22 times>, __align = 0}}, entries = 0x3fffb6b8ca10, size = 251, n_elements = 105, free = 0x3fffb7fd2300 <free>}, _ns_debug = {r_version = 0, r_map = 0x0, r_brk = 0, r_state = RT_CONSISTENT, r_ldbase = 0}}, {_ns_loaded = 0x0, _ns_nloaded = 0, _ns_main_searchlist = 0x0, _ns_global_scope_alloc = 0, _ns_unique_sym_table = {lock = {mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}}, entries = 0x0, size = 0, n_elements = 0, free = 0x0}, _ns_debug = {r_version = 0, r_map = 0x0, r_brk = 0, r_state = RT_CONSISTENT, r_ldbase = 0}} <repeats 15 times>}, _dl_nns = 1, _dl_load_lock = {mutex = { __data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 1, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 16 times>, "\001", '\000' <repeats 22 times>, __align = 0}}, _dl_load_write_lock = {mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 1, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 16 times>, "\001", '\000' <repeats 22 times>, __align = 0}}, _dl_load_adds = 58, _dl_initfirst = 0x3fffb7ff5ba0, _dl_cpuclock_offset = 349196429060435, _dl_profile_map = 0x0, _dl_num_relocations = 5667, _dl_num_cache_relocations = 2829, _dl_all_dirs = 0x3fffb7ff5ae0, _dl_rtld_map = {l_addr = 70367535824896, l_name = 0x20000238 "/lib64/ld64.so.2", l_ld = 0x3fffb7fefc10, l_next = 0x3fffb7ff5ba0, l_prev = 0x3fffb7ff5580, l_real = 0x3fffb7ff0990 <_rtld_local+2448>, l_ns = 0, l_libname = 0x3fffb7ff1190 <_dl_rtld_libname>, l_info = {0x0, 0x0, 0x3fffb7fefc90, 0x3fffb7fefc80, 0x3fffb7fefc20, 0x3fffb7fefc40, 0x3fffb7fefc50, 0x3fffb7fefce0, 0x3fffb7fefcf0, 0x3fffb7fefd00, 0x3fffb7fefc60, 0x3fffb7fefc70, 0x0, 0x0, 0x3fffb7fefc10, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3fffb7fefca0, 0x0, 0x0, 0x3fffb7fefcb0, 0x3fffb7fefd30, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3fffb7fefcc0, 0x0, 0x0, 0x3fffb7fefcd0, 0x0, 0x0, 0x3fffb7fefd20, 0x3fffb7fefd10, 0x3fffb7fefd40, 0x0, 0x3fffb7fefd60, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3fffb7fefd50, 0x0 <repeats 25 times>, 0x3fffb7fefc30}, l_phdr = 0x3fffb7fa0040, l_entry = 0, l_phnum = 7, l_ldnum = 0, l_searchlist = {r_list = 0x0, r_nlist = 0}, l_symbolic_searchlist = {r_list = 0x0, r_nlist = 0}, l_loader = 0x0, l_versions = 0x3fffb6b864d0, l_nversions = 6, l_nbuckets = 17, l_gnu_bitmask_idxbits = 3, l_gnu_shift = 8, l_gnu_bitmask = 0x3fffb7fa02d8, { l_gnu_buckets = 0x3fffb7fa02f8, l_chain = 0x3fffb7fa02f8}, {l_gnu_chain_zero = 0x3fffb7fa0330, l_buckets = 0x3fffb7fa0330}, l_direct_opencount = 0, l_type = lt_library, l_relocated = 1, l_init_called = 0, l_global = 1, l_reserved = 0, l_phdr_allocated = 0, l_soname_added = 0, l_faked = 0, l_need_tls_init = 0, l_auditing = 0, l_audit_any_plt = 0, l_removed = 0, l_contiguous = 0, l_symbolic_in_local_scope = 0, l_free_initfini = 0, l_rpath_dirs = {dirs = 0x0, malloced = 0}, l_reloc_result = 0x0, l_versyms = 0x3fffb7fa08de, l_origin = 0x0, l_map_start = 70367535824896, l_map_end = 70367536157432, l_text_end = 70367536050744, l_scope_mem = {0x0, 0x0, 0x0, 0x0}, l_scope_max = 0, l_scope = 0x0, l_local_scope = {0x0, 0x0}, l_file_id = {dev = 0, ino = 0}, l_runpath_dirs = {dirs = 0x0, malloced = 0}, l_initfini = 0x0, l_reldeps = 0x0, l_reldepsmax = 0, l_used = 1, l_feature_1 = 0, l_flags_1 = 0, l_flags = 0, l_idx = 0, l_mach = {<No data fields>}, l_lookup_cache = {sym = 0x0, type_class = 0, value = 0x0, ret = 0x0}, l_tls_initimage = 0x0, l_tls_initimage_size = 0, l_tls_blocksize = 0, l_tls_align = 0, l_tls_firstbyte_offset = 0, l_tls_offset = -1, l_tls_modid = 0, l_tls_dtor_count = 0, l_relro_addr = 325464, l_relro_size = 2216, l_serial = 0, l_audit = 0x3fffb7ff0e08 <_rtld_local+3592>}, audit_data = {{cookie = 0, bindflags = 0} <repeats 16 times>}, _dl_rtld_lock_recursive = 0x3fffb7fa1640 <rtld_lock_default_lock_recursive>, _dl_rtld_unlock_recursive = 0x3fffb7fa1660 <rtld_lock_default_unlock_recursive>, _dl_make_stack_executable_hook = 0x3fffb7fc4af0 <__GI__dl_make_stack_executable>, _dl_stack_flags = 6, _dl_tls_dtv_gaps = false, _dl_tls_max_dtv_idx = 7, _dl_tls_dtv_slotinfo_list = 0x3fffb6b894f0, _dl_tls_static_nelem = 7, _dl_tls_static_size = 3168, _dl_tls_static_used = 1504, _dl_tls_static_align = 16, _dl_initial_dtv = 0x3fffb6b8ad50, _dl_tls_generation = 0, _dl_init_static_tls = 0x3fffb7fb9080 <_dl_nothread_init_static_tls>, _dl_wait_lookup_done = 0x0, _dl_scope_free_list = 0x0} __GI__dl_argv = 0x3ffffffff558 __pointer_chk_guard_local = 11505981894542216821 _rtld_global_ro = {_dl_debug_mask = 0, _dl_osversion = 264719, _dl_platform = 0x3ffffffff7c9 "power8", _dl_platformlen = 6, _dl_pagesize = 65536, _dl_inhibit_cache = 0, _dl_initial_searchlist = {r_list = 0x3fffb6b85510, r_nlist = 57}, _dl_clktck = 100, _dl_verbose = 0, _dl_debug_fd = 2, _dl_lazy = 1, _dl_bind_not = 0, _dl_dynamic_weak = 0, _dl_fpu_control = 0, _dl_correct_cache_id = 1283, _dl_hwcap = 3691013570, _dl_hwcap_mask = 268436480, _dl_auxv = 0x3ffffffff608, _dl_powerpc_cap_flags = {"ppcle\000\000\000\000", "true_le\000\000", "\000\000\000\000\000\000\000\000\000", "\000\000\000\000\000\000\000\000\000", "\000\000\000\000\000\000\000\000\000", "\000\000\000\000\000\000\000\000\000", "archpmu\000\000", "vsx\000\000\000\000\000\000", "arch_2_06", "power6x\000\000", "dfp\000\000\000\000\000\000", "pa6t\000\000\000\000\000", "arch_2_05", "ic_snoop\000", "smt\000\000\000\000\000\000", "booke\000\000\000\000", "cellbe\000\000\000", "power5+\000\000", "power5\000\000\000", "power4\000\000\000", "notb\000\000\000\000\000", "efpdouble", "efpsingle", "spe\000\000\000\000\000\000", "ucache\000\000\000", "4xxmac\000\000\000", "mmu\000\000\000\000\000\000", "fpu\000\000\000\000\000\000", "altivec\000\000", "ppc601\000\000\000", "ppc64\000\000\000\000", "ppc32\000\000\000\000", "\000\000\000\000\000\000\000\000\000" <repeats 22 times>, "ieee128\000\000", "arch_3_00", "htm-nosc\000", "vcrypto\000\000", "tar\000\000\000\000\000\000", "isel\000\000\000\000\000", "ebb\000\000\000\000\000\000", "dscr\000\000\000\000\000", "htm\000\000\000\000\000\000", "arch_2_07"}, _dl_inhibit_rpath = 0x0, _dl_origin_path = 0x0, _dl_use_load_bias = 0, _dl_profile = 0x0, _dl_profile_output = 0x3fffb7fd7498 "/var/tmp", _dl_trace_prelink = 0x0, _dl_trace_prelink_map = 0x0, _dl_init_all_dirs = 0x3fffb7ff5ae0, _dl_sysinfo_dso = 0x3fffb7f80000, _dl_sysinfo_map = 0x3fffb7ff18a0, _dl_hwcap2 = 4278190080, _dl_debug_printf = 0x3fffb7fc00b0 <_dl_debug_printf>, _dl_mcount = 0x3fffb7fc1bd0 <__GI__dl_mcount>, _dl_lookup_symbol_x = 0x3fffb7fb7500 <_dl_lookup_symbol_x>, _dl_check_caller = 0x3fffb7fc4c10 <_dl_check_caller>, _dl_open = 0x3fffb7fc9b80 <_dl_open>, _dl_close = 0x3fffb7fcbe20 <_dl_close>, _dl_tls_get_addr_soft = 0x3fffb7fc4460 <_dl_tls_get_addr_soft>, _dl_discover_osversion = 0x3fffb7fd1cf0 <_dl_discover_osversion>, _dl_audit = 0x0, _dl_naudit = 0} _dl_argc = 1 (gdb) q A debugging session is active. Inferior 1 [process 10697] will be killed. Quit anyway? (y or n) y
I think it is a problem with ld.so/glibc or gcc. We aren't even get to the point that main() from IPA's test is executed.
Moving to glibc to investigate what's wrong with a loader.
==11306== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0) ==11306== ==11306== 1 errors in context 1 of 2: ==11306== Jump to the invalid address stated on the next line ==11306== at 0x0: ??? ==11306== by 0x549863F: ??? (in /usr/lib64/libtcmalloc.so.4.4.1) ==11306== by 0x549A543: operator delete(void*, unsigned long) (in /usr/lib64/libtcmalloc.so.4.4.1) ==11306== by 0x401A267: resolve_ifunc (dl-machine.h:666) ==11306== by 0x401A267: elf_machine_rela (dl-machine.h:708) ==11306== by 0x401A267: elf_dynamic_do_Rela (do-rel.h:137) ==11306== by 0x401A267: _dl_relocate_object (dl-reloc.c:259) ==11306== by 0x400588B: dl_main (rtld.c:2047) ==11306== by 0x40318B3: _dl_sysdep_start (dl-sysdep.c:253) ==11306== by 0x4001DE7: _dl_start_final (rtld.c:303) ==11306== by 0x40074B3: _dl_start (rtld.c:411) ==11306== by 0x4001577: _start (in /usr/lib64/ld-2.25.90.so) ==11306== Address 0x0 is not stack'd, malloc'd or (recently) free'd This issue is resolved in 389-ds-base-1.3.6.6-2.fc27
389-ds-base-1.3.6.6-2.fc27 temporarily disabled tcmalloc on ppc64/ppc64le to stop its own build problems - that's why its "fixed". So there is still an issue with tcmalloc.