Bug 557266 - Compilation of static markers is broken in rawhide
Summary: Compilation of static markers is broken in rawhide
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: postgresql
Version: rawhide
Hardware: All
OS: Linux
high
high
Target Milestone: ---
Assignee: Tom Lane
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-01-20 21:08 UTC by Tom Lane
Modified: 2013-07-03 03:26 UTC (History)
11 users (show)

Fixed In Version:
Clone Of:
: 557297 (view as bug list)
Environment:
Last Closed: 2010-01-20 22:50:16 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Tom Lane 2010-01-20 21:08:19 UTC
Description of problem:
I just tried to rebuild postgresql in rawhide, and it fails with an "undefined reference" linker error for each systemtap static marker.  This worked last week, and I have not changed the postgresql code, so something went pear-shaped on the systemtap side.

Version-Release number of selected component (if applicable):
systemtap-sdt-devel-1.1-1.fc13.x86_64

How reproducible:
Probably 100%, but I didn't retry

Steps to Reproduce:
1. Try to build postgresql
  
Actual results:
See tail of build log here:
http://koji.fedoraproject.org/koji/getfile?taskID=1934768&name=build.log

Expected results:
Should build

Additional info:
This is a bit urgent since it blocks rebuilds of postgresql, and presumably every other package with static markers.  If I have to, I'll disable static marker support, but I'd rather not ...

Comment 2 Mark Wielaard 2010-01-20 21:36:43 UTC
Stan, maybe this rings a bell for you? This is because references to the semaphores are missing:

[...]
make[3]: Entering directory `/builddir/build/BUILD/postgresql-8.4.2/src/backend/utils'
dtrace -C -h -s probes.d -o probes.h.tmp
sed -e 's/POSTGRESQL_/TRACE_POSTGRESQL_/g' probes.h.tmp >probes.h
rm probes.h.tmp
make[3]: Leaving directory `/builddir/build/BUILD/postgresql-8.4.2/src/backend/utils'
cd ../../src/include/utils/ && rm -f probes.h && \
	    ln -s ../../../src/backend/utils/probes.h .
make[3]: Leaving directory `/builddir/build/BUILD/postgresql-8.4.2/src/backend/utils'
[...]
gcc -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -DLINUX_OOM_ADJ=0 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing -fwrapv -L../../src/port -Wl,--as-needed -Wl,-E access/common/heaptuple.o access/common/indextuple.o access/common/printtup.o access/common/reloptions.o access/common/scankey.o access/common/tupdesc.o access/gist/gist.o access/gist/gistutil.o access/gist/gistxlog.o access/gist/gistvacuum.o access/gist/gistget.o access/gist/gistscan.o access/gist/gistproc.o access/gist/gistsplit.o access/hash/hash.o access/hash/hashfunc.o access/hash/hashinsert.o access/hash/hashovfl.o access/hash/hashpage.o access/hash/hashscan.o access/hash/hashsearch.o access/hash/hashsort.o access/hash/hashutil.o access/heap/heapam.o access/heap/hio.o access/heap/pruneheap.o access/heap/rewriteheap.o access/heap/syncscan.o access/heap/tuptoaster.o access/heap/visibilitymap.o access/index/genam.o access/index/indexam.o access/nbtree/nbtcompare.o access/nbtree/nbtinsert.o access/nbtree/nbtpage.o access/nbtree/nbtree.o access/nbtree/nbtsearch.o access/nbtree/nbtutils.o access/nbtree/nbtsort.o access/nbtree/nbtxlog.o access/transam/clog.o access/transam/transam.o access/transam/varsup.o access/transam/xact.o access/transam/xlog.o access/transam/xlogutils.o access/transam/rmgr.o access/transam/slru.o access/transam/subtrans.o access/transam/multixact.o access/transam/twophase.o access/transam/twophase_rmgr.o access/gin/ginutil.o access/gin/gininsert.o access/gin/ginxlog.o access/gin/ginentrypage.o access/gin/gindatapage.o access/gin/ginbtree.o access/gin/ginscan.o access/gin/ginget.o access/gin/ginvacuum.o access/gin/ginarrayproc.o access/gin/ginbulk.o access/gin/ginfast.o bootstrap/bootparse.o bootstrap/bootstrap.o catalog/catalog.o catalog/dependency.o catalog/heap.o catalog/index.o catalog/indexing.o catalog/namespace.o catalog/aclchk.o catalog/pg_aggregate.o catalog/pg_constraint.o catalog/pg_conversion.o catalog/pg_depend.o catalog/pg_enum.o catalog/pg_inherits.o catalog/pg_largeobject.o catalog/pg_namespace.o catalog/pg_operator.o catalog/pg_proc.o catalog/pg_shdepend.o catalog/pg_type.o catalog/storage.o catalog/toasting.o parser/analyze.o parser/gram.o parser/keywords.o parser/parser.o parser/parse_agg.o parser/parse_cte.o parser/parse_clause.o parser/parse_expr.o parser/parse_func.o parser/parse_node.o parser/parse_oper.o parser/parse_relation.o parser/parse_type.o parser/parse_coerce.o parser/parse_target.o parser/parse_utilcmd.o parser/scansup.o parser/kwlookup.o commands/aggregatecmds.o commands/alter.o commands/analyze.o commands/async.o commands/cluster.o commands/comment.o commands/conversioncmds.o commands/copy.o commands/dbcommands.o commands/define.o commands/discard.o commands/explain.o commands/foreigncmds.o commands/functioncmds.o commands/indexcmds.o commands/lockcmds.o commands/operatorcmds.o commands/opclasscmds.o commands/portalcmds.o commands/prepare.o commands/proclang.o commands/schemacmds.o commands/sequence.o commands/tablecmds.o commands/tablespace.o commands/trigger.o commands/tsearchcmds.o commands/typecmds.o commands/user.o commands/vacuum.o commands/vacuumlazy.o commands/variable.o commands/view.o executor/execAmi.o executor/execCurrent.o executor/execGrouping.o executor/execJunk.o executor/execMain.o executor/execProcnode.o executor/execQual.o executor/execScan.o executor/execTuples.o executor/execUtils.o executor/functions.o executor/instrument.o executor/nodeAppend.o executor/nodeAgg.o executor/nodeBitmapAnd.o executor/nodeBitmapOr.o executor/nodeBitmapHeapscan.o executor/nodeBitmapIndexscan.o executor/nodeHash.o executor/nodeHashjoin.o executor/nodeIndexscan.o executor/nodeMaterial.o executor/nodeMergejoin.o executor/nodeNestloop.o executor/nodeFunctionscan.o executor/nodeRecursiveunion.o executor/nodeResult.o executor/nodeSeqscan.o executor/nodeSetOp.o executor/nodeSort.o executor/nodeUnique.o executor/nodeValuesscan.o executor/nodeCtescan.o executor/nodeWorktablescan.o executor/nodeLimit.o executor/nodeGroup.o executor/nodeSubplan.o executor/nodeSubqueryscan.o executor/nodeTidscan.o executor/nodeWindowAgg.o executor/tstoreReceiver.o executor/spi.o foreign/foreign.o lib/dllist.o lib/stringinfo.o libpq/be-fsstubs.o libpq/be-secure.o libpq/auth.o libpq/crypt.o libpq/hba.o libpq/ip.o libpq/md5.o libpq/pqcomm.o libpq/pqformat.o libpq/pqsignal.o main/main.o nodes/nodeFuncs.o nodes/nodes.o nodes/list.o nodes/bitmapset.o nodes/tidbitmap.o nodes/copyfuncs.o nodes/equalfuncs.o nodes/makefuncs.o nodes/outfuncs.o nodes/readfuncs.o nodes/print.o nodes/read.o nodes/params.o nodes/value.o optimizer/geqo/geqo_copy.o optimizer/geqo/geqo_eval.o optimizer/geqo/geqo_main.o optimizer/geqo/geqo_misc.o optimizer/geqo/geqo_mutation.o optimizer/geqo/geqo_pool.o optimizer/geqo/geqo_recombination.o optimizer/geqo/geqo_selection.o optimizer/geqo/geqo_erx.o optimizer/geqo/geqo_pmx.o optimizer/geqo/geqo_cx.o optimizer/geqo/geqo_px.o optimizer/geqo/geqo_ox1.o optimizer/geqo/geqo_ox2.o optimizer/path/allpaths.o optimizer/path/clausesel.o optimizer/path/costsize.o optimizer/path/equivclass.o optimizer/path/indxpath.o optimizer/path/joinpath.o optimizer/path/joinrels.o optimizer/path/orindxpath.o optimizer/path/pathkeys.o optimizer/path/tidpath.o optimizer/plan/createplan.o optimizer/plan/initsplan.o optimizer/plan/planagg.o optimizer/plan/planmain.o optimizer/plan/planner.o optimizer/plan/setrefs.o optimizer/plan/subselect.o optimizer/prep/prepjointree.o optimizer/prep/prepqual.o optimizer/prep/preptlist.o optimizer/prep/prepunion.o optimizer/util/clauses.o optimizer/util/joininfo.o optimizer/util/pathnode.o optimizer/util/placeholder.o optimizer/util/plancat.o optimizer/util/predtest.o optimizer/util/relnode.o optimizer/util/restrictinfo.o optimizer/util/tlist.o optimizer/util/var.o port/dynloader.o port/pg_sema.o port/pg_shmem.o postmaster/autovacuum.o postmaster/bgwriter.o postmaster/fork_process.o postmaster/pgarch.o postmaster/pgstat.o postmaster/postmaster.o postmaster/syslogger.o postmaster/walwriter.o regex/regcomp.o regex/regerror.o regex/regexec.o regex/regfree.o rewrite/rewriteRemove.o rewrite/rewriteDefine.o rewrite/rewriteHandler.o rewrite/rewriteManip.o rewrite/rewriteSupport.o storage/buffer/buf_table.o storage/buffer/buf_init.o storage/buffer/bufmgr.o storage/buffer/freelist.o storage/buffer/localbuf.o storage/file/fd.o storage/file/buffile.o storage/freespace/freespace.o storage/freespace/fsmpage.o storage/freespace/indexfsm.o storage/ipc/ipc.o storage/ipc/ipci.o storage/ipc/pmsignal.o storage/ipc/procarray.o storage/ipc/shmem.o storage/ipc/shmqueue.o storage/ipc/sinval.o storage/ipc/sinvaladt.o storage/large_object/inv_api.o storage/lmgr/lmgr.o storage/lmgr/lock.o storage/lmgr/proc.o storage/lmgr/deadlock.o storage/lmgr/lwlock.o storage/lmgr/spin.o storage/lmgr/s_lock.o storage/page/bufpage.o storage/page/itemptr.o storage/smgr/md.o storage/smgr/smgr.o storage/smgr/smgrtype.o tcop/dest.o tcop/fastpath.o tcop/postgres.o tcop/pquery.o tcop/utility.o tsearch/ts_locale.o tsearch/ts_parse.o tsearch/wparser.o tsearch/wparser_def.o tsearch/dict.o tsearch/dict_simple.o tsearch/dict_synonym.o tsearch/dict_thesaurus.o tsearch/dict_ispell.o tsearch/regis.o tsearch/spell.o tsearch/to_tsany.o tsearch/ts_selfuncs.o tsearch/ts_typanalyze.o tsearch/ts_utils.o utils/adt/acl.o utils/adt/arrayfuncs.o utils/adt/array_userfuncs.o utils/adt/arrayutils.o utils/adt/bool.o utils/adt/cash.o utils/adt/char.o utils/adt/date.o utils/adt/datetime.o utils/adt/datum.o utils/adt/domains.o utils/adt/enum.o utils/adt/float.o utils/adt/format_type.o utils/adt/geo_ops.o utils/adt/geo_selfuncs.o utils/adt/int.o utils/adt/int8.o utils/adt/like.o utils/adt/lockfuncs.o utils/adt/misc.o utils/adt/nabstime.o utils/adt/name.o utils/adt/numeric.o utils/adt/numutils.o utils/adt/oid.o utils/adt/oracle_compat.o utils/adt/pseudotypes.o utils/adt/rowtypes.o utils/adt/regexp.o utils/adt/regproc.o utils/adt/ruleutils.o utils/adt/selfuncs.o utils/adt/tid.o utils/adt/timestamp.o utils/adt/varbit.o utils/adt/varchar.o utils/adt/varlena.o utils/adt/version.o utils/adt/xid.o utils/adt/network.o utils/adt/mac.o utils/adt/inet_net_ntop.o utils/adt/inet_net_pton.o utils/adt/ri_triggers.o utils/adt/pg_lzcompress.o utils/adt/pg_locale.o utils/adt/formatting.o utils/adt/ascii.o utils/adt/quote.o utils/adt/pgstatfuncs.o utils/adt/encode.o utils/adt/dbsize.o utils/adt/genfile.o utils/adt/trigfuncs.o utils/adt/tsginidx.o utils/adt/tsgistidx.o utils/adt/tsquery.o utils/adt/tsquery_cleanup.o utils/adt/tsquery_gist.o utils/adt/tsquery_op.o utils/adt/tsquery_rewrite.o utils/adt/tsquery_util.o utils/adt/tsrank.o utils/adt/tsvector.o utils/adt/tsvector_op.o utils/adt/tsvector_parser.o utils/adt/txid.o utils/adt/uuid.o utils/adt/windowfuncs.o utils/adt/xml.o utils/cache/catcache.o utils/cache/inval.o utils/cache/plancache.o utils/cache/relcache.o utils/cache/syscache.o utils/cache/lsyscache.o utils/cache/typcache.o utils/cache/ts_cache.o utils/error/assert.o utils/error/elog.o utils/fmgr/dfmgr.o utils/fmgr/fmgr.o utils/fmgr/funcapi.o utils/hash/dynahash.o utils/hash/hashfn.o utils/hash/pg_crc.o utils/init/flatfiles.o utils/init/globals.o utils/init/miscinit.o utils/init/postinit.o utils/mb/encnames.o utils/mb/conv.o utils/mb/mbutils.o utils/mb/wchar.o utils/mb/wstrcmp.o utils/mb/wstrncmp.o utils/misc/guc.o utils/misc/help_config.o utils/misc/pg_rusage.o utils/misc/ps_status.o utils/misc/superuser.o utils/misc/tzparser.o utils/mmgr/aset.o utils/mmgr/mcxt.o utils/mmgr/portalmem.o utils/resowner/resowner.o utils/sort/logtape.o utils/sort/tuplesort.o utils/sort/tuplestore.o utils/time/combocid.o utils/time/tqual.o utils/time/snapmgr.o utils/fmgrtab.o ../../src/timezone/localtime.o ../../src/timezone/strftime.o ../../src/timezone/pgtz.o ../../src/port/libpgport_srv.a -lxslt -lxml2 -lpam -lssl -lcrypto -lgssapi_krb5 -lcrypt -ldl -lm -lldap -o postgres
access/transam/clog.o: In function `CheckPointCLOG':
/builddir/build/BUILD/postgresql-8.4.2/src/backend/access/transam/clog.c:545: undefined reference to `clog__checkpoint__start_semaphore'
/builddir/build/BUILD/postgresql-8.4.2/src/backend/access/transam/clog.c:547: undefined reference to `clog__checkpoint__done_semaphore'
access/transam/clog.o: In function `ShutdownCLOG':
/builddir/build/BUILD/postgresql-8.4.2/src/backend/access/transam/clog.c:533: undefined reference to `clog__checkpoint__start_semaphore'
/builddir/build/BUILD/postgresql-8.4.2/src/backend/access/transam/clog.c:535: undefined reference to `clog__checkpoint__done_semaphore'
[... etc ...]

Comment 3 Frank Ch. Eigler 2010-01-20 21:37:42 UTC
Tom, was this with the postgres makefile hack that turns off the probe.o 
generation/linking (dtrace -G)?

Comment 4 Frank Ch. Eigler 2010-01-20 22:00:02 UTC
Specifically, this bit in src/backend/Makefile et al.

# As of 9/2009:
# * The probes.o file is necessary for dtrace support on Solaris.
# * OS X's dtrace doesn't use it and doesn't even recognize the -G option.
# * Systemtap's dtrace will take -G, but it produces a useless empty file.
# So, build probes.o only on Solaris.
# This will likely need adjustment as other platforms add dtrace support.
ifeq ($(PORTNAME), solaris)
ifeq ($(enable_dtrace), yes)
LOCALOBJS += utils/probes.o
endif
endif


appears to be counterproductive on current systemtap.

Comment 5 Tom Lane 2010-01-20 22:08:08 UTC
So we should now do the -G part?  I had gathered from the last conversation that that would be useless for the foreseeable future, but ...

Comment 6 Stan Cox 2010-01-20 22:29:49 UTC
The dtrace -G step now creates probe semaphore definitions.  So for example:
provider sdt_misc {
        probe test_probe_0 ();
};
 
Will create with dtrace -G probes.d
a definition 'test_probe_0_semaphore' in probes.o.  This value is used to avoid some of the overhead of setting up the probe when systemtap is not being used.  The value is set by systemtap itself whenever stap is run.  So the use of the dtrace script now more or less parallels the solaris usage,  -G builds probes.o and -h builds probes.h.

Comment 7 Tom Lane 2010-01-20 22:32:41 UTC
OK, that makes this a postgres bug not a systemtap bug.  Thanks for your advice.

Oh, one other question: is this version of systemtap going to appear in Fedora 11 and/or 12?

Comment 8 Frank Ch. Eigler 2010-01-20 22:41:41 UTC
It's already in F11 and F12.

(Later versions will probably not require -G to make this stuff work,
though should still support it.)

Comment 9 Tom Lane 2010-01-20 22:50:16 UTC
Well, given that Solaris has to have the -G file, there's not a lot of advantage to having systemtap not need it.  In any case, please don't break this again ;-)

Comment 10 Mark Wielaard 2010-01-21 08:08:55 UTC
Sorry about that. The problem really comes from macos, they don't support the dtrace -G option. We tried to be compatible and support it (the Sun variant doesn't work without it), but indeed in the past it wasn't required (and it would be nice if we didn't require it again in the future). Python for example has the following configure check to detect how to build:

# Check for dtrace support
AC_MSG_CHECKING(for --with-dtrace)
AC_ARG_WITH(dtrace,
            AC_HELP_STRING(--with(out)-dtrace, disable/enable dtrace support))

if test ! -z "$with_dtrace"
then
    if dtrace -G -o /dev/null -s $srcdir/Include/pydtrace.d 2>/dev/null
    then
	AC_DEFINE(WITH_DTRACE, 1, 
	 [Define if you want to compile in Dtrace support])
	with_dtrace="Sun"
	DTRACEOBJS="Python/phelper.o Python/dtrace.o"
	DTRADEHDRS=""
    elif dtrace -h -o /dev/null -s $srcdir/Include/pydtrace.d
    then
	AC_DEFINE(WITH_DTRACE, 1, 
	 [Define if you want to compile in Dtrace support])
	with_dtrace="Apple"
	DTRACEOBJS=""
	DTRADEHDRS="phelper.h pydtrace.h"
    else
	with_dtrace="no"
    fi
else
    with_dtrace="no"
fi

AC_MSG_RESULT($with_dtrace)
AC_SUBST(DTRACEOBJS)
AC_SUBST(DTRACEHDRS)

SystemTap is then detected as the "Sun" variant.

Comment 11 Stan Cox 2010-01-21 14:49:45 UTC
The stap testsuite builds postgres with --enable-dtrace and runs the postgres
tests.  I grab it from upstream though because it is so easy to automate:
 wget http://wwwmaster.postgresql.org/...
 tar -x -f postgresql-${postgresrelease}.tar.bz2


Note You need to log in before you can comment on or make changes to this bug.