Bug 166800 - Review Request: sbcl: The Steel Bank Common Lisp development environment
Review Request: sbcl: The Steel Bank Common Lisp development environment
Status: CLOSED NEXTRELEASE
Product: Fedora
Classification: Fedora
Component: Package Review (Show other bugs)
rawhide
All Linux
medium Severity medium
: ---
: ---
Assigned To: Gérard Milmeister
David Lawrence
http://sbcl.sourceforge.net/
:
Depends On:
Blocks: FE-ACCEPT
  Show dependency treegraph
 
Reported: 2005-08-25 16:11 EDT by Rex Dieter
Modified: 2010-11-19 07:52 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2005-09-20 10:19:41 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Description Rex Dieter 2005-08-25 16:11:35 EDT
Spec Name or Url: http://apt.kde-redhat.org/apt/fedora/SPECS/sbcl-0.9.3-1.spec
SRPM Name or Url: http://apt.kde-redhat.org/apt/fedora/all/SRPMS/stable/sbcl-0.9.3-1.src.rpm
Description:
Steel Bank Common Lisp (SBCL) is a Open Source development environment
for Common Lisp. It includes an integrated native compiler,
interpreter, and debugger.
Comment 1 Gérard Milmeister 2005-08-25 16:28:53 EDT
How is sbcl boottrapped?
You are aware that sbcl on FC4 man startup correctly or not...?
It is guaranteed to startup only with setarch -R.
Comment 2 Rex Dieter 2005-08-25 16:40:43 EDT
Here's the bootstrap options in the specfile (lightly tested):

## define to be one of the following
# use our own/local bootstrap?
%define bootstrap_sbcl_local 1
#define bootstrap_sbcl sbcl
#define bootstrap_clisp clisp

Comment 3 Rex Dieter 2005-08-25 16:41:48 EDT
Wasn't aware of the startup issues on fc4 (had only lightly tested on fc3 and
rhel4 so far).
Comment 5 Gérard Milmeister 2005-08-29 09:17:34 EDT
I cannot download
http://apt.kde-redhat.org/apt/fedora/all/SRPMS/stable/sbcl-0.9.4-1.src.rpm
(object not found)
Comment 6 Rex Dieter 2005-08-29 09:19:16 EDT
My bad for posting before I actually put the files in place.  Should be good now.
Comment 7 Gérard Milmeister 2005-08-29 09:25:15 EDT
This should be SRPMS.stable, not SRPMS/stable.
Comment 8 Rex Dieter 2005-08-29 09:27:33 EDT
SRPM Name or Url:
http://apt.kde-redhat.org/apt/fedora/all/SRPMS.stable/sbcl-0.9.4-1.src.rpm
Comment 9 Gérard Milmeister 2005-08-29 09:43:01 EDT
* The spec file should be named sbcl.spec
* Shouldn't the src rpm contain bootstrapping binaries for all architectures?
  AFAIK src rpms are not architecture dependent.
* You may request a ppc binary from upstream, so that all fedora platforms
  are supported.
* Requires(post): /sbin/install-info
  Requires(preun): /sbin/install-info
* setarch -R is required on FC4 to start sbcl, you might create a script
  /usr/bin/sbcl that calls a renamed /usr/bin/sbcl.bin with setarch.
  Better yet, communicate with upstream to investigate the problem.
* Have you tried clisp to bootstrap sbcl? I did not succeed. If you do, the
  binaries are not necessary.
Comment 10 Rex Dieter 2005-08-29 09:50:35 EDT
specfile names will certainly be fixed when imported to svn.

> setarch -R is required on FC4 to start sbcl,

I figured if it was *built* with that option, then the resulting binary would
inherit that behavior.  Turns out you're right.

> Shouldn't the src rpm contain bootstrapping binaries for all architectures?
> AFAIK src rpms are not architecture dependent.

Hmm, good point, probably.

> Have you tried clisp to bootstrap sbcl? I did not succeed. If you do, the
> binaries are not necessary.

Tried, it failed.
Comment 11 Gérard Milmeister 2005-08-29 10:19:33 EDT
Where is %{?fedora} defined?
Comment 12 Rex Dieter 2005-08-29 10:20:59 EDT
%{fedora} is defined by the Extras buildsystem.  It's set to 3 on Fedora Core 3,
4 on Fedora Core 4.
Comment 13 Gérard Milmeister 2005-08-29 12:40:41 EDT
But %{fedora} is not defined in fc4's rpmbuild, i.e., the src rpm may not build
correctly for a user.

output of rpmlint:
W: sbcl devel-file-in-non-devel-package /usr/lib/sbcl/sb-bsd-sockets/alien/undefs.c
W: sbcl devel-file-in-non-devel-package /usr/lib/sbcl/sb-bsd-sockets/foo.c
W: sbcl devel-file-in-non-devel-package
/usr/lib/sbcl/sb-bsd-sockets/alien/get-h-errno.c
E: sbcl script-without-shellbang /usr/lib/sbcl/sb-bsd-sockets/alien/get-h-errno.c

Are these files necessary?
At least chmod 644 get-h-errno.c

E: sbcl info-dir-file /usr/share/info/dir

this must be removed

W: sbcl devel-file-in-non-devel-package /usr/lib/sbcl/sb-posix/alien/stat-macros.c
W: sbcl devel-file-in-non-devel-package
/usr/lib/sbcl/sb-posix/alien/waitpid-macros.c
W: sbcl devel-file-in-non-devel-package /usr/lib/sbcl/sb-posix/foo.c

Are these necessary?
Comment 14 Rex Dieter 2005-08-29 13:01:22 EDT
> But %{fedora} is not defined in fc4's rpmbuild, i.e., the src rpm may not build
> correctly for a user.

True, and eventually the default case could to make it work for end-users, but
don't forget that the primary target here is the Fedora Extras buildsystem.

> W: sbcl devel-file-in-non-devel-package 
> /usr/lib/sbcl/sb-posix/alien/stat-macros.c
> Are these necessary?

Dunno, probably not, just packaging what upstream does.
Comment 15 Rex Dieter 2005-08-29 13:11:58 EDT
%changelog
* Mon Aug 29 2005 Rex Dieter <rexdieter[AT]users.sf.net> 0.9.4-2
- rm -f /usr/share/info/dir
- fix perms on packaged .c files
- include all bootstrap binaries
- Requires(post,preun): /sbin/install-info
- sbcl.sh app-wrapper (when using setarch)

Spec Name or Url: http://apt.kde-redhat.org/apt/fedora/SPECS/sbcl-0.9.4-2.spec
SRPM Name or Url:
http://apt.kde-redhat.org/apt/fedora/all/SRPMS.stable/sbcl-0.9.4-2.src.rpm
Comment 16 Gérard Milmeister 2005-08-30 07:02:08 EDT
0.9.4-2 builds fine in mock (fc-4).
There is a little error in the sbcl shell script:
Instead of:
SBCL_SETARCH=setarch i386 -R
it must be:
SBCL_SETARCH="setarch i386 -R"

There is a ppc binary for sbcl 0.8.15 on sbcl.sf.net.
Maybe this can be used to bootstrap 0.9.4 on ppc.
Unfortunately I have no means to try this out.
Comment 17 Gérard Milmeister 2005-08-30 07:07:25 EDT
How did you build the src rpm in order to include all binaries?
There are still conditional for including sources.
Comment 18 Rex Dieter 2005-08-30 07:46:14 EDT
> How did you build the src rpm in order to include all binaries?
> There are still conditional for including sources.

The conditional tells it which src to use, but all bootstraps are
unconditionally included:

# local Bootstrap binaries
Source10:
http://dl.sourceforge.net/sourceforge/sbcl/sbcl-%{version}-x86-linux-binary.tar.bz2
%ifarch %{ix86}
%define bootstrap_src -a 10
%endif
Source11:
http://dl.sourceforge.net/sourceforge/sbcl/sbcl-%{version}-x86-64-linux-binary.tar.bz2
%ifarch %{x86_64}
%define bootstrap_src -a 11
%endif


Comment 19 Rex Dieter 2005-08-30 09:32:54 EDT
* Tue Aug 30 2005 Rex Dieter <rexdieter[AT]users.sf.net> 0.9.4-3
- patch to avoid need to use setarch/app-wrapper
- fix app-wrapper (quote SBCL_SETARCH)
- include ppc bootstrap (oldish 0.8.15, untested)

Spec Name or Url: http://apt.kde-redhat.org/apt/fedora/SPECS/sbcl-0.9.4-3.spec
SRPM Name or Url:
http://apt.kde-redhat.org/apt/fedora/all/SRPMS.stable/sbcl-0.9.4-3.src.rpm
Comment 20 Rex Dieter 2005-08-30 09:37:21 EDT
I'm fairly certain the old ppc binary won't work to bootstrap.  I'll request
upstream to provide an updated ppc binary.
Comment 21 Gérard Milmeister 2005-08-30 12:53:54 EDT
The build of sbcl-0.9.4-3.src.rpm fails, because of %check.
run-tests.sh needs to be run within setarch.
The patch you added essentially does the same thing as
a wrapper script, but relies on /proc being present. However
this doesn't seem to when building with mock.
Comment 22 Rex Dieter 2005-08-30 13:09:45 EDT
%changelog
* Tue Aug 30 2005 Rex Dieter <rexdieter[AT]users.sf.net> 0.9.4-4
- safer NO_ADDR_RANDOMIZE patch
- use %%{?setarch} in %%check too
               
Spec Name or Url: http://apt.kde-redhat.org/apt/fedora/SPECS/sbcl-0.9.4-4.spec
SRPM Name or Url:
http://apt.kde-redhat.org/apt/fedora/all/SRPMS.stable/sbcl-0.9.4-4.src.rpm

Comment 23 Juho Snellman 2005-08-31 13:53:51 EDT
> Source10:
http://dl.sourceforge.net/sourceforge/sbcl/sbcl-%{version}-x86-linux-binary.tar.bz2
> Source11:
http://dl.sourceforge.net/sourceforge/sbcl/sbcl-%{version}-x86-64-linux-binary.tar.bz2

Upstream doesn't necessarily make binaries for all architectures for
all releases. I suspect it would be better to hardcode a specific
version here. The intent is that building SBCL always requires only a
reasonably compliant Common Lisp implementation, so there's no need to use the
latest release as the host compiler.

> ## Is sb-thread really limited to these archs only?  Don't think so, removing
limitation, for now. --Rex 
> #ifarch %{ix86} %{x86_64}
> sed -i -e "s|; :sb-thread|:sb-thread|" base-target-features.lisp-expr
> #endif

It's limited to just x86 and x86-64.

Comment 24 Rex Dieter 2005-08-31 15:05:40 EDT
>Upstream doesn't necessarily make binaries for all architectures for
>all releases. I suspect it would be better to hardcode a specific
>version here.

Plan is to eventually use native (Fedora Extras) bootstraps, and drop those bits
altogether.
Comment 25 Gérard Milmeister 2005-08-31 17:01:06 EDT
(In reply to comment #24)
> Plan is to eventually use native (Fedora Extras) bootstraps, and drop those bits
> altogether.
This might work for updated packages on a given platform, but for every new
platform (fc5, fc6, ...), bootstrapping is needed again.

Comment 26 Gérard Milmeister 2005-09-10 14:13:50 EDT
I think you can import the package. You can use the build system to try to build
a ppc version. If this does work without much problems, you can include it,
otherwise create a bug report about this and assign it to yourself.
Comment 27 Rex Dieter 2005-09-13 08:01:28 EDT
Imported into cvs, but now the build(s) are failing.  On FC4(and
rawhide/FC5)/i386, the build hangs late in the build process:
...
//testing for consistency of first and second GENESIS passes
//header files match between first and second GENESIS -- good

real    28m7.484s
user    26m47.750s
sys     0m35.730s
//entering make-target-2.sh
//doing warm init
... hang ...

  I had successfully built sbcl not too long ago, so I'm guessing some recent
update (kernel or glibc?) changed things. ??

The build (seems to) succeed on x86_64, but it fails some 'make check' tests:
http://buildsys.fedoraproject.org/logs//development/994-sbcl-0.9.4-9.fc5/

Arg.
Comment 28 Gérard Milmeister 2005-09-13 15:32:14 EDT
I have observed your heroic efforts fighting the build system :-)
I checked out the cvs and made "make i386" and "make mockbuild".
The first builds without errors and results in an rpm.
The second builds in a mock chroot environment and fails.
I get a long list of errors ("Help! 11 nested errors.
SB-KERNEL:*MAXIMUM-ERROR-DEPTH* exceeded.")
I don't know what goes wrong. However in the mock environment
there may be stronger ulimits or other conditions that do not
apply in a normal environment.
The gcc and glibc versions used are the same for both environment,
so this can't be the problem.
Maybe you ask for help on fedora-extras list.
Comment 29 Rex Dieter 2005-09-13 15:38:30 EDT
I've gotten it to build reliably for me on FC4, at least.  (-:

Gérard, the latest ADDR_NO_RANDOMIZE patch sbcl is building with atm is using
argv[0] instead of relying upon /proc/self/exe.  Do you see anything
wrong/dangerous in this approach?
Comment 30 Rex Dieter 2005-09-13 15:41:44 EDT
Woo hoo!  Time to go dance a jig.
-------------
 1047 (sbcl): Build on target development succeeded.
     Build logs may be found at
http://buildsys.fedoraproject.org/logs//development/1047-sbcl-0.9.4-13.fc5/
-------------

Should we close the bug now?  (I'll get FC-3/FC-4 targets built as soon as the
branches get made in cvs) 
Comment 31 Gérard Milmeister 2005-09-13 17:30:08 EDT
(In reply to comment #29)
> Gérard, the latest ADDR_NO_RANDOMIZE patch sbcl is building with atm is using
> argv[0] instead of relying upon /proc/self/exe.  Do you see anything
> wrong/dangerous in this approach?
GCL uses the same approach, so it's probably ok. I hope that memory management
for these LISP compilers while sometime adapted to work without this hack,
but I am sceptical.
Try to build maxima with GCL and SBCL. If the hack doesn't cause any problems
it seems good enough.
Comment 32 Rex Dieter 2005-09-13 21:07:45 EDT
Yep, maxima (5.9.1.9) builds with all lisps: clisp, gcl, sbcl, cmucl. 
Comment 33 Gérard Milmeister 2005-09-16 12:45:00 EDT
What was the problem that caused the build to fail?
Comment 34 Rex Dieter 2005-09-16 12:49:28 EDT
To what failed build are you referring? 
AFAIK, sbcl built fine (on fc5/devel), and I've been able to build the (newer)
maxima with sbcl on fc4.
Comment 35 Gérard Milmeister 2005-09-16 12:53:34 EDT
I meant the one mentioned in comment #27.
BTW, what about fc3 and fc4?
Comment 36 Rex Dieter 2005-09-16 12:56:30 EDT
Fixed FC4+ per comment #29, my local FC-3 builds are flaky (mostly not working),
but that's probably due to a lack of setarch -R there.
Comment 37 Rex Dieter 2005-09-16 16:31:45 EDT
sbcl-0.9.4-15.fc4 build succeeded (omitting pcc).
Comment 38 Gérard Milmeister 2005-09-17 06:36:04 EDT
The randomize patch of sbcl-0.9.4-15.fc4 doesn't seem to work in the
released build:
WARNING: Couldn't re-execute SBCL with the proper personality flags.  Trying to
continue anyway.

A previous build that I did myself in mock worked flawlessly.
I will reopen this bug now, maybe we create a new one to resolve this issue.
Comment 39 Rex Dieter 2005-09-17 06:43:41 EDT
I wonder how sbcl passed the 'make check' (which is quite thorough) section of
the specfile if the build is bad. (??).
Comment 40 Gérard Milmeister 2005-09-17 06:47:27 EDT
Somehow the patch is simply ineffectual and sbcl reverts to the previous
behaviour, i.e., sometimes it works, sometimes it doesn't. I think, it
was simply luck that made sbcl succeed the build :-(
Comment 41 Rex Dieter 2005-09-17 06:48:50 EDT
For the record, my FC3 builds (comment #36) were flaky for the same reason you
described... it complained about being unable to set personality.  Perhaps using
argv[0] in the exec call *is* unreliable, and we should stick to trying to use
/proc/self/exe instead.  I'll see about whipping up a replacement patch to do
one of the following:
1.  Use /proc/self/exe, if it exists, else use argv[0]
2.  Try to use argv[0], and if the personality re-exec fails, fall back to
trying /proc/self/exe instead.
Comment 42 Gérard Milmeister 2005-09-17 08:29:34 EDT
Yes, argv[0] *is* the problem:
It is set to "sbcl", however execve requires a full path, i.e., "/usr/bin/sbcl".
So either usr /proc/self/exe as before as you suggested, or create a script
that calls, e.g., /usr/lib/sbcl/bin/sbcl or something similar.
The argv[0] patch works with gcl, because gcl is a script that calls
/usr/lib/gcl-2.6.7/unixport/saved_ansi_gcl, whose argv[0] is of course
a full path.
Comment 43 Rex Dieter 2005-09-20 10:19:41 EDT
OK, should be all sorted out in sbcl-0.9.4-17
Comment 44 Anthony Green 2010-11-19 07:51:19 EST
Package Change Request
======================
Package Name: sbcl
New Branches: el6
Owners: green
InitialCC: rdieter
Comment 45 Anthony Green 2010-11-19 07:52:11 EST
(In reply to comment #44)
> Package Change Request
> ======================
> Package Name: sbcl
> New Branches: el6
> Owners: green
> InitialCC: rdieter

Oops. Ignore this.  Need coffee before I type.

Note You need to log in before you can comment on or make changes to this bug.