Bug 166800 - Review Request: sbcl: The Steel Bank Common Lisp development environment
Summary: Review Request: sbcl: The Steel Bank Common Lisp development environment
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Gérard Milmeister
QA Contact: David Lawrence
URL: http://sbcl.sourceforge.net/
Whiteboard:
Depends On:
Blocks: FE-ACCEPT
TreeView+ depends on / blocked
 
Reported: 2005-08-25 20:11 UTC by Rex Dieter
Modified: 2010-11-19 12:52 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2005-09-20 14:19:41 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Rex Dieter 2005-08-25 20:11:35 UTC
Spec Name or Url: http://apt.kde-redhat.org/apt/fedora/SPECS/sbcl-0.9.3-1.spec
SRPM Name or Url: http://apt.kde-redhat.org/apt/fedora/all/SRPMS/stable/sbcl-0.9.3-1.src.rpm
Description:
Steel Bank Common Lisp (SBCL) is a Open Source development environment
for Common Lisp. It includes an integrated native compiler,
interpreter, and debugger.

Comment 1 Gérard Milmeister 2005-08-25 20:28:53 UTC
How is sbcl boottrapped?
You are aware that sbcl on FC4 man startup correctly or not...?
It is guaranteed to startup only with setarch -R.

Comment 2 Rex Dieter 2005-08-25 20:40:43 UTC
Here's the bootstrap options in the specfile (lightly tested):

## define to be one of the following
# use our own/local bootstrap?
%define bootstrap_sbcl_local 1
#define bootstrap_sbcl sbcl
#define bootstrap_clisp clisp



Comment 3 Rex Dieter 2005-08-25 20:41:48 UTC
Wasn't aware of the startup issues on fc4 (had only lightly tested on fc3 and
rhel4 so far).

Comment 5 Gérard Milmeister 2005-08-29 13:17:34 UTC
I cannot download
http://apt.kde-redhat.org/apt/fedora/all/SRPMS/stable/sbcl-0.9.4-1.src.rpm
(object not found)

Comment 6 Rex Dieter 2005-08-29 13:19:16 UTC
My bad for posting before I actually put the files in place.  Should be good now.

Comment 7 Gérard Milmeister 2005-08-29 13:25:15 UTC
This should be SRPMS.stable, not SRPMS/stable.

Comment 8 Rex Dieter 2005-08-29 13:27:33 UTC
SRPM Name or Url:
http://apt.kde-redhat.org/apt/fedora/all/SRPMS.stable/sbcl-0.9.4-1.src.rpm

Comment 9 Gérard Milmeister 2005-08-29 13:43:01 UTC
* The spec file should be named sbcl.spec
* Shouldn't the src rpm contain bootstrapping binaries for all architectures?
  AFAIK src rpms are not architecture dependent.
* You may request a ppc binary from upstream, so that all fedora platforms
  are supported.
* Requires(post): /sbin/install-info
  Requires(preun): /sbin/install-info
* setarch -R is required on FC4 to start sbcl, you might create a script
  /usr/bin/sbcl that calls a renamed /usr/bin/sbcl.bin with setarch.
  Better yet, communicate with upstream to investigate the problem.
* Have you tried clisp to bootstrap sbcl? I did not succeed. If you do, the
  binaries are not necessary.

Comment 10 Rex Dieter 2005-08-29 13:50:35 UTC
specfile names will certainly be fixed when imported to svn.

> setarch -R is required on FC4 to start sbcl,

I figured if it was *built* with that option, then the resulting binary would
inherit that behavior.  Turns out you're right.

> Shouldn't the src rpm contain bootstrapping binaries for all architectures?
> AFAIK src rpms are not architecture dependent.

Hmm, good point, probably.

> Have you tried clisp to bootstrap sbcl? I did not succeed. If you do, the
> binaries are not necessary.

Tried, it failed.

Comment 11 Gérard Milmeister 2005-08-29 14:19:33 UTC
Where is %{?fedora} defined?

Comment 12 Rex Dieter 2005-08-29 14:20:59 UTC
%{fedora} is defined by the Extras buildsystem.  It's set to 3 on Fedora Core 3,
4 on Fedora Core 4.

Comment 13 Gérard Milmeister 2005-08-29 16:40:41 UTC
But %{fedora} is not defined in fc4's rpmbuild, i.e., the src rpm may not build
correctly for a user.

output of rpmlint:
W: sbcl devel-file-in-non-devel-package /usr/lib/sbcl/sb-bsd-sockets/alien/undefs.c
W: sbcl devel-file-in-non-devel-package /usr/lib/sbcl/sb-bsd-sockets/foo.c
W: sbcl devel-file-in-non-devel-package
/usr/lib/sbcl/sb-bsd-sockets/alien/get-h-errno.c
E: sbcl script-without-shellbang /usr/lib/sbcl/sb-bsd-sockets/alien/get-h-errno.c

Are these files necessary?
At least chmod 644 get-h-errno.c

E: sbcl info-dir-file /usr/share/info/dir

this must be removed

W: sbcl devel-file-in-non-devel-package /usr/lib/sbcl/sb-posix/alien/stat-macros.c
W: sbcl devel-file-in-non-devel-package
/usr/lib/sbcl/sb-posix/alien/waitpid-macros.c
W: sbcl devel-file-in-non-devel-package /usr/lib/sbcl/sb-posix/foo.c

Are these necessary?

Comment 14 Rex Dieter 2005-08-29 17:01:22 UTC
> But %{fedora} is not defined in fc4's rpmbuild, i.e., the src rpm may not build
> correctly for a user.

True, and eventually the default case could to make it work for end-users, but
don't forget that the primary target here is the Fedora Extras buildsystem.

> W: sbcl devel-file-in-non-devel-package 
> /usr/lib/sbcl/sb-posix/alien/stat-macros.c
> Are these necessary?

Dunno, probably not, just packaging what upstream does.

Comment 15 Rex Dieter 2005-08-29 17:11:58 UTC
%changelog
* Mon Aug 29 2005 Rex Dieter <rexdieter[AT]users.sf.net> 0.9.4-2
- rm -f /usr/share/info/dir
- fix perms on packaged .c files
- include all bootstrap binaries
- Requires(post,preun): /sbin/install-info
- sbcl.sh app-wrapper (when using setarch)

Spec Name or Url: http://apt.kde-redhat.org/apt/fedora/SPECS/sbcl-0.9.4-2.spec
SRPM Name or Url:
http://apt.kde-redhat.org/apt/fedora/all/SRPMS.stable/sbcl-0.9.4-2.src.rpm

Comment 16 Gérard Milmeister 2005-08-30 11:02:08 UTC
0.9.4-2 builds fine in mock (fc-4).
There is a little error in the sbcl shell script:
Instead of:
SBCL_SETARCH=setarch i386 -R
it must be:
SBCL_SETARCH="setarch i386 -R"

There is a ppc binary for sbcl 0.8.15 on sbcl.sf.net.
Maybe this can be used to bootstrap 0.9.4 on ppc.
Unfortunately I have no means to try this out.


Comment 17 Gérard Milmeister 2005-08-30 11:07:25 UTC
How did you build the src rpm in order to include all binaries?
There are still conditional for including sources.

Comment 18 Rex Dieter 2005-08-30 11:46:14 UTC
> How did you build the src rpm in order to include all binaries?
> There are still conditional for including sources.

The conditional tells it which src to use, but all bootstraps are
unconditionally included:

# local Bootstrap binaries
Source10:
http://dl.sourceforge.net/sourceforge/sbcl/sbcl-%{version}-x86-linux-binary.tar.bz2
%ifarch %{ix86}
%define bootstrap_src -a 10
%endif
Source11:
http://dl.sourceforge.net/sourceforge/sbcl/sbcl-%{version}-x86-64-linux-binary.tar.bz2
%ifarch %{x86_64}
%define bootstrap_src -a 11
%endif




Comment 19 Rex Dieter 2005-08-30 13:32:54 UTC
* Tue Aug 30 2005 Rex Dieter <rexdieter[AT]users.sf.net> 0.9.4-3
- patch to avoid need to use setarch/app-wrapper
- fix app-wrapper (quote SBCL_SETARCH)
- include ppc bootstrap (oldish 0.8.15, untested)

Spec Name or Url: http://apt.kde-redhat.org/apt/fedora/SPECS/sbcl-0.9.4-3.spec
SRPM Name or Url:
http://apt.kde-redhat.org/apt/fedora/all/SRPMS.stable/sbcl-0.9.4-3.src.rpm


Comment 20 Rex Dieter 2005-08-30 13:37:21 UTC
I'm fairly certain the old ppc binary won't work to bootstrap.  I'll request
upstream to provide an updated ppc binary.

Comment 21 Gérard Milmeister 2005-08-30 16:53:54 UTC
The build of sbcl-0.9.4-3.src.rpm fails, because of %check.
run-tests.sh needs to be run within setarch.
The patch you added essentially does the same thing as
a wrapper script, but relies on /proc being present. However
this doesn't seem to when building with mock.

Comment 22 Rex Dieter 2005-08-30 17:09:45 UTC
%changelog
* Tue Aug 30 2005 Rex Dieter <rexdieter[AT]users.sf.net> 0.9.4-4
- safer NO_ADDR_RANDOMIZE patch
- use %%{?setarch} in %%check too
               
Spec Name or Url: http://apt.kde-redhat.org/apt/fedora/SPECS/sbcl-0.9.4-4.spec
SRPM Name or Url:
http://apt.kde-redhat.org/apt/fedora/all/SRPMS.stable/sbcl-0.9.4-4.src.rpm



Comment 23 Juho Snellman 2005-08-31 17:53:51 UTC
> Source10:
http://dl.sourceforge.net/sourceforge/sbcl/sbcl-%{version}-x86-linux-binary.tar.bz2
> Source11:
http://dl.sourceforge.net/sourceforge/sbcl/sbcl-%{version}-x86-64-linux-binary.tar.bz2

Upstream doesn't necessarily make binaries for all architectures for
all releases. I suspect it would be better to hardcode a specific
version here. The intent is that building SBCL always requires only a
reasonably compliant Common Lisp implementation, so there's no need to use the
latest release as the host compiler.

> ## Is sb-thread really limited to these archs only?  Don't think so, removing
limitation, for now. --Rex 
> #ifarch %{ix86} %{x86_64}
> sed -i -e "s|; :sb-thread|:sb-thread|" base-target-features.lisp-expr
> #endif

It's limited to just x86 and x86-64.



Comment 24 Rex Dieter 2005-08-31 19:05:40 UTC
>Upstream doesn't necessarily make binaries for all architectures for
>all releases. I suspect it would be better to hardcode a specific
>version here.

Plan is to eventually use native (Fedora Extras) bootstraps, and drop those bits
altogether.

Comment 25 Gérard Milmeister 2005-08-31 21:01:06 UTC
(In reply to comment #24)
> Plan is to eventually use native (Fedora Extras) bootstraps, and drop those bits
> altogether.
This might work for updated packages on a given platform, but for every new
platform (fc5, fc6, ...), bootstrapping is needed again.



Comment 26 Gérard Milmeister 2005-09-10 18:13:50 UTC
I think you can import the package. You can use the build system to try to build
a ppc version. If this does work without much problems, you can include it,
otherwise create a bug report about this and assign it to yourself.

Comment 27 Rex Dieter 2005-09-13 12:01:28 UTC
Imported into cvs, but now the build(s) are failing.  On FC4(and
rawhide/FC5)/i386, the build hangs late in the build process:
...
//testing for consistency of first and second GENESIS passes
//header files match between first and second GENESIS -- good

real    28m7.484s
user    26m47.750s
sys     0m35.730s
//entering make-target-2.sh
//doing warm init
... hang ...

  I had successfully built sbcl not too long ago, so I'm guessing some recent
update (kernel or glibc?) changed things. ??

The build (seems to) succeed on x86_64, but it fails some 'make check' tests:
http://buildsys.fedoraproject.org/logs//development/994-sbcl-0.9.4-9.fc5/

Arg.

Comment 28 Gérard Milmeister 2005-09-13 19:32:14 UTC
I have observed your heroic efforts fighting the build system :-)
I checked out the cvs and made "make i386" and "make mockbuild".
The first builds without errors and results in an rpm.
The second builds in a mock chroot environment and fails.
I get a long list of errors ("Help! 11 nested errors.
SB-KERNEL:*MAXIMUM-ERROR-DEPTH* exceeded.")
I don't know what goes wrong. However in the mock environment
there may be stronger ulimits or other conditions that do not
apply in a normal environment.
The gcc and glibc versions used are the same for both environment,
so this can't be the problem.
Maybe you ask for help on fedora-extras list.


Comment 29 Rex Dieter 2005-09-13 19:38:30 UTC
I've gotten it to build reliably for me on FC4, at least.  (-:

Gérard, the latest ADDR_NO_RANDOMIZE patch sbcl is building with atm is using
argv[0] instead of relying upon /proc/self/exe.  Do you see anything
wrong/dangerous in this approach?

Comment 30 Rex Dieter 2005-09-13 19:41:44 UTC
Woo hoo!  Time to go dance a jig.
-------------
 1047 (sbcl): Build on target development succeeded.
     Build logs may be found at
http://buildsys.fedoraproject.org/logs//development/1047-sbcl-0.9.4-13.fc5/
-------------

Should we close the bug now?  (I'll get FC-3/FC-4 targets built as soon as the
branches get made in cvs) 

Comment 31 Gérard Milmeister 2005-09-13 21:30:08 UTC
(In reply to comment #29)
> Gérard, the latest ADDR_NO_RANDOMIZE patch sbcl is building with atm is using
> argv[0] instead of relying upon /proc/self/exe.  Do you see anything
> wrong/dangerous in this approach?
GCL uses the same approach, so it's probably ok. I hope that memory management
for these LISP compilers while sometime adapted to work without this hack,
but I am sceptical.
Try to build maxima with GCL and SBCL. If the hack doesn't cause any problems
it seems good enough.

Comment 32 Rex Dieter 2005-09-14 01:07:45 UTC
Yep, maxima (5.9.1.9) builds with all lisps: clisp, gcl, sbcl, cmucl. 

Comment 33 Gérard Milmeister 2005-09-16 16:45:00 UTC
What was the problem that caused the build to fail?

Comment 34 Rex Dieter 2005-09-16 16:49:28 UTC
To what failed build are you referring? 
AFAIK, sbcl built fine (on fc5/devel), and I've been able to build the (newer)
maxima with sbcl on fc4.

Comment 35 Gérard Milmeister 2005-09-16 16:53:34 UTC
I meant the one mentioned in comment #27.
BTW, what about fc3 and fc4?

Comment 36 Rex Dieter 2005-09-16 16:56:30 UTC
Fixed FC4+ per comment #29, my local FC-3 builds are flaky (mostly not working),
but that's probably due to a lack of setarch -R there.

Comment 37 Rex Dieter 2005-09-16 20:31:45 UTC
sbcl-0.9.4-15.fc4 build succeeded (omitting pcc).

Comment 38 Gérard Milmeister 2005-09-17 10:36:04 UTC
The randomize patch of sbcl-0.9.4-15.fc4 doesn't seem to work in the
released build:
WARNING: Couldn't re-execute SBCL with the proper personality flags.  Trying to
continue anyway.

A previous build that I did myself in mock worked flawlessly.
I will reopen this bug now, maybe we create a new one to resolve this issue.

Comment 39 Rex Dieter 2005-09-17 10:43:41 UTC
I wonder how sbcl passed the 'make check' (which is quite thorough) section of
the specfile if the build is bad. (??).

Comment 40 Gérard Milmeister 2005-09-17 10:47:27 UTC
Somehow the patch is simply ineffectual and sbcl reverts to the previous
behaviour, i.e., sometimes it works, sometimes it doesn't. I think, it
was simply luck that made sbcl succeed the build :-(

Comment 41 Rex Dieter 2005-09-17 10:48:50 UTC
For the record, my FC3 builds (comment #36) were flaky for the same reason you
described... it complained about being unable to set personality.  Perhaps using
argv[0] in the exec call *is* unreliable, and we should stick to trying to use
/proc/self/exe instead.  I'll see about whipping up a replacement patch to do
one of the following:
1.  Use /proc/self/exe, if it exists, else use argv[0]
2.  Try to use argv[0], and if the personality re-exec fails, fall back to
trying /proc/self/exe instead.

Comment 42 Gérard Milmeister 2005-09-17 12:29:34 UTC
Yes, argv[0] *is* the problem:
It is set to "sbcl", however execve requires a full path, i.e., "/usr/bin/sbcl".
So either usr /proc/self/exe as before as you suggested, or create a script
that calls, e.g., /usr/lib/sbcl/bin/sbcl or something similar.
The argv[0] patch works with gcl, because gcl is a script that calls
/usr/lib/gcl-2.6.7/unixport/saved_ansi_gcl, whose argv[0] is of course
a full path.

Comment 43 Rex Dieter 2005-09-20 14:19:41 UTC
OK, should be all sorted out in sbcl-0.9.4-17

Comment 44 Anthony Green 2010-11-19 12:51:19 UTC
Package Change Request
======================
Package Name: sbcl
New Branches: el6
Owners: green
InitialCC: rdieter

Comment 45 Anthony Green 2010-11-19 12:52:11 UTC
(In reply to comment #44)
> Package Change Request
> ======================
> Package Name: sbcl
> New Branches: el6
> Owners: green
> InitialCC: rdieter

Oops. Ignore this.  Need coffee before I type.


Note You need to log in before you can comment on or make changes to this bug.