Bug 815263

Lua 5.2 is a major update including a soname bump and introduces some fairly major changes to the language, requiring porting efforts on all/most dependant packages. Perhaps most notably, rpm needs to be ported to work with lua-5.2 before this can be considered even in rawhide (or alternatively, lua-5.2 could be introduced as a differently named package similar to python3, or lua-5.1 renamed to a compatibility package, but these approaches have issues of their own)

lua >= 5.2 is needed for nmap >= 6.25
right now, not even rawhide has lua 5.2.x

And for a good reason. It breaks a lot of things. We will most likely propose this as a feature in the F-19 or F-20 time frame.

That sounds reasonable to me.

FWIW, rpm >= 4.11 (currenly in rawhide) supports Lua 5.2 so from that POV starting to plan to move ahead is ok now, but since Lua is a dependency of rpm itself, soname bumps are tricky. And then there are questions like whether any of the <lua> scripts in Fedora package set are affected by the changes in Lua language itself (unlikely but technically possible), and how to deal with that on upgrades from older releases etc. Plus of course the dozens of other packages depending on Lua. So definitely "feature" material.

Please don't forget that Feature Submission Deadline for Fedora 19 is 2013-01-29.

FPC has just granted a temporary bundling exception for nmap to bundle a newer lua through F19 (which works out to F18 and F19 in practice).  In order to not go through this same thing for F20 (possibly with more painful results) we wanted to ask what the plan for updating lua is.

Going to propose this as an F20 feature as soon as F19 branches?  (a week or so away)?  That would seem to be the best plan.

This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle.
Changing version to '19'.

(As we did not run this process for some time, it could affect also pre-Fedora 19 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19

*** Bug 913417 has been marked as a duplicate of this bug. ***

As I said on the other bug, I have a 5.2.2 package (derived from current 5.1.4 package) that I'm happy with here:

http://konradm.fedorapeople.org/fedora/SPECS/lua.spec
http://konradm.fedorapeople.org/fedora/SRPMS/lua-5.2.2-1.fc17.src.rpm

I stripped out the autotooling and instead dropped a few simple patches on upstream's makefile-based buildsystem, which should make for easier updates...

Can we get a Koji branch to start updating Lua and seeing what breaks? Who do I need to bribe around here to see this happen? =)

releng ticket is the way to get a koji branch:

https://fedorahosted.org/rel-eng/newticket

Why drop the autoconf patches? They have served us well and forward porting has been easy in the past. They ensure building with proper flags etc. I'd rather not like them to be dropped out of ignorance. Is there a particular reason? Afterall we need to carry patches to any build system.

I appreciate your help, but we haven't had any contact before. So also before I approve the commit ACLs (I'm happy to have someone co-maintain Lua!) I'd like to know a little more. Maybe contact me off-bugzilla to discuss this a little and let's keep the other Lua maintainers in the loop.

(In reply to comment #11)
> Why drop the autoconf patches?

They don't apply anything like cleanly when things change (e.g. 5.1.4 -> 5.2.2). And my patches are a total of 10-20 lines of changes, vs 1000s of lines of autotools diff. It seems like a reasonable reduction of complexity.

> They have served us well and forward porting
> has been easy in the past.

I didn't find it easy to forward port them, but I'm no autotools expert. How have you done it in the past?

> They ensure building with proper flags etc.

Have you looked at the makefile / build.log? Lua is only ~40 files and it's pretty easy to examine the one Makefile (src/Makefile) and confirm that the correct flags are used (mostly just by setting MYCFLAGS).

> I'd
> rather not like them to be dropped out of ignorance. Is there a particular
> reason? Afterall we need to carry patches to any build system.

Sure, I am ignorant. What's the benefit? Lua has an existing build system; it's simple, just two fairly straightforward Makefiles. And I found it only needed a couple small patches to be suitable for Fedora (correct %optflags, install into RPM_BUILD_ROOT, link only bin/lua against -lreadline, ...).
 
> I appreciate your help, but we haven't had any contact before. So also
> before I approve the commit ACLs (I'm happy to have someone co-maintain
> Lua!) I'd like to know a little more. Maybe contact me off-bugzilla to
> discuss this a little and let's keep the other Lua maintainers in the loop.

I don't need commit access -- I don't particularly care whether we get lua-5.2.x with autotools or without, and I don't want to step on your toes if you plan to update Lua to 5.2 with the autotooling. I just saw that the last update was in 2011 and assumed you were an inactive maintainer (maybe looking for a comaintainer).

I do want us to start making progress on it. 5.2 came out back in December 2011! Fedora is really late to the party on this one.

(In reply to comment #11)
> Why drop the autoconf patches? They have served us well and forward porting
> has been easy in the past. They ensure building with proper flags etc. I'd
> rather not like them to be dropped out of ignorance. Is there a particular
> reason? Afterall we need to carry patches to any build system.

I personally have a bit of an issue with a patch larger than the entire uncompressed Lua source:
[pmatilai@localhost lua]$ ls -l lua-5.1.4-autotoolize.patch lua-5.1.4.tar
-rw-rw-r--. 1 pmatilai pmatilai 1443402 Mar  7  2012 lua-5.1.4-autotoolize.patch
-rw-rw-r--. 1 pmatilai pmatilai  880640 Sep  3  2008 lua-5.1.4.tar

Of course that's just auto*foo insanity wrt generated sources, the actual change to switch to autotools is fairly simple in reality. It'd be a whole lot less scarier if the autotoolize patch only carried the auto*foo "sources" with spec doing 'autoreconf' to get all the generated goo in the place, or alternatively carry the autogenerated stuff in a separate patch.

+1 to removing generated autotools files and running autoreconf when building. Since upstream doesn't use autotools, I don't see a need to include the generated stuff.

As far as removing autotools entirely, I disagree. The lua-5.1.4-autotoolize.patch patch size is very misleading. If you just look at the source code for each build system, the Autotools one is actually shorter: 176 lines of configure.ac + Makefile.am's versus 234 lines of upstream Makefiles.

To demonstrate, I divided the patch into separate patches, each patch being a specific purpose. Here's the breakdown:

95.5% - Autoconf-generated files
 3.0% - luaconf.h move (the vast majority of this patch is just a file move)
 0.5% - removing Lua's build system
 0.4% - Configuration for Autoconf

For the especially curious, here's the raw numbers:

-rw-rw-r--. 1 dafrito dafrito 1378523 Apr 10 13:47 autotools.patch
-rw-rw-r--. 1 dafrito dafrito 1443402 Apr 10 13:46 lua-5.1.4-autotoolize.patch
-rw-rw-r--. 1 dafrito dafrito   46425 Apr 10 13:43 luaconf.patch
-rw-rw-r--. 1 dafrito dafrito    5846 Apr 10 13:42 real-config.patch
-rw-rw-r--. 1 dafrito dafrito    7673 Apr 10 13:42 removals.patch

This discussion isn't entirely germane to getting Lua upgraded to 5.2, but I figure I'd chip in my two cents here. If desired, I'll whip up a branch of http://pkgs.fedoraproject.org/git/lua.git that wouldn't have the generated files in there.

I cloned http://pkgs.fedoraproject.org/git/lua.git and updated it to not include sources generated from the Autotools. This is still for Lua 5.1.4, but I figure it'd be good to get synced up before moving forward with Lua 5.2:

    git clone git://github.com/dafrito/fedora-lua.git

I personally find large patchfiles to be terrible for transparency, so to ensure proper review, I automatically generate those patches from another git repo I made, found here:

    git clone git://github.com/dafrito/fedora-lua-src.git
    cd fedora-lua-src
    ./rpm/make-patches.sh
    cd rpm/patches
    ls -l
total 40
-rw-rw-r--. 1 dafrito dafrito  5872 Apr 10 17:02 lua-5.1.4-2.patch
-rw-rw-r--. 1 dafrito dafrito 21481 Apr 10 17:02 lua-5.1.4-autotoolize.patch
-rw-rw-r--. 1 dafrito dafrito   288 Apr 10 17:02 lua-5.1.4-idsize.patch
-rw-rw-r--. 1 dafrito dafrito   389 Apr 10 17:02 lua-5.1.4-lunatic.patch

make-patches.sh is a 20-line shell script that uses 'git diff' to create the patches listed above. You can look at the commit history and generate the patches yourself to ensure everything created works properly.

If this changes are reasonable, I'll use the same repo, rebase off lua 5.2, and see where I end up.

(In reply to comment #15)
> I cloned http://pkgs.fedoraproject.org/git/lua.git and updated it to not
> include sources generated from the Autotools.
>
> ... snip ...
>
> If this changes are reasonable, I'll use the same repo, rebase off lua 5.2,
> and see where I end up.

This sounds quite reasonable to me, let us know how it goes =). +1

I finished up rebasing off 5.2.2. The specfile and patches can be found here (a clone of the pkgs.fp.o lua repo):

https://github.com/dafrito/fedora-lua/tree/lua-5.2

As done during my 5.1.4 rework, I based the patches roughly off a series of git commits, so if you're interested in reviewing what was changed from the upstream tarball, look at the commits in this repo:

https://github.com/dafrito/fedora-lua-src/commits/lua-5.2

Be advised that if you're checking out those repos, you'll need to switch to the lua-5.2 branch to see 5.2 stuff.

### Changes ###

My approach was to be somewhat conservative in changing the old Autotools layout, though I did make minor changes in how a couple things were done. The changes include:

* readline inclusion is now more specific, and no changes are needed from the specfile
* luac statically links only against liblua, rather than using -static
* moved lua.pc.in to src (etc/ no longer exists in upstream)

The git history is the best place to look if you want to know everything that was changed. I did not produce a diff against the 5.1.4 Autotools arrangement, though I could do so fairly easily.

### rpmlint ### 

Just for completeness (since this isn't a full package review), I also ran rpmlint against the built rpm and specfile:

$ rpmlint ~/rpmbuild/RPMS/x86_64/lua-5.2.2-1.fc18.x86_64.rpm
lua.x86_64: W: spelling-error %description -l en_US bytecodes -> byte codes, byte-codes, postcodes
lua.x86_64: W: shared-lib-calls-exit /usr/lib64/liblua-5.2.so exit.5
1 packages and 0 specfiles checked; 0 errors, 2 warnings.

$ rpmlint lua.spec 
0 packages and 1 specfiles checked; 0 errors, 0 warnings.

The shared-lib-calls-exit refers to an exit() call in src/loslib.c, which implements the os.exit() call from Lua. As a result, I believe this warning can be ignored. For the curious, Lua's panic handling code calls abort() rather than exit().

### Next steps ###

Please let me know if this stuff looks acceptable. I'm familiar with RPM and I've been reading up on Fedora's packaging docs, but I've probably made some mistakes in the process. Thanks in advance!

LGTM. Tim?

Shameless bump to see if there's been any movement on this? I emailed Tim for co-maintainership but didn't hear anything back. Tim, if you're just busy at the moment, that's fine, but I am curious what the next step is here.

Any update on this?

Spot is working on this in rawhide (f20).

Lua 5.2.2 is now available in rawhide repository thanks to Spot and Aaron Faanes for rebasing the newest version. This bug report is now closed.

*** Bug 986703 has been marked as a duplicate of this bug. ***