Bug 679197

Summary: RFE - Blender svn snapshot work
Product: [Fedora] Fedora Reporter: Richard Shaw <hobbes1069>
Component: blenderAssignee: Jochen Schmitt <jochen>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: jochen, kwizart, promac
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-05-04 16:26:54 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Richard Shaw 2011-02-21 21:03:42 UTC
Description of problem:
When trying to build the blender beta on F14 from the F14 source RPM an error occurs. (Really two, but I fixed one myself)

Version-Release number of selected component (if applicable):
blender-2.56-7.fc15.src.rpm

How reproducible:
Every time.

Steps to Reproduce:
1. Install source rpm
2. rpmbuild -ba blender.spec
3.
  
Actual results:
source/gameengine/GameLogic/SCA_PythonController.cpp: In member function 'virtual void SCA_PythonController::Trigger(SCA_LogicManager*)':
source/gameengine/GameLogic/SCA_PythonController.cpp:411:70: error: cannot convert 'PyObject*' to 'PyCodeObject*' for argument '1' to 'PyObject* PyEval_EvalCode(PyCodeObject*, PyObject*, PyObject*)'
scons: *** [build/linux2/source/gameengine/GameLogic/SCA_PythonController.o] Error 1
scons: building terminated because of errors.
error: Bad exit status from /var/tmp/rpm-tmp.pkEHOS (%build)


RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.pkEHOS (%build)

Expected results:
RPM Package built

Additional info:
I fixed one problem in SOURCES/blender-2.56.config where the line read:
BF_PYTHON_INC = '${BF_PYTHON}/include/python${BF_PYTHON_VERSION}mu'

I'm not sure if the 'mu' is supposed to be at the end but if I did not remove it then building crapped out trying to compile node.c because it was looking for Python.h but could not find it. With the 'mu' removed it gets much further before running into the above error which is beyond my ability to fix.

Comment 1 Nicolas Chauvet (kwizart) 2011-02-21 21:22:46 UTC
IIRC there is a fix needed from python (or used in blender for the python version in f15).
That been said, we do not support this version in F-14 so the bug is invalid.

Comment 2 Richard Shaw 2011-02-21 21:29:10 UTC
I considered posting to the devel list but wasn't sure that was appropriate either. Oh well... The pre-compiled version from blender.org seems to work. 

Hopefully my fix for the config file is still useful.

Comment 3 Richard Shaw 2011-02-21 22:34:48 UTC
Nevermind. I see that the 'mu' is necessary since installing the python3 packages from rawhide in order to see if I can get it to build.

Comment 4 Paulo Roma Cavalcanti 2011-02-22 01:03:44 UTC
There is an F14 build here:

http://packages.atrpms.net/dist/f14/blender-2.56/

If you want you can take the patch from the .src.rpm.

The problem is that the menus are not showing up and I do not know
why yet. But maybe you have a better luck ...

Comment 5 Richard Shaw 2011-02-22 03:22:16 UTC
Well I'm not much of a programmer so I doubt I'll have much luck.

One question... When is blender-repack.sh called? In reviewing the file it seems to be hard coded for version 2.49b so I don't see how it would work probably with 2.56.

Also, the atrpms source has an extra patch: blender-2.49b-uid.patch

I'm not exactly sure what it's function is.

I have verified that the precompiled binary directly from blender.org seems to work including the menus showing up. Is their build system using a different version of python3 than we are?

Comment 6 Paulo Roma Cavalcanti 2011-02-22 09:24:42 UTC
(In reply to comment #5)
> Well I'm not much of a programmer so I doubt I'll have much luck.
> 
> One question... When is blender-repack.sh called? In reviewing the file it
> seems to be hard coded for version 2.49b so I don't see how it would work
> probably with 2.56.

blender-repack is not called. This is a full version with everything, including ffmpeg, so one can produce compressed avis.

> 
> Also, the atrpms source has an extra patch: blender-2.49b-uid.patch

Don't look at the spec. It is for blender 2.49b. Use the .src.rpm
The problem is that F14 has python3 3.1 and not 3.2. F14 should be built using
--without py32.

The updated spec can be found here:

http://orion.lcg.ufrj.br/RPMS/SPECS/blender.spec

> 
> I'm not exactly sure what it's function is.
> 
> I have verified that the precompiled binary directly from blender.org seems to
> work including the menus showing up. Is their build system using a different
> version of python3 than we are?

I did not spend time trying to see why I have no menus, and I also did not install F15/rawhide yet to test. 

But it seems to be this bug:

https://bugzilla.redhat.com/show_bug.cgi?id=671284

Maybe someone with F15 installed can check whether this bug is specific to F14 or not?

Comment 7 Richard Shaw 2011-02-22 14:18:52 UTC
(In reply to comment #6)
> blender-repack is not called. This is a full version with everything, including
> ffmpeg, so one can produce compressed avis.

Ok, so I can just ignore that.


> > Also, the atrpms source has an extra patch: blender-2.49b-uid.patch
> 
> Don't look at the spec. It is for blender 2.49b. Use the .src.rpm
> The problem is that F14 has python3 3.1 and not 3.2. F14 should be built using
> --without py32.

Where do I pass that option? Is it part of "rpmbuild..."? Or is it what I saw in the spec file?


> The updated spec can be found here:
> 
> http://orion.lcg.ufrj.br/RPMS/SPECS/blender.spec
> 
> > 
> > I'm not exactly sure what it's function is.
> > 
> > I have verified that the precompiled binary directly from blender.org seems to
> > work including the menus showing up. Is their build system using a different
> > version of python3 than we are?
> 
> I did not spend time trying to see why I have no menus, and I also did not
> install F15/rawhide yet to test. 
> 
> But it seems to be this bug:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=671284

I think I have a clue on that one. I ran blender with debug output and it says it couldn't find the "scripts/modules" directory, specifically the bpy_types.py file. 

I believe this is installed in /usr/share/blender/scripts/modules" and for some reason it's not looking there. I stayed up too late last night trying to figure out how the function that tries to find the modules determines where to look.

The function is "BLI_get_folder" in bpy.c:
---
void BPy_init_modules( void )
{
        extern BPy_StructRNA *bpy_context_module;
        PointerRNA ctx_ptr;
        PyObject *mod;

        /* Needs to be first since this dir is needed for future modules */
        char *modpath= BLI_get_folder(BLENDER_SCRIPTS, "modules");
        if(modpath) {
                // printf("bpy: found module path '%s'.\n", modpath);
                PyObject *sys_path= PySys_GetObject("path"); /* borrow */
                PyObject *py_modpath= PyUnicode_FromString(modpath);
                PyList_Insert(sys_path, 0, py_modpath); /* add first */
                Py_DECREF(py_modpath);
        }
        else {
                printf("bpy: couldnt find 'scripts/modules', blender probably wont start.\n");
        }
---
With debug output on I get the 'else' output to the console so I assume here is where the problem is.


> Maybe someone with F15 installed can check whether this bug is specific to F14
> or not?

Probably not since the reporter of the other bug was not using F14 but rawhide, correct?

I wasn't aware you could compile without python 3.2 so I installed the rawhide packages and I still don't get menus.

Comment 8 Richard Shaw 2011-02-22 14:58:46 UTC
I may be getting a little closer. A bug reported for archlinux mentioned that the .py files in /usr/share/blender/modules/scripts are not byte-compiled and since regular users don't have write access then they can not be done on first run.

How would I go about manually byte-compiling the files?

Comment 9 Richard Shaw 2011-02-22 16:03:55 UTC
Ok, I'm not sure why the files are not getting automatically byte-compiled but I even tried forcing it to by using "%py_byte_compile" but it still didn't work. I'm lost at this point.

Comment 10 Richard Shaw 2011-02-22 16:36:39 UTC
Ok, now I'm really confused. The files are getting byte-compiled but they are being put in the "__pycache__" subdirectory. Is this a python3 thing? Also, they are getting byte-compiled by python 3.2 even though I'm using the spec file you referenced that was supposed to disable it.

Since this is probably the problem with the menus not showing up I'm going to move all future comments to bug 671284

Comment 11 Paulo Roma Cavalcanti 2011-02-22 17:14:37 UTC
Your have to use the option for not using python 3.2:


rpmbuild .... --without py32


The default is with python 3.2

Comment 12 Richard Shaw 2011-02-22 18:59:55 UTC
I got the figured out but the with or without conditionals are very confusing at first.

I get a good build now using "--without py32" but when I try to install via yum (via local repo) I still get the following:

# yum install blender
Loaded plugins: changelog, presto, refresh-packagekit, remove-with-leaves, rpm-
              : warm-cache
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package blender.x86_64 0:2.56-8.fc14 set to be installed
--> Processing Dependency: libpython3.2mu.so.1.0()(64bit) for package: blender-2.56-8.fc14.x86_64
--> Finished Dependency Resolution
Error: Package: blender-2.56-8.fc14.x86_64 (local)
           Requires: libpython3.2mu.so.1.0()(64bit)
---

So something it still causing rpmbuild to pickup a dependency for python3.2.

Comment 13 Richard Shaw 2011-02-22 19:06:27 UTC
Nevermind... I updated the repo data but apparently yum was still using the cache. Installing now.

Comment 14 Richard Shaw 2011-02-22 21:22:13 UTC
I finally forced the python files in /usr/share/blender/scripts/modules/ to byte-compile but it didn't help. I'm back to thinking the problem is that it's not looking in /usr/share for the files.

Comment 15 Richard Shaw 2011-02-27 17:17:44 UTC
At the suggestion of others on the blender-linux mailing list, I'm going to use SVN instead of the 2.56-beta source. Apparently there were a lot of bugs in the released beta source. Also trying to convert from using scons to cmake which should make things easier (like setting the install dir to /usr instead of /usr/local).

Comment 16 Richard Shaw 2011-03-11 00:49:18 UTC
As I reported on the other bug I now have Blender 2.56 building well on Fedora 14 including reverse patching the current svn version to remove dependency on Python 3.2. 

I have been collaborating with the Suse package maintainer and have it using the now recommended cmake over scons.

It's building under mock and the rpmlint output is good. I've made sure it's Fedora compliant to the best of my ability and have put over 40 hours into this so I would appreciate a serious review.

Is there any interest in making this package officially available?

My plan is to make sure F14/i686 and F15 builds properly.

Comment 17 Paulo Roma Cavalcanti 2011-03-11 14:10:53 UTC
Please, post a link to the .src.rpm

Blender 2.56 has to be fixed before F15 goes out, so
the interface appears ...

Comment 18 Richard Shaw 2011-03-11 14:18:51 UTC
Just got it uploaded last night. My upload capacity on my DSL is pitiful.

https://docs.google.com/leaf?id=0B4A6of0Rl4nUN2E5MmQyNGUtYjk3MS00YWRlLTgxZmMtYzIxY2VhN2VmMmQw&hl=en

Comment 19 Nicolas Chauvet (kwizart) 2011-03-11 14:21:07 UTC
It also does matter to ensure that all packages using blender (such as image renderers) are tested against this version of blender 2.56. So the package would eventually be backported to F-14 along with a newer blender.

Comment 20 Paulo Roma Cavalcanti 2011-03-11 14:34:15 UTC
(In reply to comment #18)
> Just got it uploaded last night. My upload capacity on my DSL is pitiful.
> 
> https://docs.google.com/leaf?id=0B4A6of0Rl4nUN2E5MmQyNGUtYjk3MS00YWRlLTgxZmMtYzIxY2VhN2VmMmQw&hl=en

This is the wrong link. It is openshot-1.3.0 ....

Comment 21 Richard Shaw 2011-03-11 14:56:29 UTC
Crap. It's supposed to be the whole folder, not just the file. 

This should do it:

https://docs.google.com/leaf?id=0B4A6of0Rl4nUZDkxZWQ2NzItZTMxYi00NjU2LThkNWItNzM4YzEwNTJkZjQ2&hl=en

Also, this build currently requires openCOLLADA which can be disabled and it not currently in the Fedora repos. It's MIT licensed so I would think it would be acceptable.

Comment 22 Paulo Roma Cavalcanti 2011-03-11 19:13:49 UTC
I still don't see it. Now there is a mlt and mlt-python.

Don't you have a dropbox account or something outside google?

Comment 23 Richard Shaw 2011-03-11 19:26:06 UTC
Sorry, still getting a handle on the share settings. I set the "Collection" to anyone but apparently I have to do the same things with the individual files. Should work now. I tried the link with another browser that was not logged into Google.

Comment 24 Richard Shaw 2011-03-22 21:03:07 UTC
I've uploaded a newer svn build to the same link. I've also got rid of the quicktime dependency.

Comment 25 Nicolas Chauvet (kwizart) 2011-03-22 21:21:31 UTC
Please upload a diff against the current rawhide package from dist-git.
At least the way to describe the svn version doesn't comply with the guidelines:
https://fedoraproject.org/wiki/Packaging:NamingGuidelines#Snapshot_packages

Comment 26 Richard Shaw 2011-03-22 22:04:21 UTC
I may need some help with the naming issue. 

Since I have the svn revision number as a part of the source root directory name it needs to be in the "Version:" field unless I override the diectory name in %setup, right? 

The other issue is the release number needs to be the current release number for 2.56 beta +1 for post release packages. I didn't catch that the first time I read the guidelines.

So the current release number for F15 is 7 so mine should be 8, right?

As far as the diff, do you mean a binary diff? Or a deltarpm?

Comment 27 Richard Shaw 2011-03-23 15:53:48 UTC
I've now got current SVN also building for F15 (Python 3.2/gcc 4.6) under mock. 

Also:
- Requested that the gcc 4.6 patch (mine is updated from the one in the beta source rpm) gets integrated upstream but I haven't gotten a response yet.
- Updated the package name. It now is "blender-2.56-8.svn35722.<dist>.<arch>.rpm"
- Removed a %{__rm} macro as I saw the updated packaging guideline email.

Should I try building for i686 or leave that to koji when the package is considered accepted?

The current spec log is from the Suse package I based the build on since it was updated to use cmake. Should I "zero" it out and start fresh?

Comment 28 Paulo Roma Cavalcanti 2011-03-23 16:42:59 UTC
I looked at you spec and it is far from being acceptable in Fedora.
I am not saying it does not work, but that is not the point.

I think some of your patches could be used, but in my opinion, we should wait until an official 2.5 version is released. Maybe it is better to ship version 2.49b in F15, while version 2.5 is still in heavily development.

What do Jochen and Nicolas think?

Comment 29 Richard Shaw 2011-03-23 17:54:24 UTC
Let me know what needs to be corrected in the spec file. I assume %{__install} will need to be replaced with just "install". I'm not sure why the Suse maintainer used the macro.

I'm not sure if all the python macros are necessary for Fedora. Fedora seems to have a much more mature macro base than Suse.

Assuming we can get the package into an acceptable state, would it be appropriate to put this in the testing repo? That way people who want to new version can get it from an official source and those who don't care can stick with 2.49b.

Comment 30 Jochen Schmitt 2011-03-23 20:58:44 UTC
I have create blender-2.56-8.svn35722, but this is only a developement release which should not introduced into F-15.

I hope to get blender-2.57 oder later until begin May for F-15.

Comment 31 Richard Shaw 2011-03-30 14:10:19 UTC
2.57 RC1 was just announced on the development list using svn r35899 (currently on r35889 as of this morning). If I can get a good build after beating the spec file into compliance, can we get an official package at least into updates-testing for F14?

Comment 32 Nicolas Chauvet (kwizart) 2011-03-30 14:30:35 UTC
I will need to update LuxRender to work with blender from 2.57.
So given that it will break compatibilities and dependencies, this is "no" to backport anythinkg to f14 from my side at this step.

BTW! the 2.5x branch is a development branch. So is the 2.57 version: a step of development for 2.60 future stable release.

The blender stable version remains 2.49.

@Richard,
You seems to be happy to contribute to the package, but I don't see much diff proposal against our git version in devel until now...
(I don't read docs.google.com I expect a diff, not a src.rpm - so I still cannot comment your work).

Comment 33 Richard Shaw 2011-03-30 14:49:26 UTC
I am happy to help but I requested what exactly was meant by "diff" but I haven't gotten any clarification. 

I'm relatively new to packaging and am not an official packager for Fedora (or any of the 3rd party repos) so I'm still on the steep part of the learning curve.

I've made a first pass through:

http://fedoraproject.org/wiki/Package_update_HOWTO#Working_with_packages_in_the_stable_branches

But I don't yet understand it completely. I see the reference to a "diff" for the "Building for Rawhide" section, but not in the "Working with packages in the stable branches" section.

I'm currently attempting to do another F14 build with a new patch for Python 3.1 compatibility. I assume that if I follow the directions on the above mentioned link that I will not hurt anything? Being unfamiliar with the process I'm exercising a great deal of caution.

Comment 34 Richard Shaw 2011-04-01 03:33:15 UTC
One question regarding the gcc 4.6 patches. 

I asked about including them upstream and all the #include <stddef.h>'s have been added to the offending files. The only thing left in the path are the "struct" statements.

I got this response from the maintainer:

http://lists.blender.org/pipermail/bf-committers/2011-March/031203.html

So do we really need them?

Comment 35 Richard Shaw 2011-04-01 20:23:58 UTC
Something I just thought of... I've been building blender with openCOLLADA support.

The Suse maintainer has already created a source RPM for it to build it as a library. It's usually built in statically but makes the build time much longer and it doesn't revise nearly as often as Blender.

Since this would be considered a "new" package for Fedora what exactly needs to happen? 

The library version doesn't really have a "version" in the usual sense so I'm just using the svn revision number as the version. i.e.:

openCOLLADA-836-1.fc14.src.rpm

This actually builds a package called "libOpenCOLLADA0" and one called "openCOLLADA-devel". I'm not sure why he needed a 0 on the end of the library package but I didn't want to change it until getting input here.

Comment 36 Richard Shaw 2011-04-05 18:19:50 UTC
I did a git checkout of the current blender.

The main difference between it and my package is that I'm using cmake instead of scons. I'll continue to try and beat my spec file into compliance.

One thing I noticed is that the git build is still using wrapper scripts to load blender. I don't think this is necessary anymore as blender will create the proper directories in the users home directory the first time they save their configuration.

If we want to make sure the right directories are there before that happens then at a minimum the scripts need to be updated. There's no longer a "bpydata" directory and blender creates a directory including the blender version. For instance, on my system I have:

/home/richard/.blender/2.56/config

I'm not sure what the current status of loading system and user modules and plugins but the symbolic linking may not be necessary anymore either.

Comment 37 Richard Shaw 2011-04-06 14:05:13 UTC
As of r36024 the patch to use system glew should no longer be necessary.

Campbell has added a cmake flag WITH_BUILTIN_GLEW that can be turned off to use the system glew and the file:

/source/blender/editors/include/BIF_gl.h

has been updated to match the patch, i.e.: "#INCLUDE <GL/glew.h>"

I have built a packaged without error using this new flag. I also updated my spec to include removing the same directories from 'extern' as the official spec file.

Comment 38 Jochen Schmitt 2011-04-06 18:35:14 UTC
I have updated my package on rawhide to svn36007 which should show a proper UI after startup.

Comment 39 Nicolas Chauvet (kwizart) 2011-04-06 20:32:03 UTC
(In reply to comment #35)
> Something I just thought of... I've been building blender with openCOLLADA
> support.
Can you create a fedora package review for this. I'm usually more open to spec file created from rpmdev-newspec
Suffixing with 0 is wrong, it will suggest that we will support different ABI of a same library at runtime which is an exeption, not the rule. So we don't follow this logic in fedora. We do this prefixing with compat- the package name, search into the repository for reference.

Comment 40 Richard Shaw 2011-04-06 20:54:17 UTC
(In reply to comment #39)
> (In reply to comment #35)
> > Something I just thought of... I've been building blender with openCOLLADA
> > support.
> Can you create a fedora package review for this. I'm usually more open to spec
> file created from rpmdev-newspec
> Suffixing with 0 is wrong, it will suggest that we will support different ABI
> of a same library at runtime which is an exeption, not the rule. So we don't
> follow this logic in fedora. We do this prefixing with compat- the package
> name, search into the repository for reference.

I found out the 0 suffix is a Suse requirement. Something to do with figuring out when an incompatible API/ABI change is made. I've removed it from my latest build.

I'm uploading the SRPM to my google docs. Currently the package is named

openCOLLADA-836-1.fc14.src.rpm with the library being libOpenCOLLADA-836-1.fc14.x86_64.rpm

There isn't really "version" as they seem to go strictly by the svn rev and since the version field can only be numeric I just left it as 836 instead of faking a version like 8.3.6 so let me know what you think. Since svn rev's only increment then a newer svn rev should always update properly with yum.

I'm working on the packaging review now.

Comment 41 Richard Shaw 2011-04-06 21:05:18 UTC
Here it is:

https://bugzilla.redhat.com/show_bug.cgi?id=694287

Comment 42 Jochen Schmitt 2011-05-04 16:26:54 UTC
I will close this bug, because we have blender-2.57b on rawhide as an official release.

Comment 43 Richard Shaw 2011-05-04 17:14:25 UTC
It will probably be a separate bug report but now that openCOLLADA is an official Fedora package we need to look at updating the current blender build to use it. I'm not sure if that will require moving from scons to cmake or not but it would probably be a good idea anyway.