Bug 921226 - mongodb fails to build on ARM
Summary: mongodb fails to build on ARM
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: mongodb
Version: rawhide
Hardware: arm
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Troy Dawson
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-03-13 18:19 UTC by D. Marlin
Modified: 2013-05-19 02:32 UTC (History)
6 users (show)

Fixed In Version: mongodb-2.2.4-2.fc18
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-05-16 13:19:55 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description D. Marlin 2013-03-13 18:19:00 UTC
Description of problem:

mongodb fails to build on ARM.  The build terminates with an error (see below).  The same build completes successfully on x86.


Version-Release number of selected component (if applicable):

  mongodb-2.2.3-3.fc19


How reproducible:

  consistently


Steps to Reproduce:
1.  set up a rawhide mock environment on an ARM builder
2.  perform a mock build of mongodb
3.
  

Actual results:

  build fails with errors (see below).


Expected results:

  build succeeds.


Additional info:

scons: *** [/root/rpmbuild/BUILDROOT/mongodb-2.2.3-3.fc18.0.arm1.arm/usr/lib/libmongoclient.a] AssertionError : Installing source ['libmongoclient.a', 'build/linux2/extrapath__usr/ssl/use-system-all/usev8/client_build/libmongoclient.a'] into target ['/root/rpmbuild/BUILDROOT/mongodb-2.2.3-3.fc18.0.arm1.arm/usr/lib/libmongoclient.a']: target and source lists must have same length.


I can provide access to a test environment for reproducing this error.

I found that the same error had been reported when building on OSX, but I did not see a resolution:

  https://jira.mongodb.org/browse/SERVER-5575

Comment 1 Troy Dawson 2013-03-13 19:08:46 UTC
I believe this bug is more more in line with the problem and possible solution.

https://jira.mongodb.org/browse/SERVER-1811

You'll notice that is currently isn't supported and currently has no timetable for support.  But, there are also several patches in there that supposedly allow it to build.

Comment 2 D. Marlin 2013-03-13 22:32:03 UTC
If I am following the jira issue correctly, that is referring to the atomic intrinsics issue for ARM.  The test build that I am working on includes a patch for that issue, but the build fails due to a different (presumably unrelated) issue.  Please note that this error is an scons _install_ error (libmongoclient.a built successfully, but fails to install), and not the compile error that was encountered before with the unpatched sources, i.e.,

original build error:
----------
In file included from src/mongo/platform/atomic_intrinsics.h:46:0,
                 from src/mongo/platform/atomic_word.h:20,
                 from src/mongo/bson/oid.cpp:22:
src/mongo/platform/atomic_intrinsics_gcc.h: In member function 'void mongo::OID::initSequential()':
src/mongo/platform/atomic_intrinsics_gcc.h:123:44: error: impossible constraint in 'asm'
src/mongo/platform/atomic_intrinsics_gcc.h:123:44: error: impossible constraint in 'asm'
scons: *** [build/linux2/extrapath__usr/ssl/use-system-all/usev8/mongo/bson/oid.o] Error 1

In file included from src/mongo/platform/atomic_intrinsics.h:46:0,
                 from src/mongo/platform/atomic_word.h:20,
                 from src/mongo/db/lockstat.h:23,
                 from src/mongo/db/d_concurrency.h:27,
                 from src/mongo/db/client.h:36,
                 from src/mongo/db/db.cpp:25:
src/mongo/platform/atomic_intrinsics_gcc.h: In static member function 'static T mongo::AtomicIntrinsics<T, typename boost::disable_if_c<(sizeof (T) <= 4u)>::type>::fetchAndAdd(volatile T*, T) [with T = long long int]':
src/mongo/platform/atomic_intrinsics_gcc.h:123:44: error: impossible constraint in 'asm'
src/mongo/platform/atomic_intrinsics_gcc.h:123:44: error: impossible constraint in 'asm'
scons: *** [build/linux2/extrapath__usr/ssl/use-system-all/usev8/mongo/db/db.o] Error 1
----------

The patched SRPM (for ARM atomic intrinsics) still builds correctly on x86, but produces an scons install error on ARM.

I can provide a full build log of failed the ARM build, or the test SRPM including the ARM atomic intrinsics patch, if that would be helpful.

Comment 3 Troy Dawson 2013-03-22 22:05:50 UTC
Sorry for the delay, it's been a very busy week.
I think the test SRPMS, including the ARM patch would be most helpful.
The log wouldn't hurt, but the patch is what will help the most.

Comment 4 D. Marlin 2013-03-23 00:00:18 UTC
I managed to get this to build for ARM using the previously mentioned patch from Jon Masters and a small change to the spec file.  A scratch build using these changes is in arm.koji:

  http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=1609623

The SRPM that includes the patch and spec file change is available there.

So the issue now is, when I try to start the mongod service (daemon) to test, it just times out.  

----------
# systemctl start mongod.service
Job for mongod.service failed. See 'systemctl status mongod.service' and 'journalctl -xn' for details.

---

 systemd[1]: Starting High-performance, schema-free document-oriented database...
 systemd[1]: mongod.service operation timed out. Terminating.
 systemd[1]: Failed to start High-performance, schema-free document-oriente...ase.
 systemd[1]: Unit mongod.service entered failed state
----------


Should we close this issue and open a new one, or should we make sure the patch is not the cause of this problem before closing?

Comment 5 D. Marlin 2013-04-25 22:08:28 UTC
I did some more testing and found the patch was not working quite as intended.  I have rewritten the patch to use the gcc built-in atomic operations (libatomic) to replace the architecture-specific ones, and it seems to work now (tested on both x86_64 and ARM).

  http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=1746365

Again, the SRPM includes the new ARM patch and spec file change.

Please let me know if this approach looks acceptable, or if you need any additional information.

Comment 6 Troy Dawson 2013-04-26 14:15:00 UTC
This is great.  I'm going to run it through some tests, but my pre-tests are looking good.

I notice you didn't put anything in the changelog.  Do you mind if I give you credit in the changelog?

Comment 7 D. Marlin 2013-04-26 15:28:04 UTC
That would be fine.  Do I need to write up a changelog entry for it summarizing the change.  Sorry, I wasn't sure what process to follow.  Please let me know if I need to submit a patch made from git with an ACK by, or anything like that.

Comment 8 Troy Dawson 2013-04-26 19:02:13 UTC
I put this in the changlog

* Fri Apr 26 2013 David Marlin <dmarlin> - 2.2.3-5
- Patch to build on ARM (#921226)

The problem I am currently having is the %if statement that tells it to only apply the patch if it's building on an arm architecture.

I've tried a couple of options like

%ifarch %{arm}

The builds worked, even though the patch wasn't applied.

http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=1748059

But it probrubly has the same problem you mentioned before.

Comment 9 D. Marlin 2013-04-26 19:39:46 UTC
Is there a reason to only apply this patch for ARM?  It builds runs for me on ARM and x86_64, but if you want to avoid making changes to other archs, I can change the patch to only affect ARM.

Comment 10 D. Marlin 2013-04-30 22:16:56 UTC
I found that gcc-4.7 is the first version that includes gcc built-in atomic operations support, which is not an issue for F18/F19, but RHEL uses the same source version of mongodb and it uses gcc-4.4.  I have updated the patch to only use the gcc built-in atomic operations if the gcc version is 4.7 or newer.  The updated patch uses to the previous (platform-specific) atomic operations for any version of gcc < 4.7.

I took the SRPM from taskID=1748059, applied the new patch, and did a scratch build on F19 (i686, x86_64, armv7hl),

  http://koji.fedoraproject.org/koji/taskinfo?taskID=5318973
  http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=1762134

and a local build on RHEL6 (x86_64).  They all completed successfully.

The patch is applied unconditionally in the spec file, but it conditionally selects the atomic operation definitions in the header file, depending on gcc version.

Please let me know if this is more acceptable.

Comment 11 Troy Dawson 2013-05-01 13:48:41 UTC
That does make me feel better.
I'm going to try it on some RHEL builds and see how it does.

Comment 12 Fedora Update System 2013-05-02 22:23:31 UTC
mongodb-2.2.4-1.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/mongodb-2.2.4-1.fc19

Comment 13 Fedora Update System 2013-05-02 22:24:44 UTC
mongodb-2.2.4-1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/mongodb-2.2.4-1.fc18

Comment 14 Fedora Update System 2013-05-03 15:16:36 UTC
Package mongodb-2.2.4-1.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing mongodb-2.2.4-1.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-7311/mongodb-2.2.4-1.fc19
then log in and leave karma (feedback).

Comment 15 D. Marlin 2013-05-03 22:29:46 UTC
(In reply to comment #13)
> mongodb-2.2.4-1.fc18 has been submitted as an update for Fedora 18.
> https://admin.fedoraproject.org/updates/mongodb-2.2.4-1.fc18

This still fails to build on F18 ARM for armv5tel (ARMv5) _only_.  It works as designed on F18 armv7hl (ARMv7) and F19.  The issue is that __GCC_HAVE_SYNC_COMPARE_AND_SWAP_* is not supported in gcc for ARMv5, but it is supported for ARMv7.  This did not show up in our F19 test builds because ARMv5 is no longer supported (after F18).

I have added a patch for the relevant code that replaces the __sync_* operations with __atomic_* operations when gcc >= 4.7 (similar to the previous patch).  This allows mongodb to build for both ARMv5 and ARMv7 on F18 and F19.  I performed scratch builds on F18 and F19 for ARM and x86, and they all now build:

F18:
  http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=1774276
  http://koji.fedoraproject.org/koji/taskinfo?taskID=5329757

F19
  http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=1774282
  http://koji.fedoraproject.org/koji/taskinfo?taskID=5329551

I apologize for the oversight.

Comment 16 Troy Dawson 2013-05-04 12:59:22 UTC
Better to find it now than later, I'll make the change.

How do you get your arm builds to build on both armv5 and armv7?  I ran test builds and they only built on armv7.

Comment 17 D. Marlin 2013-05-04 17:03:17 UTC
F19 only builds ARMv7, but F18 should automatically build both ARMv5 and ARMv7.  The command I use to submit an F18 scratch build is:

  arm-koji build f18 --scratch <SRPM>

Comment 18 D. Marlin 2013-05-04 17:06:37 UTC
(In reply to comment #17)
> F19 only builds ARMv7, but F18 should automatically build both ARMv5 and
> ARMv7.  The command I use to submit an F18 scratch build is:
> 
>   arm-koji build f18 --scratch <SRPM>

Also, if you are building locally (not using koji), ARM will only build for the host platform (ARMv7 target on an ARMv7 build host).  If you want to build ARMv5 you must use an ARMv5 build host.

Comment 19 Fedora Update System 2013-05-06 16:34:12 UTC
mongodb-2.2.4-2.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/mongodb-2.2.4-2.fc18

Comment 20 Fedora Update System 2013-05-06 16:35:05 UTC
mongodb-2.2.4-2.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/mongodb-2.2.4-2.fc19

Comment 21 Fedora Update System 2013-05-15 15:44:35 UTC
Package mongodb-2.2.4-2.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing mongodb-2.2.4-2.fc18'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-7505/mongodb-2.2.4-2.fc18
then log in and leave karma (feedback).

Comment 22 Troy Dawson 2013-05-15 20:06:31 UTC
Why was this bug re-opened?  Does something else need to happen?

Comment 23 Fedora Update System 2013-05-19 02:32:40 UTC
mongodb-2.2.4-2.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.