Bug 162416 - GDBM database dump/restore utility to migrate files between architectures
GDBM database dump/restore utility to migrate files between architectures
Status: ASSIGNED
Product: Fedora
Classification: Fedora
Component: gdbm (Show other bugs)
rawhide
x86_64 Linux
medium Severity medium
: ---
: ---
Assigned To: Marek Skalický
Fedora Extras Quality Assurance
: FutureFeature
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2005-07-04 07:31 EDT by Matthias Saou
Modified: 2015-12-09 12:18 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Output from strace dumpgdbm (3.32 KB, text/plain)
2005-07-04 09:28 EDT, Matthias Saou
no flags Details

  None (edit)
Description Matthias Saou 2005-07-04 07:31:10 EDT
Description of problem:
PHP recompiled with GDBM support didin't seem to work on RHEL4 x86_64, and
trying to access the GDBM files by other means seems to fail all the time.

Version-Release number of selected component (if applicable):
gdbm-1.8.0-24.x86_64

How reproducible:
Always.

Steps to Reproduce:
1. Download and compile dumpgdbm from http://krabulator.free.fr/devel/dumpgdbm.html
2. Run dumpgdbm on a test GDBM file, it segfaults
3. Now compile dumpgdbm on i386, install gdbm.i386, readline.i386 and
libtermcap.i386 (all provided in RHEL4 x86_64), run that dumpgdbm on the same
GDBM file, it works
  
Actual results:
Access to GDBM files from x86_64 applications doesn't work, from i386 it does.

Expected results:
GDBM file access should work natively from x86_64 applications.

Additional info:
I've also tried installing the latest FC Development x86_64 package, same
result. I also tried with gdbm 1.8.3 (since all RH/FC products are still at
1.8.0) and again, same result.
Comment 1 Matthias Saou 2005-07-04 09:26:44 EDT
Some more information on the problem :

- With testgdbm, on a file that works fine on an i386 host :
./testgdbm -g ~/stats.db
gdbm_open failed, File seek error

- Strace of dumpgdbm on that same file :
"lseek(3, 35184372097024, SEEK_SET)      = -1 EINVAL (Invalid argument)" (see
the attached strace.txt file)

So it seems to be some kind of file seek error, for whatever that might mean.
Comment 2 Matthias Saou 2005-07-04 09:28:17 EDT
Created attachment 116328 [details]
Output from strace dumpgdbm
Comment 3 Matthias Saou 2005-07-04 09:54:44 EDT
More digging into the issue :
- Recompile the "testgdbm" program with "make progs" in the gdbm sources.
- On the x86_64 machine, run "testgdbm -g test.gdbm" and insert 2 entries.
- Copy the file over to the i386 machine, run "testgdbm -g test.gdbm", it seems
to work, but the database doesn't contain the inserted entries, it only contains
one empty key/value pair.

The other way around :
Doing the exact same operation but inserting on i386 and trying to read on
x86_64 results in testgdbm's "gdbm_open failed, File seek error" error message.

So it seems that GDBM files created on i386 and x86_64 are incompatible. This is
probably not normal, as endianess is the same, so sompatibility could probably
be achieved.
Comment 4 Bill Nottingham 2005-07-04 20:52:38 EDT
gdbm databases are, afaik, wordsize specific. Cc'ing Nalin, who I believe looked
at this once.
Comment 5 Matthias Saou 2005-07-05 05:46:06 EDT
Unfortunately (for me), you are right. I got this reply from the current
maintainer :

"Of course it works.  You're accessing a 32bit GDBM file with a 32bit library.
You can not, however, access a 32bit GDBM file with a *64bit* library.
GDBM files are not portable between different architectures."

So, I guess I could nevertheless change this bug to an RFE, to include some kind
of dump/restore utility that would allow moving GDBM files across architectures?
I'm pretty sure I won't be the last to move GDBM files from i386 to x86_64
servers...

The utility that did it for me was gdbmtext :
http://starynkevitch.net/Basile/gdbmtext.c

Which requires the following change to not segfault in some common cases when
importing :

--- gdbmtext.c.orig     2005-07-04 17:15:06.963692448 +0200
+++ gdbmtext.c  2005-07-04 19:10:02.505410768 +0200
@@ -71,6 +71,7 @@ indatum (char **ppc)
       xbuf[1] = pc[2];
       xbuf[2] = xbuf[3] = 0;
       *(wp++) = (char) strtol (xbuf, 0, 16);
+      pc += 2;
     } else
       *(wp++) = *pc;
   };
Comment 6 Matthias Saou 2005-11-07 06:45:49 EST
Anyone willing to review that gdbmtext.c piece of code and get it included
either in upstream gdbm or at least in the Fedora/RH packages? Some basic
dump/restore functionnality is something really valuable when the format is
binary incompatible across supported architectures...

Changing this to a RFE while I'm at it.
Comment 7 Paul Nasrat 2006-08-08 18:29:22 EDT
Move to Fedora
Comment 8 Matthias Saou 2007-02-05 08:59:00 EST
Is anyone still interested in providing a gdbm "migration tool"?
Comment 9 Red Hat Bugzilla 2007-08-21 01:20:29 EDT
User pnasrat@redhat.com's account has been closed
Comment 10 Ondrej Dvoracek 2007-09-05 18:51:27 EDT
Hi,
I am new owner of this bug. 
I will have a look at the gdbmtext.c and will decide whether we should try to
add this package to the distribution.

Cheers Ondrej
Comment 11 Sergio Monteiro Basto 2010-01-19 17:17:12 EST
(In reply to comment #8)
> Is anyone still interested in providing a gdbm "migration tool"?    

Hi , I am facing this kind of problem , but I need a dbm which is not architecture dependent , because I need read on 32 and 64-bit platforms , 

Any suggestion ? 
Thanks,
Comment 12 Karel Klíč 2010-03-02 08:30:24 EST
Hi Sergio,
please ask on a public mailing list, only very few people read Bugzilla entries.
Comment 13 Fedora Admin XMLRPC Client 2011-01-07 09:10:37 EST
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 14 Honza Horak 2011-10-05 07:32:45 EDT
Note, that a new gdbm-1.9.1 (already in Rawhide) provides different magic values for 32 and 64 bits, so we can discover what system the file was created on if we use this new version. Even though it is not a solution, I hope it can help at least a bit.
Comment 15 Fedora Admin XMLRPC Client 2014-09-11 11:13:42 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 16 Sergio Monteiro Basto 2015-12-02 20:33:38 EST
Some weeks ago I found that sqlite3 is a very good alternative and also seems that is platform independent [1] 

[1] http://stackoverflow.com/questions/15024658/are-sqlite3-databases-are-platform-independent
Comment 17 Honza Horak 2015-12-09 12:18:43 EST
Well, sqlite3 might be suitable alternative in some cases, but it is generally very different product.

Note You need to log in before you can comment on or make changes to this bug.