Description of problem: After a "fedup" from FC17 to FC18, the slave NIS database seems to be unusable by the FC18 ypserv process. The NIS Master is Fedora-17. Version-Release number of selected component (if applicable): Slave FC18 Host: kernel-3.7.2-201.fc18.i686 ypbind-1.36-7.fc18.i686 ypserv-2.29-3.fc18.i686 yp-tools-2.12-11.fc18.i686 Master FC17 Host: kernel-PAE-3.6.11-1.fc17.i686 ypbind-1.36-7.fc17.i686 ypserv-2.29-1.fc17.i686 yp-tools-2.12-9.fc17.i686 How reproducible: Always Steps to Reproduce: 1. Master NIS is Fedora-17 2. Slave NIS is Fedora-18 3. From the FC17 master: yppush ypservers ypservers->fedora18.localdomain: Callback timed out Actual results: ypservers->fedora18.localdomain: Callback timed out Expected results: fedora18.localdomain: Master's version not newer Additional info: From the Fedora18 host: ypwhich -m Can't find master for map ypservers. Reason: NIS map database is bad Work around is to have ypbind use the Fedora-17 master and not run the fedora-18 slave ypserv.
## A partial fix -- ypxfrd checks that the Master and Slave are using the same database packages; it will refuse to send the data if they are not. cd /var/yp rm -f My.Domain/* /usr/lib/yp/ypinit -s Fedora17_NIS_Master_Host ## this will replace the local slave's NIS database files ## although, ypxfrd still has errors ## then restart the NIS services... systemctl restart ypserv.service systemctl restart ypbind.service ## synchronization via ypxfrd still does NOT work, but the ## local slave is functional. ## ## So the real "fix" is to update ALL NIS hosts.
Reference: man ypxfrd If the on-disk format of the database on both machines is not the same, rpc.ypxfrd will refuse to send the data. After upgrading all NIS Master/Slave hosts to Fedora-18, ypxfrd works as expected.
Well, you are totally correct that if we want to use rpc.ypxfrd to speed up map transfers, then we need to have the same database formats on master and slave, because we simply copy pure binary files (quite hackish, but very fast). A similar issue would appear if master and slave would have different endianness. Since ypserv in Fedora 18 uses Tokyo Cabinet as database back-end now, it became incompatible with older versions and thus it is not possible any more to use rpc.ypxfrd to speed up transfer between F17 and F18 builds of ypserv. The reason of the change was license incompatibility between gdbm-1.9 (GPLv3+) and ypserv (GPLv2 only). However, database format incompatibility shouldn't do such harm, like you're reporting, because in case the format is different, a backup mechanism should be used, which means basically copying data and creating maps on slaves from scratch. I've tested it and this is actually broken.
Created attachment 686003 [details] Fix mode when opening Tokyo Cabinet files This patch fixes mode when opening Tokyo Cabinet files and fixes permission of newly created map files. Besides that also a script for re-building maps during upgrade has been enhanced, so it should handle master and slave configuration. Important notice: In case anyone sees issues with opening map files, please, consider rebuilding maps using /usr/lib/yp/ypinit utility, according your configuration (master, slave).
ypserv-2.29-6.fc18 has been submitted as an update for Fedora 18. https://admin.fedoraproject.org/updates/ypserv-2.29-6.fc18
Package ypserv-2.29-6.fc18: * should fix your issue, * was pushed to the Fedora 18 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing ypserv-2.29-6.fc18' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2013-1404/ypserv-2.29-6.fc18 then log in and leave karma (feedback).
I can confirm the issue: NIS master server on a Fedora 17 and NIS slave server on Fedora 18 (newly upgraded from Fedora 17), yppush does no longer work (even after a fresy ypinit -s <master_server>). I still have to test the proposed update. Maurizio
The proposed update *does* fix the problem, thank you!
Interesting... I have a strange "hang" in YUM when I try to install this update. I have updated ALL machines to Fedora-18, so I can't really prove that the proposed update fixes the F17 -> F18 compatibility issue. So, I think we can/should use Mauizio's testing as proof; and close this issue as Fixed. My YUM issue looks like this: yum -y install --enablerepo=updates-testing ypserv-2.29-6.fc18 Loaded plugins: langpacks, presto, refresh-packagekit Resolving Dependencies --> Running transaction check ---> Package ypserv.i686 0:2.29-3.fc18 will be updated ---> Package ypserv.i686 0:2.29-6.fc18 will be an update --> Finished Dependency Resolution Dependencies Resolved Package Arch Version Repository Size Updating: ypserv i686 2.29-6.fc18 updates-testing 158 k Transaction Summary Upgrade 1 Package Total download size: 158 k Downloading Packages: Setting up and reading Presto delta metadata ypserv-2.29-6.fc18.i686.rpm | 158 kB 00:00:05 Running Transaction Check Running Transaction Test Transaction Test Succeeded Running Transaction ----- HANG ---- No disk or CPU activity Hit ^C error: %pre(ypserv-2.29-6.fc18.i686) scriptlet failed, signal 2 Error in PREIN scriptlet in rpm package ypserv-2.29-6.fc18.i686 Verifying : ypserv-2.29-6.fc18.i686 The /var/log/yum.log entry is odd also: Jan 27 11:59:32 ypserv-2.29-6.fc18.i686: 100 The package did not install. I followed with a yum reinstall from the "fedora" repositor and all is OK again.
Actually I can confirm: I tested the update on two fedora 18 machines and on one of these I also experienced this "hang". I thought that it was due to the extensive trials and experiments that we performed on that machine because of the compatibility issue, so I did not mentioned the problem. Everything went fine on the other machine. It seems there is some problem in the preinstall scriptlet, but I did not investigate. (In reply to comment #9) > Interesting... I have a strange "hang" in YUM when I try to install this > update. I have updated ALL machines to Fedora-18, so I can't really prove > that > the proposed update fixes the F17 -> F18 compatibility issue. So, I think we > can/should use Mauizio's testing as proof; and close this issue as Fixed. > > My YUM issue looks like this: > > yum -y install --enablerepo=updates-testing ypserv-2.29-6.fc18 > Loaded plugins: langpacks, presto, refresh-packagekit > Resolving Dependencies > --> Running transaction check > ---> Package ypserv.i686 0:2.29-3.fc18 will be updated > ---> Package ypserv.i686 0:2.29-6.fc18 will be an update > --> Finished Dependency Resolution > > Dependencies Resolved > > Package Arch Version Repository Size > > Updating: > ypserv i686 2.29-6.fc18 updates-testing 158 k > > Transaction Summary > Upgrade 1 Package > > Total download size: 158 k > Downloading Packages: > Setting up and reading Presto delta metadata > ypserv-2.29-6.fc18.i686.rpm | 158 kB 00:00:05 > Running Transaction Check > Running Transaction Test > Transaction Test Succeeded > Running Transaction > > ----- HANG ---- No disk or CPU activity > Hit ^C > > error: %pre(ypserv-2.29-6.fc18.i686) scriptlet failed, signal 2 > Error in PREIN scriptlet in rpm package ypserv-2.29-6.fc18.i686 > Verifying : ypserv-2.29-6.fc18.i686 > > The /var/log/yum.log entry is odd also: > Jan 27 11:59:32 ypserv-2.29-6.fc18.i686: 100 > > The package did not install. I followed with a yum reinstall from > the "fedora" repositor and all is OK again.
(In reply to comment #10) > Actually I can confirm: I tested the update on two fedora 18 machines and on > one of these I also experienced this "hang". I thought that it was due to > the extensive trials and experiments that we performed on that machine > because of the compatibility issue, so I did not mentioned the problem. > Everything went fine on the other machine. > > It seems there is some problem in the preinstall scriptlet, but I did not > investigate. Thank you for the feedback, it seems to be indeed caused by deadlock when opening a locked map file during preinstall script. I think we have to imitate GDBM locking style -- which means non-blocking reading every-time. Map files are only read or created from scratch anyway, so non-blocking read and blocking one-time writing should be safe enough. I'll submit an update in few minutes.
(In reply to comment #11) > I'll submit an update in few minutes. The updated package is in Bodhi since yesterday [1], but still pending for testing. Feel free to test it. [1] https://admin.fedoraproject.org/updates/FEDORA-2013-1404/ypserv-2.29-7.fc18
Package ypserv-2.29-7.fc18: * should fix your issue, * was pushed to the Fedora 18 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing ypserv-2.29-7.fc18' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2013-1404/ypserv-2.29-7.fc18 then log in and leave karma (feedback).
I downloaded the ypserv-2.29-7.fc18.i686.rpm. To install this package the ypserv.service MUST be manually shut down first! If not, the yum install process will hang, see below. Note: All hosts have been updated to Fedora-18, so the original issue of a Fedora-17/18 compatibility, was not re-tested. After installation of the 2.29-7 version; ypserv, ypbind, yptest, yppush all seem to be happy with each other. With the exception that one host saw a Segmentation fault, during the YUM installation. /var/tmp/rpm-tmp.x50L2T: line 18: 15248 Segmentation fault (core dumped) /usr/lib/yp/yphelper -i "$map" > /dev/null 2>&1 This installation did install successfully??? YUM Hang snip : Running Transaction Check Running Transaction Test Transaction Test Succeeded Running Transaction ----- HANG ---- No disk or CPU activity Hit ^C error: %pre(ypserv-2.29-6.fc18.i686) scriptlet failed, signal 2 Error in PREIN scriptlet in rpm package ypserv-2.29-6.fc18.i686
(In reply to comment #14) > I downloaded the ypserv-2.29-7.fc18.i686.rpm. To install this > package the ypserv.service MUST be manually shut down first! If > not, the yum install process will hang, see below. Thanks for pointing to that. This can indeed happen, but only if the old version is between ypserv-2.29-1.fc18 and ypserv-2.29-5.fc18. I've added a stop/start commands during %pre section to not fall into this issue. > Note: All hosts have been updated to Fedora-18, so the original > issue of a Fedora-17/18 compatibility, was not re-tested. > > After installation of the 2.29-7 version; ypserv, ypbind, yptest, > yppush all seem to be happy with each other. With the > exception that one host saw a Segmentation fault, during the YUM > installation. > > /var/tmp/rpm-tmp.x50L2T: line 18: 15248 Segmentation fault > (core dumped) /usr/lib/yp/yphelper -i "$map" > /dev/null 2>&1 Hm, I can't reproduce this. In case you see it happen again, could you provide a backtrace of the failure? > This installation did install successfully??? When it ended during %pre section, then install probably wasn't successful. > YUM Hang snip : > > Running Transaction Check > Running Transaction Test > Transaction Test Succeeded > Running Transaction > > ----- HANG ---- No disk or CPU activity > Hit ^C > > error: %pre(ypserv-2.29-6.fc18.i686) scriptlet failed, signal 2 > Error in PREIN scriptlet in rpm package ypserv-2.29-6.fc18.i686 This will be fixed in ypserv-2.29-8.fc18, which is now built in koji: https://koji.fedoraproject.org/koji/buildinfo?buildID=382089 and is heading into bodhi right now.
(In reply to comment #15) > (In reply to comment #14) > > I downloaded the ypserv-2.29-7.fc18.i686.rpm. To install this > > package the ypserv.service MUST be manually shut down first! If > > not, the yum install process will hang, see below. Yes, the current distro-sync is ypserv-2.29-3.fc18.i686 > > Thanks for pointing to that. This can indeed happen, but only if the old > version is between ypserv-2.29-1.fc18 and ypserv-2.29-5.fc18. I've added a > stop/start commands during %pre section to not fall into this issue. > > > Note: All hosts have been updated to Fedora-18, so the original > > issue of a Fedora-17/18 compatibility, was not re-tested. > > > > After installation of the 2.29-7 version; ypserv, ypbind, yptest, > > yppush all seem to be happy with each other. With the > > exception that one host saw a Segmentation fault, during the YUM > > installation. > > > > /var/tmp/rpm-tmp.x50L2T: line 18: 15248 Segmentation fault > > (core dumped) /usr/lib/yp/yphelper -i "$map" > /dev/null 2>&1 > > Hm, I can't reproduce this. In case you see it happen again, could you > provide a backtrace of the failure? One more thing, this Seg Fault was on the NIS MASTER host. I don't know if that has anything to do with it? None of the Slave hosts had a problem. > > > This installation did install successfully??? > > When it ended during %pre section, then install probably wasn't successful. > > > YUM Hang snip : > > > > Running Transaction Check > > Running Transaction Test > > Transaction Test Succeeded > > Running Transaction > > > > ----- HANG ---- No disk or CPU activity > > Hit ^C > > > > error: %pre(ypserv-2.29-6.fc18.i686) scriptlet failed, signal 2 > > Error in PREIN scriptlet in rpm package ypserv-2.29-6.fc18.i686 > > This will be fixed in ypserv-2.29-8.fc18, which is now built in koji: > https://koji.fedoraproject.org/koji/buildinfo?buildID=382089 > and is heading into bodhi right now.
> Hm, I can't reproduce this. In case you see it happen again, could you > provide a backtrace of the failure? How do I get the "backtrace" of this failure? So, I re-tested the installation on the NIS MASTER and the Seg-Fault happened again, see following: Downloading Packages: Running Transaction Check Running Transaction Test Transaction Test Succeeded Running Transaction /var/tmp/rpm-tmp.7BiidK: line 18: 3553 Segmentation fault (core dumped) /usr/lib/yp/yphelper -i "$map" > /dev/null 2>&1 Updating : ypserv-2.29-7.fc18.i686 1/2 Cleanup : ypserv-2.29-3.fc18.i686 2/2 Verifying : ypserv-2.29-7.fc18.i686 1/2 Verifying : ypserv-2.29-3.fc18.i686 2/2 Updated: ypserv.i686 0:2.29-7.fc18 Complete! ## It appears to have installed correctly: rpm -qa | grep ypserv ypserv-2.29-7.fc18.i686
(In reply to comment #17) > > Hm, I can't reproduce this. In case you see it happen again, could you > > provide a backtrace of the failure? > How do I get the "backtrace" of this failure? Getting core file is not easy in systemd, but I managed to do that with the following steps: 1) add "LimitCORE=500000" into /lib/systemd/system/ypserv.service 2) # systemctl daemon-reload 3) # systemctl restart ypserv.service 4) # sysctl kernel.core_pattern=/core 5) do the upgrade After that you should see core files in root / and you should be able to get the backtrace using gdb ("gdb sbin/ypserv /core.XXXX" and then provide output of "backtrace" command). For getting the backtrace it would be better to downgrade back to the ypserv-2.29-3.fc18 (and install corresponding -debuginfo package) to debug the version that actually crashed (it was during %pre section, so old ypserv package was installed at that time). Anyway, thanks for your time, we really appreciate your help.
Your not going to like this... NO Segmentation Fault??? I did the following twice with NO problems? yum -y distro-sync rpm -q ypserv ## ypserv-2.29-3.fc18.i686 ulimit -c unlimited vim /usr/lib/systemd/system/ypserv.service ## added to the [Unit] section: LimitCORE=500000 systemctl --system daemon-reload systemctl stop ypserv.service sysctl kernel.core_pattern=/core yum -y install /var/tmp/Temp/ypserv-2.29-7.fc18.i686.rpm ## NO error (no Seg Fault) rpm -q ypserv ## ypserv-2.29-7.fc18.i686 systemctl start ypserv.service systemctl status ypserv.service yptest ##All tests passed
Package ypserv-2.29-8.fc18: * should fix your issue, * was pushed to the Fedora 18 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing ypserv-2.29-8.fc18' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2013-1404/ypserv-2.29-8.fc18 then log in and leave karma (feedback).
(In reply to comment #19) > Your not going to like this... NO Segmentation Fault??? > I did the following twice with NO problems? Never mind, just in case you'll hit this issue again in the future, we'll appreciate any lead what could be wrong.
ypserv-2.29-8.fc18 has been pushed to the Fedora 18 stable repository. If problems still persist, please make note of it in this bug report.