Bug 25109

Can you reproduce it? It doesn't happen here....

Certainly can reproduce it.  Here it is again on a different system, different 
installation...

[root@zelda mysql]# rpm -qa | grep mysql
mysql-3.23.29-1
mysqlclient9-3.23.22-3
php-mysql-4.0.4pl1-3.pdflib3
mysql-server-3.23.29-1
[root@zelda mysql]# /sbin/service mysqld status
mysqld (pid 9544 724 723 710) is running...
[root@zelda mysql]# /sbin/service mysqld stop
[  OK  ] down MySQL server[  OK  ]
[root@zelda mysql]# dir
mysql-3.23.32-1.7.i386.rpm        mysql-server-3.23.32-1.7.i386.rpm
mysql-devel-3.23.32-1.7.i386.rpm  mysqlclient9-3.23.22-3.i386.rpm
[root@zelda mysql]# rpm -Fvh mysql*
mysql                       ##################################################
mysql-server                ##################################################
[root@zelda mysql]# /sbin/service mysqld status
mysqld is stopped
[root@zelda mysql]# /sbin/service mysqld start
Starting MySQL:  [  OK  ]
[root@zelda mysql]# /sbin/service mysqld status
mysqld (pid 9815 9814 9812) is running...
[root@zelda mysql]# /sbin/service mysqld restart
Stopping MySQL:  [FAILED]
Starting MySQL:  [  OK  ]
[root@zelda mysql]# /sbin/service mysqld stop
Stopping MySQL:  [FAILED]
[root@zelda mysql]# /sbin/service mysqld start
Starting MySQL:  [  OK  ]
[root@zelda mysql]# mysql -u root -p
Enter password:
ERROR 1045: Access denied for user: 'root@localhost' (Using password: YES)
[root@zelda mysql]# dir ../oldmysql
mysql-3.23.29-1.i386.rpm        mysql-server-3.23.29-1.i386.rpm
mysql-devel-3.23.29-1.i386.rpm  mysqlclient9-3.23.22-3.i386.rpm
[root@zelda mysql]# rpm -Uvh --oldpackage ../oldmysql/mysql-3.23.29* ../oldmysql
/mysql-server*
mysql                       ##################################################
mysql-server                ##################################################
cannot remove /var/run/mysqld - directory not empty
[root@zelda mysql]# /sbin/service mysqld stop
[  OK  ] down MySQL server[  OK  ]
[root@zelda mysql]# /sbin/service mysqld restart
[FAILED] down MySQL server[FAILED]
Starting MySQL:  [  OK  ]
[root@zelda mysql]# /sbin/service mysqld restart
[  OK  ] down MySQL server[  OK  ]
Starting MySQL:  [  OK  ]
[root@zelda mysql]# /sbin/service mysqld stop
[  OK  ] down MySQL server[  OK  ]
[root@zelda mysql]# /sbin/service mysqld start
Starting MySQL:  [  OK  ]
[root@zelda mysql]# /sbin/service mysqld restart
[  OK  ] down MySQL server[  OK  ]
Starting MySQL:  [  OK  ]
[root@zelda mysql]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 3.23.29-gamma-log

Type 'help;' or '\h' for help. Type '\c' to clear the buffer

mysql> quit
Bye
[root@zelda mysql]# rpm -qa | grep mysql
mysqlclient9-3.23.22-3
php-mysql-4.0.4pl1-3.pdflib3
mysql-server-3.23.29-1
mysql-3.23.29-1
[root@zelda mysql]#

Well, I just tried building the RPMs locally with the SRPM, and I don't have 
any more success.  The only non-RedHat standard things that I've got on these 
two systems are pdflib-3.03 and kernel-2.4.0 (both built locally).  But, mind 
you, the old MySQL RPMS work fine in the environment.  Here's some more bash-
log for you to maul over.

[root@providence i386]# ps -e | grep mysql
[root@providence i386]# dir
mysql-3.23.32-1.7.i386.rpm        mysql-server-3.23.32-1.7.i386.rpm
mysql-devel-3.23.32-1.7.i386.rpm
[root@providence i386]# rpm -Fvh mysql*
mysql                       ##################################################
mysql-devel                 ##################################################
mysql-server                ##################################################
[root@providence i386]# ps -e | grep mysql
[root@providence i386]# /sbin/service mysqld start
Starting MySQL:  [  OK  ]
[root@providence i386]# /sbin/service mysqld stop
Stopping MySQL:  [FAILED]
[root@providence i386]# ps -e | grep mysql
 7232 pts/0    00:00:00 safe_mysqld
 7259 pts/0    00:00:00 mysqld
 7261 pts/0    00:00:00 mysqld
 7262 pts/0    00:00:00 mysqld
[root@providence i386]# /sbin/service mysqld status
mysqld (pid 7262 7261 7259) is running...
[root@providence i386]# killall mysqld
[root@providence i386]# ps -e | grep mysql
[root@providence i386]# /sbin/service mysqld status
mysqld dead but subsys locked
[root@providence i386]# rm /var/lock/subsys/mysqld
rm: remove `/var/lock/subsys/mysqld'? y
[root@providence i386]# /sbin/service mysqld status
mysqld is stopped
[root@providence i386]# /sbin/service mysqld start
Starting MySQL:  [  OK  ]
[root@providence i386]# /sbin/service mysqld stop
Stopping MySQL:  [FAILED]
[root@providence i386]# /sbin/service mysqld status
mysqld (pid 7421 7420 7418) is running...
[root@providence i386]# killall mysqld
[root@providence i386]# /sbin/service mysqld status
mysqld dead but subsys locked
[root@providence i386]# rpm -Uvh --oldpackage /home/lee/mysql/*
mysql                       ##################################################
mysql-devel                 ##################################################
mysql-server                ##################################################
[root@providence i386]# /sbin/service mysqld start
Starting MySQL:  [  OK  ]
[root@providence i386]# /sbin/service mysqld restart
[  OK  ] down MySQL server[  OK  ]
Starting MySQL:  [  OK  ]
[root@providence i386]#

If you still can't reproduce these problems there, then I'm tempted to try the 
RPMs from MySQL.com http://www.mysql.com/downloads/mysql-3.23.html for testing 
purposes.  Would you like me to do that?  What differences between the packages 
should I consider (i.e. paths)?

Try the standard kernel, and see if this works better... stock 2.4 is full of
problems. If this works, try the ac patch to 2.4 which fixes a huge pile of
problems.

Okay, I just did a fresh, full installation of Guinness on a third machine 
(this being a PII-233 and the other two being 1GHz Athlon and K6-2/500).

I loaded all of the errata updates minus the MySQL RPMs in question.  I did not 
do anything with the kernel (so I'm using kernel-2.2.16-22), nor did I install 
pdflib.

So, we're talking about a non-configured Guinness setup here... and same 
results exactly.  I won't bother you by posting them again.

I know that others aren't having this problem... so what to do from here?

Okay, so I restarted the system via 'shutdown -r now' and when it came up the 
new MySQL version began working properly.  So, I guess the question is now, 
what did rebooting do that the RPM installation didn't do?

Hmm.... did you have apache etc. running with connections to mysql?

In this last case, the answer is no, there were no apache connectins running to 
MySQL at the time.  But this further begs the question as to why the old RPMs 
worked fine when upgrading/downgrading, but not the new ones until I rebooted.

I don't think that it was rebooting, but it was the fact that I had run:

rpm -e `rpm -qa | grep mysql` (which removed php-mysql also)
rpm -Uvh mysql*  (to reinstall the new packages)
shutdown -r now

Somehow the order of the installation between php-mysql, mysql-3.23.29, and 
mysql-3.23.32 causes this.  I haven't been thorough in my analysis of it, 
though, but with this third machine I have gotten it to cause FAILED service 
stops twice between various installations of MySQL.  Now I've got it running 
fine.  But I believe that you can duplicate the problem if you...

1) install Guinness
2) update errata for mysql-3.23.29
3) update errata for php-4.0.4pl1
4) update errata for mysql-3.23.32

I haven't gone back to that, though to test, but it seems that if you have 
installed php-4.0.4pl1 before installing mysql-3.23.32 you can uninstall php-
mysql, install mysql-3.23.32, and then install php-mysql and it works.  Very 
curious...  I'm still working on getting the 1GHz Athlon to take the new MySQL 
installation... can't figure it out, though.

Okay, I got the Athlon running mysql-3.23.32 now.  I had to:

1) rpm -e `rpm -qa | grep mysql`
2) shutdown -r now (I *had* to do this part)
3) rpm -Uvh mysql* (installing the 3.23.32 RPMs)
4) rpm -Uvh php-mysql*

Then it works.  The problem must have had something to do with the installation 
order as I mentioned.

Nope I was wrong about the problem (again)... it has to do with my old MySQL 
database (in the example is known as mysql.backup):

[root@zelda lib]# rpm -q mysql
mysql-3.23.32-1.7
[root@zelda lib]# /sbin/service mysqld restart
Stopping MySQL:  [  OK  ]
Starting MySQL:  [  OK  ]
[root@zelda lib]# /sbin/service mysqld restart
Stopping MySQL:  [  OK  ]
Starting MySQL:  [  OK  ]
[root@zelda lib]# /sbin/service mysqld stop
Stopping MySQL:  [  OK  ]
[root@zelda lib]# dir
games  logrotate.status  misc  mysql  mysql.backup  nfs  rpm  slocate  xdm  xkb
[root@zelda lib]# rm -rf mysql
[root@zelda lib]# mv mysql.backup mysql
[root@zelda lib]# ls -l
total 36
drwxr-xr-x    2 root     root         4096 Feb  6  1996 games
-rw-r--r--    1 root     root          615 Jan 29 04:02 logrotate.status
drwxr-xr-x    2 root     root         4096 May 15  2000 misc
drwxr-xr-x    3 mysql    mysql        4096 Jan 29 18:43 mysql
drwxr-xr-x    3 root     root         4096 Jan 27 13:30 nfs
drwxr-xr-x    2 root     root         4096 Feb 14  1996 rpm
drwxr-x---    2 root     slocate      4096 Jan 29 04:02 slocate
drwxr-xr-x    2 root     root         4096 Aug 30 13:55 xdm
drwxr-xr-x    2 root     root         4096 Jan 27 13:32 xkb
[root@zelda lib]# /sbin/service mysqld start
Starting MySQL:  [  OK  ]
[root@zelda lib]# /sbin/service mysqld restart
Stopping MySQL:  [FAILED]
Starting MySQL:  [  OK  ]
[root@zelda lib]# /sbin/service mysqld restart
Stopping MySQL:  [FAILED]
Starting MySQL:  [  OK  ]
[root@zelda lib]# mv mysql mysql.backup
[root@zelda lib]# /sbin/service mysqld restart
Stopping MySQL:  [FAILED]
Initializing MySQL database [  OK  ]
Starting MySQL:  [  OK  ]
[root@zelda lib]# /sbin/service mysqld restart
Stopping MySQL:  [  OK  ]
Starting MySQL:  [  OK  ]
[root@zelda lib]# /sbin/service mysqld restart
Stopping MySQL:  [  OK  ]
Starting MySQL:  [  OK  ]
[root@zelda lib]#

What changed such that the old (empty) database would do this?

Here's the problem with the old database: it has a root password.
Take a look at this:

[root@zelda lib]# dir
games  logrotate.status  misc  nfs  rpm  slocate  xdm  xkb
[root@zelda lib]# rpm -q mysql
mysql-3.23.32-1.7
[root@zelda lib]# /sbin/service mysqld start
Initializing MySQL database [  OK  ]
Starting MySQL:  [  OK  ]
[root@zelda lib]# /sbin/service mysqld restart
Stopping MySQL:  [  OK  ]
Starting MySQL:  [  OK  ]
[root@zelda lib]# mysqladmin password thisisabadpassword
[root@zelda lib]# /sbin/service mysqld restart
Stopping MySQL:  [FAILED]
Starting MySQL:  [  OK  ]
[root@zelda lib]# /sbin/service mysqld stop
Stopping MySQL:  [FAILED]
[root@zelda lib]# /sbin/service mysqld status
mysqld (pid 930 929 927) is running...
[root@zelda lib]# ps -e | grep mysql
  900 pts/0    00:00:00 safe_mysqld
  927 pts/0    00:00:00 mysqld
  929 pts/0    00:00:00 mysqld
  930 pts/0    00:00:00 mysqld
[root@zelda lib]# rm -rf mysql
[root@zelda lib]# killall mysqld
[root@zelda lib]# ps -e | grep mysql
[root@zelda lib]# /sbin/service mysqld start
Initializing MySQL database [  OK  ]
Starting MySQL:  [  OK  ]
[root@zelda lib]# /sbin/service mysqld restart
Stopping MySQL:  [  OK  ]
Starting MySQL:  [  OK  ]
[root@zelda lib]# /sbin/service mysqld restart
Stopping MySQL:  [  OK  ]
Starting MySQL:  [  OK  ]
[root@zelda lib]# mysqladmin password
mysqladmin: Too few arguments to change password
[root@zelda lib]# mysqladmin password A1b2C3d4E5
[root@zelda lib]# /sbin/service mysqld restart
Stopping MySQL:  [FAILED]
Starting MySQL:  [  OK  ]
[root@zelda lib]#

Can you reproduce this?

Try putting this in /root/.my.cnf, and see if that helps...

Put what? "mysqladmin password somepassword" ?  Okay, so I did that, here's 
what I get when I attempt to start the daemon:

[root@zelda /root]# cd /var/lib
[root@zelda lib]# dir
games  logrotate.status  misc  mysql  nfs  rpm  slocate  xdm  xkb
[root@zelda lib]# rm -rf mysql
[root@zelda lib]# /sbin/service mysqld restart
Stopping MySQL:  [FAILED]
Initializing MySQL database error: Found option without preceding group in confi
g file: /root/.my.cnf at line: 1
Program aborted
Preparing db table
Preparing host table
Preparing user table
Preparing func table
Preparing tables_priv table
Preparing columns_priv table
Installing all prepared tables
Installation of grant tables failed!

Examine the logs in /var/lib/mysql for more information.
You can also try to start the mysqld demon with:
/usr/libexec/mysqld --skip-grant &
You can use the command line tool
/usr/bin/mysql to connect to the mysql
database and look at the grant tables:

shell> /usr/bin/mysql -u root mysql
mysql> show tables

Try 'mysqld --help' if you have problems with paths. Using --log
gives you a log in /var/lib/mysql that may be helpful.

The latest information about MySQL is available on the web at
http://www.mysql.com
Please consult the MySQL manual section: 'Problems running mysql_install_db',
and the manual section that describes problems on your OS.
Another information source is the MySQL email archive.
Please check all of the above before mailing us!
And if you do mail us, you MUST use the /usr/bin/mysqlbug script!
error: Found option without preceding group in config file: /root/.my.cnf at lin
e: 1
Program aborted
[FAILED]
[root@zelda lib]#

Where's the man page on the proper format of ~/.my.cnf ?  MySQL Reference 
Manual, chapter 11.5 doesn't seem to help much.

Here it is on a fourth unique system... a fresh install of Guinness, updated:

[root@metroid lib]# rpm -qa | grep mysql
mysql-devel-3.23.32-1.7
mysql-server-3.23.32-1.7
php-mysql-4.0.4pl1-3
mysql-3.23.32-1.7
mysqlclient9-3.23.22-3
[root@metroid lib]# /sbin/service mysqld start
Initializing MySQL database [  OK  ]
Starting MySQL:  [  OK  ]
[root@metroid lib]# /sbin/service mysqld restart
Stopping MySQL:  [  OK  ]
Starting MySQL:  [  OK  ]
[root@metroid lib]# mysqladmin password somelousypassword
[root@metroid lib]# /sbin/service mysqld restart
Stopping MySQL:  [FAILED]
Starting MySQL:  [  OK  ]
[root@metroid lib]# /sbin/service mysqld stop
Stopping MySQL:  [FAILED]
[root@metroid lib]# killall mysqld
[root@metroid lib]# ps -e | grep mysql
[root@metroid lib]# rm -rf /var/lib/mysql
[root@metroid lib]# /sbin/service mysqld start
Initializing MySQL database [  OK  ]
Starting MySQL:  [  OK  ]
[root@metroid lib]# /sbin/service mysqld restart
Stopping MySQL:  [  OK  ]
Starting MySQL:  [  OK  ]
[root@metroid lib]#

Can you duplicate this there?  It seems quite consistent here.

I haven't tried duplicating the last two days, but I tried a couple of times
before that - and we upgraded a couple of machines before the errata release.

I've changed the way mysql shuts down now... could you try 3.23.32-5 from
Rawhide and see if you can still reproduce this?

I don't see mysql-3.23.32-5 at
ftp://rawhide.redhat.com/pub/rawhide/i386/RedHat/RPMS
or
ftp://rawhide.redhat.com/pub/rawhide/SRPMS/SRPMS

Where can I find it?

It should be there, but since the site is probably swamped by now, take a look
at http://people.redhat.com/teg/

[deanox@providence deanox]$ ftp people.redhat.com
Connected to people.redhat.com.
220 charlotte.redhat.com FTP server (Version wu-2.6.1(1) Wed Aug 9 05:54:50 EDT
2000) ready.
530 Please login with USER and PASS.
530 Please login with USER and PASS.
KERBEROS_V4 rejected as an authentication type
Name (people.redhat.com:deanox): anonymous
331 Guest login ok, send your complete e-mail address as password.
Password:
230 Guest login ok, access restrictions apply.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> dir
227 Entering Passive Mode (199,183,24,237,3,76)
150 Opening ASCII mode data connection for /bin/ls.
total 176
drwxr-xr-x    2 5014     5014         4096 Oct 18 19:19 alexl
drwxr-xr-x    2 aviro    aviro        4096 Mar 30  2000 aviro
....
drwxrwxr-x    4 sopwith  sopwith      4096 Nov 15 20:13 sopwith
drwxr-xr-x    2 tfox     tfox         4096 Jul 31  2000 tfox
drwxr-xr-x   22 twaugh   twaugh       4096 Nov 30 15:24 twaugh
drwxr-xr-x    2 2911     2920         4096 Oct 17 18:40 vallande
drwxrwxr-x    6 wanger   wanger       4096 Apr 27  2000 wanger
drwxr-xr-x    4 wil      wil          4096 Dec 15 17:31 wil
drwxr-xr-x    2 2950     2950         4096 Jan 25 16:23 wjones
drwxr-xr-x    2 wolfgang wolfgang     4096 Aug  1  2000 wolfgang
226 Transfer complete.
ftp>

Where's the teg directory?  Also, I checked Rawhide again now for the mysql-
3.23.32-5 RPM or SRPM - and it's not there.  Any ideas?

http, not ftp - as for Rawhide, it's probably only synced to some servers yet.
Not sure why.

mysql*-3.23.32-5 RPMs work fine as far as I can tell.  Thanks!

Note that the existing mysqld service *must* be stopped before updating, 
otherwise stopping the services is next to impossible without rebooting.

Thanks again.