Description of problem: As a side-effect of an update of man-db the following showed up in a cron output: /etc/cron.daily/man-db.cron: col: Invalid or incomplete multibyte or wide character OTOH 'man col' seems to display just fine and a cursory look at a content of /usr/share/man/man1/col.1.gz does not make anything jump on me as "obvious". What has really a problem here? Version-Release number of selected component (if applicable): util-linux-ng-2.18-5.fc15.x86_64 man-db-2.5.9-1.fc15.x86_64
Hm, it dawned on me only now that maybe this is not a complaint about col.1.gz but /usr/bin/mandb is running col which is unhappy about something. It is not obvious on quick glance, though, as /usr/bin/mandb is ELF 64-bit LSB executable.
Could you arrange for mandb to be run with the -d (or --debug) option the next time cron invokes it, and send me the output?
(In reply to comment #2) > Could you arrange for mandb to be run with the -d (or --debug) option the next > time cron invokes it, and send me the output? It appears that mandb is too smart to fall for that. :-) After a check what it is doing is pretty obvious that it does not bother with pages already registered in its database. OTOH dropping all index.db files from /var/cache/man and running it again forces mandb to check all available manpages. With --debug this produces for me 35M of output (with both stdin and sterr redirected to a file). Only there is no it this "Invalid" line from my report; so I have no idea where this came from. Maybe a real offender was already replaced? Still I collected 27 lines of "whatis parse for <something> failed" sort and two "ignoring bogus filename" (for manl/intro_blas1.man.gz and manl/ssyrk.txt_new.gz). The later two already reported in April as bug #583182. These "failed" lines will show up also when not using --debug option and an output then is only 12K long. Anybody wants to look at any of these two files? For the record - these manpages make mandb unhappy: Apache::TestServer.3pm.gz checkXML.1.gz CPAN::Debug.3pm.gz CPAN::HandleConfig.3pm.gz CPAN::Queue.3pm.gz CPAN::Tarzip.3pm.gz drpmsync.8.gz faxformat.1.gz kbuildsycoca4.8.gz kcookiejar4.8.gz kde4-config.1.gz kded4.8.gz kdeinit4.8.gz kdeoptions.7.gz kdesu.1.gz kjs.1.gz kjscmd.1.gz kross.1.gz lapack.l.gz libpm.3.gz meinproc4.8.gz mpd.1.gz nepomukserver.8.gz nepomukservicestub.8.gz PDA::Pilot.3pm.gz qtoptions.7.gz wireshark-filter.4.gz At least the last one is bug #635878.
I'm seeing this, too. Every day, cron.daily issues the complaint from col. Running "mandb -c -d" from the command line does not show the complaint. Doing some random checking in the language-specific man files turned up this: % zcat /usr/share/man/bg/man1/deja-dup-preferences.1.gz | enca -L bulgarian Universal transformation format 8 bits; UTF-8 Doubly-encoded to UTF-8 from CP1251 Could doubly-encoded UTF-8 cause the error message?
I determined what was causing the "col: Invalid or incomplete multibyte or wide character" message on my system by making a change to the man-db.cron job. To locate the problem I used debug mode and redirected output of stdout and stderr. In man-db.cron the command change looks like this. #mandb $OPTS mandb -d > /root/stdout 2> /root/stderr I then set up a temporary man-db.cron job to run using crontab. When the job finished I searched stderr for the col message and this is what I found. ... Testing for existence: /usr/share/man/man8/groupadd.8 Testing for existence: /usr/share/man/man8/inadyn.8 inadyn(8) is not in the db. Starting pipeline: gzip -dc /var/cache/man/cat8/inadyn.8.gz | /usr/libexec/man-d b/manconv -f UTF-8:ISO-8859-1 -t UTF-8//IGNORE | col -bx [input: {0, /var/cache/ man/cat8/inadyn.8.gz}, output: {-1, NULL}] Started "gzip", pid 11976 Started "/usr/libexec/man-db/manconv -f UTF-8:ISO-8859-1 -t UTF-8//IGNORE", pid 11977 Started "col", pid 11978 Starting pipeline: [input: {6, NULL}, output: {-1, NULL}] trying encoding UTF-8 -> UTF-8//IGNORE Waiting for pipeline: [input: {6, NULL}, output: {-1, NULL}] col: Invalid or incomplete multibyte or wide character Waiting for pipeline: gzip -dc /var/cache/man/cat8/inadyn.8.gz | /usr/libexec/man-db/manconv -f UTF-8:ISO-8859-1 -t UTF-8//IGNORE | col -bx [input: {0, /var/cache/man/cat8/inadyn.8.gz}, output: {-1, NULL}] Active processes (3): "gzip" (11976) -> -1 "/usr/libexec/man-db/manconv -f UTF-8:ISO-8859-1 -t UTF-8//IGNORE" (11977) -> 0 "col" (11978) -> -1 Active processes (2): "gzip" (11976) -> 0 "col" (11978) -> -1 Active processes (1): "col" (11978) -> 256 record = 'inadyn-mt - a client for open DNS servers.' base = 'inadyn' name = 'inadyn-mt', id = E name = 'inadyn', id = D Testing for existence: /usr/share/man/man1/tftp.1 ... In my case it was the man file inadyn.8.gz. However, the inadyn-mt package was only on my system a short period of time and is no longer there. What remained behind was a copy of the man file inside the cache [/var/cache/man/cat8/inadyn.8.gz]. The col message stopped appearing after I removed that file from the cache. I don't claim to know exactly what man-db is doing, but I do know that removing the offending file from the cache made the message go away. Perhaps mandb should clear the cache when a manpage is no longer in the db.
I was able to reproduce this bug by moving a man file from /usr/share/man that was in /var/cache/man. I first checked /var/cache/man/cat8 and found that mandb.8.gz was in the cache. Next I moved mandb.8.gz from /usr/share/man/man8 to a temporary location. Then it was just a matter of running my modified cronjob and take a look at the results. ... Testing for existence: /usr/share/man/man8/shutdown.8 Testing for existence: /usr/share/man/man8/mandb.8 mandb(8) is not in the db. Starting pipeline: gzip -dc /var/cache/man/cat8/mandb.8.gz | /usr/libexec/man-db /manconv -f UTF-8:ISO-8859-1 -t UTF-8//IGNORE | col -bx [input: {0, /var/cache/m an/cat8/mandb.8.gz}, output: {-1, NULL}] Started "gzip", pid 9718 Started "/usr/libexec/man-db/manconv -f UTF-8:ISO-8859-1 -t UTF-8//IGNORE", pid 9719 Started "col", pid 9720 Starting pipeline: [input: {6, NULL}, output: {-1, NULL}] trying encoding UTF-8 -> UTF-8//IGNORE Waiting for pipeline: [input: {6, NULL}, output: {-1, NULL}] col: Invalid or incomplete multibyte or wide character Waiting for pipeline: gzip -dc /var/cache/man/cat8/mandb.8.gz | /usr/libexec/man-db/manconv -f UTF-8:ISO-8859-1 -t UTF-8//IGNORE | col -bx [input: {0, /var/cache/man/cat8/mandb.8.gz}, output: {-1, NULL}] Active processes (3): "gzip" (9718) -> -1 "/usr/libexec/man-db/manconv -f UTF-8:ISO-8859-1 -t UTF-8//IGNORE" (9719) -> 0 "col" (9720) -> -1 Active processes (2): "gzip" (9718) -> -1 "col" (9720) -> 256 Active processes (1): "gzip" (9718) -> 0 record = 'mandb - create or update the manual page index caches' base = 'mandb' name = 'mandb', id = D Testing for existence: /usr/share/man/man8/cron.8 Testing for existence: /usr/share/man/man8/usermod.8 ... You'll notice that the "col: Invalid or incomplete multibyte or wide character" message appeared. If I remove mandb.8.gz from the cache or restore it to /usr/share/man/man8 the message goes away. This condition can occur if a package with a man file is installed and the man file viewed thereby putting it in the cache. After which the package is removed, but a man file from that package remains in the cache. In my opinion this is not a problem with col itself, but how man-db is using col. I feel that this bug should be updated to make the component man-db and the Fedora version 14.
Is this related to the col man page using an endash instead of a dash? i.e.: .Dd May 2011 " " .Dt COL(1) "" "User Commands" .Os util-linux .Sh NAME .Nm col .Nd filter reverse line feeds from input I've just seen this on Fedora 16.
The bug title is wrong - the error message is from an invocation of the col program by mandb, and has nothing to do with the col(1) manual page. I haven't had a chance to pick through the logs yet, sorry; but, at least, William Makowski is correct that this should be reassigned to man-db.
(In reply to comment #8) > The bug title is wrong - the error message is from an invocation of the col > program by mandb, and has nothing to do with the col(1) manual page. Yes. See a follow up in a comment #1.
I also am seeing this with Fedora 16. It has been a consistent message that I have seen in Fedora 14 and 15 previously. So please update this to reflect Fedora 16 vice Fedora 14.
This bug has been reported elsewhere, possibly: - http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=446939 - http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=555331 - https://bugs.archlinux.org/task/18722
William, do you think you could try to reproduce this bug with this build and see whether it is fixed? http://koji.fedoraproject.org/koji/buildinfo?buildID=311838 Thanks, peter
I should have some time to run it through this weekend and will follow-up afterwards.
I downloaded the source from the build in comment #12 and performed a rebuild on a Fedora 16 box. It was also necessary to rebuild groff since groff-base was a dependency, looks like this split out as of Fedora 17. Sorry, the "col: Invalid or incomplete multibyte or wide character" message continues to be a problem in man-db-2.6.1-1. I reproduced the bug using the method outlined in comment #6. The message also appears with the current version of man in Fedora 16, man-db-2.6.0.2-2.
William, what's your locale? Please post output of: $ locale Thanks, peter
Here is the output from locale. [makowski@frodo ~]$ locale LANG=en_US.UTF-8 LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_PAPER="en_US.UTF-8" LC_NAME="en_US.UTF-8" LC_ADDRESS="en_US.UTF-8" LC_TELEPHONE="en_US.UTF-8" LC_MEASUREMENT="en_US.UTF-8" LC_IDENTIFICATION="en_US.UTF-8" LC_ALL=
Thanks William. I have another task for you. Please, add these 2 lines to your modified man-db.cron file just before the mandb command and let me know, whether the error message still persists: export LC_ALL="en_US.UTF-8" export LANG="en_US.UTF-8" Thanks, peter
I had the same problem, and the following solved it for me: rm -r /var/cache/man/* /bin/bash /etc/cron.daily/man-db.cron
Peter, You are onto something with setting LC_ALL and/or LANG within man-db.cron. I ran both with and without setting the variables using version man-db-2.6.0.2-3.fc16.i686. When they were set the error message did not occur. When not set the "col: Invalid or incomplete multibyte or wide character" message came back. Bill
Thank you William. I've prepared testing build which could fix this. You can find it here: http://koji.fedoraproject.org/koji/taskinfo?taskID=4176965 Shame is that I can't reproduce this bug anyhow. So, please, could you try to test it? Thanks, peter
My apologies for the delay. I do not see the src inside the testing build. Did I wait too long to pick them up? Bill
Yeah, scratch builds last only for a week. I created another one here: http://koji.fedoraproject.org/koji/taskinfo?taskID=4215401 peter
Success! I performed a yum update using the man-db-2.6.2-0.1.fc18.i686.rpm on a Fedora 17 installation and it worked. The message is no longer coming up. Prior to the update I ran a test using man-db-2.6.0.2-6.fc17.i686 and the "col: Invalid or incomplete multibyte or wide character" message showed up. Bill
fixed in: man-db-2.6.2-1.fc18 http://koji.fedoraproject.org/koji/buildinfo?buildID=329909
man-db-2.6.0.2-7.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/man-db-2.6.0.2-7.fc17
man-db-2.6.0.2-4.fc16 has been submitted as an update for Fedora 16. https://admin.fedoraproject.org/updates/man-db-2.6.0.2-4.fc16
Package man-db-2.6.0.2-7.fc17: * should fix your issue, * was pushed to the Fedora 17 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing man-db-2.6.0.2-7.fc17' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2012-10644/man-db-2.6.0.2-7.fc17 then log in and leave karma (feedback).
man-db-2.6.0.2-7.fc17 has been pushed to the Fedora 17 stable repository. If problems still persist, please make note of it in this bug report.
man-db-2.6.0.2-4.fc16 has been pushed to the Fedora 16 stable repository. If problems still persist, please make note of it in this bug report.