Red Hat Bugzilla – Bug 186456
Copy text file to a ssh vfs add garbage at the top of the file
Last modified: 2013-07-02 19:15:10 EDT
Description of problem:
When copy text file to a ssh vfs, some garbage (like a help of a command) is
appended at the top of the destination file. The garbage is: "usage: head [-n #]
Version-Release number of selected component (if applicable): FC5 - MC
Steps to Reproduce:
1. Connect to another machine with ssh
2. Copy a local filesystem text-file to the ssh VFS
3. Open remote text file and see the garbage a the top of the file.
The copied text file have garbage (usage: head [-n #] [-#] [filename...])
The file copied correctly
Connecting to Solaris. Only in text files. The garbage sounds like a help for a
"usage: head [-n #] [-#] [filename...]"
I suspect that Solaris uses a non-GNU implementation of head so it lacks some
options that FISH vfs in mc requires for communication. I'm quite sure about
this because the help of GNU head looks like this:
Usage: head [OPTION]... [FILE]...
Print the first 10 lines of each FILE to standard output.
With more than one FILE, precede each with a header giving the file name.
With no FILE, or when FILE is -, read standard input.
In the code I found:
/* First, try this as stor:
* ( head -c number ) | ( cat > file; cat >/dev/null )
* If `head' is not present on the remote system, `dd' will be used.
* Unfortunately, we cannot trust most non-GNU `head' implementations
* even if `-c' options is supported. Therefore, we separate GNU head
* (and other modern heads?) using `-q' and `-' . This causes another
* implementations to fail (because of "incorrect options").
* Fallback is:
* while [ $rest -gt 0 ]
* cnt=`expr \( $rest + 255 \) / 256`
* n=`dd bs=256 count=$cnt | tee -a <target_file> | wc -c`
* rest=`expr $rest - $n`
* `dd' was not designed for full filling of input buffers,
* and does not report exact number of bytes (not blocks).
* Therefore a more complex shell script is needed.
So you can eventually use this fallback. Hope that helps.
Since this is not a bug in Fedora, closing NOTABUG.
Jindrich, which code are you referring to when stating "In the code I found:"?
That usage of head is in FC's fish upload patch. Original code only uses dd
You might use "2> /dev/null" to get rid of the error generated by head on Solaris.
Created attachment 126654 [details]
Pipe error output from head to /dev/null
Pipe error output from head to /dev/null. This patch should be applied against
the fish upload patch.
Otto, does this fix your issue?
Hm. I now see the error output of head already is piped to /dev/null (duh).
Maybe Solaris' head behaves oddly with respect to this?
I have no access to Solaris. Otto, maybe you could figure out how to pipe the
head error output to /dev/null and let us know?
Ummm... i think that >/dev/null is the correct way to pipe the head error output
to null in Solaris too. I dont have Solaris rigth now to try this. I try it later.
What about this line:
head -c %lu -q - || echo DD >&3\n
This one may cause the problem?
Hi Leonard, the note I presented here is in vfs/fish.c in fish_file_store().
Confirmed it comes from the Dmitry's patch (bug 140750) so adding him here to
join the discussion.
comment #6: yes, this is likely the source of problem seen on solaris since the
non-GNU implementations lacks the -c option. In the current situation I'd better
not apply the speedup patch by default but make it optional by a macro in spec
file (disabled by default) so that mc user can recompile mc when he's sure that
he accesses only boxes with GNU head installed and want to use the FISH transfer
Ok. I think another solution: when connect to ssh, "test" if -c option is
available and set some "global" flag. When starting copy choose (checking the
flag) if apply the fish transfer speedup or not. It is posible? This is to avoid
recompile, specialy for end users.
From what I can see the problem is that Solaris' head does not write its error
output to stderr but to stdout instead (this needs to be confirmed though). I
believe this is why the 2> /dev/null fails and the error output ends up in the
file. If my guess is right the problem is Solaris' head, not the discussed patch.
Well, guys, I am here.
> In the current situation I'd better not apply the speedup patch by default
Jindrich, don't hurry up to make a decision - we shall think up something. :)
Anyway, this patch is not "some speedup" patch, it is "speedup in several times"
patch. On the other side, the Solaris issue IMHO is a corner case. But I hope we
can find a solution here...
When you get access to some Solaris system, could you first run some tests:
echo foobar | head -c 1 2>/dev/null
to make sure "head" reports to stdout instead of stderr (comment #10),
echo foobar | head -c 1 >/dev/null 2>&1 ; echo $?
to check what is an exit status in this case.
3) What a vesrion of Solaris causes this issue? Is it OpenSolaris? Another
words, can I obtain someway a source code of its "head" implementation?
Note, that the patch discussed was checked on some non-GNU-head systems (IBM
AIX, some old SYSV etc), i.e. it is not a "all non-GNU-head problem", it is just
"Solaris head" problem.
I've obtained OpenSolaris source, and now I see the problem. (I hope the
"Closed" Solaris has the same or similar source too :)).
On any cmdline error the Solaris "head" call a "Usage()" function, which is:
(void) printf(gettext("usage: head [-n #] [-#] [filename...]\n"));
i.e., it writes to stdout instead of stderr. Urhhh...
Fortunately, they return good exit status.
Unfortunately, the source I had obtained is some recent version (20 march) -- it
can mean that the issue exists in all prior versions too, and even in the ALL
You can skip the tests I wrote -- IMHO all is clean now.
Created attachment 126813 [details]
Full fish-upload patch v2: handle Solaris
OK, solution is found.
When upload, use ">/%s" in the DD fallback code to zero the target file anyway.
Previously ">/%s" was used before the whole code, which was extra for the
"head" way (as it always do "cat ... >/%s") and was not robust for DD fallback
(which was showed in Solaris). Now all should be robust.
For "file appending" variant (the second half of the patch) we cannot do
">/%s", as it zeroes the target file. It seems we cannot use head here at
But "appending to target file" is obviously the corner case, therefore I assume
we can drop "head" usage for "appending" variant and just use DD-fallback for
Could you check this patch against Solaris?
(Just replace the original one in the MC source rpm).
I'll wait for Otto's response and then apply your newer version of the patch.
Thanks for the quick reply ;)
Confirmed: the patch is cool! :) this work very well!!
SunOS iwaylab 5.9 Generic_117171-15 sun4u sparc SUNW,Sun-Fire-V240
Thanks to all!
Just to be sure: Do you have checked both upload-"creation/overwrite" and
upload-"append" ? (I'm paranoid a little :))
As you did some works anyway, could you test this patch against some more
non-GNU-head systems? (i.e., not Solaris, or some another version of Solaris
etc.)? While all the things are hot, it is a good moment for this... :)
Yes, i have checked create, overwrite and append and works ok.
I can't test against others non-GNU-head systems because i don't have more ;)
Shouldn't this behaviour of Solaris' head be reported as a bug with Sun?
Maybe the patch should be extended with a comment explaining the "> /%s\n" is
needed because of a faulty head implementation on Solaris?
> Shouldn't this behaviour of Solaris' head be reported as a bug with Sun?
Look into the head.c file (you can obtain it from the on-src-YYYYMMDD.tar.bz2
tarball from the OpenSolaris site). It seems to me that it is not just a code of
Sun, IMHO it began since "1984 AT&T" time...
In other words, it seems that this bug can exist in another UN*Xes too, and
already exists there long time.
In such a situation the fixing of this bug by Sun for OpenSolaris will not take
much sense, as all the previous Sun systems, as well as some another UNIX
systems, are affected too. Anyway we should reckon with such systems in this patch.
> Maybe the patch should be extended with a comment explaining the "> /%s\n" is
> needed because of a faulty head implementation on Solaris?
Well, You're right.
"> /%s\n" is needed for DD-fallback in any case, but not using of head for the
file append case should be commented.
Created attachment 126971 [details]
Fish upload patch: final variant
with an additional comment added
Dmitry pointed me to this discussion since I was not aware that a newer version
of the fish upload patch exists. Since I have applied the older patch upstream
yesterday I'll apply this one too.
Btw. if someone needs access to Solaris system for developing / testing MC under
Solaris just drop me a letter. I own a Solaris box and I'll add acount for you.
Issue is fixed in FC5.
Closing again. Changes since June 8th got lost due to a bugzilla crash.