From Bugzilla Helper: User-Agent: Mozilla/4.76 [en] (X11; U; Linux 2.2.16-22enterprise i686) Description of problem: One process is writing text lines to a file. Another process is reading those lines as they are written. Using 2.2.x kernels and/or 2.4.x uniprocessor kernels, the reading program always reads complete lines. Using a 2.4.x SMP kernel, the reading process sometimes reads partial lines. This happens regardless of the i/o type (stream or binary) used., and mandatory file locking does not appear to help. Version-Release number of selected component (if applicable): How reproducible: Always Steps to Reproduce: 1. The writing program loops: opens the file for append, writes a single complete line of text, and then closes the file. The reading program: opens the file for input, then loops reading from the file. The lines are a fixed size (80 chars) terminated with a newline. 2. A modified version of the writing program tries to use mandatory file locking to protect the write operation, without success. 3. Actual Results: Under 2.2 kernels, or 2.4.x uniprocessor, the reading program always reads a complete line. Under an SMP 2.4.x kernel, the reading program usually reads a complete line, but occasionally (depending on the speed of both programs) reads only a partial line. Expected Results: The expected behavior without mandatory file locking does not appear to be documented, but it should be consistent between uni- and multi-processor kernels. Using mandatory file locking in the writing program should insure that the reading program always reads complete lines. Additional info: This has been tried with the version 2.4.2-2 kernel on the retail 7.1 CDs, and also with the updated 2.4.3-12 kernel. Test programs for both the reading and writing side are attached (simread.c and simtst.c). In actual use, the reading program is a large commercial application ; we have no access to the sources. The writing program was written locally; we have the sources and can make any useful changes to it.
Created attachment 31776 [details] reader program (self explanatory)
Created attachment 31777 [details] writer program (self explanatory)
I hope you still have warranty on the big program; read() and write() are ALWAYS allowed to make partial progress. and always have been. (posix specification) Jakub: just to make sure, could you check the program as well ? The program doesn't check read bytes so that looks buggy to me.
The behaviour on SMP _is_ the same as on UP. Neither UP nor SMP makes the gaurantee that writes are atomic with respect to reads. If you are running on an otherwise quiescent system so that the write system call happens to be able to proceed without having to do any IO or hitting a kernel lock which is already taken, then on UP you'll be lucky and finish the write before the read syscall can be scheduled. However, on a busy system, even UP will show non-atomic writes. This is correct behaviour: neither POSIX nor the SingleUnix specification require writes to be atomic, except for the special case of small writes to pipes. Mandatory locking will not fix your problem. Taking a mandatory lock in the write path won't necessarily wait for an already-running read to complete: it only prevents new reads from beginning. You'd need cooperative locking between both processes to make the required atomicity guarantees. If you have no access to the source for the reading application, then the only solution I can think of is to write a small library stub wrapper around the read syscall and LD_PRELOAD that.