While troubleshooting code from another programmer that suddenly failed on 
a newer release of Linux, I found what I believe to be a bug in the 
implementation of fcntl.  

Here is a copy of the bug report minus the boring stuff:

Introduction

After review of an error referenced during the testing and operation of a 
keylock device attached through a serial port to the ******* product, I 
have discovered a bug in the use of fcntl  in Caldera 2.2 and Redhat 7.0.  
This bug affects the ability of fcntl to modify open flags on an open file 
descriptor.

Intent Of Code Sequence

In **********, most serial port operations occur within serial.c.  It 
contains a series of functions that ********* calls to open, close, read 
and write from serial ports.

The function within serial.c opens all ports in a non-blocking mode.  This 
means that when a read operation is called, the point of execution will 
not stop at the read even if the input buffer is empty.
    
When a read operation executes and there is a timeout value (in this case, 
a 15 second timeout) the program will call the fcntl function.  This 
function call should remove the O_NDELAY (NONBLOCK) flag from the set of 
parameters being used.  At this point in execution, all read operations 
for that file descriptor are set to blocking mode, meaning the point of 
execution will stop at the read until the input buffer contains data.

The timeout period is timed through alarm, which will generate a SIGALRM 
signal when it concludes its count-down.  Using signal and through some 
other techniques we wont describe here, the point of execution will 
return one line past the read function call.


Discovery

In our tests of serial.prg, we found that if the fcntl function call for 
both the F_GETFL and F_SETFL were called from separate lines, the fcntl 
call would fail to remove the O_NDELAY (NONBLOCK) flag.  When we combined 
the two steps into the same command sequence, it worked.  

Here is how the two sequences were written:

CODE THAT FAILED

{
   int I;
   I = fcntl(fd,F_GETFL);
   I &= ~O_NDELAY;
   fcntl(fd,F_SETFL,&I);
}

CODE THAT SUCCEEDED
	
fcntl(fd, F_SETFL, fcntl(fd,F_GETFL) & ~O_NDELAY):

Logically, both sequences should have worked.  I have found documents 
supporting the use of both methods.  There seems to be no explanation why 
the original code sequence failed to work on later versions of GLIBC.
Test Code

The following is sample test code I used to determine my conclusion:

#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>



int main(void)
{
	int fd;
	int cnt;
	int rc;
	int lp;
	int i;
	unsigned char test[100];

	cnt = 0;

	fd = open("/dev/tty",O_NOCTTY | O_NDELAY);

	//********** works ***************
	rc = fcntl(fd,F_SETFL,fcntl(fd,F_GETFL) & ~O_NDELAY);

	//*********** doesnt work **************
	//i = fcntl(fd,F_GETFL);
	//i &= ~O_NDELAY;
	//rc = fcntl(fd,F_SETFL,&i);

	if(rc<0) printf("Error: unable to execute fcntl");

	while (cnt<20)
	{
	  for(lp=0;lp<sizeof(test);lp++)
	  {
		test[lp]=0;
	  }

	  rc = read(fd,(void *)test,sizeof(test)-1);

	  printf("\nOUTPUT: %s\n", test);
	  cnt++;
	}	
	close(fd);	
}


Conclusion

I conclude that this is likely a bug in the implementation of the newer 
GLIBC library distributed by Caldera and RedHat.  


By-the-way, I personally wouldn't have written code like this.  I usually 
keep everything blocking, and use 'select'.  Nevertheless, it does seem 
odd that one of the methods work and the other does not.