Hide Forgot
When the store is set up with --jfile-size 32768 on a 32-bit RHEL, the process of creating the store files fails with: JERR_FCNTL_WRITE: Unable to write to file. (wr_size=2097152 errno=22 (Invalid argument)) This does not happen on 64-bit systems. Setting to medium priority, as 32-bit OSs are not common for these large-file use-cases.
Created attachment 539772 [details] simple reproducer Simple reproducer.
The simple reproducer seems to show that this is a kernel limit of write size in 32-bit kernels imposed by using the O_DIRECT flag. The man (2) page for open() indicates that limits on file offsets are possible when O_DIRECT is in use. We could solve this particular problem by not using the O_DIRECT flag at all. As the file is being formatted, a regular file handle may suffice.
As this error occurs exactly on the maximum allowable file size (jfile-size-pgs) store parameter, the most prudent fix is to reduce this limit by one from 32768 to 32767. Fixed in r.4485. NOTE: This may change the numbers for max file size in the Messaging User's Guide.
QE: to reproduce, run on a 32-bit RHEL. ./qpidd --auth no --load-module /abs/path/to/msgstore.so --store-dir /tmp/store --jfile-size 32768 --num-jfiles 4 --log-enable info+ ./qpid-perftest --count 10 --durable yes will result in a failure in the broker: JERR_FCNTL_WRITE: Unable to write to file. (wr_size=2097152 errno=22 (Invalid argument)) When fixed, --jfile-size 32768 will be too big by 1, and the broker will change it with a log warning to 32767. The test will complete without an error. As each journal file in this test is ~2GB, the formatting of the files will take some time - be patient.
This is in the 0.14 rebase
CLOSED/CRELEASE -> ASSIGNED -> ON_QA The defect has to go through QA process.
Tested on RHEL5.8 and RHEL6.2. This problem was fixed. Packages used for testing: RHEL5.8: qpid-cpp-client-0.14-16.el5 qpid-cpp-client-devel-0.14-16.el5 qpid-cpp-client-devel-docs-0.14-16.el5 qpid-cpp-client-rdma-0.14-16.el5 qpid-cpp-client-ssl-0.14-16.el5 qpid-cpp-mrg-debuginfo-0.14-16.el5 qpid-cpp-server-0.14-16.el5 qpid-cpp-server-cluster-0.14-16.el5 qpid-cpp-server-devel-0.14-16.el5 qpid-cpp-server-rdma-0.14-16.el5 qpid-cpp-server-ssl-0.14-16.el5 qpid-cpp-server-store-0.14-16.el5 qpid-cpp-server-xml-0.14-16.el5 rh-qpid-cpp-tests-0.14-16.el5 RHEL6.2: qpid-cpp-client-0.14-15.el6.i686 qpid-cpp-client-devel-0.14-15.el6.i686 qpid-cpp-client-devel-docs-0.14-15.el6.noarch qpid-cpp-client-rdma-0.14-15.el6.i686 qpid-cpp-client-ssl-0.14-15.el6.i686 qpid-cpp-debuginfo-0.14-15.el6.i686 qpid-cpp-server-0.14-15.el6.i686 qpid-cpp-server-cluster-0.14-15.el6.i686 qpid-cpp-server-devel-0.14-15.el6.i686 qpid-cpp-server-rdma-0.14-15.el6.i686 qpid-cpp-server-ssl-0.14-15.el6.i686 qpid-cpp-server-store-0.14-15.el6.i686 qpid-cpp-server-xml-0.14-15.el6.i686 rh-qpid-cpp-tests-0.14-15.el6.i686 -> VERIFIED
Technical note added. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: When creating a journal with file-size=32768 on 32-bit systems, the store fails with error 22. The reson is that on 32-bit systems, this size exceeds the maximum allowed by the kernel when using the O_DIRECT flag. The error occurs only at this value, which is coincidentally the maximum allowed value for the store. This problem was fixed by reducing the maximum allowed value to 32767.