Bug 1089652
Summary: | [RFE]: Configuration option for linear store to delete the used journal files instead of recycling them. | ||||||
---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise MRG | Reporter: | Christos Triantafyllidis <ctrianta> | ||||
Component: | qpid-cpp | Assignee: | Kim van der Riet <kim.vdriet> | ||||
Status: | CLOSED ERRATA | QA Contact: | Zdenek Kraus <zkraus> | ||||
Severity: | medium | Docs Contact: | |||||
Priority: | medium | ||||||
Version: | 3.0 | CC: | esammons, iboverma, jbuchta, jross, kim.vdriet, mcressma, pmoravec, zkraus | ||||
Target Milestone: | 3.1 | Keywords: | Documentation, FutureFeature, Improvement | ||||
Target Release: | --- | ||||||
Hardware: | All | ||||||
OS: | Linux | ||||||
Whiteboard: | |||||||
Fixed In Version: | qpid-cpp-0.30-4 | Doc Type: | Enhancement | ||||
Doc Text: |
Because customers required used journal files to be cleaned of all legacy user data, and returned to the pool for re-use, a linearstore file overwrite option is implemented by adding the --overwrite-before-return option flag when starting the broker (with the linearstore module loaded).
When using the --overwrite-before-return option, the store overwrites the data portion of the file (the entire file excluding the header) with `\0`, which erases the previous contents of the file. This will prevent any legacy data from existing in the Empty File Pool (EFP). Using this option will have a performance impact because it takes time to overwrite each file. This option should not be used if security considerations do not require it.
|
Story Points: | --- | ||||
Clone Of: | Environment: |
[Needinfo] Awaiting Docs text from Kim. Bumped for info 6th Mar.
|
|||||
Last Closed: | 2015-04-14 13:47:57 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Attachments: |
|
Comment 2
Justin Ross
2014-06-16 15:12:27 UTC
This should be relatively easy to implement. If security is the issue, then another option would be to overwrite the entire file before replacing it into the EFP. This is probably more expensive than deleting the file and re-creating it later. However, when deleting only, there is no guarantee that a new file will not contain some of the old data as the kernel does not wipe old file data on delete. Created attachment 919932 [details]
Patch proposal (incomplete)
Patch proposal, though incomplete.
What remains to do:
1) storing and recovering the option. Currently broker restart forgets to set the option to the queue's journal. I attempted to fix it in MessageStoreImpl but without a luck.
2) Parsing property value: see cpp/src/qpid/linearstore/MessageStoreImpl.cpp, lines starting "qpid::framing::FieldTable::ValuePtr value =". Currently the only accepted values are "0" or "true". The problem here is, args_ provides the value as string everytime (so we have to translate it to bool).
3) Think about if the trick with EmptyFilePool::createEmptyFile used is safe.
4) JournalImpl.cpp has some updates to _mgmtObject that is in commented out code - it is worth having mgmtObject updated to be able to see what queues have this option (un)set.
Proposed solution: r.1620426 This solution adds an option --overwrite-before-return which causes each used journal file to be completely overwritten with /0 chars before being placed back into the EFP. To test, start broker using --truncate yes. Then send and receive a number of messages to a queue: ./qpid-send --address "tq ; {create: always, node: {durable: true}}" --tx 0 --messages 10000 --content-size 1000 --durable yes ./qpid-receive --address "tq" --print-headers no --print-content no --messages 10000 Once this is complete, there should be a number of files in the EFP: tree <store-dir> The contents can be checked with a simple script like this: for f in <store-dir>/p001/efp/2048k/*.jrnl; do echo $f; hexdump -C -n 8192 $f; done If the --overwrite-before-return option is used, then the files should all be blank except for the header: qls/p001/efp/2048k/456ca263-4fff-4f75-bd65-53ddd8545c8c.jrnl 00000000 51 4c 53 66 02 00 00 00 00 00 00 00 00 00 00 00 |QLSf............| 00000010 00 00 00 00 00 00 00 00 01 00 01 00 00 00 00 00 |................| 00000020 00 08 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00002000 otherwise data will be present starting at address 00001000. This was tested on RHEL 6 i686 and x86_64 with following packages: qpid-cpp-server-0.30-4 qpid-cpp-server-devel-0.30-4 qpid-cpp-server-xml-0.30-4 qpid-cpp-server-linearstore-0.30-4 Fix work as expected. ->VERIFIED Fixing Assignee and QA contact Fields, sorry for that. Hey Kim, is there docs text available for this ticket considering that it was customer-initiated (based on the Customer Portal ticket)? I apologize for taking so long to get to this! Doc text provided. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2015-0805.html |