Bug 1420483
| Summary: | [linearstore] Enhancements to linearstore options | ||
|---|---|---|---|
| Product: | Red Hat Enterprise MRG | Reporter: | Kim van der Riet <kim.vdriet> |
| Component: | qpid-cpp | Assignee: | Kim van der Riet <kim.vdriet> |
| Status: | POST --- | QA Contact: | Messaging QE <messaging-qe-bugs> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | Development | CC: | jross |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Enhancement | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | Type: | Bug | |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Fixed (https://git-wip-us.apache.org/repos/asf?p=qpid-cpp.git;h=34cfb7f) * Added option --wcache-num-pages, removed internal table/formulas for determining this value and connected to option. * Set default for --wcache-num-pages to 16. * Changed default for --wcache-page-size from 32 to 16. * Added wiring to set the values of wcache-page-size and wcache-num-pages from qpid-config by looking for the values "qpid.wcache-page-size" and "qpid.wcache-num-pages" in the queue create arguments. This will allow qpid-config to use these as keys to set these values on queue creation. * Added options --wcache-page-size and --wcache-num-pages to qpid-config as queue create options. This allows these parameters to be set when a queue is created using qpid-config. * Tidy-up of some linearstore messages. In particular, when the broker is stopped, the linearstore no longer uses message [Store] info Linear Store: Journal "<queue-name>": Destroyed which may be confusing (and cause some unnecessary panic to users), but rather [Store] info Linear Store: Journal "<queue-name>": Stopped * When a queue is created using parameters other than the defaults, these are now reported at info level logging. |
In applications where large numbers of persistent queues are created on a single broker, control over the default and per-queue store options are important so that resources are not wasted or become exhausted prematurely. There are two resources of concern: 1. Memory allocated per queue for buffering; 2. Disk resources consumed for journal files. Currently, the default setting for memory allocation is 1024kiB/queue (ie 32 pages of 32kiB). The only default setting available is the page size (using the wcache-page-size option), and the number of pages is set internally according to the following table: wcache-page-size | wcache-num-pages | total mmap (kiB) | (set interanlly) | (kiB) -----------------+------------------+----------- 4 | 64 | 256 8 | 64 | 512 16 | 32 | 512 32* | 32 | 1024 64 | 16 | 1024 128 | 8 | 1024 * default This gives in effect only 3 per-queue memory allocation options: 256kiB, 512kiB and 1024kiB. Currently, the values of wcache-page-size is limited to the powers of 2 between 4 and 128. It is suggested that the wcache-num-pages value be set as as an option rather than internally, which would give the user much greater control over the per-queue memory footprint and the buffer geometry (which can be optimized for the disk hardware and queue load characteristics). In addition, the upper value caps should be removed (as they are arbitrary limits) and the default value of wcache-page-size should be revised. Further, it would be valuable to add these parameters to qpid-config so that if a queue is created using this tool, the default values for the store parameters can be overidden on a per-queue basis.