Back to bug 709325
| Who | When | What | Removed | Added |
|---|---|---|---|---|
| RHEL Program Management | 2011-05-31 12:11:52 UTC | Flags | needinfo? | |
| RHEL Program Management | 2011-05-31 12:12:01 UTC | Flags | needinfo? | |
| J.H.M. Dassen (Ray) | 2011-07-22 09:22:31 UTC | CC | rdassen | |
| Link ID | Red Hat Knowledge Base 53822 | |||
| J.H.M. Dassen (Ray) | 2012-01-27 13:26:01 UTC | Blocks | 785156 | |
| Irina Boverman | 2012-03-15 16:11:11 UTC | Blocks | 803771 | |
| Irina Boverman | 2012-03-15 18:24:51 UTC | Target Milestone | --- | 2.2 |
| Justin Ross | 2012-04-05 18:33:57 UTC | CC | jross | |
| Assignee | rhm-maint-list | kgiusti | ||
| Flags | needinfo?(kgiusti) | |||
| Justin Ross | 2012-04-05 18:36:12 UTC | Assignee | kgiusti | kim.vdriet |
| Gordon Sim | 2012-04-05 19:05:52 UTC | CC | gsim | |
| Justin Ross | 2012-04-10 13:45:08 UTC | Flags | needinfo?(kgiusti) | needinfo?(kim.vdriet) |
| Kim van der Riet | 2012-04-10 14:27:12 UTC | Flags | needinfo?(kim.vdriet) | |
| Justin Ross | 2012-04-20 19:29:23 UTC | Target Milestone | 2.2 | Next Version |
| Irina Boverman | 2012-08-07 16:06:33 UTC | Blocks | 808578 | |
| Irina Boverman | 2012-08-07 16:26:19 UTC | Blocks | 808578 | |
| Justin Ross | 2012-12-04 21:05:59 UTC | Target Milestone | Next Version | --- |
| Justin Ross | 2012-12-12 19:30:36 UTC | Target Milestone | --- | 2.4 |
| J.H.M. Dassen (Ray) | 2013-02-25 08:47:33 UTC | Link ID | Red Hat Knowledge Base (Legacy) 53822 | Red Hat Knowledge Base (Solution) 53822 |
| Zdenek Kraus | 2013-04-11 12:10:49 UTC | CC | zkraus | |
| Simon Green | 2013-07-03 04:42:25 UTC | CC | rdassen | rbinkhor |
| Martin Tessun | 2013-07-19 07:48:16 UTC | CC | mtessun | |
| Leonid Zhaldybin | 2013-07-22 14:52:20 UTC | CC | lzhaldyb | |
| Leonid Zhaldybin | 2013-07-22 14:53:24 UTC | QA Contact | mrgqe-bugs | lzhaldyb |
| Justin Ross | 2013-07-22 14:54:31 UTC | Status | NEW | ASSIGNED |
| Target Milestone | 2.4 | 3.0 | ||
| Eric Sammons | 2013-07-25 12:16:39 UTC | CC | esammons | |
| Joshua Wulf | 2013-08-06 23:43:11 UTC | CC | jwulf | |
| Joshua Wulf | 2013-08-07 19:38:57 UTC | Blocks | 994701 | |
| Joshua Wulf | 2013-10-03 05:41:16 UTC | Flags | needinfo?(jross) | |
| Justin Ross | 2013-10-03 17:24:42 UTC | CC | kim.vdriet | |
| Flags | needinfo?(jross) | needinfo?(kim.vdriet) | ||
| Kim van der Riet | 2013-11-20 16:22:11 UTC | Status | ASSIGNED | POST |
| Flags | needinfo?(kim.vdriet) | |||
| Zdenek Kraus | 2013-11-20 16:30:37 UTC | QA Contact | lzhaldyb | zkraus |
| Irina Boverman | 2013-11-20 19:07:15 UTC | CC | iboverma | |
| Frantisek Reznicek | 2013-11-21 15:13:10 UTC | CC | freznice | |
| Irina Boverman | 2013-11-22 15:29:08 UTC | Status | POST | MODIFIED |
| Fixed In Version | qpid-cpp-0.22-27 | |||
| errata-xmlrpc | 2013-11-22 15:56:51 UTC | Status | MODIFIED | ON_QA |
| Zdenek Kraus | 2013-11-28 14:05:42 UTC | Depends On | 1035802 | |
| Zdenek Kraus | 2013-11-29 11:05:44 UTC | Depends On | 1036071 | |
| Zdenek Kraus | 2013-11-29 11:05:56 UTC | Depends On | 1036026 | |
| Zdenek Kraus | 2013-12-02 12:42:52 UTC | Depends On | 1035843 | |
| Zdenek Kraus | 2013-12-05 13:04:17 UTC | Depends On | 1038599 | |
| Zdenek Kraus | 2013-12-09 12:06:26 UTC | Depends On | 1039522 | |
| Zdenek Kraus | 2013-12-09 12:06:29 UTC | Depends On | 1039525 | |
| Frantisek Reznicek | 2013-12-10 12:13:08 UTC | Depends On | 1039949 | |
| Stuart Auchterlonie | 2014-02-07 11:26:39 UTC | CC | sauchter | |
| Frantisek Reznicek | 2014-02-12 07:59:03 UTC | Depends On | 1064181 | |
| Frantisek Reznicek | 2014-02-12 09:45:01 UTC | Depends On | 1064230 | |
| Zdenek Kraus | 2014-02-12 11:25:40 UTC | Depends On | 1049870 | |
| Zdenek Kraus | 2014-02-19 05:41:07 UTC | Depends On | 1066256 | |
| Zdenek Kraus | 2014-02-19 05:41:12 UTC | Depends On | 1051097 | |
| Zdenek Kraus | 2014-02-19 05:41:16 UTC | Depends On | 1051924 | |
| Zdenek Kraus | 2014-02-19 05:41:28 UTC | Depends On | 1052518 | |
| Zdenek Kraus | 2014-02-19 06:06:50 UTC | Depends On | 1063700 | |
| Zdenek Kraus | 2014-02-19 06:06:51 UTC | Depends On | 1052445 | |
| Zdenek Kraus | 2014-02-19 06:06:55 UTC | Depends On | 1053701 | |
| Zdenek Kraus | 2014-02-19 06:06:59 UTC | Depends On | 1053749 | |
| Zdenek Kraus | 2014-02-19 06:07:04 UTC | Depends On | 1052727 | |
| Zdenek Kraus | 2014-02-19 06:07:09 UTC | Depends On | 1052775 | |
| Zdenek Kraus | 2014-02-19 06:07:13 UTC | Depends On | 1054448 | |
| Zdenek Kraus | 2014-02-20 12:59:17 UTC | Depends On | 1067429 | |
| Zdenek Kraus | 2014-02-20 14:16:03 UTC | Depends On | 1067480 | |
| Zdenek Kraus | 2014-02-20 14:30:18 UTC | Depends On | 1067482 | |
| Zdenek Kraus | 2014-03-17 12:42:36 UTC | Depends On | 1049870 | |
| Frantisek Reznicek | 2014-03-19 09:13:13 UTC | Depends On | 1078142 | |
| Zdenek Kraus | 2014-05-29 14:49:13 UTC | Depends On | 1098118 | |
| Zdenek Kraus | 2014-06-26 08:31:26 UTC | Depends On | 1066256, 1067429, 1067480, 1067482 | |
| Zdenek Kraus | 2014-06-26 08:54:47 UTC | Blocks | 994701 | |
| Depends On | 994701 | |||
| Zdenek Kraus | 2014-08-08 08:22:44 UTC | Status | ON_QA | VERIFIED |
| Petr Matousek | 2014-09-10 13:57:10 UTC | Status | VERIFIED | MODIFIED |
| CC | jmorgan, pematous | |||
| Flags | needinfo?(kim.vdriet) | |||
| Petr Matousek | 2014-09-10 14:36:18 UTC | Flags | needinfo?(jross) | |
| Justin Ross | 2014-09-11 14:13:51 UTC | Flags | needinfo?(kim.vdriet) needinfo?(jross) | |
| errata-xmlrpc | 2014-09-15 13:17:49 UTC | Status | MODIFIED | ON_QA |
| Zdenek Kraus | 2014-09-17 17:36:20 UTC | Flags | needinfo?(jross) | |
| Zdenek Kraus | 2014-09-17 17:36:37 UTC | Flags | needinfo?(kim.vdriet) | |
| Kim van der Riet | 2014-09-19 17:19:35 UTC | Doc Text | Feature: Linearstore Reason: The original Legacystore has several serious limitations, one of which is the fact that it uses fixed circular file buffers for queue journals. This system is optimized for speed in systems where messages are consumed strictly in-order, but has the following disadvantages: 1. If messages are consumed out-of-order or a message is orphaned (abandoned), then it will cause the buffer to eventually fill, and will cuase the queue to become inoperable for adding messages until blocking message(s) are consumed. 2. It is difficult to size a fixed buffer correctly. The user must estimate the worst-case scenario for queue depth and size the buffer accordingly. This can be a wasteful use of disk space and makes broker recovery unnecessarily long. If there are many queues, then this can quickly drain the available disk space. Result: A new store has been developed which is largely based on the previous legacystore. However, it keeps a pool of empty files of fixed size from which it takes journal files as needed and appends them to a queue journal in a linear manner. Once a journal file has all its messages consumed and there are no preceding journal files still with messages, the file is returned to the pool. This store is known as the linearstore. Notes: 1. There is no compatibility between legacystore and linearstore journals. A user which upgrades from legacystore to linearstore must do so with a clean store. 2. The Empty File Pool (EFP) contains empty files of the correct size and format from which journal files are taken. If the EFP is empty when the linearstore needs a file, it will create and format one automatically and use it in the queue. However, this causes the store to run a little more slowly than if old files are re-used. 3. Ideally, the EFP should be maintained by an external process to keep the number of empty formatted files topped up for immediate use by the store. However, currenlty there is no such utility. Users will need to rely on the automatic creation ability of the store for the time being. For a fixed number of queues, this create an initial burden of file creation when queues are first created and used, but over time the EFP will fill up with used files and will be circulated as needed. 4. There are currently no limits on the maximum size of any journal. Currently, a queue can contain an unlimited number of journal files. This will be addressed in a future enhancement of the linearstore. Under normal usage patterns, a queue will maintain a reasonable number of journal files. However, under some unusual circumstances, such as orphaned persistent mesages, the number of files in the queue will grow. It is up to the user to monitor file usage. 5. There are currently no QMF abilities in linearstore. This will be addressed in a later enhancement. 6. The legacystore geometry parameters --jfile-size-pgs and --num-jfiles (and the TPL equivalents --tpl-jfile-size-pgs and --tpl-num-jfiles) are no longer used in linearstore. Journal files are now a fixed size and the number is automatically adjusted to the number of messages theat must be stored. 7. The linearstore --partition parameter is not fully functional, and should be ignored for all practical purposes. A future enhancement will add the ability to specify a disk partition for a queue journal, thus allowing the user to place some queues on high-performance media (such as solid state drives) while others may be on regular magnetic disk. Currently the --partition parameter will select a partition directory from the store tree structure, but lacks the ability to actually span several real partitions. The default partition is p001, and this should be the only one used for linearstore until the enhancement is complete. 8. The --overwrite-before-return parameter will force each journal file to be completely overwritten before it is returned to the EFP. This parameter should only be used where security concerns dictate that old message data should not be present in the EFP. Using this parameter comes with an as-yet undetermined performance penalty owing to the additional write activity needed for each file as it is reutnred to the EFP. 9. The current directory structure of the store below the --store-dir parameter is as follows: store-dir--qls +-dat <BDB database> +-p001 | +-efp | | +-2048k | | +-8192k | | +- <other file sizes> | | | +-jrnl | +-queue_1 | +-queue_2 | ... | +-queue_n | +-p002 <dir structure as above> ... +-pnnn <dir structure as above> | |
| Flags | needinfo?(jross) needinfo?(kim.vdriet) | |||
| Leonid Zhaldybin | 2014-09-22 09:14:48 UTC | Group | support, devel | |
| Jared MORGAN | 2014-09-23 03:35:51 UTC | Doc Text | Feature: Linearstore Reason: The original Legacystore has several serious limitations, one of which is the fact that it uses fixed circular file buffers for queue journals. This system is optimized for speed in systems where messages are consumed strictly in-order, but has the following disadvantages: 1. If messages are consumed out-of-order or a message is orphaned (abandoned), then it will cause the buffer to eventually fill, and will cuase the queue to become inoperable for adding messages until blocking message(s) are consumed. 2. It is difficult to size a fixed buffer correctly. The user must estimate the worst-case scenario for queue depth and size the buffer accordingly. This can be a wasteful use of disk space and makes broker recovery unnecessarily long. If there are many queues, then this can quickly drain the available disk space. Result: A new store has been developed which is largely based on the previous legacystore. However, it keeps a pool of empty files of fixed size from which it takes journal files as needed and appends them to a queue journal in a linear manner. Once a journal file has all its messages consumed and there are no preceding journal files still with messages, the file is returned to the pool. This store is known as the linearstore. Notes: 1. There is no compatibility between legacystore and linearstore journals. A user which upgrades from legacystore to linearstore must do so with a clean store. 2. The Empty File Pool (EFP) contains empty files of the correct size and format from which journal files are taken. If the EFP is empty when the linearstore needs a file, it will create and format one automatically and use it in the queue. However, this causes the store to run a little more slowly than if old files are re-used. 3. Ideally, the EFP should be maintained by an external process to keep the number of empty formatted files topped up for immediate use by the store. However, currenlty there is no such utility. Users will need to rely on the automatic creation ability of the store for the time being. For a fixed number of queues, this create an initial burden of file creation when queues are first created and used, but over time the EFP will fill up with used files and will be circulated as needed. 4. There are currently no limits on the maximum size of any journal. Currently, a queue can contain an unlimited number of journal files. This will be addressed in a future enhancement of the linearstore. Under normal usage patterns, a queue will maintain a reasonable number of journal files. However, under some unusual circumstances, such as orphaned persistent mesages, the number of files in the queue will grow. It is up to the user to monitor file usage. 5. There are currently no QMF abilities in linearstore. This will be addressed in a later enhancement. 6. The legacystore geometry parameters --jfile-size-pgs and --num-jfiles (and the TPL equivalents --tpl-jfile-size-pgs and --tpl-num-jfiles) are no longer used in linearstore. Journal files are now a fixed size and the number is automatically adjusted to the number of messages theat must be stored. 7. The linearstore --partition parameter is not fully functional, and should be ignored for all practical purposes. A future enhancement will add the ability to specify a disk partition for a queue journal, thus allowing the user to place some queues on high-performance media (such as solid state drives) while others may be on regular magnetic disk. Currently the --partition parameter will select a partition directory from the store tree structure, but lacks the ability to actually span several real partitions. The default partition is p001, and this should be the only one used for linearstore until the enhancement is complete. 8. The --overwrite-before-return parameter will force each journal file to be completely overwritten before it is returned to the EFP. This parameter should only be used where security concerns dictate that old message data should not be present in the EFP. Using this parameter comes with an as-yet undetermined performance penalty owing to the additional write activity needed for each file as it is reutnred to the EFP. 9. The current directory structure of the store below the --store-dir parameter is as follows: store-dir--qls +-dat <BDB database> +-p001 | +-efp | | +-2048k | | +-8192k | | +- <other file sizes> | | | +-jrnl | +-queue_1 | +-queue_2 | ... | +-queue_n | +-p002 <dir structure as above> ... +-pnnn <dir structure as above> | A new store implementation named Linearstore replaces Legacystore in this release. Linearstore addresses limitations imposed by Legacystore, specifically its use of fixed circular file buffers for queue journals. Linearstore utilizes a pool of fixed-size empty journal files and appends the journal files to a queue journal in a linear manner. The file is returned to the pool once a journal file is cleared of messages, and has no preceding journal files with messages waiting. There are a number of changes between Legacystore and Linearstore users must be aware of. See Comment 33 in the original Bugzilla Enhancement for further usage caveats and migration limitations. This information will be migrated to formal user documentation shortly after the 3.0 Release. |
| Zdenek Kraus | 2014-09-23 07:36:08 UTC | Status | ON_QA | VERIFIED |
| errata-xmlrpc | 2014-09-24 14:40:44 UTC | Status | VERIFIED | RELEASE_PENDING |
| errata-xmlrpc | 2014-09-24 15:02:59 UTC | Status | RELEASE_PENDING | CLOSED |
| Resolution | --- | ERRATA | ||
| Last Closed | 2014-09-24 11:02:59 UTC | |||
| John Skeoch | 2015-09-14 00:22:45 UTC | CC | mnewsome |
Back to bug 709325