In the below log you can see that adding storage to haproxy-1.4 cartridge works but then removing it does not work. It reports successful operation but storage keeps staying at 11GB. Doing remove on the php-5.3 cartridge does the right thing though. I'm not sure why one would add/remove storage to/from the haproxy gear but since this is listed in the test plan I'm reporting it. Tested with devenv_4399. I'm pretty sure I did the same with an earlier build a couple of days ago and it was working. But now it does not. > [avalon@localhost ~]$ rhc cartridge storage -c haproxy-1.4 -a cphp0s0 --add 4gb > Set storage on cartridge ... set to 7GB > > Storage Info > ------------ > Base Gear Storage: 1GB > Additional Gear Storage: 7GB > [avalon@localhost ~]$ rhc cartridge storage -c haproxy-1.4 -a cphp0s0 --add 4 > Set storage on cartridge ... set to 11GB > > Storage Info > ------------ > Base Gear Storage: 1GB > Additional Gear Storage: 11GB > [avalon@localhost ~]$ rhc cartridge storage -c haproxy-1.4 -a cphp0s0 --show > RESULT: > Web Load Balancer > ----------------- > Base Gear Storage: 1GB > Additional Gear Storage: 11GB > > [avalon@localhost ~]$ rhc cartridge storage -a cphp0s0 > RESULT: > > Web Load Balancer > ----------------- > Base Gear Storage: 1GB > Additional Gear Storage: 11GB > > PHP 5.3 > ------- > Base Gear Storage: 1GB > Additional Gear Storage: 11GB > > MongoDB 2.4 > ----------- > Base Gear Storage: 1GB > Additional Gear Storage: 4GB > [avalon@localhost ~]$ rhc cartridge storage -c haproxy-1.4 -a cphp0s0 --remove 1gb > Set storage on cartridge ... set to 10GB > > Storage Info > ------------ > Base Gear Storage: 1GB > Additional Gear Storage: 11GB > [avalon@localhost ~]$ rhc cartridge storage -c haproxy-1.4 -a cphp0s0 --remove 1gb > Set storage on cartridge ... set to 10GB > > Storage Info > ------------ > Base Gear Storage: 1GB > Additional Gear Storage: 11GB > [avalon@localhost ~]$ rhc cartridge storage -c haproxy-1.4 -a cphp0s0 --remove 1gb > Set storage on cartridge ... set to 10GB > > Storage Info > ------------ > Base Gear Storage: 1GB > Additional Gear Storage: 11GB > > [avalon@localhost ~]$ rhc cartridge storage -c php-5.3 -a cphp0s0 --remove 1gb > Set storage on cartridge ... set to 10GB > > Storage Info > ------------ > Base Gear Storage: 1GB > Additional Gear Storage: 10GB
Broker does not seem to send the instructions to node in case of --remove.
The workaround for this issue, as stated, is to add/remove the storage on the web_framework cartridge. There are two alternative approaches: 1. Block sparse cartridges from setting additional storage. - This will need to be handled in case haproxy is in "standalone" mode (Windows application) - If a sparse cart is the ONLY cart within a group instance, an exception could be made to this rule 2. We could make storage part of component specific override as well. - This creates confusion when both sparse and non-sparse carts, within the same group instance, specify storage Given the issues with #2, I would be tempted to go with approach #1 and make it obvious to the user by specifying the exact reason for the failure to add storage to the haproxy cart. The response could include the cartridge that the user could specify the storage on.
Given that we are moving away from multiple carts residing on the same gear in 3.x I suggest we lower the priority on this bug.
Why not just allow for the cart to have quota changed? And I couldn't understand how "moving away from multiple carts residing on the same gear in 3.x" would change anything about priority of this bug. We won't need to support --remove for haproxy in this case? Could you elaborate?
In current scheme of things a happroxy and php-5.3 cartridge can reside on the same gear groups. When a storage quota is changed, it is set on the gear group and not on the cartridge itself. So when you set quota on haproxy or php-5.3 you are setting the quota on the gear group containing both cartridges. The happroxy is a sparse cartridge and will reside only on some of the gears within the gear group (actually one in case of haproxy but by definition a sparse cartridge can be on every nth gear). So you could say in the current schema we cannot support setting quota on a sparse cartridge. Because we have no way of setting the quota on specific gears (where the sparse cartridge resides). We only have a away of setting a quota on a gear group (which may contain other cartridges). We could go ahead and build this into 2.x but given that all this is going to change in 3.x. It seems hardly worth it. Alternatively, we could prevent both add/remove storage functionality for sparse cartridges but then we prevent users from setting quota on sparse cartridge when the sparse cartridge is the only cartridges (see examples given in comment 3 above) The workaround for this issue is for the user to remove quota on web framework cartridge instead of the web proxy cartridge.
I see your point but the question I still can't understand is why --add storage to haproxy should work but remove not? IMO the simplest solution is to make --remove also work for haproxy. It has worked in the past, so also makes sense from backward compatibility point of view. Moreover it's absolutely not acceptable for the command to exit with success status and quata to still be unchanged. Any technical difficulties to make --remove work with haproxy given --add already works? IMHO +1 for the above proposed complicated changes to wait for OSO 3.x
I agree with you that the command should not have a success status when it has not been successful in changing the storage.
https://github.com/openshift/origin-server/pull/5458
Fixed with --> https://github.com/openshift/origin-server/pull/5458
Commit pushed to master at https://github.com/openshift/origin-server https://github.com/openshift/origin-server/commit/4de9bb52836a24d34832f61496186f5d753e048e Bug 1067404: Handling additional storage correctly at the group level
Verified on devenv_4874, the cartridge storage remove command now works on haproxy-1.4 as well. % rhc cartridge storage --add 8 -c haproxy-1.4 -a php1s Set storage on cartridge ... set to 8GB Storage Info ------------ Base Gear Storage: 1GB Additional Gear Storage: 8GB 0 % rhc cartridge storage --show -c haproxy-1.4 -a php1s RESULT: Web Load Balancer ----------------- Base Gear Storage: 1GB Additional Gear Storage: 8GB 0 % rhc cartridge storage --remove 4 -c haproxy-1.4 -a php1s Set storage on cartridge ... set to 4GB Storage Info ------------ Base Gear Storage: 1GB Additional Gear Storage: 4GB 0 % rhc cartridge storage --show -a php1s RESULT: Web Load Balancer ----------------- Base Gear Storage: 1GB Additional Gear Storage: 4GB PHP 5.3 ------- Base Gear Storage: 1GB Additional Gear Storage: 4GB 0 % rhc cartridge storage --remove 4 -c haproxy-1.4 -a php1s Set storage on cartridge ... set to 0GB Storage Info ------------ Base Gear Storage: 1GB Additional Gear Storage: None hjw@hjw-sixmachine ~ 0 % rhc cartridge storage --show -a php1s RESULT: Web Load Balancer ----------------- Base Gear Storage: 1GB Additional Gear Storage: None PHP 5.3 ------- Base Gear Storage: 1GB Additional Gear Storage: None