Bug 1013594 - swift: conflicting commands between horizon and cli cause issues with containers
swift: conflicting commands between horizon and cli cause issues with containers
Status: CLOSED INSUFFICIENT_DATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-swift (Show other bugs)
4.0
x86_64 Linux
unspecified Severity medium
: ---
: 5.0 (RHEL 7)
Assigned To: Pete Zaitcev
Dafna Ron
storage
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-09-30 08:13 EDT by Dafna Ron
Modified: 2016-04-26 12:27 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-12-09 20:45:11 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
logs (1.87 KB, application/x-gzip)
2013-09-30 08:13 EDT, Dafna Ron
no flags Details
a test script (6.10 KB, text/plain)
2013-11-07 20:06 EST, Pete Zaitcev
no flags Details

  None (edit)
Description Dafna Ron 2013-09-30 08:13:13 EDT
Created attachment 805170 [details]
logs

Description of problem:

I uploaded a file in cli while deleting the container from horizon. 
it seems that the container is there but we cannot retrieve the files and when we try to delete the container from swift cli we get an error. 
 
Version-Release number of selected component (if applicable):

openstack-swift-1.8.0-6.el6ost.noarch

How reproducible:

100%

Steps to Reproduce:
1. delete swift empty container from horizon while uploading file to it in cli
2.
3.

Actual results:

it appears that the container is not deleted but we cannot list objects or delete it 

Expected results:

we should have a locking mechanized for conflicting commands

Additional info:

this is the upload command from cli while I was deleting the container in horizon: 

[root@nott-vdsa tmp(keystone_admin)]# swift upload  bla packstack-answers-20130924-140637.txt 
packstack-answers-20130924-140637.txt

if we run swift list we can see the container but cannot delete it: 
[root@nott-vdsa tmp(keystone_admin)]# swift list 
bla
[root@nott-vdsa tmp(keystone_admin)]# swift delete bla
Container 'bla' not found

list the container does not work as well: 

[root@nott-vdsa tmp(keystone_admin)]# swift list bla
Container 'bla' not found

The only way to remove it is to run swift delete all
Comment 1 Dafna Ron 2013-09-30 09:01:08 EDT
https://bugs.launchpad.net/swift/+bug/1233111
Comment 2 Pete Zaitcev 2013-09-30 14:04:36 EDT
Yeah, I know... The usual workaround is to do  swift post bla  first,
then delete. You do not always want to blow every container with -a.
Comment 3 Pete Zaitcev 2013-11-07 20:06:00 EST
Created attachment 821367 [details]
a test script
Comment 4 Pete Zaitcev 2013-11-07 20:22:23 EST
I was unable to reproduce the problem thus far. I hit something similar
one time, but it was related to poor cluster health.
Comment 5 Pete Zaitcev 2013-11-13 11:41:55 EST
Dafna: I understand you're busy, but do you happen to have a reproducer
for this? I need to know what the health of the cluster was when it happened,
in particular if there was an out of space condition. My tests show that
we primarily rely on updaters to resolve these inconsistencies (not auditors),
so running out of space in the "pending" directory would cause havoc.
Comment 6 Dafna Ron 2013-11-15 05:45:18 EST
I did have a case of low space on my data servers. 
I don't have that setup any more and the setup I have has 1 zone (the other one had 2 zones and 5 data servers). 
I think that the only way to reproduce this is if the delete will take a few more seconds than it does in the 1 zone setup I have now. 

I did notice that we have a conflict error when we do things the other way around (delete from cli and upload from UI) but when I upload from cli and delete from horizon we will create the container if it does not exist. 
is it possible that we have an exit in the code when deleting from cli but we do not have that in horizon? 

[root@nott-vdsa ~(keystone_admin)]# swift delete bla
haha10
Container DELETE failed: http://10.35.64.69:8080/v1/AUTH_f316d4aeb27b43979c3ec3543ae37985/bla 409 Conflict  [first 60 chars of response] <html><h1>Conflict</h1><p>There was a conflict when trying t
Comment 9 Pete Zaitcev 2014-01-22 10:51:10 EST
Closing with INSUFFICIENT_DATA.

Note You need to log in before you can comment on or make changes to this bug.