Bug 1612909

Summary: Concurrent creation of subnets can lead to records with duplicate names
Product: Red Hat Satellite Reporter: Ivan Necas <inecas>
Component: NetworkingAssignee: Ivan Necas <inecas>
Status: CLOSED ERRATA QA Contact: Jitendra Yejare <jyejare>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.3.2CC: andrew.schofield, egolov, inecas, jyejare, omankame, simeon.hoch, sthirugn
Target Milestone: 6.4.0Keywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: foreman-1.18.0.20-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-10-16 19:02:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ivan Necas 2018-08-06 13:59:43 UTC
Description of problem:
Concurrent creation of subnets can lead to records with duplicate names

Version-Release number of selected component (if applicable):
Verified in 6.3, as well as in upstream, has been around probably from the beginning

How reproducible:
Under concurrent requets

for x in {1..20}; do
   echo "Attempt $x"
   for y in {1..4}; do
     curl -k -u admin:changeme -X POST -d '{"name":"subnet-'$x'","network":"192.168.0.1","mask":"255.255.255.0"}' -H 'Content-Type: application/json' http://localhost:300/api/v2/subnets & 
   done
   sleep 1
 done

Actual results:
  
    select name, count(*) from subnets group by name having count(*) > 1;

returns multiple records

Expected results:

    select name, count(*) from subnets group by name having count(*) > 1;

returns no record.

Additional info:

Testing notes: when verifying, please first generate multiple subnets with an installation without this fix and try to migrate to the version with it.

Comment 1 Ivan Necas 2018-08-06 14:42:33 UTC
Created redmine issue https://projects.theforeman.org/issues/24546 from this bug

Comment 2 Ivan Necas 2018-08-06 14:55:42 UTC
Here is the cleanup script to merge the duplicate subnet records:

cat <<EOF | foreman-rake console
  duplicate_names = Subnet.unscoped.select(:name).group(:name).having('count(name) > 1').pluck(:name)
  Rails.logger.info("Found #{duplicate_names} duplicate subnet names.")
  duplicate_names.each do |name|
    winner, *loosers = Subnet.unscoped.where(name: name)
    Rails.logger.info("Merging duplicate subnets with name '#{name}', " \
                      "keeping #{winner.id}, merging with #{loosers.map(&:id)}")
    Hostgroup.unscoped.where(:subnet_id => loosers.map(&:id)).update_all(:subnet_id => winner.id)
    Nic::Base.unscoped.where(:subnet_id => loosers.map(&:id)).update_all(:subnet_id => winner.id)
    SubnetParameter.unscoped.where(:reference_id => loosers.map(&:id)).update_all(:reference_id => winner.id)
    loosers.each(&:destroy!)
  end
EOF

Comment 4 Satellite Program 2018-08-06 16:02:40 UTC
Upstream bug assigned to inecas

Comment 5 Satellite Program 2018-08-06 16:02:43 UTC
Upstream bug assigned to inecas

Comment 8 Satellite Program 2018-09-05 18:01:51 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/24546 has been resolved.

Comment 10 Jitendra Yejare 2018-09-11 10:08:09 UTC
Verified!

@ Satellite 6.4 snap 21

Steps:

1. Run API command to create concurrent subnets with same name(using the same script mentioned in Description of this bug)


Observation:

1. The concurrent subnets are created but not duplicates.
2. An error is returned on attempting to create duplicate subnet as:
"error": {"id":null,"errors":{"name":["has already been taken"]},"full_messages":["Name has already been taken"]}

Hence the bug is now verified.

Also, I observed sometimes the incorrect/db error is redirected while cresting duplicate subnets concurrently as:
"error": {"message":"PG::UniqueViolation: ERROR:  duplicate key value violates unique constraint \"index_subnets_on_name\"\nDETAIL:  Key (name)=(bug_subnet-19) already exists.\n: INSERT INTO \"subnets\" (\"network\", \"mask\", \"name\", \"created_at\", \"updated_at\") VALUES ($1, $2, $3, $4, $5) RETURNING \"id\""}

I would raise another bz for same.

Comment 13 Bryan Kearney 2018-10-16 19:02:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2927