Bug 979277

Summary: [RFE] Specify the formatting requirements for the CSV file so that Beaker can successfully import the data
Product: [Retired] Beaker Reporter: wangjing <jingwang>
Component: web UIAssignee: Amit Saha <asaha>
Status: CLOSED CURRENTRELEASE QA Contact: tools-bugs <tools-bugs>
Severity: low Docs Contact:
Priority: unspecified    
Version: 0.13CC: aigao, asaha, dcallagh, ebaak, jingwang, llim, qwan, rmancy
Target Milestone: 0.15.3Keywords: FutureFeature, Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-02-03 04:51:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
screenshot for comment0
none
attached csv files with quotes none

Description wangjing 2013-06-28 06:54:52 UTC
Created attachment 766430 [details]
screenshot for comment0

Description of problem:
can't import CSV 

Version-Release number of selected component (if applicable):
beaker-devel Version - 0.13.1

How reproducible:
always

Steps to Reproduce:
1. Admin->import
2. browse one CSV(have attached)
3. click import.

Actual results:
always return 'Missing csv_type from record' in error log, but it was in the CSV.

Expected results:
can import CSV.

Additional info:
the example CSV attached.

Comment 1 Raymond Mancy 2013-07-01 03:42:51 UTC
The attachment seems to be a tab separated values. Beaker supports comma separated values.

Comment 2 wangjing 2013-07-10 03:43:00 UTC
(In reply to Raymond Mancy from comment #1)
> The attachment seems to be a tab separated values. Beaker supports comma
> separated values.

the attachment file was exported by Beaker, so it should be imported to Beaker too.

Comment 4 Dan Callaghan 2013-07-10 06:44:27 UTC
Can you please describe how you exported the CSV containing tabs as separators? Beaker is configured to produce the Python standard CSV dialect on output and expect it on input, and this is not it.

Comment 5 wangjing 2013-07-31 05:48:40 UTC
(In reply to Dan Callaghan from comment #4)
> Can you please describe how you exported the CSV containing tabs as
> separators? Beaker is configured to produce the Python standard CSV dialect
> on output and expect it on input, and this is not it.

I see the problem:

steps for explaining:
1. Admin->Export, export one file, such as user_group.csv.
2. open user_group.csv via LibreOffice calc, a window pop up to set the parameters, including the *Separator options*, and we must only check *Comma* here.
3. add some user and group in this file, then import.

the point is step2, in comment0, I didn't know that we must only check *Comma*  in  *Separator options* section, so maybe it contained 'tab' in front of 'Comma' by default, then I met the error message that confused me.

now I know how to import. I'd like to change this bug to an RFE to add some text on web UI for indicating this requirement of the csv file when imported.

thanks!

Comment 6 Amit Saha 2014-01-07 00:41:38 UTC
Is it sufficient to document this in the (going to be written) admin guide or do we also need a hint in the Web UI ?

Comment 7 Dan Callaghan 2014-01-07 00:50:25 UTC
(In reply to Amit Saha from comment #6)
> Is it sufficient to document this in the (going to be written) admin guide
> or do we also need a hint in the Web UI ?

Maybe the web UI could have a reference to the docs, just so that users realise that "CSV import" doesn't mean it will magically accept CSV in any of its weird and wonderful forms. I don't think it is worth trying to describe the exact CSV format there in the web UI though.

Comment 8 Amit Saha 2014-01-07 05:15:11 UTC
http://gerrit.beaker-project.org/#/c/2649/

Comment 11 wangjing 2014-01-09 09:54:58 UTC
Created attachment 847549 [details]
attached csv files with quotes

Comment 18 Amit Saha 2014-01-10 06:31:47 UTC
(In reply to wangjing from comment #10)

> 
> 2. checking the links on the CSV import/export page-->fail.
> steps:
> 1)visit page: Admin->Import(Export)
> 2)check the links refer to the docs.
> 
> Actual results:
> sometimes displaying correctly like:
> 'Refer to the documentation to learn more about the exported data.'
> and 
> 'Refer to the documentation for details about the supported CSV format.'
> but sometimes nothing above displaying on web, such as freshing the page.
> 
> Expected results:
> hope the link to the docs provided on the CSV import page and export page
> are displayed stably.



http://gerrit.beaker-project.org/#/c/2658/1

Comment 22 Amit Saha 2014-01-10 08:14:48 UTC
(In reply to wangjing from comment #21)
> tested on
> beaker-devel(beaker-server-0.15.3-0.git.18.2d0d7d8.el6eng.noarch)-->partly
> fail
> 
> 1. checking the
> doc(http://beaker-project.org/docs-release-0.15/admin-guide/interface.
> html#import)-->fail
> 
> questions need to confirm: 
> 1) the doc was not added what we talked in scrum meeting(complete CSV
> examples in comment14) temporarily? 
> 2) seems importing field value: "wj test,"""csv""" ""wjabc"""""  was not
> correct, it can be imported and result is: wj test,"csv""" ""wjabc"""""
> 
> based on comment16, suppose odd number quotes won't be imported.

That is Python's CSV module behaviour from what I can see which is what Beaker's import feature also uses.

Using your example from comment 16 (wj test,"csv" "c"""""""). The output for this is:

wj test,"csv" "c"""""""

Which is what you get as well.

Comment 27 wangjing 2014-01-13 06:20:15 UTC
based on the discussions in comment21-26, 
verified on beaker-devel(beaker-server-0.15.3-0.git.18.2d0d7d8.el6eng.noarch)-->pass

steps:
1. checking the doc(http://beaker-project.org/docs-release-0.15/admin-guide/interface.html#import)-->pass

2. checking the links on the CSV import/export page-->pass

Additional:
the question of importing invalid odd number quotes was filed a low bug 1052012 for tracking.

Comment 28 Nick Coghlan 2014-02-03 04:51:55 UTC
This change is included in the Beaker 0.15.3 maintenance release:

http://beaker-project.org/docs/whats-new/release-0.15.html#beaker-0-15-3