Bug 979277 - [RFE] Specify the formatting requirements for the CSV file so that Beaker can successfully import the data
[RFE] Specify the formatting requirements for the CSV file so that Beaker can...
Status: CLOSED CURRENTRELEASE
Product: Beaker
Classification: Community
Component: web UI (Show other bugs)
0.13
Unspecified Unspecified
unspecified Severity low (vote)
: 0.15.3
: ---
Assigned To: Amit Saha
tools-bugs
: FutureFeature, Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-06-28 02:54 EDT by wangjing
Modified: 2015-07-26 18:15 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-02-02 23:51:55 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
screenshot for comment0 (62 bytes, text/csv)
2013-06-28 02:54 EDT, wangjing
no flags Details
attached csv files with quotes (162 bytes, text/csv)
2014-01-09 04:54 EST, wangjing
no flags Details

  None (edit)
Description wangjing 2013-06-28 02:54:52 EDT
Created attachment 766430 [details]
screenshot for comment0

Description of problem:
can't import CSV 

Version-Release number of selected component (if applicable):
beaker-devel Version - 0.13.1

How reproducible:
always

Steps to Reproduce:
1. Admin->import
2. browse one CSV(have attached)
3. click import.

Actual results:
always return 'Missing csv_type from record' in error log, but it was in the CSV.

Expected results:
can import CSV.

Additional info:
the example CSV attached.
Comment 1 Raymond Mancy 2013-06-30 23:42:51 EDT
The attachment seems to be a tab separated values. Beaker supports comma separated values.
Comment 2 wangjing 2013-07-09 23:43:00 EDT
(In reply to Raymond Mancy from comment #1)
> The attachment seems to be a tab separated values. Beaker supports comma
> separated values.

the attachment file was exported by Beaker, so it should be imported to Beaker too.
Comment 4 Dan Callaghan 2013-07-10 02:44:27 EDT
Can you please describe how you exported the CSV containing tabs as separators? Beaker is configured to produce the Python standard CSV dialect on output and expect it on input, and this is not it.
Comment 5 wangjing 2013-07-31 01:48:40 EDT
(In reply to Dan Callaghan from comment #4)
> Can you please describe how you exported the CSV containing tabs as
> separators? Beaker is configured to produce the Python standard CSV dialect
> on output and expect it on input, and this is not it.

I see the problem:

steps for explaining:
1. Admin->Export, export one file, such as user_group.csv.
2. open user_group.csv via LibreOffice calc, a window pop up to set the parameters, including the *Separator options*, and we must only check *Comma* here.
3. add some user and group in this file, then import.

the point is step2, in comment0, I didn't know that we must only check *Comma*  in  *Separator options* section, so maybe it contained 'tab' in front of 'Comma' by default, then I met the error message that confused me.

now I know how to import. I'd like to change this bug to an RFE to add some text on web UI for indicating this requirement of the csv file when imported.

thanks!
Comment 6 Amit Saha 2014-01-06 19:41:38 EST
Is it sufficient to document this in the (going to be written) admin guide or do we also need a hint in the Web UI ?
Comment 7 Dan Callaghan 2014-01-06 19:50:25 EST
(In reply to Amit Saha from comment #6)
> Is it sufficient to document this in the (going to be written) admin guide
> or do we also need a hint in the Web UI ?

Maybe the web UI could have a reference to the docs, just so that users realise that "CSV import" doesn't mean it will magically accept CSV in any of its weird and wonderful forms. I don't think it is worth trying to describe the exact CSV format there in the web UI though.
Comment 8 Amit Saha 2014-01-07 00:15:11 EST
http://gerrit.beaker-project.org/#/c/2649/
Comment 11 wangjing 2014-01-09 04:54:58 EST
Created attachment 847549 [details]
attached csv files with quotes
Comment 18 Amit Saha 2014-01-10 01:31:47 EST
(In reply to wangjing from comment #10)

> 
> 2. checking the links on the CSV import/export page-->fail.
> steps:
> 1)visit page: Admin->Import(Export)
> 2)check the links refer to the docs.
> 
> Actual results:
> sometimes displaying correctly like:
> 'Refer to the documentation to learn more about the exported data.'
> and 
> 'Refer to the documentation for details about the supported CSV format.'
> but sometimes nothing above displaying on web, such as freshing the page.
> 
> Expected results:
> hope the link to the docs provided on the CSV import page and export page
> are displayed stably.



http://gerrit.beaker-project.org/#/c/2658/1
Comment 22 Amit Saha 2014-01-10 03:14:48 EST
(In reply to wangjing from comment #21)
> tested on
> beaker-devel(beaker-server-0.15.3-0.git.18.2d0d7d8.el6eng.noarch)-->partly
> fail
> 
> 1. checking the
> doc(http://beaker-project.org/docs-release-0.15/admin-guide/interface.
> html#import)-->fail
> 
> questions need to confirm: 
> 1) the doc was not added what we talked in scrum meeting(complete CSV
> examples in comment14) temporarily? 
> 2) seems importing field value: "wj test,"""csv""" ""wjabc"""""  was not
> correct, it can be imported and result is: wj test,"csv""" ""wjabc"""""
> 
> based on comment16, suppose odd number quotes won't be imported.

That is Python's CSV module behaviour from what I can see which is what Beaker's import feature also uses.

Using your example from comment 16 (wj test,"csv" "c"""""""). The output for this is:

wj test,"csv" "c"""""""

Which is what you get as well.
Comment 27 wangjing 2014-01-13 01:20:15 EST
based on the discussions in comment21-26, 
verified on beaker-devel(beaker-server-0.15.3-0.git.18.2d0d7d8.el6eng.noarch)-->pass

steps:
1. checking the doc(http://beaker-project.org/docs-release-0.15/admin-guide/interface.html#import)-->pass

2. checking the links on the CSV import/export page-->pass

Additional:
the question of importing invalid odd number quotes was filed a low bug 1052012 for tracking.
Comment 28 Nick Coghlan 2014-02-02 23:51:55 EST
This change is included in the Beaker 0.15.3 maintenance release:

http://beaker-project.org/docs/whats-new/release-0.15.html#beaker-0-15-3

Note You need to log in before you can comment on or make changes to this bug.