Bug 1963213

Summary: Memsource download script running into API error
Product: OpenShift Container Platform Reporter: ralpert
Component: Management ConsoleAssignee: ralpert
Status: CLOSED ERRATA QA Contact: Yadan Pei <yapei>
Severity: low Docs Contact:
Priority: unspecified    
Version: 4.8CC: aos-bugs, jokerman, yapei
Target Milestone: ---   
Target Release: 4.8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-07-27 23:09:50 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description ralpert 2021-05-21 16:54:10 UTC
Description of problem:
I'm getting an error when I try to use the Memsource script to download the latest Sprint 200 project:

Memsource API Error
Error: Invalid parameters
Error description: Job "sdsTPziVr4rHedDUU7fPR0 88kwpZ59qWqwaFsssR0UU6 2GBCUePZGIVcfV77FM8WN0 ChGCwVGoN5qW2lB1uT2MK2 ydyxxZSDRVUR4xn851iU85 flwWPytGrypGe5q9nYf4a0 VqW3051bRVBk24PGfG5xd0 v0j1hEdi7MyMsAwGQDnrcc Cu90D8lg0EwDskdMulj0iv yBS5hw7EgE973F2zBJkT10 xFpoqv0089KdeAGSioAYy5 KAaVv907AIhSlqcrXG1XGa L3E57bNeYN8c99Y0MpBKQ6 jJ2jSAPvgTXc8p3AaQiRN1 SZvkyFpRcyEBcNaAbextX1 MMtJDq0l8cjz4WNM0X2qRb ViMmyOU8fhEE7FhpheElF1 DcRKnsi3r5r44lK6Trm1R7 lBvtedAd4bXgahtZGTCb61 nmtIJE0m2JOcsyHB8RkEr3 2lVN3RqTQ0k1tk0t2ZnkTv UxFJx2SaKAXpeRdLyRczc3 OJDxArk1mbR37yLCPI1orf dSVagR6Pch95eVKhzauk40 GNQjw1nE7VM6ry6Dwt5KO6 4mbJfPolia974t8jXVhK73 OCrb0HVygvM33iW0sFM29e ZSFbfbFyOVEOd5qZf7owB0 9EZh2z9mPdoD4RVeNJIHt2 DpGQqTihd4V71Gcu5di2a7 01suXHocQCAy5k8siHXLW6 lgZn06Qg7rnlsWrFzHuFP0 4lYVYohhXb8ucJsCjUlTZ0 8pwD3ZIouOWg7FXatnCei1 " not imported correctly or import is in progress
Error Reason: 
Response status: 400

HTTP response headers: HTTPHeaderDict({'Memsource-Action-Id': '0b24fca0103c40a1839ea44666422fe1', 'X-Frame-Options': 'DENY', 'X-Content-Type-Options': 'nosniff', 'Referrer-Policy': 'strict-origin-when-cross-origin', 'X-XSS-Protection': '1; mode=block', 'X-Permitted-Cross-Domain-Policies': 'none', 'Content-Security-Policy': "img-src 'self' * data:; child-src 'self' * blob:; frame-src 'self' * blob:; worker-src 'self' blob:; default-src 'self' data: 'unsafe-inline' 'unsafe-eval' https://*.zdassets.com https://*.zendesk.com https://*.google.com https://*.googleapis.com https://*.googleadservices.com https://*.googletagmanager.com https://*.googleanalytics.com https://*.google-analytics.com https://*.doubleclick.net https://*.gstatic.com https://*.pardot.com https://*.memsource.com https://*.userpilot.io wss://api.userpilot.io https://api.ipstack.com https://*.chargebee.com", 'Permissions-Policy': 'accelerometer=(), ambient-light-sensor=(), battery=(), camera=(), display-capture=(), fullscreen=(), geolocation=(), gyroscope=(), magnetometer=(), microphone=(), midi=(), payment=(), screen-wake-lock=(), usb=(), web-share=()', 'Cache-Control': 'no-store, private, max-age=0', 'Expires': 'Fri, 21 May 2021 15:52:25 GMT', 'Last-Modified': 'Fri, 21 May 2021 16:52:25 GMT', 'Date': 'Fri, 21 May 2021 16:52:25 GMT', 'Vary': 'Accept-Encoding', 'Content-Type': 'application/json', 'Transfer-Encoding': 'chunked', 'Server': '', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'})

We should confirm that we're uploading files correctly and investigate the source of this issue. In testing, we were able to download files from projects created by the Globalization team with no trouble. 

Version-Release number of selected component (if applicable):
4.8

How reproducible:
Always

Steps to Reproduce:
1. Run yarn memsource-download -p 88AFVAwGg3qLblJiBfMBS3 with Memsource credentials turned on

Actual results:
Error message; files not downloaded.

Expected results:
Files are downloaded and imported correctly.

Additional info:

Comment 1 ralpert 2021-05-21 16:54:44 UTC
For the time being, I used the old po-to-i18n script. We're not totally blocked, but it's important to fix this.

Comment 3 Yadan Pei 2021-06-01 02:47:58 UTC
Hi Rebecca,

I have memsource account, but seems I don't have permission to download the translation files. Could you help verify the bug? 

Thank you

Comment 4 ralpert 2021-06-01 15:46:07 UTC
Did you configure the CLI tool? The instructions are here: https://github.com/unofficial-memsource/memsource-cli-client#configuration-red-hat-enterprise-linux-derivatives

I'm realizing I didn't include that in the README; I should add that in.

I can reach out to Terry and see if he can check Memsource also.

Comment 5 Yadan Pei 2021-06-02 06:37:06 UTC
I've successfully setup memsource CLI but still got `403 Access denied error`

this is the detailed error message 

(.memsource) [yapei@localhost ~]$ export MEMSOURCE_URL="https://cloud.memsource.com/web"
(.memsource) [yapei@localhost ~]$ export MEMSOURCE_USERNAME=yapei
(.memsource) [yapei@localhost ~]$ export MEMSOURCE_PASSWORD=<password>
(.memsource) [yapei@localhost ~]$ export MEMSOURCE_TOKEN=$(memsource auth login --user-name $MEMSOURCE_USERNAME --password "${MEMSOURCE_PASSWORD}" -c token -f value)
(.memsource) [yapei@localhost ~]$ echo $MEMSOURCE_TOKEN
46vBB2mZNM0V0HEY5Kxxxxxxxxxxx
 
(.memsource) [yapei@localhost ~]$ cd ~/go/src/github.com/openshift/console/frontend/ 
(.memsource) [yapei@localhost frontend]$ yarn memsource-download -p 88AFVAwGg3qLblJiBfMBS3
yarn run v1.22.10
$ ./i18n-scripts/memsource-download.sh -p 88AFVAwGg3qLblJiBfMBS3
+ source ./i18n-scripts/languages.sh
++ LANGUAGES=('ja' 'zh-cn' 'ko')
++ export LANGUAGES
+ getopts p: flag
+ case "${flag}" in
+ PROJECT_ID=88AFVAwGg3qLblJiBfMBS3
+ getopts p: flag
+ echo 'Checking if git workspace is clean'
Checking if git workspace is clean
++ git status --short --untracked-files -- public/locales packages/ceph-storage-plugin/locales packages/console-app/locales packages/console-shared/locales packages/container-security/locales packages/dev-console/locales packages/gitops-plugin/locales packages/git-service/locales packages/helm-plugin/locales packages/insights-plugin/locales packages/knative-plugin/locales packages/kubevirt-plugin/locales packages/local-storage-operator-plugin/locales packages/metal3-plugin/locales packages/network-attachment-definition-plugin/locales packages/operator-lifecycle-manager/locales packages/patternfly/locales packages/pipelines-plugin/locales packages/rhoas-plugin/locales packages/topology/locales
+ GIT_STATUS=
+ '[' -n '' ']'
+ echo 'Downloading PO files from Project ID "88AFVAwGg3qLblJiBfMBS3"'
Downloading PO files from Project ID "88AFVAwGg3qLblJiBfMBS3"
++ mktemp -d
+ DOWNLOAD_PATH=/tmp/tmp.01Q8Czu1Up
+ for i in "${LANGUAGES[@]}"
+ COUNTER=0
+ CURRENT_PAGE=($(memsource job list --project-id "$PROJECT_ID" --target-lang "$i" -f value --page-number 0 -c uid))
++ memsource job list --project-id 88AFVAwGg3qLblJiBfMBS3 --target-lang ja -f value --page-number 0 -c uid
Memsource API Error
Error: 403
Error description: Access denied
Error Reason: 
Response status: 403

HTTP response headers: HTTPHeaderDict({'Memsource-Action-Id': 'e8f79856d8284f9b97117982278a4c11', 'X-Frame-Options': 'DENY', 'X-Content-Type-Options': 'nosniff', 'Referrer-Policy': 'strict-origin-when-cross-origin', 'X-XSS-Protection': '1; mode=block', 'X-Permitted-Cross-Domain-Policies': 'none', 'Content-Security-Policy': "img-src 'self' * data:; child-src 'self' * blob:; frame-src 'self' * blob:; worker-src 'self' blob:; default-src 'self' data: 'unsafe-inline' 'unsafe-eval' https://*.zdassets.com https://*.zendesk.com https://*.google.com https://*.googleapis.com https://*.googleadservices.com https://*.googletagmanager.com https://*.googleanalytics.com https://*.google-analytics.com https://*.doubleclick.net https://*.gstatic.com https://*.pardot.com https://*.memsource.com https://*.userpilot.io wss://api.userpilot.io https://api.ipstack.com https://*.chargebee.com", 'Permissions-Policy': 'accelerometer=(), ambient-light-sensor=(), battery=(), camera=(), display-capture=(), fullscreen=(), geolocation=(), gyroscope=(), magnetometer=(), microphone=(), midi=(), payment=(), screen-wake-lock=(), usb=(), web-share=()', 'Cache-Control': 'no-store, private, max-age=0', 'Expires': 'Wed, 02 Jun 2021 05:33:36 GMT', 'Last-Modified': 'Wed, 02 Jun 2021 06:33:36 GMT', 'Date': 'Wed, 02 Jun 2021 06:33:36 GMT', 'Vary': 'Accept-Encoding', 'Content-Type': 'application/json', 'Transfer-Encoding': 'chunked', 'Server': '', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'})

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Comment 6 ralpert 2021-06-02 20:01:17 UTC
That's weird. Let me reach out to Terry and see if he knows what the issue is. I'll CC you. Everything you have there looks good to me.

Comment 7 Yadan Pei 2021-06-04 03:11:47 UTC
With sufficient permissions, `yarn memsource-download -p ` succeeded without errors

$ yarn memsource-download -p 88AFVAwGg3qLblJiBfMBS3
yarn run v1.22.10
$ ./i18n-scripts/memsource-download.sh -p 88AFVAwGg3qLblJiBfMBS3
+ source ./i18n-scripts/languages.sh
++ LANGUAGES=('ja' 'zh-cn' 'ko')
++ export LANGUAGES
+ getopts p: flag
+ case "${flag}" in
+ PROJECT_ID=88AFVAwGg3qLblJiBfMBS3
+ getopts p: flag
+ echo 'Checking if git workspace is clean'
Checking if git workspace is clean
++ git status --short --untracked-files -- public/locales packages/ceph-storage-plugin/locales packages/console-app/locales packages/console-shared/locales packages/container-security/locales packages/dev-console/locales packages/git-service/locales packages/gitops-plugin/locales packages/helm-plugin/locales packages/insights-plugin/locales packages/knative-plugin/locales packages/kubevirt-plugin/locales packages/local-storage-operator-plugin/locales packages/metal3-plugin/locales packages/network-attachment-definition-plugin/locales packages/operator-lifecycle-manager/locales packages/patternfly/locales packages/pipelines-plugin/locales packages/rhoas-plugin/locales packages/topology/locales
+ GIT_STATUS=
+ '[' -n '' ']'
+ echo 'Downloading PO files from Project ID "88AFVAwGg3qLblJiBfMBS3"'
Downloading PO files from Project ID "88AFVAwGg3qLblJiBfMBS3"
++ mktemp -d
+ DOWNLOAD_PATH=/var/folders/jw/mtghbhx152l1zfffkwqtcdfr0000gn/T/tmp.uwnOfcfc
+ for i in '"${LANGUAGES[@]}"'
+ COUNTER=0
+ CURRENT_PAGE=($(memsource job list --project-id "$PROJECT_ID" --target-lang "$i" -f value --page-number 0 -c uid))
++ memsource job list --project-id 88AFVAwGg3qLblJiBfMBS3 --target-lang ja -f value --page-number 0 -c uid
+ '[' -z sdsTPziVr4rHedDUU7fPR0 ']'
+ (( COUNTER++ ))
+ echo Downloading page 1
Downloading page 1
+ memsource job download --project-id 88AFVAwGg3qLblJiBfMBS3 --output-dir /var/folders/jw/mtghbhx152l1zfffkwqtcdfr0000gn/T/tmp.uwnOfcfc/ja --job-id sdsTPziVr4rHedDUU7fPR0 88kwpZ59qWqwaFsssR0UU6 2GBCUePZGIVcfV77FM8WN0 ChGCwVGoN5qW2lB1uT2MK2 ydyxxZSDRVUR4xn851iU85 flwWPytGrypGe5q9nYf4a0 VqW3051bRVBk24PGfG5xd0 v0j1hEdi7M
......
Adding latest translations from Memsource project https://cloud.memsource.com/web/project2/show/88AFVAwGg3qLblJiBfMBS3'
husky > pre-commit (node v14.16.0)
✔ Preparing...
✔ Running tasks...
✔ Applying modifications...
✔ Cleaning up... 
[ipaddr b0adbde677] chore(i18n): update translations
 8 files changed, 832 insertions(+), 4 deletions(-)
 create mode 100644 frontend/packages/console-app_console-app/locales/ja/undefined.json
 create mode 100644 frontend/packages/error/locales/ja/undefined.json
 create mode 100644 frontend/packages/error/locales/ko/undefined.json
 create mode 100644 frontend/packages/public_public/locales/ja/undefined.json
✨  Done in 147.89s.


Moving to VERIFIED

Comment 10 errata-xmlrpc 2021-07-27 23:09:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438