Bug 1963213 - Memsource download script running into API error
Summary: Memsource download script running into API error
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Management Console
Version: 4.8
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ---
: 4.8.0
Assignee: ralpert
QA Contact: Yadan Pei
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-05-21 16:54 UTC by ralpert
Modified: 2021-07-27 23:10 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-07-27 23:09:50 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift console pull 9024 0 None open Bug 1963213: Fixed bug in Memsource download script 2021-05-24 15:59:32 UTC
Red Hat Product Errata RHSA-2021:2438 0 None None None 2021-07-27 23:10:03 UTC

Description ralpert 2021-05-21 16:54:10 UTC
Description of problem:
I'm getting an error when I try to use the Memsource script to download the latest Sprint 200 project:

Memsource API Error
Error: Invalid parameters
Error description: Job "sdsTPziVr4rHedDUU7fPR0 88kwpZ59qWqwaFsssR0UU6 2GBCUePZGIVcfV77FM8WN0 ChGCwVGoN5qW2lB1uT2MK2 ydyxxZSDRVUR4xn851iU85 flwWPytGrypGe5q9nYf4a0 VqW3051bRVBk24PGfG5xd0 v0j1hEdi7MyMsAwGQDnrcc Cu90D8lg0EwDskdMulj0iv yBS5hw7EgE973F2zBJkT10 xFpoqv0089KdeAGSioAYy5 KAaVv907AIhSlqcrXG1XGa L3E57bNeYN8c99Y0MpBKQ6 jJ2jSAPvgTXc8p3AaQiRN1 SZvkyFpRcyEBcNaAbextX1 MMtJDq0l8cjz4WNM0X2qRb ViMmyOU8fhEE7FhpheElF1 DcRKnsi3r5r44lK6Trm1R7 lBvtedAd4bXgahtZGTCb61 nmtIJE0m2JOcsyHB8RkEr3 2lVN3RqTQ0k1tk0t2ZnkTv UxFJx2SaKAXpeRdLyRczc3 OJDxArk1mbR37yLCPI1orf dSVagR6Pch95eVKhzauk40 GNQjw1nE7VM6ry6Dwt5KO6 4mbJfPolia974t8jXVhK73 OCrb0HVygvM33iW0sFM29e ZSFbfbFyOVEOd5qZf7owB0 9EZh2z9mPdoD4RVeNJIHt2 DpGQqTihd4V71Gcu5di2a7 01suXHocQCAy5k8siHXLW6 lgZn06Qg7rnlsWrFzHuFP0 4lYVYohhXb8ucJsCjUlTZ0 8pwD3ZIouOWg7FXatnCei1 " not imported correctly or import is in progress
Error Reason: 
Response status: 400

HTTP response headers: HTTPHeaderDict({'Memsource-Action-Id': '0b24fca0103c40a1839ea44666422fe1', 'X-Frame-Options': 'DENY', 'X-Content-Type-Options': 'nosniff', 'Referrer-Policy': 'strict-origin-when-cross-origin', 'X-XSS-Protection': '1; mode=block', 'X-Permitted-Cross-Domain-Policies': 'none', 'Content-Security-Policy': "img-src 'self' * data:; child-src 'self' * blob:; frame-src 'self' * blob:; worker-src 'self' blob:; default-src 'self' data: 'unsafe-inline' 'unsafe-eval' https://*.zdassets.com https://*.zendesk.com https://*.google.com https://*.googleapis.com https://*.googleadservices.com https://*.googletagmanager.com https://*.googleanalytics.com https://*.google-analytics.com https://*.doubleclick.net https://*.gstatic.com https://*.pardot.com https://*.memsource.com https://*.userpilot.io wss://api.userpilot.io https://api.ipstack.com https://*.chargebee.com", 'Permissions-Policy': 'accelerometer=(), ambient-light-sensor=(), battery=(), camera=(), display-capture=(), fullscreen=(), geolocation=(), gyroscope=(), magnetometer=(), microphone=(), midi=(), payment=(), screen-wake-lock=(), usb=(), web-share=()', 'Cache-Control': 'no-store, private, max-age=0', 'Expires': 'Fri, 21 May 2021 15:52:25 GMT', 'Last-Modified': 'Fri, 21 May 2021 16:52:25 GMT', 'Date': 'Fri, 21 May 2021 16:52:25 GMT', 'Vary': 'Accept-Encoding', 'Content-Type': 'application/json', 'Transfer-Encoding': 'chunked', 'Server': '', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'})

We should confirm that we're uploading files correctly and investigate the source of this issue. In testing, we were able to download files from projects created by the Globalization team with no trouble. 

Version-Release number of selected component (if applicable):
4.8

How reproducible:
Always

Steps to Reproduce:
1. Run yarn memsource-download -p 88AFVAwGg3qLblJiBfMBS3 with Memsource credentials turned on

Actual results:
Error message; files not downloaded.

Expected results:
Files are downloaded and imported correctly.

Additional info:

Comment 1 ralpert 2021-05-21 16:54:44 UTC
For the time being, I used the old po-to-i18n script. We're not totally blocked, but it's important to fix this.

Comment 3 Yadan Pei 2021-06-01 02:47:58 UTC
Hi Rebecca,

I have memsource account, but seems I don't have permission to download the translation files. Could you help verify the bug? 

Thank you

Comment 4 ralpert 2021-06-01 15:46:07 UTC
Did you configure the CLI tool? The instructions are here: https://github.com/unofficial-memsource/memsource-cli-client#configuration-red-hat-enterprise-linux-derivatives

I'm realizing I didn't include that in the README; I should add that in.

I can reach out to Terry and see if he can check Memsource also.

Comment 5 Yadan Pei 2021-06-02 06:37:06 UTC
I've successfully setup memsource CLI but still got `403 Access denied error`

this is the detailed error message 

(.memsource) [yapei@localhost ~]$ export MEMSOURCE_URL="https://cloud.memsource.com/web"
(.memsource) [yapei@localhost ~]$ export MEMSOURCE_USERNAME=yapei
(.memsource) [yapei@localhost ~]$ export MEMSOURCE_PASSWORD=<password>
(.memsource) [yapei@localhost ~]$ export MEMSOURCE_TOKEN=$(memsource auth login --user-name $MEMSOURCE_USERNAME --password "${MEMSOURCE_PASSWORD}" -c token -f value)
(.memsource) [yapei@localhost ~]$ echo $MEMSOURCE_TOKEN
46vBB2mZNM0V0HEY5Kxxxxxxxxxxx
 
(.memsource) [yapei@localhost ~]$ cd ~/go/src/github.com/openshift/console/frontend/ 
(.memsource) [yapei@localhost frontend]$ yarn memsource-download -p 88AFVAwGg3qLblJiBfMBS3
yarn run v1.22.10
$ ./i18n-scripts/memsource-download.sh -p 88AFVAwGg3qLblJiBfMBS3
+ source ./i18n-scripts/languages.sh
++ LANGUAGES=('ja' 'zh-cn' 'ko')
++ export LANGUAGES
+ getopts p: flag
+ case "${flag}" in
+ PROJECT_ID=88AFVAwGg3qLblJiBfMBS3
+ getopts p: flag
+ echo 'Checking if git workspace is clean'
Checking if git workspace is clean
++ git status --short --untracked-files -- public/locales packages/ceph-storage-plugin/locales packages/console-app/locales packages/console-shared/locales packages/container-security/locales packages/dev-console/locales packages/gitops-plugin/locales packages/git-service/locales packages/helm-plugin/locales packages/insights-plugin/locales packages/knative-plugin/locales packages/kubevirt-plugin/locales packages/local-storage-operator-plugin/locales packages/metal3-plugin/locales packages/network-attachment-definition-plugin/locales packages/operator-lifecycle-manager/locales packages/patternfly/locales packages/pipelines-plugin/locales packages/rhoas-plugin/locales packages/topology/locales
+ GIT_STATUS=
+ '[' -n '' ']'
+ echo 'Downloading PO files from Project ID "88AFVAwGg3qLblJiBfMBS3"'
Downloading PO files from Project ID "88AFVAwGg3qLblJiBfMBS3"
++ mktemp -d
+ DOWNLOAD_PATH=/tmp/tmp.01Q8Czu1Up
+ for i in "${LANGUAGES[@]}"
+ COUNTER=0
+ CURRENT_PAGE=($(memsource job list --project-id "$PROJECT_ID" --target-lang "$i" -f value --page-number 0 -c uid))
++ memsource job list --project-id 88AFVAwGg3qLblJiBfMBS3 --target-lang ja -f value --page-number 0 -c uid
Memsource API Error
Error: 403
Error description: Access denied
Error Reason: 
Response status: 403

HTTP response headers: HTTPHeaderDict({'Memsource-Action-Id': 'e8f79856d8284f9b97117982278a4c11', 'X-Frame-Options': 'DENY', 'X-Content-Type-Options': 'nosniff', 'Referrer-Policy': 'strict-origin-when-cross-origin', 'X-XSS-Protection': '1; mode=block', 'X-Permitted-Cross-Domain-Policies': 'none', 'Content-Security-Policy': "img-src 'self' * data:; child-src 'self' * blob:; frame-src 'self' * blob:; worker-src 'self' blob:; default-src 'self' data: 'unsafe-inline' 'unsafe-eval' https://*.zdassets.com https://*.zendesk.com https://*.google.com https://*.googleapis.com https://*.googleadservices.com https://*.googletagmanager.com https://*.googleanalytics.com https://*.google-analytics.com https://*.doubleclick.net https://*.gstatic.com https://*.pardot.com https://*.memsource.com https://*.userpilot.io wss://api.userpilot.io https://api.ipstack.com https://*.chargebee.com", 'Permissions-Policy': 'accelerometer=(), ambient-light-sensor=(), battery=(), camera=(), display-capture=(), fullscreen=(), geolocation=(), gyroscope=(), magnetometer=(), microphone=(), midi=(), payment=(), screen-wake-lock=(), usb=(), web-share=()', 'Cache-Control': 'no-store, private, max-age=0', 'Expires': 'Wed, 02 Jun 2021 05:33:36 GMT', 'Last-Modified': 'Wed, 02 Jun 2021 06:33:36 GMT', 'Date': 'Wed, 02 Jun 2021 06:33:36 GMT', 'Vary': 'Accept-Encoding', 'Content-Type': 'application/json', 'Transfer-Encoding': 'chunked', 'Server': '', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'})

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Comment 6 ralpert 2021-06-02 20:01:17 UTC
That's weird. Let me reach out to Terry and see if he knows what the issue is. I'll CC you. Everything you have there looks good to me.

Comment 7 Yadan Pei 2021-06-04 03:11:47 UTC
With sufficient permissions, `yarn memsource-download -p ` succeeded without errors

$ yarn memsource-download -p 88AFVAwGg3qLblJiBfMBS3
yarn run v1.22.10
$ ./i18n-scripts/memsource-download.sh -p 88AFVAwGg3qLblJiBfMBS3
+ source ./i18n-scripts/languages.sh
++ LANGUAGES=('ja' 'zh-cn' 'ko')
++ export LANGUAGES
+ getopts p: flag
+ case "${flag}" in
+ PROJECT_ID=88AFVAwGg3qLblJiBfMBS3
+ getopts p: flag
+ echo 'Checking if git workspace is clean'
Checking if git workspace is clean
++ git status --short --untracked-files -- public/locales packages/ceph-storage-plugin/locales packages/console-app/locales packages/console-shared/locales packages/container-security/locales packages/dev-console/locales packages/git-service/locales packages/gitops-plugin/locales packages/helm-plugin/locales packages/insights-plugin/locales packages/knative-plugin/locales packages/kubevirt-plugin/locales packages/local-storage-operator-plugin/locales packages/metal3-plugin/locales packages/network-attachment-definition-plugin/locales packages/operator-lifecycle-manager/locales packages/patternfly/locales packages/pipelines-plugin/locales packages/rhoas-plugin/locales packages/topology/locales
+ GIT_STATUS=
+ '[' -n '' ']'
+ echo 'Downloading PO files from Project ID "88AFVAwGg3qLblJiBfMBS3"'
Downloading PO files from Project ID "88AFVAwGg3qLblJiBfMBS3"
++ mktemp -d
+ DOWNLOAD_PATH=/var/folders/jw/mtghbhx152l1zfffkwqtcdfr0000gn/T/tmp.uwnOfcfc
+ for i in '"${LANGUAGES[@]}"'
+ COUNTER=0
+ CURRENT_PAGE=($(memsource job list --project-id "$PROJECT_ID" --target-lang "$i" -f value --page-number 0 -c uid))
++ memsource job list --project-id 88AFVAwGg3qLblJiBfMBS3 --target-lang ja -f value --page-number 0 -c uid
+ '[' -z sdsTPziVr4rHedDUU7fPR0 ']'
+ (( COUNTER++ ))
+ echo Downloading page 1
Downloading page 1
+ memsource job download --project-id 88AFVAwGg3qLblJiBfMBS3 --output-dir /var/folders/jw/mtghbhx152l1zfffkwqtcdfr0000gn/T/tmp.uwnOfcfc/ja --job-id sdsTPziVr4rHedDUU7fPR0 88kwpZ59qWqwaFsssR0UU6 2GBCUePZGIVcfV77FM8WN0 ChGCwVGoN5qW2lB1uT2MK2 ydyxxZSDRVUR4xn851iU85 flwWPytGrypGe5q9nYf4a0 VqW3051bRVBk24PGfG5xd0 v0j1hEdi7M
......
Adding latest translations from Memsource project https://cloud.memsource.com/web/project2/show/88AFVAwGg3qLblJiBfMBS3'
husky > pre-commit (node v14.16.0)
✔ Preparing...
✔ Running tasks...
✔ Applying modifications...
✔ Cleaning up... 
[ipaddr b0adbde677] chore(i18n): update translations
 8 files changed, 832 insertions(+), 4 deletions(-)
 create mode 100644 frontend/packages/console-app_console-app/locales/ja/undefined.json
 create mode 100644 frontend/packages/error/locales/ja/undefined.json
 create mode 100644 frontend/packages/error/locales/ko/undefined.json
 create mode 100644 frontend/packages/public_public/locales/ja/undefined.json
✨  Done in 147.89s.


Moving to VERIFIED

Comment 10 errata-xmlrpc 2021-07-27 23:09:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438


Note You need to log in before you can comment on or make changes to this bug.