When executing "gsutils cp" in python script I get Error message I do not understand

Question

I do the following call from within a Pyhton3 script on a master node of a GCP Dataproc cluster:

gsutil cp -r gs_folder local_folder

I get a message which I find a bit strange:

STDERR: b'ERROR: (gsutil) Failed to create the default configuration. Ensure your have the correct permissions o
n: [/home/.config/gcloud/configurations].\n  Could not create directory [/home/.config/gcloud/configurations]: P
ermission denied.\n\nPlease verify that you have permissions to write to the parent directory.\n'**********
GSUTIL COMMAND:
gsutil cp -r gs://my_source_paht /target_path

What I find odd is the location /home/.config/gcloud/configurations it mentions. Does this even make sense?

When I execute the same command from CLI of this machine as sudo then it works. So I am not sure: Is my current user not allowed to use gutil or 2) is it a permission problem regarding Google Storage?

Answer 1

I don't think the install has worked correctly. One would expect the install to setup configuration within ${HOME}/.config/gcloud. e.g. /home/mike/.config/cloud. But your install appears to be in /home itself. System wide configurations are normally located within /etc not /home.

Running gsutil gives an error because your user does not permission to write to /home (which is owned by root). Or permission to write to a subdirectory of /home. gsutil is trying to write to /home/.config/gcloud/configurations but does not have permission.