I have successfully added the ability to connect and transfer files from a GCP instance using my own local account on the GCP-instance using a by adding the API access for a Google Cloud service-account.

I have a service, NGINX, running on the GCP instance and it is running with a local account privileges (nginx) on the instance. However as the user 'nginx' I cannot use the gsutil command. What I'm trying to do is to transfer log files from the GCP-instance to a bucket for safe keeping so that I can free up the space on the instance.

Now to my question: How can I set up the permissions or environment so that the user so that local accounts on GCP instances can use the gsutil and/or gcloud (or similar) commands to copy data from the instance to a bucket using an GCP service account?


A GCE instance will authenticate API calls using a service account, which is abstracted away from any local user on the server - credentials are essentially embedded into the instance's metadata. You should not use secondary service accounts or user credentials on a GCE instance - this is most likely why your personal user is able to transfer files, and the nginx user is not.

To enable your instance to write data to a GCS bucket, you will most likely need to launch the instance with the GCS read/write scope (docs) - GCS access is disabled by default.

Note that if you created your instance using an instance template, these cannot be edited, so the template will need to be destroyed and recreated, and any instances based on the old template will also need to be reprovisioned.

| improve this answer | |
  • I have enabled the Cloud Storage API as Read and Write for the GCE instance. I did that prior to asking the question. And I have added the service account for the project to the instance with access to GCS thats what was necessary to be able to use gsutil as my user. Do you mean that the GCE instance have it's own service account that is added when the instance is created and I should use that instead? I could only find one GCP service account in the project and I assigned that to the GCE instance since it was not present there from the start. – Fredrik Westermarck Jun 13 '18 at 17:38
  • Each GCP project will have its own "default" compute service account. This account is automatically used by any running GCE instance. – Craig Watson Jun 14 '18 at 10:31

Some ideas for what might be wrong:

  • The configuration telling gsutil to use application default credentials is in /etc/boto.cfg. Make sure that your nginx user can read that file.
  • gsutil's settings can be overridden. See if there's a BOTO_CONFIG environment variable or a ~nginx/.boto.cfg file replacing the default settings.

I believe any user on a GCE instance should be able to use gsutil. The credentials are granted via an HTTP call. You could test that you receive them by sudo'ing into being the nginx user and running this:

curl "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token" \
-H "Metadata-Flavor: Google"
| improve this answer | |
  • I checked the /etc/boto.cfg and it is readable for all users. Neither my user account on the instance or the nginx local account has an overridden $BOTO_CONFIG. Running the curlcommand gives me an JSON response with an accesstoken and some more data. I can acutally run the gsutil command as the instance local account but it looks for it's credentials from my user account /home/<username>/.config/gcloud/active_config and that is not readable. Q: It is possible to set up credentials on the local instance account instead of granting permissions to my unser account? – Fredrik Westermarck Jun 13 '18 at 7:02
  • Or is it better to use https://github.com/GoogleCloudPlatform/gcsfuse to be able to move the files from the GCP instance to a Cloud Storage bucket? – Fredrik Westermarck Jun 13 '18 at 12:08

I solved the issue by adding a group and providing the .config and .gsutil directories with the appropriate permissions for the group and then added the nginx user to the group. But that was really not the way I preferred to resolve it - but it does the trick.

| improve this answer | |
  • As I mentioned in my answer, you should not use local credentials. Creating those directories suggests that you are manually feeding credentials to the binary, which will override the instance's service account. – Craig Watson Jun 14 '18 at 10:29
  • It's the default GCP service account that is being used (XXXXXXXXX-compute@developer.gserviceaccount.com). However the local account nginx on the server were trying to read the credentials from my user account on the local machine when running gsutils. I would have preferred another way of adding the credentials to the nginx-account but I could not find a way to set the credentials as the local user account and gsutil did, by default, look for the credentials in my local user account. – Fredrik Westermarck Jun 14 '18 at 21:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.