Questions tagged [gsutil]

The tag has no usage guidance.

1
vote
1answer
80 views

How/why would ALL our Google Cloud Platform Service account keys suddenly disappear/be deleted?

We had a major issue with Google Cloud Platform some time between 2019-04-16T11:00:00Z and 2019-04-17T02:30:00Z, when all of our service accounts lost their access keys for an unknown reason, so our ...
0
votes
0answers
69 views

gcloud-rsync skip files which other process is currently using

I am running gcloud rsync command where I am trying to sync my local folder with google cloud bucket. Is there any way I can skip those files from local folder which is currently in write mode by ...
1
vote
1answer
265 views

How to use JSON keys with google cloud gsutil to manage multiple Keys

We have multiple GCP Service account keys from different environment such as DEV,STAGING,..PROD. I would like to run a command in my jenkins box which is there in Dev environment to create access DEV ...
0
votes
0answers
15 views

Error gsutil cp Service Exception: Transfer invalid cannot have start index greater than total size

I'm trying to download a lot of files from a Google Bucket. I run the command gsutil cp -r gs://bucket-link-info/files/ . and the download starts fine. However, after a while, it stops with the error ...
0
votes
0answers
50 views

Why would gsutil come up short when transferring files into a bucket?

I have a very large directory tree sitting in the file system of a Google Compute instance. If I do a "find foo -type f | wc -l" on it, it comes back with 34577, and I have no reason to assume that ...
0
votes
1answer
66 views

Specify account for single gsutil call

I want to be able to perform simultaneous gsutil calls against Google Cloud Storage, using different accounts. For that purpose, I'd like to know if there is a way to specify a Google Cloud account ...
0
votes
1answer
448 views

CommandException: arg […] does not name a directory, bucket, or bucket subdir. for a directory on Compute Engine

I am trying to use gsutils command to deploy changes from CircleCI to a directory on my Compute Engine instance. The command I am using is: gsutil -m rsync -d -r dev/ [VM_INSTANCE_NAME]:/var/www/dev/ ...
0
votes
3answers
294 views

How to run gsutil rsync without auth?

aws s3 sync provides a parameter named --no-sign-request, so user can sync a publicly-visible bucket without any auth in advance. gsutil help anon says that: gsutil users can access publicly ...
0
votes
1answer
247 views

gsutil “Copying mtime from src to dst” having moved source files?

I use gsutil to backup photos from a dedicated box to google nearline storage. Recently I moved all of my photos from that dedicated box to a new dedicated box. I made sure to use the relevant ...
-2
votes
1answer
367 views

CommandException: Caught non-retryable exception - aborting rsync

..... At destination listing 8360000... CommandException: Caught non-retryable exception - aborting rsync ..... I tried to locate the files with this sync problem but I am not able to do so. Is there ...
0
votes
1answer
1k views

Access denied 403 when trying to download from the Google Cloud Platform, but user has all access

We are just trying to use gsutil from the command line to download some DCM data from the Google Cloud Platform: gsutil -m cp -R gs://dcdt_-dcm_account75701/dcm_account75701_activity_201803* C:\Users\...
0
votes
2answers
107 views

Dedicated Interconnect to GCP Storage (gsutil)

We are have a dedicated interconnect setup from our data center to GCP using a cloud router. We have a RHEL linux server setup in the datacenter with a nic port connected to the public Internet and a ...
0
votes
0answers
258 views

Why does gsutil rsync on CoreOS fail with 'CommandException: arg () does not name a directory, bucket, or bucket subdir'

On a recent Google Cloud Platform CoreOS VM, the command gsutil rsync -r gs://some-bucket-here dest fails with: CommandException: arg (dest) does not name a directory, bucket, or bucket subdir. $ ...
1
vote
1answer
202 views

Google Storage ACL - write only for public, but don't allow read, list, delete or update operations

I need to setup Google Storage access for 3rd parties (public) to allow them to upload one or more files, but they shouldn't be allowed to read, list, delete or update an existing file. This structure ...
1
vote
1answer
930 views

Copy list of files (-I flag) with gsutil preserving path

I am trying to copy all pictures and static files to a bucket of mine in Google Cloud Platform. I am attempting this command from the root dir of my app: find -regextype posix-extended -iregex ".*\.(...
-1
votes
2answers
237 views

Convert filenames to lowercase [Google Cloud Storage] [gsutil]

We recently moved all product images from our website to Google Cloud Storage. Due to bad naming policies in the past the casing (lowercase, uppercase) in the file names does not always match the way ...
2
votes
1answer
752 views

gsutil ls -l fails when gsutil mb succeeded

I get the following when I attempt to list the buckets for the account I'm logged in with: $ gsutil ls -l gs://bucket AccessDeniedException: 403 Forbidden $ gcloud config list Your active ...
6
votes
2answers
8k views

Using JSON keys with google cloud gsutil

I have a ssh private key in a key.json file, and I want to use this credential to access a storage bucket using gsutil. I can't seem to find anything about how to include json keys as a ...
1
vote
1answer
632 views

Setting up logging for buckets in Google Cloud storage

Trying to set up logging on a GCS bucket. every time i enter the command to set logging i get the error below. "Command exception: "logging" command does not support "file://" URLs. did you mean to ...
2
votes
1answer
196 views

gsutil cp error: cannot have start index greater than total size

I'm trying to download a lot of images from a Google Bucket that my employer's web host (not so) helpfully set up. I run the command gsutil cp -r gs://bucket-link-info/images/ . and the download ...
5
votes
1answer
1k views

Slow synchronisation stage on gsutil rsync?

I've just started to use GCS as backup for my web servers. One server has 1.2 million JPEGS (3.5TB) and this all rsynced over flawlessly over 10 hours or so. The other has 2.5 million JPEGS (just ...