Questions tagged [gsutil]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
-1
votes
0answers
51 views

Google Cloud storage failed at certain files

how to check which folders/files cause my backup failed from the local server to the bucket? Is there any indicator/email notification if the backup fails? Thanks
1
vote
1answer
24 views

start/stop a Google Cloud VM from desktop

I have a VM running on Google cloud that I would like to start and stop. I am able to start/stop it from another VM in the same project using the command sudo gcloud compute instances stop my-daily-...
0
votes
2answers
155 views

gcloud auth login: Limit Oauth2 scope

How can I limit the Oauth2 scope when logging in with my user principal using (non service account): gcloud auth login Background: I created a Gsuite Takeout, got the Takeout Bucket. Now I want to ...
0
votes
1answer
41 views

remotely connect to files via gs path [closed]

I'm using an R package that requires a list of file locations to work. Normally, if I have files stored in remote servers I provide the ftp, http or ssh paths and it works. If I do the same providing ...
0
votes
1answer
87 views

gsutil error /bootstrapping/gsutil.py

gsutil is giving me the following error. It is installed in its latest version on a linux suse 12. Can you help me? gsutil Traceback (most recent call last): File "/root/google-cloud-sdk/bin/...
2
votes
0answers
187 views

gsutil rsync fails on Ubuntu with UnicodeDecodeError

I'm trying to rsync between google cloud storage and amazon s3. On my local machine (MacOS) this command works gsutil rsync -d -r gs://mira-internal/data/exports/20191108 s3://mira-temp/raw/20191108 ...
3
votes
1answer
412 views

gsutil rsync “too many values to unpack” error

I'm trying to sync a folder with over 3k files to a bucket on Google Cloud with gsutil. The problem is that less than half of the files are uploaded and then I get this error: too many values to ...
1
vote
1answer
132 views

How/why would ALL our Google Cloud Platform Service account keys suddenly disappear/be deleted?

We had a major issue with Google Cloud Platform some time between 2019-04-16T11:00:00Z and 2019-04-17T02:30:00Z, when all of our service accounts lost their access keys for an unknown reason, so our ...
3
votes
1answer
775 views

How to use JSON keys with google cloud gsutil to manage multiple Keys

We have multiple GCP Service account keys from different environment such as DEV,STAGING,..PROD. I would like to run a command in my jenkins box which is there in Dev environment to create access DEV ...
0
votes
0answers
79 views

Why would gsutil come up short when transferring files into a bucket?

I have a very large directory tree sitting in the file system of a Google Compute instance. If I do a "find foo -type f | wc -l" on it, it comes back with 34577, and I have no reason to assume that ...
0
votes
1answer
315 views

Specify account for single gsutil call

I want to be able to perform simultaneous gsutil calls against Google Cloud Storage, using different accounts. For that purpose, I'd like to know if there is a way to specify a Google Cloud account ...
0
votes
1answer
1k views

CommandException: arg […] does not name a directory, bucket, or bucket subdir. for a directory on Compute Engine

I am trying to use gsutils command to deploy changes from CircleCI to a directory on my Compute Engine instance. The command I am using is: gsutil -m rsync -d -r dev/ [VM_INSTANCE_NAME]:/var/www/dev/ ...
0
votes
3answers
567 views

How to run gsutil rsync without auth?

aws s3 sync provides a parameter named --no-sign-request, so user can sync a publicly-visible bucket without any auth in advance. gsutil help anon says that: gsutil users can access publicly ...
0
votes
1answer
513 views

gsutil “Copying mtime from src to dst” having moved source files?

I use gsutil to backup photos from a dedicated box to google nearline storage. Recently I moved all of my photos from that dedicated box to a new dedicated box. I made sure to use the relevant ...
-2
votes
1answer
586 views

CommandException: Caught non-retryable exception - aborting rsync

..... At destination listing 8360000... CommandException: Caught non-retryable exception - aborting rsync ..... I tried to locate the files with this sync problem but I am not able to do so. Is there ...
1
vote
1answer
2k views

Access denied 403 when trying to download from the Google Cloud Platform, but user has all access

We are just trying to use gsutil from the command line to download some DCM data from the Google Cloud Platform: gsutil -m cp -R gs://dcdt_-dcm_account75701/dcm_account75701_activity_201803* C:\Users\...
0
votes
2answers
234 views

Dedicated Interconnect to GCP Storage (gsutil)

We are have a dedicated interconnect setup from our data center to GCP using a cloud router. We have a RHEL linux server setup in the datacenter with a nic port connected to the public Internet and a ...
0
votes
0answers
356 views

Why does gsutil rsync on CoreOS fail with 'CommandException: arg () does not name a directory, bucket, or bucket subdir'

On a recent Google Cloud Platform CoreOS VM, the command gsutil rsync -r gs://some-bucket-here dest fails with: CommandException: arg (dest) does not name a directory, bucket, or bucket subdir. $ ...
2
votes
1answer
351 views

Google Storage ACL - write only for public, but don't allow read, list, delete or update operations

I need to setup Google Storage access for 3rd parties (public) to allow them to upload one or more files, but they shouldn't be allowed to read, list, delete or update an existing file. This structure ...
1
vote
1answer
2k views

Copy list of files (-I flag) with gsutil preserving path

I am trying to copy all pictures and static files to a bucket of mine in Google Cloud Platform. I am attempting this command from the root dir of my app: find -regextype posix-extended -iregex ".*\.(...
-1
votes
2answers
510 views

Convert filenames to lowercase [Google Cloud Storage] [gsutil]

We recently moved all product images from our website to Google Cloud Storage. Due to bad naming policies in the past the casing (lowercase, uppercase) in the file names does not always match the way ...
2
votes
1answer
819 views

gsutil ls -l fails when gsutil mb succeeded

I get the following when I attempt to list the buckets for the account I'm logged in with: $ gsutil ls -l gs://bucket AccessDeniedException: 403 Forbidden $ gcloud config list Your active ...
14
votes
2answers
15k views

Using JSON keys with google cloud gsutil

I have a ssh private key in a key.json file, and I want to use this credential to access a storage bucket using gsutil. I can't seem to find anything about how to include json keys as a ...
1
vote
1answer
895 views

Setting up logging for buckets in Google Cloud storage

Trying to set up logging on a GCS bucket. every time i enter the command to set logging i get the error below. "Command exception: "logging" command does not support "file://" URLs. did you mean to ...
2
votes
1answer
251 views

gsutil cp error: cannot have start index greater than total size

I'm trying to download a lot of images from a Google Bucket that my employer's web host (not so) helpfully set up. I run the command gsutil cp -r gs://bucket-link-info/images/ . and the download ...
6
votes
1answer
2k views

Slow synchronisation stage on gsutil rsync?

I've just started to use GCS as backup for my web servers. One server has 1.2 million JPEGS (3.5TB) and this all rsynced over flawlessly over 10 hours or so. The other has 2.5 million JPEGS (just ...