Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

Questions tagged [google-cloud-storage]

Google Cloud Storage, also called GCS, is a cloud service for storing and accessing arbitrarily large objects.

Filter by
Sorted by
Tagged with
0 votes
0 answers
21 views

Google Cloud Storage video streaming vs Local storage?

I am working on an application using React that makes it possible for users to edit videos. What I am trying to figure out is the pros and cons of providing storage for the users. If I wrap the React ...
user avatar
0 votes
2 answers
73 views

Download from URL into GCP bucket

Given a URL that points to a large file, is there a shortcut to getting that file into a bucket, without downloading it locally and then uploading? Is there a simpler way to do it, other than building ...
user avatar
  • 115
0 votes
1 answer
38 views

GCS - Internet Egress Costs

Does anyone know where can I find the pricing for Google Cloud Storage pricing for Internet Traffic? Doing some design work and I need to work out what the cost of multiple GCS Buckets serving content ...
user avatar
0 votes
0 answers
18 views

Files in Bucket no longer appear

I'm new to GC and while following the directions to install an SSL on a project it appears the bucket for the project was wiped. No more files are showing up. One, can anyone explain what I may have ...
user avatar
0 votes
0 answers
26 views

Google Cloud Storage: what happens to billing after storage class change on an object

I would need to better understand what it happens to billing when I change the storage class of an object. I have a bucket on Google Cloud Storage; inside the bucket I have an object of 100Gb saved as ...
user avatar
  • 101
1 vote
1 answer
184 views

How to trigger an event after saving a chosen csv file in Google Cloud Storage bucket

Trying to make a synchronous pipeline, I need to copy a csv file from Google Cloud Storage after it has been saved in Google Cloud Storage. The copy job does not have to be triggered right after the ...
user avatar
0 votes
1 answer
330 views

Reading only the metadata of a file in a Google Cloud Storage bucket into a Cloud Function in Python (without loading the file or its data!)

I need something like Cloud Storage for Firebase: download metadata of all files, just not in Angular but in Python and just for a chosen file instead. The aim is to return this information when the ...
user avatar
0 votes
0 answers
62 views

Google cloud storage --- Upload and download a public bucket

I have a question about Google Cloud Storage. A company wants to share some data with us. They want to upload the data in a cloud storage and then we download them. My questions are: 1- Download the ...
user avatar
  • 1
0 votes
1 answer
251 views

Unable to access GCS Object with storage.objects.get

I have a bucket with uniform permissions (no object level ACLs) and my account has the Owner role on the project which should give full access to all resources. I have even tried adding Storage Admin /...
user avatar
  • 11
0 votes
0 answers
73 views

GKE autoscaler sometimes doesn't scale pods

We have a deployment configured with HPA based on the CPU metric. It can work fine for days, scaling pods up and down. And then at some point looks that it ignores metric and scales to some small ...
user avatar
0 votes
0 answers
100 views

Bucket mounted using gcsfuse doesn't match what's in my cloud storage bucket

I have a Google Cloud Storage bucket and am experimenting with creating a website in Google Cloud Platform (using Python+Flask FWIW). For my initial experiment, I'm using gcsfuse to mount my GCS ...
user avatar
0 votes
1 answer
104 views

How to sync GCP Cloud Storage Bucket metadata to a database?

I have a large number of objects, currently around 1 million, stored in a GCP Cloud Storage Bucket. Objects are added at a rate of 1-2 thousand per day. I would like to efficiently run queries to ...
user avatar
  • 362
0 votes
0 answers
78 views

Google cloud function - Unable to deploy

I have lots of cloud function deployed and working fine. (I am the project owner) Now I am not able to deploy any function and keep on getting this error - ERROR: gcloud crashed (ConnectionError): ('...
user avatar
0 votes
0 answers
38 views

Configure the default log fields in Google Cloud Storage when enable Logging for a bucket

ON GCP, when "Enable logging for a bucket" (1) is there any option to configure the default log fields (2) ? (eg: I would like to drop by default the field: c_ip_region). Thank you. (1) ...
user avatar
0 votes
0 answers
26 views

GCS egress monitoring not working

When I switch to "Monitoring" I see nothing but a message: "no data available for US-east1 during the selected time range. Make sure you have enabled the storage.googleapis.com API for ...
user avatar
1 vote
1 answer
431 views

Service account does not have storage.buckets.get access to the Google Cloud Storage bucket

I am running Ansible with a service the following configuration which should create a storage bucket on Google Cloud. - name: "Create {{ environ.name }}-cluster-backups" google.cloud....
user avatar
2 votes
3 answers
1k views

googleapi: Error 403: 567x@cloudbuild.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object

I stay trying my project based on this: https://github.com/GoogleCloudPlatform/iot-smart-home-cloud My issue is: cloudfunctions.googleapis.com google.cloud.functions.v1.CloudFunctionsService....
user avatar
0 votes
1 answer
221 views

GCP deployment to create storage bucket fails on missing storage.buckets.get access

I want to create a cloud storage bucket programatically using deployment manager, but the deployment fails with the following error: ERROR: (gcloud.deployment-manager.deployments.create) Error in ...
user avatar
0 votes
1 answer
53 views

Bucket retention policies and malicious actors

I am thinking about how one would implement a good system for guaranteeing that some files are available for a certain set of time. In this particular case it is a backup catalog of some 150 MiB per ...
user avatar
-1 votes
1 answer
58 views

Lost RDP session on gcp VM(windows server 2016) afer changing firewall settings

I lost my RDP session after setting up some firewall settings on my GCP windows VM and now that is unreachable via IP or RDP or URL. I tried pinging that IP but request timed out. Can someone please ...
user avatar
0 votes
0 answers
19 views

Google nearline, coldline, what is the media use to store data

We want to know if we are subscribing to the Nearline or Coldline offering, what type of media is use to store the data? Hard Disk 7200rpm Tape - LTO7 or 8 Blu-Ray disk
user avatar
  • 3
0 votes
0 answers
325 views

Google HTTP Load Balancer and Google Cloud Storage: Apparently random 403 in PUT requests

I have Google Cloud Storage configured with a CNAME according to the documentation, so basically I have my Google Load Balancer handling the requests. In almost 2 years with this configuration I didn'...
user avatar
  • 101
0 votes
1 answer
307 views

GCP storage refusing me access to a bucket on Cloud Storage even though I apparently have the necessary permissions

A colleague of mine has created a google cloud platform project for which we would like to use the Cloud Storage product. They have made me co-owner and storage admin in the IAM permissions. According ...
user avatar
  • 113
0 votes
1 answer
114 views

Google Cloud Monitoring storage dashboard not showing Object Count or Object size for bucket

I'm trying to see Object Count and Object Size in the Cloud Monitoring dashboard for Cloud Storage. For some buckets, the Object Count and Object size data are not populating. All I'm seeing is "...
user avatar
2 votes
1 answer
134 views

Restrict Viewer Access to Cloud Storage Objects via ACL

I'm hosting a static site in a Cloud Storage Bucket with fine-grained permissions set. I'm attempting to set by-object ACL permissions so users can only access specific resources if they have a ...
user avatar
-1 votes
2 answers
260 views

Allow multiple service accounts to access multiple storage buckets

I have some devices, and each will be handled to the customers. I need each device to have read-access to some Google Cloud storage buckets. I would like each of device (or at least each customer) to ...
user avatar
  • 99
-3 votes
1 answer
47 views

Google Cloud Storage free 50GB and pricing [closed]

I am using Google Cloud Storage for my node.js. I wonder after using the 50GB of free storage, the price will be *0.20$/gb? or starting from 0.2/gb? Thank you
user avatar
0 votes
1 answer
117 views

transferring/downloading files to bucket within a google cloud bucket causes disconnection

I am trying to download a file using wget into my GCP bucket, and just after the download is finished the following error shows: Transport endpoint is not connected Then the file is empty. Using scp ...
user avatar
0 votes
1 answer
161 views

GCSFuse v0.33.2 - Large objects (11GB) upload to GCSBucket Fails

Receiving below errors when we try to upload 11 GB file to bucket; also, filesystem is going into "????????" as we can't list the objects available in the bucket. This requires umount and ...
user avatar
1 vote
1 answer
325 views

export gcsfuse mount so a third client can access files in a storage bucket

I mount a storage bucket to a local directory. /share Then I try to make this directory - now populated with contents of the object store - available to another machine. The goal is, that this other ...
user avatar
  • 35
0 votes
1 answer
142 views

How do I go about VMs in the cloud I only need for specific times?

I am currently doing processing of videos on a local server which I'd like outsource to the Cloud. I am quite familiar with the Google Cloud which is why I would choose Google. The process would ...
user avatar
  • 103
0 votes
1 answer
367 views

Cloud bucket malware security

I have implemented an antivirus system using ClamAV on one of my apps which uses Google cloud storage for uploading files. Currently what I am doing is, listening to bucket upload, download it on one ...
user avatar
0 votes
2 answers
519 views

Find out which GCP project is hosting which appengine or which GCS bucket

We have many projects inside our Organization GCP account. We have many legacy GCS buckets and many legacy AppEngines hosted inside our various projects. All these buckets and AppEngines are served ...
user avatar
  • 121
0 votes
1 answer
147 views

change storage class a folder gcp

change storage class in a folder into a bucket in GCP, I try with gsutil rewrite -s coldline gs://bucket-name/folder/ I see the gsutil stat gs://bucket-name/folder/ what am I doing wrong?
user avatar
0 votes
1 answer
153 views

Google cloud storage favicon for PDF File

I have some pdf files hosted on Google Cloud Storage. These files are public and I open them using their public link. Is there a way I can customize the favicon shown by the browser when I view those ...
user avatar
0 votes
1 answer
8k views

Access denied (SA doesn't have storage.objects.create access) when trying to upload using a preSigned url to google cloud storage

Having issues trying to allow a client to upload a file via a presigned url. Error received <?xml version='1.0' encoding='UTF-8'?> <Error> <Code>AccessDenied</Code> <Message&...
user avatar
  • 1
0 votes
0 answers
219 views

Upload 300MB or more CSV file to GCP storage bucket

I am trying to upload a 3GB CSV file to Google Cloud Storage Bucket but every time I try it only processing 295.5MB and uploads 295.5MB. Why is this? I have set my bucket size limit to 15GB.
user avatar
1 vote
2 answers
269 views

moving 2TB files from on-premises to Google Cloud Storage

Our old infrastructure was hosted on an on-premises server, and now we are moving forward into Google Cloud Storage. The problem is, my outgoing server connection is pretty slow, any ideas about ...
user avatar
1 vote
3 answers
3k views

Recover deleted folder in Google Cloud Storage

I have accidentally deleted one folder from the Google Cloud Storage. How can I recover this folder?
user avatar
  • 11
0 votes
1 answer
129 views

I'm on GCS's free trial; how do I increase my storage from 5gb to 25 gb?

So I'm playing around in a notebook that lets me experiment with the T5 model. When I try to fine-tune the model (it has 3B parameters), I get: WARNING:tensorflow:The `3B` model is too large to use ...
user avatar
  • 3
0 votes
1 answer
82 views

Google Cloud load balancer redirect if from a certain region

I have two static pages (English, Danish) which are currently in Google Cloud Storage Bucket. This bucket contains en and dk folders for those static pages. Also, I have a Load Balancer with Google ...
user avatar
0 votes
1 answer
310 views

possible to set up GCS bucket lifecycle rule to archive rarely used objects?

GCS offers lifecycle management where you can, for example, "Downgrade the storage class of objects older than 365 days to Coldline Storage". I'd like to archive rarely used objects but keep ...
user avatar
2 votes
1 answer
1k views

Cloud Run service account does not have permission to sign

I'm working on a program that will run on Google Cloud Run and has files stored in Google Cloud storage. The problem I'm experiencing happens when attempting to generate a signed URL to download a ...
user avatar
  • 171
1 vote
2 answers
699 views

Prevent Cloud CDN from serving cached elements?

I have some problem with Cloud CDN. I've added a CDN to a bucket connected on an instance. I need to get an element from CDN, but this element is updated each 1 sec from my server. What happen is ...
user avatar
  • 201
2 votes
2 answers
12k views

How to get a download URL for files in Google Cloud Storage?

I have created a service using : storage.Client(project='project_name', credentials=credentials) https://googleapis.dev/python/storage/latest/client.html And I'm able to upload, delete and list the ...
user avatar
0 votes
1 answer
22 views

Search pdf content in google storage

could you help me with the following? How can I search pdf words hosted on google cloud storage? example: pdf1, pdf2, pdf3 hosted on storage. Look for the "father" word in all documents. Thanks
user avatar
  • 1
0 votes
1 answer
431 views

How to Change Cloud API access scopes on GCP Compute Engine with out stopping the Instance

I'm using gcsfuse inside Google Compute Engine. I mounted a folder with cloud storage bucket but, unable to create any file inside of it: touch t1 with an error: touch: cannot touch 't1': Input/...
user avatar
1 vote
1 answer
808 views

Using Google Cloud Armor to Block Requests to Google Cloud Storage

Does Google Cloud Armor support blocking requests to public Google Cloud Storage buckets? I haven't been able to determine if GC Storage counts as a backend service. The scenario is I would like to ...
user avatar
  • 429
0 votes
1 answer
71 views

GCS GoogleCredential

My application has requirement to download file from GCS bucket.I am able to download files from GCS using StorageClient DownloadObjects method. But performace is really slow. for 50MB file size it ...
user avatar
2 votes
0 answers
304 views

gsutil rsync fails on Ubuntu with UnicodeDecodeError

I'm trying to rsync between google cloud storage and amazon s3. On my local machine (MacOS) this command works gsutil rsync -d -r gs://mira-internal/data/exports/20191108 s3://mira-temp/raw/20191108 ...
user avatar