Questions tagged [google-cloud-storage]
Google Cloud Storage, also called GCS, is a cloud service for storing and accessing arbitrarily large objects.
162
questions
0
votes
0
answers
78
views
Any multi-cloud storage service that can connect to both GCP and Azure Storage?
MinIO Gateway used to provide S3 functionality where you could connect to both GCP and Azure to:
Create/delete buckets in GCP/Blob containers in Azure
Upload/download/delete files/folders
Create pre-...
- 43
0
votes
0
answers
62
views
howto edit provisioned IOPS of an Extreme Persistent Disk
I'm looking how-to change/edit the provisioned IOPS setting of an Extreme Persistent disk. I'm not able to find any suitable API to change this settings and also seems not possible from the GUI.
I'm ...
1
vote
1
answer
106
views
Transfer 200GB from client using Google Cloud
I have a client who wishes to transfer 200GB of sensitive data to us. I would like them to upload this data to a GCloud bucket.
What is the best way to set up an external user to have access to a ...
1
vote
1
answer
161
views
What's the cheapest way to find the name of the last (sorted alphabetically) "directory" in a Google Cloud Storage bucket?
At the moment I'm doing:
gsutil ls -l gs://myproject | sort | tail -n 1
It takes about 10-15 seconds, which is fine, but I worry what the cost of this operation is? If it affects the cost, there's ...
0
votes
1
answer
42
views
How to preview uploaded non-public website?
How do I preview a website I uploaded to Google Cloud? I do not want it to be publicly accessible yet and want to see first how it looks and loads. I created the CNAME record, and bucket, enabled the ...
- 1
0
votes
0
answers
11
views
GCP | Any documentation around automating and scheduling the data transformation pipelines?
I am curious to understand if there is any documentation around automating and scheduling the data transformation pipelines using GCP+BigQuery+JupyterLab.
For instance, if there are 6 BigQuery tables ...
4
votes
1
answer
105
views
shutting down gcloud service in case of DDos attacks
my company has a small and nice product on google could (using public webapp, functions and storage).
it's been hours I'm searching for a solution like a kill switch for services in case of any threat ...
0
votes
0
answers
90
views
Export VMDK Disk Files in Google Compute Platform - networks/default' was not found
I have a problem to export the VMDK files to my Cloud Storage in buckets
The command:
gcloud compute images export --image=my_vm_disk --destination-uri=gs://my_bucket/my_vm_disk.vmdk --export-format=...
0
votes
0
answers
34
views
Is it possible to determine individual data centers that store my data?
I'm using Google Cloud Platform Multi-Region storage to store data for a hobby project. I wanted to have it easily backed up without much fuss, so I selected this. My data is stored in the United ...
0
votes
1
answer
623
views
GCP Function Deployment : Zip file from local build but not Zip File from Github Action
I am trying to have a github action that build a zip file with code function source code and upload it to Google Cloud Storage
Then I have a terraform repository that deploy cloud function using this ...
0
votes
0
answers
26
views
Storageclient.ListObjects causes thread cancel error when deployed to server
I have some code that fetches files from Google Cloud Storage. The code works fine when run on my local development machine but when deployed to our production server it always "stops" ...
0
votes
1
answer
195
views
Couldn't read source object ACLs. Source bucket must not have storage.uniformBucketLevelAccess enabled and the service account must have storage.ob
Attempting to manually test creating an exact backup for a GCS bucket via Data Transfer Service in the GCP browser UI and getting error...
Couldn't read source object ACLs. Source bucket must not ...
2
votes
1
answer
980
views
service account with Storage Admin role does not have storage.buckets.get access
I'm trying to get a bucket on gcp from a pod on kubernetes using python.
Now the service account on that pod has a role of storage admin but for some reason I still get an exception when trying to ...
0
votes
1
answer
338
views
I can not run any query on BigQuery in region europe-west2 anymore (transient error n°4233314)
I can no longer query nor preview any table nor run any job in my BigQuery (region europe-west2). When I run a query in an other region, I don't get the error so it's specific to my region.
When ...
0
votes
1
answer
73
views
Compute Engine: Restricting SSH usernames
I want to use OS Login with GCP because we use IAM for scoping access to all other resources within GCP (storage buckets, SQL, Redis, etc.). I understand how to restrict users from accessing machines ...
0
votes
0
answers
239
views
Error on specific files when backing up with Iperius Backup
I'm having some trouble with my GCS lately, I have Iperius Backup and GCS configured as detination for my files, it runs everyday and those 3 files always fail with the same error:
E:\Sistema\Phoenix\...
1
vote
2
answers
3k
views
Download from URL into GCP bucket
Given a URL that points to a large file, is there a shortcut to getting that file into a bucket, without downloading it locally and then uploading?
Is there a simpler way to do it, other than building ...
0
votes
1
answer
755
views
GCS - Internet Egress Costs
Does anyone know where can I find the pricing for Google Cloud Storage pricing for Internet Traffic?
Doing some design work and I need to work out what the cost of multiple GCS Buckets serving content ...
0
votes
0
answers
23
views
Files in Bucket no longer appear
I'm new to Google Cloud Platform, and while following the directions to install an SSL on a project, it appears the bucket for the project was wiped. No more files are showing up.
One, can anyone ...
1
vote
1
answer
1k
views
How to trigger an event after saving a chosen csv file in Google Cloud Storage bucket
Trying to make a synchronous pipeline, I need to copy a csv file from Google Cloud Storage after it has been saved in Google Cloud Storage. The copy job does not have to be triggered right after the ...
0
votes
1
answer
1k
views
Reading only the metadata of a file in a Google Cloud Storage bucket into a Cloud Function in Python (without loading the file or its data!)
I need something like Cloud Storage for Firebase: download metadata of all files, just not in Angular but in Python and just for a chosen file instead.
The aim is to return this information when the ...
- 343
1
vote
1
answer
1k
views
Unable to access GCS Object with storage.objects.get
I have a bucket with uniform permissions (no object level ACLs) and my account has the Owner role on the project which should give full access to all resources. I have even tried adding Storage Admin /...
0
votes
0
answers
132
views
GKE autoscaler sometimes doesn't scale pods
We have a deployment configured with HPA based on the CPU metric. It can work fine for days, scaling pods up and down. And then at some point looks that it ignores metric and scales to some small ...
0
votes
0
answers
594
views
Bucket mounted using gcsfuse doesn't match what's in my cloud storage bucket
I have a Google Cloud Storage bucket and am experimenting with creating a website in Google Cloud Platform (using Python+Flask FWIW). For my initial experiment, I'm using gcsfuse to mount my GCS ...
0
votes
1
answer
391
views
How to sync GCP Cloud Storage Bucket metadata to a database?
I have a large number of objects, currently around 1 million, stored in a GCP Cloud Storage Bucket. Objects are added at a rate of 1-2 thousand per day. I would like to efficiently run queries to ...
0
votes
0
answers
127
views
Google cloud function - Unable to deploy
I have lots of cloud function deployed and working fine. (I am the project owner)
Now I am not able to deploy any function and keep on getting this error -
ERROR: gcloud crashed (ConnectionError): ('...
1
vote
1
answer
795
views
Service account does not have storage.buckets.get access to the Google Cloud Storage bucket
I am running Ansible with a service the following configuration which should create a storage bucket on Google Cloud.
- name: "Create {{ environ.name }}-cluster-backups"
google.cloud....
3
votes
3
answers
3k
views
googleapi: Error 403: 567x@cloudbuild.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object
I stay trying my project based on this: https://github.com/GoogleCloudPlatform/iot-smart-home-cloud
My issue is:
cloudfunctions.googleapis.com
google.cloud.functions.v1.CloudFunctionsService....
0
votes
1
answer
434
views
GCP deployment to create storage bucket fails on missing storage.buckets.get access
I want to create a cloud storage bucket programatically using deployment manager, but the deployment fails with the following error:
ERROR: (gcloud.deployment-manager.deployments.create) Error in ...
0
votes
1
answer
58
views
Bucket retention policies and malicious actors
I am thinking about how one would implement a good system for guaranteeing that some files are available for a certain set of time. In this particular case it is a backup catalog of some 150 MiB per ...
-1
votes
1
answer
229
views
Lost RDP session on gcp VM(windows server 2016) afer changing firewall settings
I lost my RDP session after setting up some firewall settings on my GCP windows VM and now that is unreachable via IP or RDP or URL. I tried pinging that IP but request timed out.
Can someone please ...
0
votes
0
answers
674
views
Google HTTP Load Balancer and Google Cloud Storage: Apparently random 403 in PUT requests
I have Google Cloud Storage configured with a CNAME according to the documentation, so basically I have my Google Load Balancer handling the requests.
In almost 2 years with this configuration I didn'...
0
votes
1
answer
933
views
GCP storage refusing me access to a bucket on Cloud Storage even though I apparently have the necessary permissions
A colleague of mine has created a google cloud platform project for which we would like to use the Cloud Storage product. They have made me co-owner and storage admin in the IAM permissions. According ...
1
vote
1
answer
536
views
Google Cloud Monitoring storage dashboard not showing Object Count or Object size for bucket
I'm trying to see Object Count and Object Size in the Cloud Monitoring dashboard for Cloud Storage. For some buckets, the Object Count and Object size data are not populating. All I'm seeing is "...
2
votes
1
answer
404
views
Restrict Viewer Access to Cloud Storage Objects via ACL
I'm hosting a static site in a Cloud Storage Bucket with fine-grained permissions set. I'm attempting to set by-object ACL permissions so users can only access specific resources if they have a ...
-1
votes
2
answers
520
views
Allow multiple service accounts to access multiple storage buckets
I have some devices, and each will be handled to the customers. I need each device to have read-access to some Google Cloud storage buckets. I would like each of device (or at least each customer) to ...
-3
votes
1
answer
58
views
Google Cloud Storage free 50GB and pricing [closed]
I am using Google Cloud Storage for my node.js.
I wonder after using the 50GB of free storage, the price will be *0.20$/gb? or starting from 0.2/gb?
Thank you
0
votes
1
answer
372
views
transferring/downloading files to bucket within a google cloud bucket causes disconnection
I am trying to download a file using wget into my GCP bucket, and just after the download is finished the following error shows: Transport endpoint is not connected
Then the file is empty.
Using scp ...
0
votes
1
answer
283
views
GCSFuse v0.33.2 - Large objects (11GB) upload to GCSBucket Fails
Receiving below errors when we try to upload 11 GB file to bucket; also, filesystem is going into "????????" as we can't list the objects available in the bucket.
This requires umount and ...
1
vote
1
answer
597
views
export gcsfuse mount so a third client can access files in a storage bucket
I mount a storage bucket to a local directory. /share
Then I try to make this directory - now populated with contents of the object store - available to another machine.
The goal is, that this other ...
0
votes
1
answer
145
views
How do I go about VMs in the cloud I only need for specific times?
I am currently doing processing of videos on a local server which I'd like outsource to the Cloud.
I am quite familiar with the Google Cloud which is why I would choose Google.
The process would ...
0
votes
1
answer
473
views
Cloud bucket malware security
I have implemented an antivirus system using ClamAV on one of my apps which uses Google cloud storage for uploading files. Currently what I am doing is, listening to bucket upload, download it on one ...
0
votes
2
answers
640
views
Find out which GCP project is hosting which appengine or which GCS bucket
We have many projects inside our Organization GCP account.
We have many legacy GCS buckets and many legacy AppEngines hosted inside our various projects.
All these buckets and AppEngines are served ...
0
votes
1
answer
288
views
change storage class a folder gcp
change storage class in a folder into a bucket in GCP, I try with
gsutil rewrite -s coldline gs://bucket-name/folder/
I see the
gsutil stat gs://bucket-name/folder/
what am I doing wrong?
0
votes
1
answer
203
views
Google cloud storage favicon for PDF File
I have some pdf files hosted on Google Cloud Storage.
These files are public and I open them using their public link.
Is there a way I can customize the favicon shown by the browser when I view those ...
0
votes
1
answer
17k
views
Access denied (SA doesn't have storage.objects.create access) when trying to upload using a preSigned url to google cloud storage
Having issues trying to allow a client to upload a file via a presigned url.
Error received
<?xml version='1.0' encoding='UTF-8'?>
<Error>
<Code>AccessDenied</Code>
<Message&...
0
votes
0
answers
434
views
Upload 300MB or more CSV file to GCP storage bucket
I am trying to upload a 3GB CSV file to Google Cloud Storage Bucket but every time I try it only processing 295.5MB and uploads 295.5MB. Why is this? I have set my bucket size limit to 15GB.
1
vote
2
answers
594
views
moving 2TB files from on-premises to Google Cloud Storage
Our old infrastructure was hosted on an on-premises server, and now we are moving forward into Google Cloud Storage. The problem is, my outgoing server connection is pretty slow, any ideas about ...
1
vote
3
answers
7k
views
Recover deleted folder in Google Cloud Storage
I have accidentally deleted one folder from the Google Cloud Storage.
How can I recover this folder?
0
votes
1
answer
177
views
I'm on GCS's free trial; how do I increase my storage from 5gb to 25 gb?
So I'm playing around in a notebook that lets me experiment with the T5 model. When I try to fine-tune the model (it has 3B parameters), I get:
WARNING:tensorflow:The `3B` model is too large to use ...
Related Tags
google-cloud-platform × 121
gcsfuse × 17
gsutil × 11
google-iam × 8
gcloud × 5
fuse × 5
kubernetes × 5
python × 5
google-cloud-sql × 3
google-cloud-cdn × 3
docker × 3
backup × 3
storage × 3
cloud × 2
deployment × 2
amazon-s3 × 2
docker-registry × 2
infrastructure × 2
mount × 2
oauth × 2
performance × 2
more related tags
Hot Network Questions
- I’m waiting for my US passport (am a dual citizen. My father is ill and booked a flight to see him - can I travel on my other passport?
- Test for inertial frame of reference
- Why is this screw on the wing of DASH-8 Q400 sticking out, is it safe?
- Is it possible for rockets to exist in a world that is only in the early stages of developing jet aircraft?
- Is Spider-Man the only Marvel character that has been represented as multiple non-human characters?
- Modify lithium battery pack balancer
- Movie in which a group of friends are driven to an abandoned warehouse full of vampires
- Does substituting electrons with muons change the atomic shell configuration?
- How does TeX know whether to eat this space if its catcode is about to change?
- Is there any evidence suggesting or refuting that Russian officials knowingly lied that Russia was not going to attack Ukraine?
- How common is it to take off from a taxiway?
- How to make a HUE colour node with cycling colours
- How appropriate is it to post a tweet saying that I am looking for postdoc positions?
- Is it OK to pray any five decades of the Rosary or do they have to be in the specific set of mysteries?
- Hydrogen Isotopes and Bronsted Lowry Acid
- Why is it "Gaudeamus igitur, *iuvenes dum* sumus!" rather than "Gaudeamus igitur, *dum iuvenes* sumus!"?
- To attain moksha, must you be born as a Hindu?
- Making three fair dice
- Ways to find a safe route on flooded roads
- Insufficient travel insurance to cover the massive medical expenses for a visitor to US?
- Noise cancels but variance sums - contradiction?
- Is there any philosophical theory behind the concept of object in computer science?
- How can I divide the contour in three parts with the same arclength?
- Complete the landscape