0

I have a Google Cloud Storage bucket and am experimenting with creating a website in Google Cloud Platform (using Python+Flask FWIW). For my initial experiment, I'm using gcsfuse to mount my GCS bucket as a filesystem:

https://github.com/GoogleCloudPlatform/gcsfuse

Following the directions I got it installed in my cloud shell and mounted it like so:

mkdir -p ~/mnt

gcsfuse <my-bucket> ~/mnt

This works, but gives me a strange (as in wildly inaccurate) view of my bucket. It shows some YYYY-MM directory structure (all empty) and a bunch of files at the top level (though far from all of them), but it doesn't look like what's really in my bucket. I can verify my actual data is still available by visiting my existing GCS-homed static website and wandering through the archived YYYY-MM directories by date and thread, viewing the messages they contain:

http://www.smontanaro.net/

(I know, it's ugly. That's one reason to use something more modern to produce a decent website.)

I am new to gcsfuse and Google Cloud Platform, so it's quite likely I'm doing something wrong. Any suggestions about where to look for clues would be much appreciated.

Edit: I upload files from my laptop using gsutil:

gsutil -m cp -I gs://<my-bucket>/path...

(It's not really big data stuff, less than 10gb. GCS was just an easy place to toss it and make it available through the web.)

Edit: I created a new bucket and tossed in a few files. I can gcsfuse mount it from a Cloud shell (for either my GCP project or GCS project) and see the files. I've been so far unsuccessful trying to mount it from my Linux laptop, getting:

SetUpBucket: OpenBucket: Bad credentials for bucket "<my-bucket>": permission denied

This when specifying a JSON key file I created for my GCP project.

6
  • Please avoid exposing your bucket in public, as it may be access by other community members to try resolve or reproduce your problem and it may cause sudden spike in your API traffic.
    – RJC
    Nov 16, 2021 at 2:27
  • How did you created and populated your bucket? Besides the official documentation, are there any guides you've followed in creating your bucket? Also, can you verify or try to create a new bucket, and mount in the same drive that you mounted your current bucket, and see if you would get the same behavior. Can you try to create a new folder on your existing bucket, and see if that newly created folder would get the same problem?
    – RJC
    Nov 16, 2021 at 5:48
  • Thanks @RJC. I tweaked my examples and detailed how I populate the bucket. I will see about creating a new bucket and mounting it later, once I'm fully caffeinated, then report back.
    – smontanaro
    Nov 16, 2021 at 13:09
  • Hello @smontanaro, -I uses stdin to specify a list of files or objects to copy. Could you try rsync to copy files to your bucket instead? In addition, the new error appears like a different problem. I suggest posting a new question about that one to isolate it.
    – RJC
    Nov 18, 2021 at 8:26
  • Thanks, yes, I understand what -I does. My file upload process finds a list of files to upload and feeds them into gsutil which expects the list of files on stdin. I think at this point I am down to (mostly) bucket/object permission issues. I will create a new question.
    – smontanaro
    Nov 18, 2021 at 18:56

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.