s3fs does not parse NO_PROXY / no_proxy env variables

Question

I am running s3fs on a node in corporate network which is behind proxy. I am trying to mount S3 bucket on another node next to it.

Both nodes are in 10.88.97.65/27 network and should be able to communicate directly.

On both nodes I have set following env variables:

HTTP_PROXY=10.88.96.153:3128
http_proxy=10.88.96.153:3128
NO_PROXY=localhost,127.0.0.1,192.168.0.0/16,10.0.0.0/8
no_proxy=localhost,127.0.0.1,192.168.0.0/16,10.0.0.0/8

s3fs version: 1.79

When I perform mount with:

s3fs mybucket /my/mount-point -o url=http://my-minio-server:9000 -o passwd_file=/mys3cred -o use_path_request_style

I get following message in /var/log/user.log: debug log

403 response in logs is from my HTTP proxy, whereas it should not go through proxy for that node at all.

Does somebody have solution for it or it is kind of bug to be fixed in the code?

Answer 1

s3fs honors the http_proxy and no_proxy environment variables via libcurl. Note that these only apply to HTTP URLs and not HTTPS and you might need to include the -o use_path_request_style flag or fully specify the hostname, e.g., no_proxy=bucketname.s3.amazonaws.com.