2

I am using the library google-auth~=2.28.1 to enable "Login with Google" in my webapp. The login works on my local computer and on the test-server (hetzner), but on the prod-server (also hetzner) it recently stopped working.

This is the part of my python webapp that throws the exception:

id_info = id_token.verify_oauth2_token(
        credential, requests.Request(), auth_config.google_client_id
)

This is the error:

google.auth.exceptions.TransportError: Could not fetch certificates at https://www.googleapis.com/oauth2/v1/certs

The webapp is running in a docker container that extends python:3.10.

After I read read this ServerFault Post I tried curl with both IPv4 and IPv6 both from the server and from within the Docker container. Opposite from the SF post, IPv6 works while IPV4 doesn't.

curl -4 'https://www.googleapis.com/oauth2/v1/certs'

Truncated Output:

<p><b>403.</b> <ins>That’s an error.</ins>
<p>Your client does not have permission to get URL <code>/oauth2/v1/certs</code> from this server.  <ins>That’s all we know.</ins>
curl -6 'https://www.googleapis.com/oauth2/v1/certs'

Truncated Output:

{"6f9777a685907798ef794062c00b65d66c240b1b": "-----BEGIN CERT....}

I had this issue in the past and it resolved itself after a few days, but I would like a more permanent solution.

2
  • open a ticket on the vendor?
    – djdomi
    Mar 11 at 18:10
  • The vendor replied and it seems like the IP was falsify flagged as Iranian which means that is was under sanctions. Swapped the IP and the issue is resolved Mar 12 at 9:45

2 Answers 2

3

Hetzner resolved the issue. It turned out that the IP of the server was falsify flagged as Iranian. (The server itself was located in Germany) Iran is currently under sanctions and blocked by Google.

Switching the primary IP resolved the issue.

3
  • nice to hear. please remember to accept (after 24h) your answer, so we will not be reminded to solve this issue until the end of the universe ;)
    – djdomi
    Mar 12 at 10:34
  • did you get any feedback from hetzner on this topic? Mar 19 at 14:40
  • Yes, we contacted their support and they informed us that our IP was falsely flagged. Unfortunately there is no easy way to swap the IP. You need to create a backup of your service and recreate it, or you need to assign a floating IP and use that to communicate to the Google API. Mar 20 at 15:49
0

We are still running into this issue.

Our bitbucket pipeline runners are hosted in a kubernetes on hetzner cloud and they dynamically scale up new nodes on demand. Sometimes it happens that builds running on those nodes get HTTP 403 when trying to contact google servers (for downloading Android SDK or Docker images from GCR)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .