I've noticed that connecting to the internet from the allocated compute node via Slurm-GCP keeps failing. For example, using wget
from the login node works successfully:
[me@gcp-login0 ~]$ wget https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.2.tar.xz
--2023-05-11 19:06:34-- https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.2.tar.xz
Resolving cdn.kernel.org (cdn.kernel.org)... 111.111.1.111, 111.111.11.111, 111.111.111.111, ...
Connecting to cdn.kernel.org (cdn.kernel.org)|111.111.1.111|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 102167060 (97M) [application/x-xz]
Saving to: ‘linux-4.17.2.tar.xz’
100%[======================================>] 102,167,060 277MB/s in 0.4s
2023-05-11 19:06:35 (277 MB/s) - ‘linux-4.17.2.tar.xz’ saved [102167060/102167060]
But on a single allocated GPU, wget
stalls and fails:
[me@gcp-compute-0-0 ~]$ wget https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.2.tar.xz
Resolving cdn.kernel.org (cdn.kernel.org)... 111.111.1.111, 111.111.11.111, 111.111.111.111, ...
Connecting to cdn.kernel.org (cdn.kernel.org)|151.101.1.176|:443...
failed: Connection timed out.
Accordingly, I was wondering if there's a way to solve this network issue for compute nodes on Slurm-GCP? I tried modifying firewall settings and VPC network details, but it seems to only affect the login node and unable to target the compute node settings on GCP.