0

I have three networks on Rackspace:

  1. Public network
  2. Service network 10.181.XXX.XXX (they call it a private network sometimes but it is not private per-se, it is datacenter-wide private so their tenants may share it)
  3. A real private network of 192.168.3.0/24 created via the networking UI

My plan is to launch a secure CoreOS cluster whose etcd peers and clients are limited to the real private networking so it is not available for the outside world.

So, I launched a CoreOS server and it had three interfaces:

  1. eth0 has the IP address of the public network - good
  2. eth1 has the IP address of the service network - good
  3. eth2 has the IP address of the private network - great!

All looked well but etcd is bound to the service network instead of the private one I created manually. Here's a piece of cloud-config:

#cloud-config
    coreos:
      etcd2:
        discovery: https://discovery.etcd.io/XXX
        advertise-client-urls: http://$private_ipv4:2379,http://$private_ipv4:4001
        initial-advertise-peer-urls: http://$private_ipv4:2380
        listen-client-urls: http://$private_ipv4:2379,http://$private_ipv4:4001
        listen-peer-urls: http://$private_ipv4:2380

Which means that $private_ipv4 variable gets expanded to eth1 IP address, here's the contents of /etc/environment:

COREOS_PUBLIC_IPV4=166.78.XXX.XXX
COREOS_PRIVATE_IPV4=10.181.XXX.XXX

Ok, looks like Rackspace injects its own networks, it is explainable. But does this mean that etcd is locked only to the first two networks and there's no way to configure it to use the real private one?

I've tried a bunch of hacks such as this:

listen-peer-urls: http://`/usr/bin/ifconfig eth2 | /usr/bin/grep --word-regexp inet | /usr/bin/awk '{print $2}'`:2380

and others but neither got properly executed nor substituted in the cloud-config.

My questions are:

  1. Am I doing something weird in terms on CoreOS cluster setup field, so that I can not achieve the target easily and naturally? If so, how would you lay out a secure cluster whose etcd operates within a real private network?
  2. Are there ways to "inject" dynamic values into cloud-init file so it gets interpolated at runtime? The issue is that the IP address is not known beforehand so it needs to get gotten and injected somehow I think.
3
  • I read this: rackspace.com/knowledge_center/frequently-asked-question/…. According to it - ServiceNet is an internal only, multi-tenant network connection within each Rackspace data center. Am I wrong? I wish I am! :)
    – Barbadoss
    Dec 28, 2015 at 19:41
  • Ok, thank you @Michael Hampton, I've submitted a support ticket to Rackspace with this particular question - whether ServiceNet is truly private or not. I'll post what they reply with here.
    – Barbadoss
    Dec 28, 2015 at 21:48
  • Rackspace support confirmed that the Service Nets are datacenter wide which means that anyone in the datacenter can run nmap against another VM's 10.x.x.x address within the datacenter.
    – Barbadoss
    Dec 29, 2015 at 5:58

2 Answers 2

4

They also provide a way to create your own private network and attach your server to it. As explained KB Article ID: 2163. If you did that and had the servicenet connected interface removed, it would no longer be an issue. However, the caveat to this would be that you would no longer have a way to utilize servicenet related services. More on intricacies about that is their KB Article ID: 2250.

Or, you could configure your firewall so that only the services that you use/need servicenet for are able to reach your server. You could even venture into the world of orchestrating a true private network through a cloud vyatta appliance and still retain access to Rackspace services as explained in KB Article ID: 3454.

All of those articles can be found at: http://www.rackspace.com/knowledge_center/

Ultimately, you'd have to weigh what option works best for you.

HTH

2

It is solved. Etcd2 is now operating on the real private network versus ServiceNet. Here's how - I created a systemd drop-in for etcd2.service where it explicitly sets environment configuration such as ETCD_ADVERTISE_CLIENT_URLS=${URL}:2379 instead of relying on variables such as $private_ipv4 and $public_ipv4.

I've written a quick tool for this job - implementation, samples and documentation can be found here. Drop a line if you have any questions or suggestions!

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .