1

I am running Linux on a t1.micro instance at Amazon EC2. Once I noticed bruteforce ssh login attemtps from a certain IP, after litle Googling I issued the two following commands (other ip):

iptables -A INPUT -s 202.54.20.22 -j DROP
iptables -A OUTPUT -d 202.54.20.22 -j DROP

Either this, or maybe some other actions like yum upgrade perhaps, caused the follwing fiasco: after rebooting the server, it came up without the Network Interface!

I only can connect to it through AWS Management Console JAVA ssh client - via local 10.x.x.x address.

Console's Attach Network Interface as well as Detach.. are greyed out for this instance.

Network Interfaces item at the left does not offer any Subnets to choose from, to create a new N.I.

Please advice, how can I recreate a Network Interface for the instance?

Upd. The instance is not accessible from outside: cannot be pinged, SSH'ed or connected by HTTP on port 80.

Here's the ifconfig output:

eth0  Link encap:Ethernet  HWaddr 12:31:39:0A:5E:06  
      inet addr:10.211.93.240  Bcast:10.211.93.255  Mask:255.255.255.0
      inet6 addr: fe80::1031:39ff:fe0a:5e06/64 Scope:Link
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:1426 errors:0 dropped:0 overruns:0 frame:0
      TX packets:1371 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000 
      RX bytes:152085 (148.5 KiB)  TX bytes:208852 (203.9 KiB)
      Interrupt:25 
lo    Link encap:Local Loopback  
      inet addr:127.0.0.1  Mask:255.0.0.0
      inet6 addr: ::1/128 Scope:Host
      UP LOOPBACK RUNNING  MTU:16436  Metric:1
      RX packets:0 errors:0 dropped:0 overruns:0 frame:0
      TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:0 
      RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

What is also unusual: a new micro instance I created from scratch, with no relation to the troubled one, was not pingable too.

7
  • 2
    a) If you can connect via 10.x.x.x then the instance has a network interface. b) EC2 instances always have a private IP associated with their interface - the public IP is NAT'ed to the private one c) You can only attach/detach an interface to a VPC instance - if this is a VPC instance, please mention that. d) Post the output of ifconfig. d) Why do you believe the instance doesn't have an interface? (a lack of connectivity doesn't mean the interface doesn't exist). e) It is possible that your public IP simply has changed - check (and post, if necessary) the output of ec2-describe-instance.
    – cyberx86
    Apr 10, 2012 at 14:52
  • By default, the EC2 security group drops all ping (ICMP) packets - so unless you specifically enable it, ping will not work. To enable ping, you need to create a 'Custom ICMP Rule' for your security group (both Echo Request and Echo Reply). The ifconfig output looks normal - you aren't missing a network interface. Your problem likely lies either with a misconfigured security group (one that is blocking everything) or with iptables. Check (and post, if necessary) the output of iptables -nvL and ec2-describe-group SECURITY_GROUP.
    – cyberx86
    Apr 10, 2012 at 15:41
  • Thank You, @cyberx86 for poining out these helpful hints. a) you were right b) got that c) not a VPC d) didn't know about AWS' icmp filtering - expected pings. e) it did change after stop/start - but I noticed the changes from AWS Console. Afterall, it seems that yum upgrade has overwritten some of my nginx configs which led to http service shutdown. Then there were no pings, and blank "Network Interface" in the AWS Console.
    – Sergiks
    Apr 10, 2012 at 15:53
  • The new installs of nginx (after v1.0, I think) add /etc/nginx/conf.d/default.conf - it is a minor change but breaks quite a few setups. (If that is the case with you, I'd suggest adding an empty default.conf so that it doesn't get recreated in the future). You seem to have explained everything except why you couldn't access SSH from the outside - was that just a result of the public IP changing?. (As you probably have guessed, the 'Network Interface' entry is always blank for EC2 instances, unless they are VPC)
    – cyberx86
    Apr 10, 2012 at 16:00
  • Right guess with the default.conf. The SSH went away due to the new ip after instance stop/start. Didn't notice that immediately, and followed the wrong path further.
    – Sergiks
    Apr 10, 2012 at 16:03

2 Answers 2

3

EC2 instances come with a single interface - eth0. This interface is mapped to your instance's private IP address and cannot be detached from the instance. (Of course, you can take the interface down, but that a function of the operating system, not AWS).

The only case in which you can attach/detach network interfaces is if your instance is running in a Virtual Private Cloud (VPC). Such instances support multiple interfaces.

Since the instance only has a single network interface, the fact that you are able to connect to it - even if it is only via the AWS console - suggests that the interface still exists and is operational (the 'Network Interface' entry in the AWS console entry is only filled in for VPC instances). It is worth noting that the public IP address of an instance will change when an instance is stopped and started (although, a 'reboot' should not affect this). Elastic IPs can be used to provide a 'static' IP address. The public IP address (whether dynamic or 'elastic') is not directly associated with the interface, rather the EC2 network will NAT the public address to the private one.

Since you still have SSH access to the instance, you can confirm the interface is operational by looking at for an eth0 entry in the output of ifconfig. You can also use the ec2-describe-instances command to get the private and public ip addresses of your instance(s).

EC2 instances typically have two causes of blocked packets - either the operating system's firewall (iptables/netfilter in this case) or the EC2 security group. To the extent possible, it is always preferable to block traffic with your security group as it prevents data from ever reaching the instance. Security groups however, are stateless, and most packages are not setup to dynamically modify them, so you will likely use iptables as well.

By default, the EC2 security group drops all ICMP packets (needed for ping) - so unless you specifically enable it, ping will not work. To enable ping from the AWS console (for your security group):

  • Create a 'Custom ICMP Rule' for your security group
  • Type: Echo Request and Type: Echo Reply (both are required)
  • Source: 0.0.0.0/0

You can see your currently configured iptables rules with: iptables -nvL and you can view your security group settings with ec2-describe-group SECURITY_GROUP

Brute force attacks and having your server scanned are, unfortunately, part of what it entails to have a public facing server today. That attacks occur is typically not something to be concerned with, but rather how your server is setup to ensure those attacks do not result in breaches. Usually, manually blocking a single IP address isn't the most effective route, since you will normally find quite an assortment of IPs as the originators of attacks against your server. I would recommend looking into fail2ban if you want a solution based on failed logins (it scans your logs and dynamically adds/removes the necessary firewall rules) or a iptables rule set based on the recent module.

Additionally, it is usually good practice, when working with iptables, to setup a cron job that will flush your iptables rules (iptables -F) after a few minutes, so that you do not accidentally lock yourself out of your server.

In this case, an update to nginx resulted in an invalid configuration, due to the addition of /etc/nginx/conf.d/default.conf. Create an empty file with the same name (instead of deleting the file) to prevent future updates from causing the same problem. While it is probably not something most people do after an update, you can always test your nginx config with service nginx configtest, which will let you know if there are any problems, before you terminate the running nginx process.

If you actually do find yourself with a disable network interface (e.g. because of ifdown eth0), you will not be able to SSH into the instance (or contact it in anyway). The solution for that scenario is to stop the instance, detach the EBS root volume, attach it as an additional volume to a new EC2 instance, fix the problem, reattach the EBS volume to your original instance, and start it back up. It is one of the definite advantages of EBS volumes.

0

Note that when restoring an instance backup, some of the references to the old network interface's MAC address may linger and cause the interface to fail to come up.

try removing /etc/udev/rules.d/70-persistent-net.rules and if that doesn't work try ensuring that the mac address in /etc/sysconfig/network-scripts/ifcfg-eth0 corresponds to the MAC address on the virtual network interface

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .