18

On a centos box, I like to dump tcp connections - I would like to see if a server tries to send requests to a certain IP. Usually tcpdump would do the trick - but tcpdump is not installed, and installing software is not an option (because of company policy). I am afraid netstat will not show me a single request.

So I was wondering what other options I have. I do have root access on the server.

4
  • Do you have root access on the server ?
    – user9517
    Jun 19, 2014 at 12:02
  • Yes, I do have root access.
    – Isaac
    Jun 19, 2014 at 12:04
  • 11
    You should also work on getting the company policy fixed. When it stops you from doing your job, the policy is broken. Jun 19, 2014 at 14:42
  • 2
    Well, change control its basically an FDA requirement - so nothing I will be changing in this life :)
    – Isaac
    Jun 20, 2014 at 5:18

6 Answers 6

18

Surely you have python?

from socket import * 
from struct import unpack 
import sys 

INTERFACE = "eth0"
TARGET = "8.8.8.8" 
 
if __name__ == "__main__": 
  sock = socket(AF_PACKET, SOCK_DGRAM, htons(0x0800)) 
  sock.bind((INTERFACE, 0x0800)) 
  while True: 
    data = sock.recvfrom(1500, 0)[0] 
    ip = inet_ntop(AF_INET, data[12:16]) 
    if ip == TARGET: 
      print("GOT TARGET")
      sys.exit(1)

This will exit with "GOT TARGET" providing the IP address coming back matches. Since TCP has to send something back during a handshake, this should catch anything from a specific target address. It doesn't care if the protocol is TCP or UDP though (nor do I check).

Don't forget to change TARGET and INTERFACE.

7
  • How would you modify this to also catch a tcp syn packet? Because if the target is not reachble, an attempted but failed connection won't show.
    – Isaac
    Jun 20, 2014 at 12:32
  • 3
    Isn't this script a new software (so is not to be installed in the system)?
    – Vi.
    Jun 20, 2014 at 15:48
  • 1
    -1 Sorry, this is a terrible solution. Not only is it killing a chicken with a machine gun (writing a program to do what there is already a solution for), but it's an infinite loop- unnecessarily consuming CPU resources (run an strace on the python process... but stand by the Control-C key!), it fails to properly solve the user's question: It will respond "GOT TARGET" to both incoming and outgoing TCP connections (user asked for "tries to send requests..."). Finally it exits with 1 (indicating failure to the shell) upon success.
    – Mike S
    Aug 1, 2016 at 17:03
  • 1
    Its a horrific solution put together in minutes! For a pretty lame 'company policy'. The real solution is have a more flexible software policy. Aug 1, 2016 at 20:05
  • 2
    very cool, but you still need elevated permissions for this. I got socket.error: (1, 'Operation not permitted') Mar 19, 2018 at 16:45
19

I'd really try to get tcpdump. That being said, some alternatives to see if a certain connection exists for an IP are:

strace:

[kbrandt@ny-kbrandt01: ~] strace -e trace=network nc 1.2.3.4 1234
...
socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(1234), sin_addr=inet_addr("1.2.3.4")}, 16) = -1 EINPROGRESS (Operation now in progress)

lsof:

[kbrandt@ny-kbrandt01: ~] nc 1.2.3.4 1234 &
[1] 11434
[kbrandt@ny-kbrandt01: ~] lsof -p 11434
....
nc      11434 kbrandt    3u  IPv4 4543149      0t0     TCP 10.7.0.78:58886->1.2.3.4:search-agent (SYN_SENT)

netstat:

[kbrandt@ny-kbrandt01: ~] nc 1.2.3.4 1234 &
[1] 11486
[kbrandt@ny-kbrandt01: ~] sudo netstat -a -p | grep 11486
tcp        0      1 10.7.0.78:58891             1.2.3.4:search-agent        SYN_SENT    11486/nc
3
  • 1
    That was the kind of answer I was hoping for. Unfortunatly strace is not installed either :/
    – Isaac
    Jun 19, 2014 at 12:24
  • I will try lsof
    – Isaac
    Jun 19, 2014 at 12:28
  • 1
    I guess the original poster liked the reply, but oddly it doesn't really answer his question ("...tcp connections - I would like to see if a server tries to send requests to a certain IP...). These will give you network connection information for a particular process... maybe he wants that? For those like me that come here looking for information about TCP connections from one machine to the next this might not do the trick. But props for showing a number of different shell command examples that allow you to gather information about your network connections.
    – Mike S
    Aug 1, 2016 at 17:33
16

Iptables has a debug capability and that can be used for traffic analysis too.

The solution is described on the URL below.

Debugging rules in Iptables

It's also worth reading the following URL to set up the logging of trace output to a file of your choice.

http://backreference.org/2010/06/11/iptables-debugging/

I would not consider this solution equal to tcpdump, but it can be done using a minimal install of Centos. You need to be careful not to fill up the disk with the logs, because tcpdump is much more efficient in disk use. Turn off the logging when it is not required.

You can use the following as a basic template in your script.

# Logging
log(){
SOURCE=a.b.c.d (IP address)
$IPT -A INPUT   -s $SOURCE -m limit --limit 50/minute -j LOG --log-level 7 --log-prefix "In: "
$IPT -A OUTPUT  -s $SOURCE -m limit --limit 50/minute -j LOG --log-level 7 --log-prefix "Out: "
$IPT -A FORWARD -s $SOURCE -m limit --limit 50/minute -j LOG --log-level 7 --log-prefix "Fw: "
$IPT -t nat -A POSTROUTING -m limit --limit 50/minute -j LOG --log-level 7 --log-prefix "Nat: "
}
#log  (remove comment to enable)

trace(){
iptables -t raw -A PREROUTING -p tcp  -j TRACE
iptables -t raw -A OUTPUT     -p tcp  -j TRACE
}
#trace (remove comment to enable)
0
8

If you need specific software to do your job, and aren't allowed to, you're either not making a good business case or selling your ideas to the right people... or you're not in control of this system...

If I were tasked to do something and needed the type of debugging/troubleshooting information that you require in this case, I'd use the right tool. That's likely tcpdump or tshark. Yes, those are pieces of software, but I'd deem them more essential utilities. In fact, they are utilities that could be installed or loaded onto the system temporarily and removed without incident (is removable media an option?...hint)

But the point is that a janky workaround to company policy probably takes more effort than getting approval for this use case.

5
  • I completly agree on using the right tool for the right job. In fact, I can install software, but we have to use change control, so the process takes a few days - I needed the dump right now, so installing was not an option. I was just wondering if I overlooked some options have not thought about yet.
    – Isaac
    Jun 19, 2014 at 12:22
  • :( about change-management. Is there any way to play a political game and get the people/parties who'd benefit from the dump to fast-track the change-management process?
    – ewwhite
    Jun 19, 2014 at 12:23
  • 1
    I am afraid not - but since its the customer who requires change-management and also tcpdump, he will have to accept the facts. I was hoping I could make him happy anyhow.
    – Isaac
    Jun 19, 2014 at 12:26
  • 3
    @ewwhite: Any good change management policy should have the ability to exempt specific, or classes of, changes from going through the full process. If this one doesn't....
    – Scott Pack
    Jun 19, 2014 at 12:27
  • 2
    Can't you compile a statically linked tcpdump elsewhere, and just copy it to /tmp and run from there?
    – che
    Jun 19, 2014 at 13:06
5

Kyle offered some great options. One more would be to use iptables:

[james@server ~]$ sudo iptables -I OUTPUT -d 1.2.3.4/32
...
[james@server ~]$ sudo iptables -L OUTPUT -n -v
Chain OUTPUT (policy ACCEPT 105 packets, 35602 bytes)
 pkts bytes target  prot opt in  out  source      destination
   87 33484 LOG     all  --  *   *    0.0.0.0/0   1.2.3.4     LOG flags 0 level 4

This is essentially an accounting rule. It doesn't explicitly permit or deny traffic, so the default policy for the OUTPUT chain is used (which defaults to ACCEPT). However, any matching packet will increment the counters for the rule.

You can optionally log details about the packet as well with the -j LOG option:

[james@server ~]$ sudo iptables -I OUTPUT -d 1.2.3.4/32 -j LOG
...
[james@server ~]@ dmesg | grep 1.2.3.4 | tail -1
IN= OUT=eth0 SRC=192.168.1.1 DST=1.2.3.4 LEN=100 TOS=0x10 PREC=0x00 TTL=64 ...

Logs will go to the kernel logging facility, so it should show up in /var/log/messages on Red Hat derivatives and /var/log/kern.log on Debian derivatives. It would also be visible in the output of dmesg, as shown. Unlike tcpdump, however, it will not log the complete contents of the packet, only the contents of the packet header.

1
  • I was thinking of iptables myself, but also not installed :/. Anyway, nice solution.
    – Isaac
    Jun 19, 2014 at 17:39
3

Since your server is connecting to a certain IP, I presume it will be to a port that you also have knowledge of?

In any case, netstat or ss are designed to do what you want. You can do the same with either command:

netstat -n -t | awk '{print $5}' | grep A.B.C.D:n
ss      -n -t | awk '{print $5}' | grep A.B.C.D:n

where A.B.C.D represents an IPv4 address, and n represents a port number that your server is connecting to on the remote side. For example:

ss      -n -t | awk '{print $5}' | grep 10.137.54.22:3389

Or, if you'd just like to know that the connection is made:

ss      -n -t | awk '{print $5}' | grep -q 10.137.54.22:3389 && echo "CONNECTION MADE"

If you don't know the port number that you are attempting to connect to, then the job would be more difficult as TCP will open a port on both sides of the conversation for the data and ACK packets. In that case you could simply grep for the IP address to show that any connection has been made, whether that's to or from.

Finally, you can loop this to your heart's content to use as your monitoring tool:

while true; do
    ss -n -t | awk '{print $5}' | grep -q A.B.C.D:n && \
        echo "CONNECTION MADE" && \
        exit 0
    sleep 1
done
1
  • Great way to monitor without needing elevated kernel privileges. Mar 19, 2018 at 16:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .