17

On a centos box, I like to dump tcp connections - I would like to see if a server tries to send requests to a certain IP. Usually tcpdump would do the trick - but tcpdump is not installed, and installing software is not an option (because of company policy). I am afraid netstat will not show me a single request.

So I was wondering what other options I have. I do have root access on the server.

  • Do you have root access on the server ? – user9517 Jun 19 '14 at 12:02
  • Yes, I do have root access. – Isaac Jun 19 '14 at 12:04
  • 11
    You should also work on getting the company policy fixed. When it stops you from doing your job, the policy is broken. – Michael Hampton Jun 19 '14 at 14:42
  • 1
    Well, change control its basically an FDA requirement - so nothing I will be changing in this life :) – Isaac Jun 20 '14 at 5:18
17

Surely you have python?

from socket import * 
from struct import unpack 
import sys 

INTERFACE = "eth0"
TARGET = "8.8.8.8" 

if __name__ == "__main__": 
  sock = socket(AF_PACKET, SOCK_DGRAM, 0x0800) 
  sock.bind((INTERFACE, 0x0800)) 
  while True: 
    data = sock.recvfrom(1500, 0)[0] 
    ip = inet_ntop(AF_INET, data[12:16]) 
    if ip == TARGET: 
      print "GOT TARGET" 
      sys.exit(1)

This will exit with "GOT TARGET" providing the IP address coming back matches. Since TCP has to send something back during a handshake, this should catch anything from a specific target address. It doesn't care if the protocol is TCP or UDP though (nor do I check).

Dont forget to change TARGET and INTERFACE.

| improve this answer | |
  • How would you modify this to also catch a tcp syn packet? Because if the target is not reachble, an attempted but failed connection won't show. – Isaac Jun 20 '14 at 12:32
  • 3
    Isn't this script a new software (so is not to be installed in the system)? – Vi. Jun 20 '14 at 15:48
  • @Vi with that logic, you can argue that any shell script is 'new software'. It depends how you define 'software'. – Matthew Ife Jun 23 '14 at 20:05
  • 1
    -1 Sorry, this is a terrible solution. Not only is it killing a chicken with a machine gun (writing a program to do what there is already a solution for), but it's an infinite loop- unnecessarily consuming CPU resources (run an strace on the python process... but stand by the Control-C key!), it fails to properly solve the user's question: It will respond "GOT TARGET" to both incoming and outgoing TCP connections (user asked for "tries to send requests..."). Finally it exits with 1 (indicating failure to the shell) upon success. – Mike S Aug 1 '16 at 17:03
  • 1
    very cool, but you still need elevated permissions for this. I got socket.error: (1, 'Operation not permitted') – Grant Bowman Mar 19 '18 at 16:45
17

I'd really try to get tcpdump. That being said, some alternatives to see if a certain connection exists for an IP are:

strace:

[kbrandt@ny-kbrandt01: ~] strace -e trace=network nc 1.2.3.4 1234
...
socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(1234), sin_addr=inet_addr("1.2.3.4")}, 16) = -1 EINPROGRESS (Operation now in progress)

lsof:

[kbrandt@ny-kbrandt01: ~] nc 1.2.3.4 1234 &
[1] 11434
[kbrandt@ny-kbrandt01: ~] lsof -p 11434
....
nc      11434 kbrandt    3u  IPv4 4543149      0t0     TCP 10.7.0.78:58886->1.2.3.4:search-agent (SYN_SENT)

netstat:

[kbrandt@ny-kbrandt01: ~] nc 1.2.3.4 1234 &
[1] 11486
[kbrandt@ny-kbrandt01: ~] sudo netstat -a -p | grep 11486
tcp        0      1 10.7.0.78:58891             1.2.3.4:search-agent        SYN_SENT    11486/nc
| improve this answer | |
  • 1
    That was the kind of answer I was hoping for. Unfortunatly strace is not installed either :/ – Isaac Jun 19 '14 at 12:24
  • I will try lsof – Isaac Jun 19 '14 at 12:28
  • 1
    I guess the original poster liked the reply, but oddly it doesn't really answer his question ("...tcp connections - I would like to see if a server tries to send requests to a certain IP...). These will give you network connection information for a particular process... maybe he wants that? For those like me that come here looking for information about TCP connections from one machine to the next this might not do the trick. But props for showing a number of different shell command examples that allow you to gather information about your network connections. – Mike S Aug 1 '16 at 17:33
15

Iptables has a debug capability and that can be used for traffic analysis too.

The solution is described on the URL below.

Debugging rules in Iptables

It's also worth reading the following URL to set up the logging of trace output to a file of your choice.

http://backreference.org/2010/06/11/iptables-debugging/

I would not consider this solution equal to tcpdump, but it can be done using a minimal install of Centos. You need to be careful not to fill up the disk with the logs, because tcpdump is much more efficient in disk use. Turn off the logging when it is not required.

You can use the following as a basic template in your script.

# Logging
log(){
SOURCE=a.b.c.d (IP address)
$IPT -A INPUT   -s $SOURCE -m limit --limit 50/minute -j LOG --log-level 7 --log-prefix "In: "
$IPT -A OUTPUT  -s $SOURCE -m limit --limit 50/minute -j LOG --log-level 7 --log-prefix "Out: "
$IPT -A FORWARD -s $SOURCE -m limit --limit 50/minute -j LOG --log-level 7 --log-prefix "Fw: "
$IPT -t nat -A POSTROUTING -m limit --limit 50/minute -j LOG --log-level 7 --log-prefix "Nat: "
}
#log  (remove comment to enable)

trace(){
iptables -t raw -A PREROUTING -p tcp  -j TRACE
iptables -t raw -A OUTPUT     -p tcp  -j TRACE
}
#trace (remove comment to enable)
| improve this answer | |
8

If you need specific software to do your job, and aren't allowed to, you're either not making a good business case or selling your ideas to the right people... or you're not in control of this system...

If I were tasked to do something and needed the type of debugging/troubleshooting information that you require in this case, I'd use the right tool. That's likely tcpdump or tshark. Yes, those are pieces of software, but I'd deem them more essential utilities. In fact, they are utilities that could be installed or loaded onto the system temporarily and removed without incident (is removable media an option?...hint)

But the point is that a janky workaround to company policy probably takes more effort than getting approval for this use case.

| improve this answer | |
  • I completly agree on using the right tool for the right job. In fact, I can install software, but we have to use change control, so the process takes a few days - I needed the dump right now, so installing was not an option. I was just wondering if I overlooked some options have not thought about yet. – Isaac Jun 19 '14 at 12:22
  • :( about change-management. Is there any way to play a political game and get the people/parties who'd benefit from the dump to fast-track the change-management process? – ewwhite Jun 19 '14 at 12:23
  • 1
    I am afraid not - but since its the customer who requires change-management and also tcpdump, he will have to accept the facts. I was hoping I could make him happy anyhow. – Isaac Jun 19 '14 at 12:26
  • 3
    @ewwhite: Any good change management policy should have the ability to exempt specific, or classes of, changes from going through the full process. If this one doesn't.... – Scott Pack Jun 19 '14 at 12:27
  • 2
    Can't you compile a statically linked tcpdump elsewhere, and just copy it to /tmp and run from there? – che Jun 19 '14 at 13:06
5

Kyle offered some great options. One more would be to use iptables:

[james@server ~]$ sudo iptables -I OUTPUT -d 1.2.3.4/32
...
[james@server ~]$ sudo iptables -L OUTPUT -n -v
Chain OUTPUT (policy ACCEPT 105 packets, 35602 bytes)
 pkts bytes target  prot opt in  out  source      destination
   87 33484 LOG     all  --  *   *    0.0.0.0/0   1.2.3.4     LOG flags 0 level 4

This is essentially an accounting rule. It doesn't explicitly permit or deny traffic, so the default policy for the OUTPUT chain is used (which defaults to ACCEPT). However, any matching packet will increment the counters for the rule.

You can optionally log details about the packet as well with the -j LOG option:

[james@server ~]$ sudo iptables -I OUTPUT -d 1.2.3.4/32 -j LOG
...
[james@server ~]@ dmesg | grep 1.2.3.4 | tail -1
IN= OUT=eth0 SRC=192.168.1.1 DST=1.2.3.4 LEN=100 TOS=0x10 PREC=0x00 TTL=64 ...

Logs will go to the kernel logging facility, so it should show up in /var/log/messages on Red Hat derivatives and /var/log/kern.log on Debian derivatives. It would also be visible in the output of dmesg, as shown. Unlike tcpdump, however, it will not log the complete contents of the packet, only the contents of the packet header.

| improve this answer | |
  • I was thinking of iptables myself, but also not installed :/. Anyway, nice solution. – Isaac Jun 19 '14 at 17:39
3

Since your server is connecting to a certain IP, I presume it will be to a port that you also have knowledge of?

In any case, netstat or ss are designed to do what you want. You can do the same with either command:

netstat -n -t | awk '{print $5}' | grep A.B.C.D:n
ss      -n -t | awk '{print $5}' | grep A.B.C.D:n

where A.B.C.D represents an IPv4 address, and n represents a port number that your server is connecting to on the remote side. For example:

ss      -n -t | awk '{print $5}' | grep 10.137.54.22:3389

Or, if you'd just like to know that the connection is made:

ss      -n -t | awk '{print $5}' | grep -q 10.137.54.22:3389 && echo "CONNECTION MADE"

If you don't know the port number that you are attempting to connect to, then the job would be more difficult as TCP will open a port on both sides of the conversation for the data and ACK packets. In that case you could simply grep for the IP address to show that any connection has been made, whether that's to or from.

Finally, you can loop this to your heart's content to use as your monitoring tool:

while true; do
    ss -n -t | awk '{print $5}' | grep -q A.B.C.D:n && \
        echo "CONNECTION MADE" && \
        exit 0
    sleep 1
done
| improve this answer | |
  • Great way to monitor without needing elevated kernel privileges. – Grant Bowman Mar 19 '18 at 16:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.