How found suspect connection in a netstat output?

Question

It seems some ssh brute force are outgoing from my debian 7.1. I'm searching how to find the source of these brute force. I'm searching in netstat output, but how can i identify trace of this hack ?

root@server:~# netstat -pa
Connexions Internet actives (serveurs et établies)
Proto Recv-Q Send-Q Adresse locale          Adresse distante        Etat        PID/Program name
tcp        0      0 *:sunrpc                *:*                     LISTEN      1778/rpcbind    
tcp        0      0 *:ftp                   *:*                     LISTEN      3344/vsftpd     
tcp        0      0 *:ssh                   *:*                     LISTEN      2853/sshd       
tcp        0      0 *:smtp                  *:*                     LISTEN      3317/master     
tcp        0      0 localhost:6502          *:*                     LISTEN      7660/murmurd    
tcp        0      0 localhost:mysql         *:*                     LISTEN      2796/mysqld     
tcp        0      0 *:43978                 *:*                     LISTEN      1809/rpc.statd  
tcp        0    384 VPS-286:ssh             lns-bzn-25-82-254:54495 ESTABLISHED 27537/sshd: bux [pr
tcp6       0      0 [::]:sunrpc             [::]:*                  LISTEN      1778/rpcbind    
tcp6       0      0 [::]:http               [::]:*                  LISTEN      20188/apache2   
tcp6       0      0 [::]:60915              [::]:*                  LISTEN      1809/rpc.statd  
tcp6       0      0 [::]:ssh                [::]:*                  LISTEN      2853/sshd       
tcp6       0      0 [::]:smtp               [::]:*                  LISTEN      3317/master     
tcp6       0      0 [::]:64738              [::]:*                  LISTEN      7660/murmurd    
tcp6       0     53 VPS-286:64738           modemcable023.125:48495 ESTABLISHED 7660/murmurd    
udp        0      0 *:sunrpc                *:*                                 1778/rpcbind    
udp        0      0 *:681                   *:*                                 1778/rpcbind    
udp        0      0 localhost:713           *:*                                 1809/rpc.statd  
udp        0      0 *:mdns                  *:*                                 2343/avahi-daemon: 
udp        0      0 *:42288                 *:*                                 2343/avahi-daemon: 
udp        0      0 *:42305                 *:*                                 1809/rpc.statd  
udp        0      0 *:1900                  *:*                                 3350/minissdpd  
udp6       0      0 [::]:sunrpc             [::]:*                              1778/rpcbind    
udp6       0      0 [::]:681                [::]:*                              1778/rpcbind    
udp6       0      0 [::]:46811              [::]:*                              1809/rpc.statd  
udp6       0      0 [::]:64738              [::]:*                              7660/murmurd    
udp6       0      0 [::]:mdns               [::]:*                              2343/avahi-daemon: 
udp6       0      0 [::]:56702              [::]:*                              2343/avahi-daemon: 
Sockets du domaine UNIX actives(serveurs et établies)
Proto RefCnt Flags       Type       State         I-Node   PID/Program name    Chemin
unix  2      [ ACC ]     STREAM     LISTENING     6257     2381/gam_server     @/tmp/fam-root-
unix  2      [ ACC ]     STREAM     LISTENING     5659     1778/rpcbind        /var/run/rpcbind.sock
unix  2      [ ACC ]     SEQPACKET  LISTENING     3360     344/udevd           /run/udev/control
unix  2      [ ACC ]     STREAM     LISTENING     6178     2343/avahi-daemon:  /var/run/avahi-daemon/socket
unix  2      [ ACC ]     STREAM     LISTENING     6237     2379/python         /var/run/fail2ban/fail2ban.sock
unix  11     [ ]         DGRAM                    6003     2134/rsyslogd       /dev/log
unix  2      [ ACC ]     STREAM     LISTENING     6031     2176/acpid          /var/run/acpid.socket
unix  2      [ ACC ]     STREAM     LISTENING     6830     2796/mysqld         /var/run/mysqld/mysqld.sock
unix  2      [ ]         DGRAM                    6005     2134/rsyslogd       /var/spool/postfix/dev/log
unix  2      [ ACC ]     STREAM     LISTENING     7527     3317/master         public/cleanup
unix  2      [ ACC ]     STREAM     LISTENING     7532     3317/master         private/tlsmgr
unix  2      [ ACC ]     STREAM     LISTENING     7535     3317/master         private/rewrite
unix  2      [ ACC ]     STREAM     LISTENING     7538     3317/master         private/bounce
unix  2      [ ACC ]     STREAM     LISTENING     7541     3317/master         private/defer
unix  2      [ ACC ]     STREAM     LISTENING     7544     3317/master         private/trace
unix  2      [ ACC ]     STREAM     LISTENING     7547     3317/master         private/verify
unix  2      [ ACC ]     STREAM     LISTENING     7550     3317/master         public/flush
unix  2      [ ACC ]     STREAM     LISTENING     7553     3317/master         private/proxymap
unix  2      [ ACC ]     STREAM     LISTENING     7556     3317/master         private/proxywrite
unix  2      [ ACC ]     STREAM     LISTENING     7559     3317/master         private/smtp
unix  2      [ ACC ]     STREAM     LISTENING     7562     3317/master         private/relay
unix  2      [ ACC ]     STREAM     LISTENING     7565     3317/master         public/showq
unix  2      [ ACC ]     STREAM     LISTENING     7568     3317/master         private/error
unix  2      [ ACC ]     STREAM     LISTENING     7571     3317/master         private/retry
unix  2      [ ACC ]     STREAM     LISTENING     7574     3317/master         private/discard
unix  2      [ ACC ]     STREAM     LISTENING     7577     3317/master         private/local
unix  2      [ ACC ]     STREAM     LISTENING     7580     3317/master         private/virtual
unix  2      [ ACC ]     STREAM     LISTENING     7583     3317/master         private/lmtp
unix  2      [ ACC ]     STREAM     LISTENING     7586     3317/master         private/anvil
unix  2      [ ACC ]     STREAM     LISTENING     7589     3317/master         private/scache
unix  2      [ ACC ]     STREAM     LISTENING     7592     3317/master         private/maildrop
unix  2      [ ACC ]     STREAM     LISTENING     7595     3317/master         private/uucp
unix  2      [ ACC ]     STREAM     LISTENING     7598     3317/master         private/ifmail
unix  2      [ ACC ]     STREAM     LISTENING     7601     3317/master         private/bsmtp
unix  2      [ ACC ]     STREAM     LISTENING     7604     3317/master         private/scalemail-backend
unix  2      [ ACC ]     STREAM     LISTENING     7607     3317/master         private/mailman
unix  2      [ ACC ]     STREAM     LISTENING     7650     3350/minissdpd      /var/run/minissdpd.sock
unix  2      [ ACC ]     STREAM     LISTENING     6135     2310/dbus-daemon    /var/run/dbus/system_bus_socket
unix  2      [ ]         DGRAM                    74140611 14274/pickup        
unix  2      [ ]         DGRAM                    74047722 27625/sudo          
unix  2      [ ]         DGRAM                    74047719 27625/sudo          
unix  3      [ ]         STREAM     CONNECTE      74047637 27537/sshd: bux [pr 
unix  3      [ ]         STREAM     CONNECTE      74047636 27539/0             
unix  2      [ ]         DGRAM                    74047635 27537/sshd: bux [pr 
unix  3      [ ]         STREAM     CONNECTE      237655   2310/dbus-daemon    /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTE      237654   7660/murmurd        
unix  3      [ ]         STREAM     CONNECTE      237652   7660/murmurd        
unix  3      [ ]         STREAM     CONNECTE      237651   7660/murmurd        
unix  3      [ ]         STREAM     CONNECTE      237650   7660/murmurd        
unix  3      [ ]         STREAM     CONNECTE      237649   7660/murmurd        
unix  3      [ ]         STREAM     CONNECTE      237632   2310/dbus-daemon    /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTE      237631   7660/murmurd        
unix  3      [ ]         STREAM     CONNECTE      237609   7660/murmurd        
unix  3      [ ]         STREAM     CONNECTE      237608   7660/murmurd        
unix  3      [ ]         STREAM     CONNECTE      237607   7660/murmurd        
unix  3      [ ]         STREAM     CONNECTE      237606   7660/murmurd        
unix  2      [ ]         DGRAM                    34557    3952/tlsmgr         
unix  2      [ ]         DGRAM                    7613     3337/qmgr           
unix  3      [ ]         STREAM     CONNECTE      7609     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7608     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7606     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7605     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7603     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7602     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7600     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7599     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7597     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7596     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7594     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7593     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7591     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7590     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7588     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7587     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7585     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7584     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7582     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7581     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7579     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7578     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7576     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7575     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7573     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7572     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7570     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7569     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7567     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7566     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7564     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7563     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7561     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7560     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7558     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7557     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7555     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7554     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7552     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7551     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7549     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7548     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7546     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7545     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7543     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7542     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7540     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7539     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7537     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7536     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7534     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7533     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7531     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7530     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7529     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7528     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7526     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7525     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7524     3317/master         
unix  3      [ ]         STREAM     CONNECTE      7523     3317/master         
unix  2      [ ]         DGRAM                    7493     3317/master         
unix  2      [ ]         DGRAM                    6746     2797/logger         
unix  3      [ ]         STREAM     CONNECTE      6357     2381/gam_server     @/tmp/fam-root-
unix  3      [ ]         STREAM     CONNECTE      6356     2379/python         
unix  3      [ ]         STREAM     CONNECTE      6321     2381/gam_server     @/tmp/fam-root-
unix  3      [ ]         STREAM     CONNECTE      6320     2379/python         
unix  3      [ ]         STREAM     CONNECTE      6261     2381/gam_server     @/tmp/fam-root-
unix  3      [ ]         STREAM     CONNECTE      6259     2379/python         
unix  3      [ ]         STREAM     CONNECTE      6181     2310/dbus-daemon    /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTE      6180     2343/avahi-daemon:  
unix  3      [ ]         STREAM     CONNECTE      6175     2344/avahi-daemon:  
unix  3      [ ]         STREAM     CONNECTE      6174     2343/avahi-daemon:  
unix  2      [ ]         DGRAM                    6172     2343/avahi-daemon:  
unix  3      [ ]         STREAM     CONNECTE      6139     2310/dbus-daemon    
unix  3      [ ]         STREAM     CONNECTE      6138     2310/dbus-daemon    
unix  2      [ ]         DGRAM                    6028     2176/acpid          
unix  3      [ ]         STREAM     CONNECTE      5774     1823/rpc.idmapd     
unix  3      [ ]         STREAM     CONNECTE      5773     1823/rpc.idmapd     
unix  3      [ ]         DGRAM                    3367     344/udevd           
unix  3      [ ]         DGRAM                    3366     344/udevd 

Answer 1

TCP Established State

The 'ESTABLISHED' means the TCP connection is established, ie the handshake has been performed on TCP/IP level. This is needed before the ssh process sees any data at all. Theoretically, the connection could be quite long in ESTABLISHED mode without sending any data depending on the timeouts set (on TCP level and/or sshd config). Expect login to occur after it.

iptraf

To look into it more, use 'iptraf' for monitoring the amount of traffic, or see /var/log/auth.log (at least, on a Debian system) for seeing who successfully logged on.

Using lsof

The lsof -i command lists all open files associated with Internet connections. It is similar in format to netstat -a -p.

lsof -i
lsof –i :22
lsof -i @linxsol.com #to check which hosts

To list information about TCP sessions on your server lsof -i tcp@hostname:22

To display all open IPv4 network files in use by the process whose PID is 1234, use:

lsof -i 4 -a -p 1234

lsof will then output all matching connections. The above examples will list connections listening or established on port 22

Using netstat

 netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

 netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr

 netstat -ntu | grep -v TIME_WAIT | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr

 netstat -an | grep :80 | awk '{print $5}' | cut -f1 -d":" | sort | uniq -c | sort -n

I hope that helps.

Answer 2

You can set iptables to log SSH out connection by following command

    iptables -A OUTPUT -p tcp --syn --dport 22 -j LOG --log-prefix "SSH OUT DETECTED!!"

And then monitor /var/log/message until you found "SSH OUT DETECTED!!" at this point you can check which process connecting out by following command

    netstat -antp | grep ESTA | grep :22 | grep -v sshd

Alternative way is to use a tool nethogs:

Answer 3

Why bother so low on the network stack? netstat can show you connections...

/var/log/auth.log Should show you a log of all the stuff the actual sshd daemon was doing. A scroll through the logs should reveal more than netstat,iptables, or any other TCP(and down) examination.