1

I have an OpenVPN between my box A, and the remote server B (both Debian jessie).

On server B, I also run Squid3.

I then configure my A-browser to use the remote proxy with the VPN address of the B-server (10.2.0.1:3128), only for port 80 (no SSL).

It works OK, I can browse the Internet. I can see with whatismyip.com that the request is originating from the public B-server address.

I also have a firewall on the B-server, with the following rules :

-A INPUT-m state --state INVALID -j NFLOG  --nflog-prefix "fp=bad_packets:1"
-A INPUT-m state --state INVALID -j DROP

I then sometimes notice some packets are discarded :

Jun 12 09:16:20 ks1 fp=bad_packets:1 IN=tun0 OUT= MAC= SRC=10.2.0.9 DST=10.2.0.1 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=58858 DF PROTO=TCP SPT=34458 DPT=3128 SEQ=3975722279 ACK=0 WINDOW=0 RST URGP=0 MARK=0

This does not prevent the browsing, but I guess there must have some errors somewhere, I just don't know how to find them.

This happens with Firefox, but also with Opera.

It's not much a "problem", it's mostly annoying, and I would like to know what is polluting my firewall logs ;)

EDIT

OK, I've run a tcpdump, refreshed a website that triggered the DROP : I see exactly what you mean, but don't have any explanation yet : DROPs happened on port 45310, so I did a grep on this port : I can see the RESET arriving, almost all on the same "seq" (sequence ?) : 3360258722 : The 1st one seems legit, then there are 6 after...Don't understand why : what's the next investigation step ?

09:58:02.133295 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 104724:105019, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 295
09:58:02.134079 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 105019:106375, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356
09:58:02.134091 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 106375:107731, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356
09:58:02.134127 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 107731:109087, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356
09:58:02.134133 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 109087:109115, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 28
09:58:02.134764 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 109115:110471, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356
09:58:02.134779 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 110471:111827, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356
09:58:02.134809 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 111827:112081, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 254
09:58:02.258393 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 106375, win 1444, options [nop,nop,TS val 1013362 ecr 128061282], length 0
09:58:02.264169 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 109087, win 1444, options [nop,nop,TS val 1013363 ecr 128061282], length 0
09:58:02.268597 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 110471, win 1444, options [nop,nop,TS val 1013364 ecr 128061282], length 0
09:58:02.273283 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 112081, win 1444, options [nop,nop,TS val 1013365 ecr 128061282], length 0
09:58:02.557676 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [P.], seq 12580:13402, ack 112081, win 1444, options [nop,nop,TS val 1013436 ecr 128061282], length 822
09:58:02.577739 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 112081:112389, ack 13402, win 678, options [nop,nop,TS val 128061393 ecr 1013436], length 308
09:58:02.577964 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 112389:113541, ack 13402, win 678, options [nop,nop,TS val 128061393 ecr 1013436], length 1152
09:58:02.593245 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 113541, win 1444, options [nop,nop,TS val 1013445 ecr 128061393], length 0
09:58:02.596042 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 113541:114897, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356
09:58:02.596081 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 114897:115211, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 314
09:58:02.596441 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 115211:116567, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356
09:58:02.596467 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 116567:117923, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356
09:58:02.596530 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 117923:118114, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 191
09:58:02.596814 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 118114:119470, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356
09:58:02.596837 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 119470:120826, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356
09:58:02.596890 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 120826:121063, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 237
09:58:02.611183 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [F.], seq 13402, ack 114897, win 1444, options [nop,nop,TS val 1013450 ecr 128061398], length 0
09:58:02.611740 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0
09:58:02.611821 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [F.], seq 121063, ack 13403, win 678, options [nop,nop,TS val 128061402 ecr 1013450], length 0
09:58:02.614315 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0
09:58:02.617520 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0
09:58:02.618641 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0
09:58:02.621415 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0
09:58:02.624966 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0
09:58:02.625256 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0
09:58:02.631926 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258723, win 0, length 0


Jun 15 09:58:02 ks1 fp=bad_packets:1 IN=tun0 OUT= MAC= SRC=10.85.0.9 DST=10.85.0.1 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=19768 DF PROTO=TCP SPT=45310 DPT=3128 SEQ=3360258722 ACK=0 WINDOW=0 RST URGP=0 MARK=0 
Jun 15 09:58:02 ks1 fp=bad_packets:1 IN=tun0 OUT= MAC= SRC=10.85.0.9 DST=10.85.0.1 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=19769 DF PROTO=TCP SPT=45310 DPT=3128 SEQ=3360258722 ACK=0 WINDOW=0 RST URGP=0 MARK=0 
Jun 15 09:58:02 ks1 fp=bad_packets:1 IN=tun0 OUT= MAC= SRC=10.85.0.9 DST=10.85.0.1 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=19770 DF PROTO=TCP SPT=45310 DPT=3128 SEQ=3360258722 ACK=0 WINDOW=0 RST URGP=0 MARK=0
Jun 15 09:58:02 ks1 fp=bad_packets:1 IN=tun0 OUT= MAC= SRC=10.85.0.9 DST=10.85.0.1 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=19771 DF PROTO=TCP SPT=45310 DPT=3128 SEQ=3360258722 ACK=0 WINDOW=0 RST URGP=0 MARK=0
Jun 15 09:58:02 ks1 fp=bad_packets:1 IN=tun0 OUT= MAC= SRC=10.85.0.9 DST=10.85.0.1 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=19772 DF PROTO=TCP SPT=45310 DPT=3128 SEQ=3360258722 ACK=0 WINDOW=0 RST URGP=0 MARK=0
Jun 15 09:58:02 ks1 fp=bad_packets:1 IN=tun0 OUT= MAC= SRC=10.85.0.9 DST=10.85.0.1 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=19773 DF PROTO=TCP SPT=45310 DPT=3128 SEQ=3360258722 ACK=0 WINDOW=0 RST URGP=0 MARK=0   

EDIT2 I've uploaded a .zip file with the tcpdump capture of both sides :

http://expirebox.com/download/80b335fb34fab6824d4e35dd3609f628.html md5sum : ba748e6c6e787e81e765b6ef83b7b2cf

Hope this will help

Note : i'll edit this message a thousand times I think to have the right syntax... (I'm a serverfault.com beginner ;)

1

Notice the RST flag being on. It seems that your browser sends RST packet to an already closed connection to your proxy server. It is hard to tell why this happens from provided information. Probably if you managed to capture the connection which causes this via wireshark/tcpdump, you could find out.

We start with what looks like normal data exchange, server sending some data to client:

09:58:02.133295 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 104724:105019, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 295
09:58:02.134079 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 105019:106375, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356
09:58:02.134091 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 106375:107731, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356
09:58:02.134127 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 107731:109087, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356
09:58:02.134133 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 109087:109115, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 28
09:58:02.134764 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 109115:110471, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356
09:58:02.134779 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 110471:111827, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356
09:58:02.134809 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 111827:112081, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 254

120ms pause and the client is acking all the received packets, still looks fine, though the client leaves window very short (1444), which could hint there is some congestion:

09:58:02.258393 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 106375, win 1444, options [nop,nop,TS val 1013362 ecr 128061282], length 0
09:58:02.264169 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 109087, win 1444, options [nop,nop,TS val 1013363 ecr 128061282], length 0
09:58:02.268597 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 110471, win 1444, options [nop,nop,TS val 1013364 ecr 128061282], length 0
09:58:02.273283 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 112081, win 1444, options [nop,nop,TS val 1013365 ecr 128061282], length 0

270 ms pause and another round of data exchange, now both ways, data ACKed (. in brackets) both ways:

09:58:02.557676 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [P.], seq 12580:13402, ack 112081, win 1444, options [nop,nop,TS val 1013436 ecr 128061282], length 822
09:58:02.577739 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 112081:112389, ack 13402, win 678, options [nop,nop,TS val 128061393 ecr 1013436], length 308
09:58:02.577964 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 112389:113541, ack 13402, win 678, options [nop,nop,TS val 128061393 ecr 1013436], length 1152
09:58:02.593245 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 113541, win 1444, options [nop,nop,TS val 1013445 ecr 128061393], length 0

Now it gets tricky. There is a lot of data (7522 bytes) from server to client, even though window is only 1444 bytes, notice it's 8 packets:

09:58:02.596042 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 113541:114897, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356
09:58:02.596081 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 114897:115211, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 314
09:58:02.596441 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 115211:116567, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356
09:58:02.596467 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 116567:117923, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356
09:58:02.596530 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 117923:118114, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 191
09:58:02.596814 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 118114:119470, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356
09:58:02.596837 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 119470:120826, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356
09:58:02.596890 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 120826:121063, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 237

Client only acks first 1356 bytes (first packet) and closes (FIN) the connection (F in brackets)

09:58:02.611183 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [F.], seq 13402, ack 114897, win 1444, options [nop,nop,TS val 1013450 ecr 128061398], length 0

Now the client sends RST with an inconsistent seq. TBH I have no idea why the seq is this wrong.

09:58:02.611740 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0

Server ACKnowledges [.] the connection close (FIN) [F].

09:58:02.611821 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [F.], seq 121063, ack 13403, win 678, options [nop,nop,TS val 128061402 ecr 1013450], length 0

Client rejects remaining seven packets sent out of window. These go into a correctly closed connection (FIN, FIN/ACK), so they do not match a valid state (thus get logged as invalid).

09:58:02.614315 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0
09:58:02.617520 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0
09:58:02.618641 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0
09:58:02.621415 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0
09:58:02.624966 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0
09:58:02.625256 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0
09:58:02.631926 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258723, win 0, length 0

I'm no security expert, but judging by the out of scope seq on RST and data sent over the window, there seems to be something wrong with your stack. It may be just some tweaking gone wild, or some misapplied filter/mangling rule, but if not justified otherwise, I'd be checking my system for malicious code at this point. Or there may be some rogue network hardware on the way.

The dump on the client side of connection may as well provide some insight into what is happening.

  • OK, I've run a tcpdump, refreshed a website that triggered the DROP : I see exactly what you mean, but don't have any explanation yet : DROPs happened on port 45310, so I did a grep on this port : I can see the RESET arriving, almost all on the same "seq" (sequence ?) : 3360258722 : The 1st one seems legit, then there are 6 after...Don't understand why : what's the next investigation step ? – Burps Jun 15 '15 at 8:06
  • @Burps, my apologies, I lost my crystal ball. Please update your question with more information (like the tcpdump of last 5-10 packets - possibly shared through pastebin, if too long), if you desire more detailed answer. – Fox Jun 15 '15 at 8:09
  • sorry, I first thought I could put everything in a simple "answer", I'm not used to serverfault.com : I edited my first post with the capture and the FW logs. – Burps Jun 15 '15 at 8:17
  • @Burps I dissected the tcpdump for you. Hopefully it helps, as I'am not sure, I will be able to provide more information on this issue. – Fox Jun 15 '15 at 8:59
  • I uploaded a .zip file that contains the tcpdump pcap files of both sides :... OK, I'll update the 1st post ;) – Burps Jun 15 '15 at 14:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.