1

my network looks like this:

ISP cable --- ISP modem/router (2 in 1) 
                    \       \--- my router --- wifi connection --- computer 1
                     \----------- cable connection ---- computer 2

on both computers, every 5 seconds i run:

ping -U -c 1 -w 1 google.com

and on computer 1 it fails every 15 minutes, at every hh:14:30, hh:29:30, hh:44:30, hh:59:30. no matter when i restarted the router. like there was some cron set up. but every existing connection works: wget, online game or radio started before failure are working correctly. but ping, loading new pages in a browser stops for a minute. same happens when i ping 8.8.8.8 so it's probably not about dns.

every time the problem takes 30-60 seconds. ping goes to isp modem (192.168.0.1) but not further. other computer works without any problems. when i connects computer 2 to my router it has the same symptoms.

i know: it's double NAT. but what is EXACTLY going on? and if i can't diagnose it's impossible to fix.

i can't bridge modem and router or use PPPoE (ISP's limitations). i tried DMZ - same result. before, with ISP mode (not modem/router) there was no such problem. currently i have no modem to diagnose if my router is broken (but i doubt it). any ideas? how can i diagnose anything?

EDIT: in case it matters:

  • isp modem/router (technicolor):

    wan: 89.78.xxx.xxx / 255.255.252.0
    lan: 192.168.0.1 / 255.255.255.0
    dhcp enabled; leasing time: 86400

  • my router (asus RT-N66U):

    wan: 192.168.0.10 / 255.255.255.0; gateway: 192.168.0.1
    lan: 192.168.1.1 / 255.255.255.0
    dhcp enabled; leasing time: 86400

  • computer 1: 192.168.1.101 / 255.255.255.0

on my router there is absolutely no additional logging activity around the time of incidents.

when i telnet to my router and do ping -c 1 -w 1 -W 1 212.77.100.101 during incident, the command exits immediately (as expected) but when i do ping -c 1 -w 1 -W 1 wp.pl (dns of above ip) it doesn't exit immediately. it waits until network is back again. but on computer 1 both commands exit immediately. not sure if that's because of network or because of older version of ping:

# ping
BusyBox v1.17.4 (2015-01-10 18:59:17 CST) multi-call binary.

i telned to my router, disabled ntp, changed its time and rebooted. yet still connection was lost at the same time. therefore it seems like it's initialised by isp router. but why computer 2 is working correctly?

my router routing table log:

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.0.1     *               255.255.255.255 UH    0      0        0 WAN
192.168.1.0     *               255.255.255.0   U     0      0        0 LAN
192.168.0.0     *               255.255.255.0   U     0      0        0 WAN
default         192.168.0.1     0.0.0.0         UG    0      0        0 WAN

connection log:

Proto NATed Address                            Destination Address                      State 
tcp   192.168.0.11:33281                       192.168.1.1:80                           ESTABLISHED
tcp   192.168.0.11:33206                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33213                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33205                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33154                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33167                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33204                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33191                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33250                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33236                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33224                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33234                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33251                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33185                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33187                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33239                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33134                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33179                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33199                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33173                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33136                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33217                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33278                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33221                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33233                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33248                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33218                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33147                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33176                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33169                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33264                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33178                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33194                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33208                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33200                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33192                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33146                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33280                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33243                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33175                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33193                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33272                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33267                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33222                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33183                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33253                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33161                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33265                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33180                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33171                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33166                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33212                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33240                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33150                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33160                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33186                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33137                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33172                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33229                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33235                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33149                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33231                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33184                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33226                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33269                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33133                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33162                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33263                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33153                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33211                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33151                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33262                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33228                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33203                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33202                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33159                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33268                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33207                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33135                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33252                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33152                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33273                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33158                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33142                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33223                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33242                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33139                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33232                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33188                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33244                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33238                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33255                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33209                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33181                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33237                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33219                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33276                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33141                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33170                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33195                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33145                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33257                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33246                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33182                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33271                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33220                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33201                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33190                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33215                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33254                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33148                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33144                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33143                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33256                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33198                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33260                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33140                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33138                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33155                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33266                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33275                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33168                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33197                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33277                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33270                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33247                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33214                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33227                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33196                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33259                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33189                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33216                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33274                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33177                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33157                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33230                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33225                       192.168.1.1:80                           TIME_WAIT  
tcp   192.168.0.11:33241                       192.168.1.1:80                           TIME_WAIT  

what other logs can help?

EDIT 2:

  • during the incident i still have access to isp router web interface without any problems.
  • when i turn off and turn on isp router, the exact time of network pauses changes but it's still every 15 minutes
  • i disabled UPnP on both routers
  • You don't mention the models of router, or if they have any logs, or the relevant subnets. It could be many things: eg the wireless router renewing a DHCP lease or changing its routing table, or an artefact of uPnP or connection tracking in either router. – Cedric Knight Feb 21 '15 at 23:22
  • @CedricKnight thx for the answer. i updated my question. maybe it helps – piotrek Feb 22 '15 at 1:08
  • Thanks, that's informative. You say lan: 192.168.0.1 / 192.168.0.1 for the wireless router - that's surely incorrect? I presume from the routing table it's 192.168.1.1 / 255.255.255.0. It's more likely to be the ARP cache on the – Cedric Knight Feb 22 '15 at 7:58
  • @CedricKnight yes, of course. i fixed the question – piotrek Feb 22 '15 at 14:03
  • @CedricKnight and i disabled upnp on both routers – piotrek Feb 22 '15 at 14:47
0

Two possible answers:

(1) The ARP cache on the Asus losing the MAC address for the modem router. Is there any other device besides the modem that has an interface on 192.168.0.1 ? You say you can ping 192.168.0.1 while WAN connectivity is interrupted, but can you see the Technicolor web interface on 192.168.0.1:80 or telnet to it ? If that changes, possibly it is a different 192.168.0.1 responding to pings. You could try setting the MAC address manually with arp -s 192.168.0.1 00:01:02:03:04:05.

(2) Connection tracking on either device, particularly if you are running a server within either LAN. Although this would account for new connections failing, it does not account for the regular 15 minute period. For example the Technicolor may well hit a limit of 2048 UDP NAT entries. You could log into that and try:

:connection timerconfig timer=udpkill value=60
:connection timerconfig timer=udpidle value=20
  • there is no other device in the network except those on the diagram. yes, i can access technicolor web interface during the network hangs (i updated the question). i can't telnet to technicolor. nmap says only http port is open. also when i turn off and on the technicolor, the network is still down every 15mins but at different moments – piotrek Feb 22 '15 at 14:24
  • OK, so things seem to be pointing to the Technicolor. Further, if you try disconnecting the ISP cable rather than resetting the modem that could indicate the time of the interruptions is related to the networking in the modem. You say ping from the Asus "exits immediately" (unsurprising that it waits for DNS resolution), but I'm guessing ping from it to 8.8.8.8 fails during the interruptions? Is there nothing at all in the Technicolor event logs? At this point, I might resort to a packet sniffer on Computer 2. – Cedric Knight Feb 22 '15 at 15:13
  • ping for 8.8.8.8 exits immediately during the interruption (as expected, because of 1s limitation) but ping to any dns (google.com) doesn't. it waits a few seconds before exiting – piotrek Feb 22 '15 at 15:33
  • It's still completely not clear from what you're saying whether you get an ICMP reply from 8.8.8.8 when pinging from the Asus. BTW did you try arp -a to check the ARP cache is correct (presumably is if you can see web interface). – Cedric Knight Feb 22 '15 at 15:54
0
when i telnet to my router and do ping -c 1 -w 1 -W 1 212.77.100.101 during incident, the command exits immediately

Of course it exits immediately, you are giving it just 1 count and 1 second to be successful.

   -c count
          Stop after sending count ECHO_REQUEST packets. With deadline option, ping waits
          for count ECHO_REPLY packets, until the timeout expires.

   -w deadline
          Specify a timeout, in seconds, before ping exits regardless of how many packets
          have been sent or received. In this case ping does not stop after count  packet
          are  sent,  it  waits  either  for  deadline  expire  or until count probes are
          answered or for some error notification from network.

   -W timeout
          Time to wait for a response, in seconds. The option  affects  only  timeout  in
          absense of any responses, otherwise ping waits for two RTTs.

But to answer your question, I have seen this behavior many times (at both the ISP level or the individual router level). If I read you correctly, your existing sessions still work during the "incident" but new ones do not. This is caused by either your ISP placing a limit on the amount of sessions an end user can establish, or because the router simply choked on the high amount of sessions established. It could also be the ISP's modem/router having a limit on the amount of sessions per IP. That would explain why your 'computer 2' with another IP continues to work fine.

ISPs place these session limits to prevent malware like bots/trojans/viruses spreading like wildfire and bringing a network to its knees. Just imagine a home user with a 50Mbps connection opening hundreds of thousands of sessions all over the place. Then multiply that by X users and you see how this cascades down into total chaos.

I suggest you retest under a more controlled environment. Maybe just one PC and in safe mode to prevent any strange software from loading that is opening the sessions (maybe a bot/virus/trojan). Then put a permanent ping (no -c or -w). I would be pretty confident that you would not see the loss of connectivity every 15 minutes.

UPDATE

Here is a new test for you to narrow down the issue. Since you say the failure lasts 40 seconds, it should give you plenty of time to do the following. As soon as the failure is detected, change the wan IP of your router (from 192.168.0.10 to maybe 192.168.0.20). If the connectivity is restored, it will prove to you the ISPs modem is limiting sessions based on IP.

  • -c -w parameters are on purpose. i have a script like while true; do ping -U -c 1 -w 1 8.8.8.8 > /dev/null || date; sleep 5; done; so when network goes down i immediately see activity on my screen. and still: why the version with dns address doesn't exit immediately? i disconnected computer 2 and run ubuntu live on computer 1 - still the same effect – piotrek Feb 22 '15 at 14:17
  • the dns version does not exit immediately because it is waiting for the dns server to resolve the name to an IP. Since there is no answer it times out after a while. – Ricardo Feb 22 '15 at 14:55
  • i did the experiment you suggested. first i changed the my router ip 90 seconds before the expected pause. that didn't prevent the pause 90s later. but when i changed the ip during the pause, the connectivity was restored immediately – piotrek Feb 22 '15 at 15:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.