I am working on assuring low delay for UDP traffic at the home gateway level. At this home gateway I have two types of traffic, TCP and UDP, and I assure differentiated treatment by using HTB. The bandwidth I am testing equals 1Mbit/s (thus it is connected to emulated ADSL 1Mbit link).

I use the following configuration:

tc qdisc add dev br0 handle 1: root htb default 15
tc class add dev br0 parent 1: classid 1:1 htb rate 1000kbit ceil 1000kbit
tc class add dev br0 parent 1:1 classid 1:14 htb rate 200kbit ceil 1000kbit prio 1
tc class add dev br0 parent 1:1 classid 1:15 htb rate 10bit ceil 1000kbit prio 2
tc filter add dev br0 parent 1: protocol ip u32 match ip tos 0xb8 0xff flowid 1:14 (UDP)
tc filter add dev br0 parent 1: protocol ip u32 match ip tos 0x00 0xff flowid 1:15 (TCP)
tc qdisc add dev br0 parent 1:14 handle 20: sfq limit 40
tc qdisc add dev br0 parent 1:15 handle 50: pfifo limit 1000 

In order to see how the traffic interacts, for UDP I use a "stairs" type of traffic, thus I start at 0bit/s and I increase the traffic every ten seconds by 100kbit/s. When I reach 1Mbit/s I decrease every 10s by 100kbit/s until I reach zero.

Alongside, I have TCP traffic, either a file upload, either a simple iperf (without any influence on observed behavior).

Normally, most of the time, I get an expected behavior. Thus I can see perfectly the TCP/UDP traffic separation and the “stairs” trend of the UDP. Additionally, UDP traffic takes over TCP (but TCP can still send – and the trend is the opposite of UDP, thus first decreasing, later increasing).

However from time to time I can see TCP completely taking over UDP traffic for a couple of seconds. It happens between 1 and 3 times during the test. In general it happens at least once, when UDP bitrate is already decreasing (about 30 seconds before reaching 0), TCP traffic completely takes over for a couple of seconds (3 to 5 seconds). I can’t really understand this behavior, because it seems that UDP traffic cannot send, but it shouldn’t be above its ceil since its bitrate is already decreasing.

I think it has something to do with HTB scheduling and blocking high priority traffic for some reason. Thus when rate is calculated for my high priority traffic, it somehow is higHer than its ceil.

  • But why would high priority traffic reach its ceil if the bitrate is lower?
  • Why could low priority traffic block the other one for several seconds, even though its assured rate is close to zero?

I have tried changing different parameters (burst, ceil, rates), but cannot see any improvement.

Thank you in advance for your help.

  • Hi, I'm trying to reproduce the issue. Can you please tell me which tool you are using to generate udp "stairs" traffic, what is your methodology to mesure bandwidth, and verify the queue length of br0 with 'ip link' ? – Sanael Apr 16 '16 at 11:10
  • Hello, for generating udp, I am just using iperf. Thus I increase the udp bandwidth every 10s. And to measure bandwidth I use Wireshark, thus I check it with Stats/IO Graphs (by using filter udp for the first graph and tcp for the second one). And I do not verify the queue length though. I just check the tc stats for that. Hope it helps. – FasolkaPoBretonsku Apr 18 '16 at 8:30
  • I'm sorry, I failed to reproduce your issue yet with your settings. I have the normal curves for both udp and tcp, though I did multiple 10 minutes tests with stairs traffic. – Sanael Apr 25 '16 at 13:46
  • Thank you. I've tried to debug this with Wireshark, and at some point I get SACK messages - it is random though. I think it is linked to the fact that I have a packet that is lost somewhere on the way, and since the queue is very long, the sender takes long time to figure it out. I also observe the congestion window (or bytes on the fly) value increase, thus since no packets are lost for TCP (given HTB properties), the sender does not know immediately that there is a congestion, it does not decrease its congestion window and it continues to send lots of packets. Do not know if it makes sense? – FasolkaPoBretonsku May 3 '16 at 8:43
  • Not really. Your idea would totally make sense if TCP was prio 1, but udp is: whatever the pfifo length, the congestion window, or the SACK option are, htb won't let tcp send. – Sanael May 24 '16 at 14:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.