Bandwidth limit for multiple network destinations with a default (slow) match with Linux TC

Question

I'm playing with Linux traffic control features to restrict KVM VM output bandwidth and got stuck while using filters to match destinations for a fast class and for a (default) slow class for a given VM.

The physical network output interface is "eth0" (1 Gbit) and is part of "bond0" bonding device. "bond0" is part of "br0" along the virtual machines virtual network cards "vmn{0,1,2,3,4}".

The path from a VM network packet to the Internet is: eth0 (vm) -> vmn0 (host) -> br0 (host) -> bond0 (host) -> nic0 (host) -> The Internet.

I'm trying to set two classes:

• Fast: 500 Mbps outgoing to destinations 10.0.1.0/24 and 10.0.2.0/24
• Slow (default): 100 Mbps outgoing to every other destination

I have tried the following approach on the host:

# If the qdisc was previously created, delete it
tc qdisc del dev vmn0 root

# New HTB qdisc. By default it goes to our slow traffic class (1:11)
tc qdisc add dev vmn0 root handle 1: htb default 11

# Classes. 1:1 default (500 Mbit) ; 1:10 -> fast (500Mbit); 1:11 -> slow (100Mbit)
tc class add dev vmn0 parent 1: classid 1:1 htb rate 500000kbit ceil 500000kbit quantum 1500
tc class add dev vmn0 parent 1:1 classid 1:10 htb rate 1000kbit ceil 500000kbit
tc class add dev vmn0 parent 1:1 classid 1:11 htb rate 1000kbit ceil 100000kbit

# Associates the fast traffic class with our desireed networks
tc filter add dev vmn0 parent 1: protocol ip prio 10 u32 match ip dst 10.0.1.0/24 flowid 1:10
tc filter add dev vmn0 parent 1: protocol ip prio 10 u32 match ip dst 10.0.2.0/24 flowid 1:10

# Default match if any of previous rules not applied to a slow rate 1:11. Superfluous like traffic goes to 1:11 by default?
tc filter add dev vmn0 parent 1: protocol ip prio 20 u32 match ip dst 0.0.0.0/0 flowid 1:11

From the VM "vma" with network device "vmn0" associated and IP address "10.0.1.2" I ran "iperf -s".

From a VM "vmb" on a different host and IP address "10.0.2.2" I ran "iperf -c 10.0.1.2" with the following results:

[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  57.8 MBytes   485 Mbits/sec    0   1.84 MBytes       
[  4]   1.00-2.00   sec  56.2 MBytes   472 Mbits/sec    0   1.84 MBytes       
[  4]   2.00-3.00   sec  56.2 MBytes   472 Mbits/sec    0   1.84 MBytes       
[  4]   3.00-4.00   sec  55.0 MBytes   461 Mbits/sec    0   1.84 MBytes       
[  4]   4.00-5.00   sec  56.2 MBytes   472 Mbits/sec    0   1.84 MBytes       

Looks good, "10.0.2.2" is rate limited to ~500 Mbit as it should.

Now, from a VM "vmc" on a different host again and IP address "192.168.1.2":

[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  57.3 MBytes   481 Mbits/sec    0   1.21 MBytes       
[  4]   1.00-2.00   sec  56.4 MBytes   473 Mbits/sec    0   1.21 MBytes       
[  4]   2.00-3.00   sec  56.7 MBytes   476 Mbits/sec    0   1.21 MBytes       
[  4]   3.00-4.00   sec  56.7 MBytes   476 Mbits/sec    0   1.21 MBytes       
[  4]   4.00-5.00   sec  56.5 MBytes   474 Mbits/sec    0   1.21 MBytes  

This isn't what I expected, as it should got ~100 Mbit (default class, and a match for any not matched previously).

Can anyone help with this?

Answer 1

Finally got it working.

These lines show what I'm doing:

tc qdisc del dev vmn0 root
tc qdisc add dev vmn0 root handle 1: htb default 11 r2q 83
tc class add dev vmn0 parent 1: classid 1:1 htb rate 100000kbit ceil 100000kbit quantum 1500
tc class add dev vmn0 parent 1:1 classid 1:10 htb rate 1000kbit ceil 500000kbit
tc class add dev vmn0 parent 1:1 classid 1:11 htb rate 1000kbit ceil 100000kbit
tc filter add dev vmn0 parent 1: protocol ip prio 10 u32 match ip src 10.0.2.0/24 flowid 1:10

After removing the filter on dst 10.0.1.0/24 traffic started to be rate limited. As expected if traffic isn't matched by the filter rule then it goes to the default class (1:11, slow one).

Traffic was always matched on rule 10.0.1.0/24 as it both performed sends and receives, so traffic went classified there and given the fast path.