linux tc htb bandwidth ceil being ignored and giving improper results

Question

I'm setting up some htb packet queues like so:

tc qdisc add dev eth0 root handle 10: htb
tc qdisc add dev eth1 root handle 10: htb

tc class add dev eth0 parent 10: classid 10:1012 htb rate 750kbps ceil 750kbps
tc filter add dev eth0 parent 10: protocol ip prio 10 handle 1: cgroup
tc class add dev eth1 parent 10: classid 10:1012 htb rate 750kbps ceil 750kbps
tc filter add dev eth1 parent 10: protocol ip prio 10 handle 1: cgroup

And am testing it by downloading a 10MB file using wget. I have verified that the packets have the correct classid on them.

When I do the download without these tc rules, I get an average download speed of ~ 9MB/s

When I apply the limit, my speed drops, but it drops to the wrong number. It tops off consistently at 3.7MB/s.

I have no idea why my limit is going over by so much especially considering I have a ceil set. I did notice that the download appears to start limited at 750kbps, but then starts incrementily gaining speed.

Any one have any ideas what I'm doing wrong?

Cheers!

Answer 1

Without further knowledge on your actual config I am going to assume you have a cgroup problem.

Try this to exclude your cgroups and just to limit the device itself.

#!/bin/bash

# Using ifb device to shape ingress traffic
modprobe ifb
ifconfig ifb0 up

# flush tc
tc qdisc del dev eth0 root &> /dev/null
tc qdisc del dev eth0 ingress &> /dev/null
tc qdisc del dev ifb0 root &> /dev/null

# limit all outbound traffic
tc qdisc add dev eth0 root handle 1: htb default 1
tc class add dev eth0 parent 1: classid 1:1 htb rate 1mbit

# forward inbound traffic to ifb
tc qdisc add dev eth0 ingress
tc filter add dev eth0 parent ffff: protocol all u32 match u32 0 0 action mirred egress redirect dev ifb0

# limit all inbound traffic via ifb egress
tc qdisc add dev ifb0 root handle 1: htb default 1
tc class add dev ifb0 parent 1: classid 1:1 htb rate 1mbit

Test and you should see that it works both inbound and outbound. If you only want to limit in one direction, just ignore the other part.