Linux Traffic Shaping with "tc": QoS filter for requests directly on the host not applied?

Question

I'm trying to reduce the interface thoughput to debug my application on slower network speed.

I've remembered a tc statement to do this on a interface which was working in the past. I used it on a dedicated (remote) development machine.

This is the statement

 tc qdisc add dev wlan0 root tbf rate 250kbit latency 300ms burst 125kbit

It works well when trying to access the local socket from another host, but when accessing a socket on that interface from my local machine, the filter is not being applied.

Interface:

# ifconfig wlan0
wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet XXX.XXX.XXX:XXX netmask 255.255.255.0  broadcast XXX.XXX.0.255
        inet6 xxxx::xxxx:xxxx:xxxx:xxxx  prefixlen 64  scopeid 0x20<link>
        ether XX:XX:XX:XX:XX:XX  txqueuelen 1000  (Ethernet)
        RX packets 31438  bytes 28025182 (26.7 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 27035  bytes 7482740 (7.1 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Socket (any):

# netstat -lntp | grep 8080
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      5990/nginx: master

Socket (lo/wlan0):

# netstat -lntp | grep 8080
tcp        0      0 127.0.0.1:8080          0.0.0.0:*               LISTEN      7432/nginx: master  
tcp        0      0 XXX.XXX.XXX.XXX:8080      0.0.0.0:*               LISTEN      7432/nginx: master

In the browser I try to access http://XXX.XXX.XXX.XXX:8080/path and the page (>1 MB content + assests) shows immediately (cache cleared before).

On another host that has access to my system all content is loading (visibly) slow.

I looked into other resources of course, but non of them is working for me:

• Port based bandwidth shaping
• Traffic shaping on Linux with HTB: weird results
• http://www.cyberciti.biz/faq/linux-traffic-shaping-using-tc-to-control-http-traffic/
• https://coderwall.com/p/zpckbg

I also tried creating a class using tc and mangling packets using iptables (as shown in the 3rd link) was not possbile on Fedora 19. The rule is ignored (no error when adding the rule, but it does not appear when issuing iptabls -n -L).

So, I suspect the filtering is only applied for packages arriving on a interface or local requests do not pass the filtering layer.

Envrionment:

• Fedora Linux 19
• Kernel 3.11.10-100.fc18.x86_64
• tc utility, iproute2-ss121001
• Nginx is the serving HTTPD

How can I get traffic shaping working for local requests?

(Tips for alternative solutions also welcome. Next I'll try out squid as reverse proxy with delay classes.)

• It should be solved directly on the host
• Additional hardware should not be required
• I prefer solutions not limited to HTTP (to test Websockets or other protocols too)

Answer 1

Packets destined for the local machine will not pass through the network interface. For example, you can flood the IP address of your eth0 interface (lets say 100Mbit link) with 1GBit of traffic while not passing any data through it. Hence the scheduler will not be used.

FWIW, it's much easier to use the netem scheduler for what you're testing instead of tbf. netem is meant to do exactly what you're trying to do. Yet it'll still not work in your case.

There's a very dirty hack that I used to do for tests that involved two interfaces. Let's say you have eth0 and eth1:

• You connect them with each other with a cable
• You assign IP address 10.0.0.1/24 on eth0 and and 10.0.1.2/24 on eth1
• You add NAT rules on both interfaces saying:
  • Traffic exiting eth0 with DST IP 10.0.0.2 is DNATed to 10.0.1.2
  • Traffic exiting eth1 with DST IP 10.0.1.1 is DNATed to 10.0.0.1
• You add static ARP entries saying that:
  • 10.0.0.2 maps to the MAC address of eth1, but associated with interface eth0
  • 10.0.1.1 maps to the MAC address of eth0, but associated with interfaces eth1

Then you may ping 10.0.0.2 and this will exit eth0 and enter back eth1. After that you can add your desired qdisc on the appropriate interface and run your tests.