I am trying to use iptables's u32 module to match the 5 highest bits of the first byte of a UDP packet payload, but it does not match everything.
iptables -A INPUT -p udp -m u32 --u32 "0>>22&0x3C@8&0xFF>>3=0x07" -j DROP
I attempt to jump the IP and UDP header, then use a mask of 0xFF to get only the first byte out of the 4 that iptables returns. Next, I shift the 5 bits I want 3 bits to the right, and compare the value with 0x07.
Given this rule, a UDP packet with the first byte of the payload set to 0x38 matches, but 0x3d does not.