Classification to tc structure example

Introduction

• nftables can replace not even iptables, but it can be used to replace very poorly documented tc filter rules and allow user to classify packets into tc class / qdisc infrastructure.
• There is also support for sets and maps (with much cleaner and more intuitive behavior than in tc filter), which has very good impact on performance in large filtering structures, because they use hashing to get correct value.
• action used to classify packets into tc structure is meta set priority "1:0x2" - 0x can be omitted, but says clearly it is hex number - double quotes are required

• Note that, unlike iptables or tc filter, you can perform several actions in one single rule and match several informations - counter is must have for debugging
• You will have to redesign your rule structure to benefit from these new nice features.
• example:

meta priority none ip saddr @priority_set meta priority set ip daddr map @group_114_prio counter packets 0 bytes 0

• packet:
  • source address 8.8.8.8
  • destination address 10.20.255.50
  • meta priority none - matches packet only when there is no priority - tc class id - set yet
  • ip saddr @priority_set - matches packet only when source IP address is listed in the set named priority_set - in our case 8.8.8.8 or 8.8.4.4 - can be subnet too
  • meta priority set ip daddr map @group_114_prio - sets priority to packet based on its destination address, which is read from map named group_114_prio - sets priority to 1:ffd9

Basic prototypes

tc class structure

+---(1:) hfsc
     +---(router:root#1:1) hfsc ls m1 0bit d 0us m2 524288Kbit ul m1 0bit d 0us m2 524288Kbit
          +---(router:shaper#1:2) hfsc ls m1 0bit d 0us m2 8192bit ul m1 0bit d 0us m2 524288Kbit
          +---(router:10.20.0.11#1:3) hfsc ls m1 0bit d 0us m2 1432bit ul m1 0bit d 0us m2 524288Kbit                                                                                                                                          
          |    +---(group:114#1:72) hfsc ls m1 0bit d 0us m2 240bit ul m1 0bit d 0us m2 62914Kbit                                                                                                                                              
          |    |    +---(shape:1141#1:ffda) hfsc rt m1 0bit d 0us m2 1572Kbit ls m1 80bit d 12.0s m2 56bit ul m1 62914Kbit d 360.0s m2 31457Kbit                                                                                               
          |    |    |    +---(prio:1141#1:ffd9) hfsc ls m1 160bit d 24.0s m2 112bit                                                                                                                                                            
          |    |    |    +---(normal:1141#1:ffd8) hfsc ls m1 80bit d 12.0s m2 56bit                                                                                                                                                            
          |    |    |                                                                                                                                                                                                                          
          |    |    +---(shape:1143#1:ffd4) hfsc rt m1 0bit d 0us m2 1572Kbit ls m1 80bit d 12.0s m2 56bit ul m1 62914Kbit d 360.0s m2 31457Kbit                                                                                               
          |    |    |    +---(prio:1143#1:ffd3) hfsc ls m1 160bit d 24.0s m2 112bit                                                                                                                                                            
          |    |    |    +---(normal:1143#1:ffd2) hfsc ls m1 80bit d 12.0s m2 56bit                                                                                                                                                            
          |
          |
          +---(router:untracked#1:ffff) hfsc ls m1 0bit d 0us m2 16bit ul m1 0bit d 0us m2 10485Kbit

tc qdisc structure

qdisc hfsc 1: root refcnt 2 default 2
qdisc fq_codel 2: parent router:shaper#1:2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
qdisc fq_codel ffd2: parent normal:1143#1:ffd2 limit 1024p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
qdisc fq_codel ffd3: parent prio:1143#1:ffd3 limit 1024p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
qdisc fq_codel ffd8: parent normal:1141#1:ffd8 limit 1024p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
qdisc fq_codel ffd9: parent prio:1141#1:ffd9 limit 1024p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
qdisc fq_codel ffff: parent router:untracked#1:ffff limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn

nftables structure

table ip filter {
   map subnet_map {
       type ipv4_addr : verdict
       flags interval
       elements = { 10.20.255.48/29 : goto group_114, 10.20.255.88/29 : goto group_114,
                    10.20.255.128/29 : goto group_114 }
   }
 
   set priority_set {                                                                                                                                                                                                                         
       type ipv4_addr
       flags interval
       elements = { 8.8.8.8, 8.8.4.4 }
   }   
 
   map group_114 {
       type ipv4_addr : classid
       flags interval
       elements = { 10.20.255.50 : 1:ffd8, 10.20.255.90 : 1:ffd5,
                     10.20.255.130 : 1:ffd2 }
   }   
 
   map group_114_prio {
       type ipv4_addr : classid
       flags interval
       elements = { 10.20.255.50 : 1:ffd9, 10.20.255.90 : 1:ffd6,
                     10.20.255.130 : 1:ffd3 }
   }   
 
   chain forward {
       type filter hook forward priority filter; policy accept;
       meta priority none ip daddr vmap @subnet_map counter packets 0 bytes 0
       meta priority none ip saddr vmap @subnet_map counter packets 0 bytes 0
       ip daddr 192.168.0.0/16 meta priority none meta priority set 1:ffff counter packets 0 bytes 0 log prefix "total - " 
       ip saddr 192.168.0.0/16 meta priority none meta priority set 1:ffff counter packets 0 bytes 0 log prefix "total - " 
       ip daddr 10.0.0.0/8 meta priority none meta priority set 1:ffff counter packets 38931 bytes 2926076 log prefix "total - " 
       ip saddr 10.0.0.0/8 meta priority none meta priority set 1:ffff counter packets 14 bytes 1064 log prefix "total - " 
       meta priority none meta priority set 1:2 counter packets 0 bytes 0 log prefix "non_shaped - " 
   }   
 
   chain input {
       type filter hook input priority filter; policy accept;
       meta priority none meta priority set 1:2 counter packets 419381 bytes 45041195
   }   
 
   chain output {
       type filter hook output priority filter; policy accept;
       meta priority none meta priority set 1:2 counter packets 507779 bytes 51809859
   }   
 
   chain group_114 {
       meta priority none ip saddr @priority_set meta priority set ip daddr map @group_114_prio counter packets 0 bytes 0
       meta priority none ip daddr @priority_set meta priority set ip saddr map @group_114_prio counter packets 0 bytes 0
       meta priority none meta priority set ip daddr map @group_114 counter packets 0 bytes 0
       meta priority none meta priority set ip saddr map @group_114 counter packets 0 bytes 0
       meta priority none meta priority set 1:ffff counter packets 0 bytes 0 log prefix "group_114 - "
   }
}

nftables commands

• executed using nft -f filename.nft - or using function nft_run_cmd_from_buffer - libnftables

add table ip filter
add chain ip filter forward { type filter hook forward priority 0; policy accept; }
add map ip filter subnet_map { type ipv4_addr : verdict; flags interval; }
add set ip filter priority_set { type ipv4_addr; flags interval; }
add element ip filter priority_set {8.8.8.8 }
add element ip filter priority_set {8.8.4.4 }
add rule ip filter forward meta priority 0 ip daddr vmap @subnet_map counter
add rule ip filter forward meta priority 0 ip saddr vmap @subnet_map counter
add rule ip filter forward ip daddr 192.168.0.0/16 meta priority 0 meta priority set "1:0xffff" counter log prefix "total - " 
add rule ip filter forward ip saddr 192.168.0.0/16 meta priority 0 meta priority set "1:0xffff" counter log prefix "total - " 
add rule ip filter forward ip daddr 10.0.0.0/8 meta priority 0 meta priority set "1:0xffff" counter log prefix "total - " 
add rule ip filter forward ip saddr 10.0.0.0/8 meta priority 0 meta priority set "1:0xffff" counter log prefix "total - " 
add rule ip filter forward meta priority 0 meta priority set "1:0x2" counter log prefix "non_shaped - " 
add chain ip filter input { type filter hook input priority 0; policy accept; }
add rule ip filter input meta priority 0 meta priority set "1:0x2" counter
add chain ip filter output { type filter hook output priority 0; policy accept; }
add rule ip filter output meta priority 0 meta priority set "1:0x2" counter                                                                                                                                                                    
add chain ip filter group_114
add map ip filter group_114 { type ipv4_addr : classid; flags interval; }
add map ip filter group_114_prio { type ipv4_addr : classid; flags interval; }
add rule ip filter group_114 meta priority 0 ip saddr @priority_set meta priority set ip daddr map @group_114_prio counter
add rule ip filter group_114 meta priority 0 ip daddr @priority_set meta priority set ip saddr map @group_114_prio counter
add rule ip filter group_114 meta priority 0 meta priority set ip daddr map @group_114 counter
add rule ip filter group_114 meta priority 0 meta priority set ip saddr map @group_114 counter
add rule ip filter group_114 meta priority 0 meta priority set "1:0xffff" counter log prefix "group_114 - " 
add element ip filter subnet_map { 10.20.255.48/29 : goto group_114 }
add element ip filter subnet_map { 10.20.255.88/29 : goto group_114 }
add element ip filter subnet_map { 10.20.255.128/29 : goto group_114 }
add element ip filter group_114_prio { 10.20.255.50/32 : "1:0xffd9" }
add element ip filter group_114 { 10.20.255.50/32 : "1:0xffd8" }
add element ip filter group_114_prio { 10.20.255.90/32 : "1:0xffd6" }
add element ip filter group_114 { 10.20.255.90/32 : "1:0xffd5" }
add element ip filter group_114_prio { 10.20.255.130/32 : "1:0xffd3" }
add element ip filter group_114 { 10.20.255.130/32 : "1:0xffd2" }

Packet processing

chain forward

   # packet passing through server
   chain forward {
       # hook forward does the magic, not the name of the chain
       # priority filter can be used in newer versions of nftables > 0.9.0
       type filter hook forward priority filter; policy accept;
       # packet is matched against subnet_map - it is verdict map = 10.20.255.48/29 : goto group_114
       meta priority none ip daddr vmap @subnet_map counter packets 0 bytes 0 # packet's dst address is looked up
       # it contains decision on where to send the packet for further processing when matched - chain group_114
       meta priority none ip saddr vmap @subnet_map counter packets 0 bytes 0 # packet's src address is looked up
       # private destination subnet without set priority is set to 1:0xffff
       ip daddr 192.168.0.0/16 meta priority none meta priority set 1:ffff counter packets 0 bytes 0 log prefix "total - "
       # private source subnet without set priority is set to 1:0xffff
       ip saddr 192.168.0.0/16 meta priority none meta priority set 1:ffff counter packets 0 bytes 0 log prefix "total - " 
       ip daddr 10.0.0.0/8 meta priority none meta priority set 1:ffff counter packets 38931 bytes 2926076 log prefix "total - " 
       ip saddr 10.0.0.0/8 meta priority none meta priority set 1:ffff counter packets 14 bytes 1064 log prefix "total - " 
       # rest of traffic is sent to separate tc class object
       meta priority none meta priority set 1:2 counter packets 0 bytes 0 log prefix "non_shaped - "
   }

chain group_114

   # subnet_map redirected the packet here
   chain group_114 {
       # packet's source / destination address is matched against set named priority_set and it can't contain any priority set
       meta priority none ip saddr @priority_set meta priority set ip daddr map @group_114_prio counter packets 0 bytes 0
       # when matched it compares destination address of the packet against group_114_prio map and sets the priority accordingly - 1:ffd9
       meta priority none ip daddr @priority_set meta priority set ip saddr map @group_114_prio counter packets 0 bytes 0
       # packets heading / originating to / from non prioritized addresses are matched in next steps
       meta priority none meta priority set ip daddr map @group_114 counter packets 0 bytes 0
       meta priority none meta priority set ip saddr map @group_114 counter packets 0 bytes 0
       # unknown traffic is set to untracked object - 1:0xffff
       meta priority none meta priority set 1:ffff counter packets 0 bytes 0 log prefix "group_114 - "
   }
 
   map group_114 {
       type ipv4_addr : classid
       flags interval
       elements = { 10.20.255.50 : 1:ffd8, 10.20.255.90 : 1:ffd5,
                     10.20.255.130 : 1:ffd2 }
   }   
 
   map group_114_prio {
       type ipv4_addr : classid
       flags interval
       elements = { 10.20.255.50 : 1:ffd9, 10.20.255.90 : 1:ffd6,
                     10.20.255.130 : 1:ffd3 }
   }

Additional documentations and articles

• this article can help a lot during definition of redesigned rule structure, it shows performance with different setup of rules, yet it tels almost nothing about classification itself
• https://developers.redhat.com/blog/2017/04/11/benchmarking-nftables/

• very good source of information is man page - it can be generated as pdf - in source code nftables/doc/build_pdfs.sh
• there is not much information about tc classification