Traffic accounting with nftables or iptables via netlink and nflog
Category: Linux Tags: Traffic Accounting, Netlink, NFLog, IPTables, NFTables
A C Project to account traffic selected in IPTables or NFTables via netlink and nflog.
If you just want the code, it's hosted at github.
Preface
I am running a custom-built router running Gentoo Linux for internet access.
I like to know who (ip address) is transferring how much data (bytes) at a given period.
Of course there tools for this.
So far used a combination of static firewall rules (per ip), ipt_account and pmacct (uacctd).
Static rules per ip are bad, as you need static ips for this and need to have rules for every ip.
ipt_account is obsolete and also would not work with nftables.
PMAcct is very powerful tool, with a lot of plugins and configuration options, but also needs a lot of computing power.
I got 500/50 internet connection and if there is high traffic (kids are playing online games, someone is downloading something, wife is watching movies via streaming service, etc.) it eats up a whole core of the quadcore processor (I admit it is a > 10 year old i5).
All this for the simple traffic accounting, I wanted to realize.
So I tried to build something on my own.
Goals
- account traffic selected by iptables or nftables.
- traffic should be dynamically accumulated per ip
- traffic data should be available for external tools (to store into a database or else)
Solution
IPTables as well as NFTables are able to log packet information via an special NFLOG-Target.
There exists a communication interface "netlink" to transfer data between kernel and user space application, which can be used to aquire the data from nflog.
In fact pmacct (uacctd in this case) is getting the input data via netlink from nflog.
Build
Well I hope your familiar with github ... so first get the code from github.
Application is pure C code. I use a simple Makefile for building, no autoconf or cmake (so far).
Application depends on libnfnetlink and libnetfilter_log
Just run make
and there should be a nflog_account binary.
If there are any issues feel free to contact me via github.
Usage
First send the traffic you want to account to nflog.
You should be familiar with iptables/nftables to do so.
# iptables -A <table> <traffic selection> -j NFLOG --nflog-group <group>
The vital part is the group here and a 16bit unsigned int. See manpage
Next start nflog_account
# nflog_account -n <net/mask> -s <socket file> -g <nflog group>
Net/mask is a network definition for which you want to account traffic per ip for.
Socket file is unix domain socket which makes the accounted data available for further processing.
Nflog group is the very group you used in the firewall rule to select traffic.
Multiple network definitions (up to 5) and multiple sockets (up to 5) are possible. Network definitions are independent of sockets. So all traffic is accounted in all sockets. If you would like to account traffic for multiple networks separately, you should start multiple instances.
Example:
# nflog_account -g 23 -n 192.168.0.0/16 -n 172.16.0.0/20 -s /var/run/nflog_account.socket -s /var/run/nflog_account2.socket
Now you can use a third party application to get the account data from the sockets. There are two commands to interact with the sockets.
- stats for simply getting the traffic data
- flush for getting the traffic and flush the data afterward
Netcat example:
ncat -U /var/run/nflog_account.socket
> stats
< 192.168.2.32 3178 299473 27670 29515977 1749825990 1749826036
< 192.168.2.22 80 8136 47 10101 1749825990 1749826035
< 192.168.2.2 48 4589 42 5456 1749825990 1749826035
< 192.168.2.42 1 76 1 76 1749826035 1749826035
stats is the command send via netcat, everything else is the reply from nflog_account.
Data format is:
<ip> <packets source> <bytes source> <packets destination> <bytes destination> <first seen> <last seen>
Socket connection is immediatly closed after executing a command. Traffic is accounted per given socket (sockets are flushed independently).
There is no option to daemonize the application (go into background).
With openrc it is very easy to setup an init-script that takes care of putting the app into background.
Same should be possible with systemd.
Conclusion
That's basically it. As you can see in the example, I use 2 sockets.
One is used to store traffic data in fixed intervals in database for statistics like this:
The other is used as data source for a websocket to display live traffic data.
Next steps
Flush traffic per ip.
I would like to flush data for ips independently. If there is no traffic for a given period for a certain ip, its and only its data shall be flushed. It would result in some kind of transfer session per ip.
More precise timestamps for first and last seen
To calculate the currently used bandwidth in short intervals (e.g. for live traffic graph) precise timestamps are needed.
So far I got those timestamps from a python script that processes the data, but it would be better if there are already precise timestamps in the data itself.
I think getimeofday() could be the way to go.
Document the inner workings of nflog_account
I am not a C developer. I am mostly doing php and javascript. So It was a big thing to get into C. Also I had to do a lot of research of how to do things.
Maybe documenting on the code could help other developers as well as might uncover errors I made.