Skip to content

Instantly share code, notes, and snippets.

@Kishi85

Kishi85/nft2ipset

Last active December 6, 2025 15:45
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save Kishi85/b7f379f9aa19f4878af28b8e1a8887ab to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save Kishi85/b7f379f9aa19f4878af28b8e1a8887ab to your computer and use it in GitHub Desktop.
Download ZIP
/etc/init.d/nft2ipset: An nftables set to ipset synchronizer for use with OpenWRT/mwan3
Raw
nft2ipset
#!/bin/sh /etc/rc.common
# Start before firewall and mwan3 which are at Prio 19
START=18
APP=nft2ipset
USE_PROCD=1
SCRIPTPATH="/tmp/nft2ipset"
write_script() {
cat > "$1" <<'EOT'
#!/bin/sh
#check if the script is already running
PID=$$
SCRIPT="$(basename $0)"
TMPDIR="/tmp"
MONITORPIDFILE="$TMPDIR/$SCRIPT-$$.nftmonitorpid"
MONITORFIFO="$TMPDIR/$SCRIPT-$$.nftmonitorfifo"
mkfifo "$MONITORFIFO"
cleanup () {
# Cleanup nft monitor subprocess
if [ -f "$MONITORPIDFILE" ]; then
MONITORPID="$(cat "$MONITORPIDFILE")"
if [ "$MONITORPID" -gt 1 ]; then
kill "$MONITORPID"
fi
fi
# Remove pid file and fifo
rm "$MONITORFIFO" "$MONITORPIDFILE"
}
trap cleanup TERM INT EXIT QUIT ABRT
create_or_update_ipset() {
# Determine ipset parameters
local DEF="$1"
local NAME="$(echo "$DEF" | cut -d' ' -f2)"
local OPTS=""
local FAMILY="inet"
if echo "$DEF" | grep -q "ipv6_addr"; then
FAMILY="inet6"
OPTS="$OPTS family $FAMILY"
fi
local TIMEOUT="$(echo "$DEF" | sed -r 's/.*timeout ([0-9]*)s.*/\1/; t; s/.*/0/')"
if [ -n "$TIMEOUT" -a "$TIMEOUT" -gt 0 ]; then
OPTS="$OPTS timeout $TIMEOUT"
fi
# Create or update ipset from nftables set
if [ "$(ipset list -n "$NAME")" = "$NAME" ]; then
CUR="$(ipset list -t "$NAME")"
if ! ( echo "$CUR" | grep -q "family $FAMILY"); then
( ipset destroy "$NAME" 2>&1 | logger -t "$SCRIPT" ) || logger -t "$SCRIPT" "WARNING: Could not destroy ipset with family != $FAMILY"
elif ! ( echo "$CUR" | grep -q "timeout $TIMEOUT"); then
# Swap current iteration of the ipset with a new iteration due to timeout mismatch
ipset create "_$NAME" hash:ip $OPTS
ipset swap "_$NAME" "$NAME"
ipset destroy "_$NAME"
logger -t "$SCRIPT" "Replaced ipset $NAME with new iteration with timeout $TIMEOUT"
fi
fi
if [ "$(ipset list -n "$NAME")" != "$NAME" ]; then
# Create a new ipset with options matching the nftables set
ipset create "$NAME" hash:ip $OPTS
# Restart mwan3 if this ipset is used by it, it is already running but the set name is not found in active rule output
if [ $? = 0 ] && grep -q "option ipset '$NAME'" /etc/config/mwan3 2>/dev/null && ( service | grep mwan3 | grep running ) && ( ! (mwan3 rules | grep -q "match-set $NAME" ) ); then
mwan3 restart
fi
logger -t "$SCRIPT" "Created new ipset $NAME with timeout $TIMEOUT"
fi
# Add already existing entries to the set
echo "$DEF" | sed -re 's/.*elements = \{ ([^\}]+) \}.*/\1/g; t; s/.*//g' | tr ',' '\n' | sed -re 's/^[ ]+//g;s/expires/timeout/g;s/s$//g' | while read LINE; do
if [ -n "$LINE" ]; then
ipset -q add "$NAME" $LINE && logger -t "$SCRIPT" "Added $LINE to $NAME upon ipset creation/update" || true
fi
done
}
# Check if ipsets exist for all currently existing nftsets or create otherwise
nft -nT list sets | tr '\n' ' ' | awk '{$1=$1;print}' | sed -r 's/ (set|table)/\n\1/g' | grep "^set" | while read DEF; do
NAME="$(echo "$DEF" | cut -d' ' -f2)"
if [ -z "$NAME" ] || ! grep -q "option ipset '$NAME'" /etc/config/mwan3; then
logger -t "$SCRIPT" "Ignored set '$NAME' as it is invalid or not used by mwan3"
else
create_or_update_ipset "$DEF"
fi
done
# Monitor nftables rule changes
nft -nT monitor > "$MONITORFIFO" 2>&1 &
echo $! > "$MONITORPIDFILE"
while read LINE; do
# Update ipsets according to specified nft monitor option (this should be always one operation per line)
if echo "$LINE" | grep -q "add element inet"; then
NAME="$(echo "$LINE" | cut -d' ' -f 5)"
if [ -z "$NAME" ] || ! grep -q "option ipset '$NAME'" /etc/config/mwan3; then
logger -t "$SCRIPT" "Ignored set '$NAME' as it is invalid or not used by mwan3"
else
# Check if ipset exists or create otherwise
if [ "$(ipset list -n $NAME)" != "$NAME" ]; then
DEF="$(nft -tnT list sets | tr '\n' ' ' | awk '{$1=$1;print}' | sed -r 's/ (set|table)/\n\1/g' | grep "^set $NAME")"
create_or_update_ipset "$DEF"
fi
# Add element to ipset
IP="$(echo "$LINE" | cut -d' ' -f 7)"
EXPIRES="$(echo "$LINE" | sed -re 's/.*expires ([0-9]+)s.*/\1/; t; s/.*/0/')"
ADDOPTS=""
if [ $EXPIRES -gt 0 ]; then
ADDOPTS="timeout $EXPIRES"
fi
if ipset -q test "$NAME" "$IP"; then
# Refresh the entry by deleting it first if already existing
ipset -q del "$NAME" "$IP"
ipset -q add "$NAME" "$IP" $ADDOPTS
else
ipset -q add "$NAME" "$IP" $ADDOPTS
logger -t "$SCRIPT" "Added $IP to ipset $NAME $ADDOPTS"
fi
fi
elif echo "$LINE" | grep -q "add set inet"; then
NAME="$(echo "$LINE" | cut -d' ' -f 5)"
if [ -z "$NAME" ] || ! grep -q "option ipset '$NAME'" /etc/config/mwan3; then
logger -t "$SCRIPT" "Ignored set '$NAME' as it is invalid or not used by mwan3"
else
# Create or update ipset
DEF="$(nft -nT list sets | tr '\n' ' ' | awk '{$1=$1;print}' | sed -r 's/ (set|table)/\n\1/g' | grep "^set $NAME")"
create_or_update_ipset "$DEF"
fi
elif echo "$LINE" | grep -q "delete set inet"; then
NAME="$(echo "$LINE" | cut -d' ' -f 5)"
if [ -z "$NAME" ] || ! grep -q "option ipset '$NAME'" /etc/config/mwan3; then
logger -t "$SCRIPT" "Ignored set '$NAME' as it is invalid or not used by mwan3"
else
# Clear and try to delete removed ipset (This will fail if it is in use by any iptables rule)
ipset clear "$NAME"
ipset destroy "$NAME" 2>&1 | logger -t "$SCRIPT"
fi
fi
done < "$MONITORFIFO"
EOT
}
start_service() {
write_script "$SCRIPTPATH"
chmod +x "$SCRIPTPATH"
procd_open_instance
procd_set_param command "$SCRIPTPATH"
procd_set_param respawn
procd_set_param pidfile /var/run/nft2ipset.pid
procd_close_instance
}
service_stopped() {
rm "$SCRIPTPATH"
}
# vim: ts=2 sw=2 et
@Kishi85
Copy link
Author

Kishi85 commented Jul 13, 2025

The real question is if there are multiple /tmp/nft2ipset-[0-9]+.monitorpid files and corresponding processes in the process list as that would indicate the script running simultaneously

Latest version seems to work properly now

  • no multiple process seen via ps w | grep -i nft
  • none mwan3 ipset use, ignored properly now

You mentioned the monitorpid.. I saw multiple files before re-testing and not sure if it had to do with previous mentioned issue (if it was the cause or an output of it) of multiple process spawned but deleted all of them before testing

Glad to hear that it works now as intended on your end as well.

The multiple monitor files could have been remnants of crashes of the script as the cleanup trap would not have been executed in that case. Not a problem as long as the pid that is saved inside the file is no longer existing with the orphaned nft monitor process.

@AcidSlide
Copy link

AcidSlide commented Jul 19, 2025 •
edited
Loading

@Kishi85 I updated my router today and somehow on my edge case, after a boot/reboot i've got multiple nft2ipset running

root@mt6000 /root [#]# ps w | grep -i nft
 1850 root      1452 S    {nft2ipset} /bin/sh /tmp/nft2ipset
 1959 root      6552 S    nft -nT monitor
20239 root      1980 S    grep --color=auto -iE post|oom|ntpd|TestNotify|nft2ipset
20328 root      1452 S    {nft2ipset} /bin/sh /tmp/nft2ipset
20329 root     46868 R    nft -nT list sets
20413 root      1812 S    grep --color=auto -i nft

root@mt6000 /root [#]# ls -laht /tmp/nft2ipset*
prw-r--r--    1 root     root           0 Jul 19 09:21 /tmp/nft2ipset-20512.nftmonitorfifo
-rwxr-xr-x    1 root     root        5.2K Jul 19 09:21 /tmp/nft2ipset
prw-r--r--    1 root     root           0 Jul 19 09:19 /tmp/nft2ipset-1850.nftmonitorfifo
-rw-r--r--    1 root     root           5 Jul 19 09:18 /tmp/nft2ipset-1850.nftmonitorpid

i'm still investigating how it's happening.. but so far it's only happening after a reboot/boot

Testing Update: ok found the issue.. doing a /etc/init.d/nft2ipset restart was the culprit. Doing a stop then start doesn't do this. It's only the restart is the issue.

Test Update #2: This is weird.. it's somehow a hit/miss thing. Mostly it happens after a reboot/boot. Or a timing issue?? When using restart.

@Kishi85
Copy link
Author

Kishi85 commented Jul 19, 2025

@Kishi85 I updated my router today and somehow on my edge case, after a boot/reboot i've got multiple nft2ipset running

root@mt6000 /root [#]# ps w | grep -i nft
 1850 root      1452 S    {nft2ipset} /bin/sh /tmp/nft2ipset
 1959 root      6552 S    nft -nT monitor
20239 root      1980 S    grep --color=auto -iE post|oom|ntpd|TestNotify|nft2ipset
20328 root      1452 S    {nft2ipset} /bin/sh /tmp/nft2ipset
20329 root     46868 R    nft -nT list sets
20413 root      1812 S    grep --color=auto -i nft

root@mt6000 /root [#]# ls -laht /tmp/nft2ipset*
prw-r--r--    1 root     root           0 Jul 19 09:21 /tmp/nft2ipset-20512.nftmonitorfifo
-rwxr-xr-x    1 root     root        5.2K Jul 19 09:21 /tmp/nft2ipset
prw-r--r--    1 root     root           0 Jul 19 09:19 /tmp/nft2ipset-1850.nftmonitorfifo
-rw-r--r--    1 root     root           5 Jul 19 09:18 /tmp/nft2ipset-1850.nftmonitorpid

i'm still investigating how it's happening.. but so far it's only happening after a reboot/boot

Testing Update: ok found the issue.. doing a /etc/init.d/nft2ipset restart was the culprit. Doing a stop then start doesn't do this. It's only the restart is the issue.

Test Update #2: This is weird.. it's somehow a hit/miss thing. Mostly it happens after a reboot/boot. Or a timing issue?? When using restart.

That is odd indeed and seems like something wrong with the procd parameters as the service and script should only be able to be started once. The script itself should be fine here but somehow the service starts twice? Would explain the restart behaviour as well. This could also be an odd racecondition in procd for the restart case?

Anyway, I've added an explicit pid file parameter to the procd configuration of the script so the real pid of the script gets stored somewhere for procd to reference. Maybe procd is just loosing track of the script pid it started without that for some odd reason? Then this should help, otherwise at least no further harm is done.

@antoniovalenzuela
Copy link

antoniovalenzuela commented Aug 13, 2025 •
edited
Loading

Works OpenWrt 24.10.2

Replace dnsmaq to dnsmasq-full to suppor nftset

  1. /luci/admin/network/firewall/ipsets
imagen
  1. /luci/admin/network/dhcp
    Necessary Family IPv4+6 (does not work only with IPv4)
imagen
  1. /luci/admin/network/mwan3/rule
    handwritten ipset
imagen
  1. /etc/init.d/nft2ipset start

Test
nft list set inet fw4 filtrado
imagen

ipset list filtrado
imagen

Works perfectly with the mwan3 rule

@tkkost
Copy link

tkkost commented Dec 6, 2025

Hello.
Excuse me. Why did you use hash:ip? I created nft ipsets with an ipset-extras incuding different Geo-IP, different ASN and I guess the nft2ipset doesn't work well with it. All the IPs aren't included to the ipset, I guess. I changed hash:ip to hash:net in script, and it looks probably good, but maybe I don't know something?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment