Skip to content

Instantly share code, notes, and snippets.

@lukasnellen

lukasnellen/00-docker-shorewall.md

Last active October 13, 2025 22:51
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save lukasnellen/20761a20286f32efc396e207d986295d to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save lukasnellen/20761a20286f32efc396e207d986295d to your computer and use it in GitHub Desktop.
Download ZIP
setup shorewall for docker networking beyond the default bridge network, e.g., for docker-compose
Raw
00-docker-shorewall.md

Docker(-compose) with shorewall

The shorewall documentation explains in http://shorewall.org/Docker.html how to configure shorewall for use with docker. The problem with the configuration is that it only allows connections from the host to the main bridge docker0. Connections to other networks on dynamically created bridges, with names starting by default with br-, is blocked. Instead of the recommended contents of /etc/shorewall/interfaces, use wild-card interface names as follows:

#ZONE	INTERFACE	OPTIONS
#dock	docker0		bridge     # disabled default recommendation
dock 	docker0		physical=docker+,routeback=1
dock 	br		physical=br-+,routeback=1

This declares interfaces with names starting with docker, including the default docker0, and starting with br- to be in the dock zone.

For the rest of the configuration, follow the shorewall documentation as is.

This setup fixes problems running composite apps set up manually or using docker-compose.

Raw
interfaces
###############################################################################
?FORMAT 2
###############################################################################
#ZONE INTERFACE OPTIONS
net eth0 dhcp,tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0
#dock docker0 bridge
dock docker0 physical=docker+,routeback=1
dock br physical=br-+,routeback=1
# and others ...
Raw
policy
###############################################################################
#SOURCE DEST POLICY LOG LEVEL LIMIT:BURST
$FW net ACCEPT
dock $FW REJECT
dock all ACCEPT
# Some applications or use case require uncommenting the next line
$FW dock ACCEPT
# THE FOLLOWING POLICY MUST BE LAST
all all REJECT info
Raw
shorewall.conf
# apply the change below, as
# ...
# LN:
#DOCKER=No
DOCKER=Yes
# ...
Raw
zones
###############################################################################
#ZONE TYPE OPTIONS IN OUT
# OPTIONS OPTIONS
fw firewall
dock ipv4
# and others ...
@lukasnellen
Copy link
Author

lukasnellen commented Feb 20, 2020

If configured with the DYNAMIC_BLACKLIST=Yes option, you can use shorewall open, shorewall close and other commands for non-persistent changes (see https://shorewall.org/manpages/shorewall.html). You still have to update the config files to be sure changes are persistent. But the persistent rules won't get verified until you restart. I recommend to shorewall compile, which catches syntax errors but won't change the active rules.

In practice, I find firewall rules pretty stable and the restart issues don't arise often. Using the shorewall commands is OK for testing which ports are needed when setting up new apps. I would still do one restart with the persistent configuration at the end. Bugs that won't show up until the next reboot are worse than one container restart.

@zhaofeng-shu33
Copy link

zhaofeng-shu33 commented Feb 25, 2020

Thank you, it works.

@rogerniesten
Copy link

rogerniesten commented Feb 28, 2020 •
edited
Loading

(I think my issue is related to the ones mentioned above, so I'll add my issue here)

I'm using Docker 19.03.6 and shorewall 5.1.12.2 and encountered situations where (creating and) starting a docker container caused the following error:

Creating network "acc_default" with the default driver
ERROR: unable to insert jump to DOCKER-ISOLATION-STAGE-1 rule in FORWARD chain: (iptables failed: iptables --wait -I FORWARD -j DOCKER-ISOLATION-STAGE-1: iptables v1.6.1: Couldn't load target `DOCKER-ISOLATION-STAGE-1':No such file or directory

How to reproduce:
$ sudo service shorewall stop
$ suso service docker stop
$ sudo docker start
$ sudo shorewall start
<< when checking iptables -L, chain DOCKER is present, chains DOCKER-ISOLATION-STAGE-1 and DOCKER-ISOLATION-STAGE-2 are not! >>
$ docker-compose up -d
<< now the error as mentioned above occurs >>

I noticed that the chains DOCKER-ISOLATION-STAGE-1 and DOCKER-ISOLATION-STAGE-2 (including references in other chains) were removed after the command "shorewall restart". As they didn't contain any rules and everything kept working (also after stopping starting containers), I didn't see this as an issue. However, when starting a new container (e.g. after it has been removed) it will fail with the error mentioned above.
When executing shorewall with the trace option, I can see mentioned DOCKER-INGRESS and DOCKER-ISOLATION, but never DOCKER-ISOLATION-STAGE-1 or DOCKER-ISOLATION-STAGE-2.

With some further test I discovered that without the DOCKER-ISOLATION-STATE-x chains, I am able to start any docker container, but can NOT create a docker network!

So my questions are:

  1. Has docker changed their iptables (are the ...-STAGE-.. chains new) and is shorewall not ready for them yet?
  2. If so, when is a new version of shorewall to be expected?
  3. Most important: what do I need to do to allow shorewall to modify iptables without having to restart the docker daemon? (we are using mrlesmithjr/ansible-shorewall to configure shorewall via ansible)

@lukasnellen
Copy link
Author

lukasnellen commented Mar 3, 2020

In my experience, shorewall has to be started before other tools that manipulate the iptables. On Debian, I can verify that the dependencies in the systemd unit files are set up in a way that makes sure that shorewall starts before docker. If you change the order of startup to

$ sudo shorewall start
$ sudo docker start

things should work as expected.

BTW: If you have other services manipulating iptables, e.g., fail2ban, make sure they get restarted after shorewall gets restarted.

@rogerniesten
Copy link

rogerniesten commented Mar 3, 2020

Thanks for your feedback, @lukasnellen.
I was under the impression that shorewall would preserve docker related chains and rules when being restarted, but apparently it doesn't :-(
That means a firewall modification will still require a restart of the docker daemon (which means an outage for our production services).

I was able to re-create the Docker Isolation chains, but the Docker chain itself is a bit more tricky...

@lukasnellen
Copy link
Author

lukasnellen commented Mar 3, 2020

In practice, the problem is that restarting docker restarts all running containers. This makes frequent changes (or trying changes) on a production machine difficult. This is not a problem for my setup here, where changes are limited (and I have a non-production machine for trying out things). This meant I never had to look into the use of shorewall open and shorewall close as means of run-time modification of shorewall's configuration (more in a comment higher up).

@aleqcz
Copy link

aleqcz commented Mar 22, 2020

Thank you very much

@schorschl
Copy link

schorschl commented Mar 31, 2020

Thanks for your helpful hints.
I am using docker-compose and shorewall. When setting rules in /etc/shorewall/rules they are not triggered.
To be able to restrict access to the docker machines I need to insert rules into the "DOCKER-USER" chain. Any idea how I can manage that with shorewall?

@lukasnellen
Copy link
Author

lukasnellen commented Mar 31, 2020

Instead of using the DOCKER-USER chain, I set up rules in shorewall that filter connections to or from the dock zone. For more fine-grained control, you can use some of docker's advanced options:

  • create separate networks, maybe with a named bridge so you can create a special zone on that bridge
  • set the IP of the containers in docker run or in docker-compose.yml
  • user shorewall's dynamic zones (https://shorewall.org/Dynamic.html), put your container's IP into the correct, dynamic zone, and filter on those zones.

The point here is to control the interface or IP address used by a container. That way, you can write specific filter rules in shorewall w/o having to inspect the container's configuration, which might change when the container gets re-created. This is very similar to how I set up filtering for full-blown virtual machines.

The alternative is to use iptables to manipulate the DOCKER-USER table, which is not managed directly by shorewall.

@phlax
Copy link

phlax commented Apr 7, 2020 •
edited
Loading

related to above comments about iptables rules on debian, it seems that the versions of shorewall and docker in debian stable and above do not work (at least not without always restarting docker after shorewall is restarted)

i rebuilt the debian shorewall packages (shorewall + shorewall-core) with the 5.2.4 version and this fixed the problem

i have raised an issue on the shorewall gitlab issue tracker

https://gitlab.com/shorewall/debian/-/issues/1

@interso
Copy link

interso commented Aug 4, 2020

Thank you very much.

@grharry
Copy link

grharry commented Oct 31, 2020

Great post!
However in my case it's missing a vital detail !
I disabled docker iptables in /etc/docker/daemon.json
{
"iptables":false
}
And after that all works Great! Docker containers are accessible from net if enabled in shorewall ( rules, zones, policy ) etc !!
Regards,
Harry

@sgrayban
Copy link

sgrayban commented Sep 9, 2021

This line is giving a Broadcast error when starting Shorewall 5.2.3.2

dock docker0 physical=docker+,routeback=1

ERROR: Invalid BROADCAST address /etc/shorewall/interfaces

@leo-fg
Copy link

leo-fg commented Sep 16, 2021

I added different and it worked as expected.

dock docker physical=docker+,routeback=1

@lukasnellen
Copy link
Author

lukasnellen commented Sep 16, 2021

The actual name should not matter, since the wildcard physical=docker+ is there to match any interface with a name starting with docker.

Details might also depend on the way how the distribution sets up networking and on the order in which components get started.

Disclaimer: My production machines are debian, versions 9-11. So details might vary. I'm glad to get additional information on what to change/fix on other distributions.

@sgrayban
Copy link

sgrayban commented Sep 16, 2021

I am using debian 10 and just cannot get this to work -- the only way to get it working for me, and I still don't think its right, is using::

dock    docker0
dock    br

shorewall version
5.2.3.2

@lukasnellen
Copy link
Author

lukasnellen commented Sep 17, 2021

Did you try running shorewall compile? This gives better diagnostics than trying to restart the service. And it allows you to detect and fix problems rather than leaving the firewall in an unreliable or diabled state (especially when restarting).

I recently migrated my debian 10 docker host to 11 and don't have a debian 10 docker host at hand to verify the behaviour.

@sgrayban
Copy link

sgrayban commented Sep 17, 2021

shorewall compile

Compiling using Shorewall 5.2.3.2...
Processing /etc/shorewall/params ...
Processing /etc/shorewall/shorewall.conf...
Loading Modules...
Compiling /etc/shorewall/zones...
Compiling /etc/shorewall/interfaces...
ERROR: Invalid BROADCAST address /etc/shorewall/interfaces (line 191)

@lukasnellen
Copy link
Author

lukasnellen commented Sep 17, 2021

Do you have the format specifier at the top of the interfaces file? The line

?FORMAT 2

is required. If I leave it out, I can reproduce the error message about the BROADCAST address in my setup. It almost looks like a comment i the file I have included at the top.

@sgrayban
Copy link

sgrayban commented Sep 17, 2021

Crap... missed that. It's working now!! Thanks mate.

@athimel
Copy link

athimel commented Mar 25, 2022 •
edited
Loading

Great post! However in my case it's missing a vital detail ! I disabled docker iptables in /etc/docker/daemon.json { "iptables":false } And after that all works Great! Docker containers are accessible from net if enabled in shorewall ( rules, zones, policy ) etc !! Regards, Harry

@grharry This is the ""detail"" I was missing! Thank you so much!

@kickerofelves
Copy link

kickerofelves commented Feb 21, 2023 •
edited
Loading

Thanks! New docker user and this gist and comments allowed me to connect to running containers.

But now running containers cannot reach the wider internet. How do I do that?

Edit: resolved with help from serverfault.com/a/631877 specifically creating a /etc/shorewall/masq file and turning on ip forwarding.

@vshuraeff
Copy link

vshuraeff commented May 30, 2023

Has anyone succeeded in restricting network access to containers for the entire Internet and only allowing access to a specific list of addresses (ipset)?

@robozb
Copy link

robozb commented Sep 11, 2023

You saved my day, thank you so much!

@abidibo
Copy link

abidibo commented Nov 17, 2023

Hero!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment