Docker User-Defined Bridge: Lost Gateway IPv4 Address

Symptom

A Docker container is running, docker ps shows it as healthy, and docker exec can reach it from inside. But accessing the published port from the host or other machines on the network hangs/times out.

$ curl -v http://localhost:2283/
*   Trying 127.0.0.1:2283...
* Connected to localhost (127.0.0.1) port 2283
> GET / HTTP/1.1
> ...
* Request completely sent off
* Operation timed out after 10001 milliseconds with 0 bytes received

Docker-proxy is running and listening on the port, but responses never come back.

Diagnosis

# docker-proxy is running and forwarding correctly
ss -tlnp | grep <port>
# → LISTEN 0.0.0.0:<port>

# Docker NAT rule exists
sudo iptables -t nat -L DOCKER -n | grep <port>
# → DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:<port> to:172.XX.0.X:<port>

# Docker filter allows traffic
sudo iptables -L DOCKER -n | grep <container_ip>

# But the host can't reach the container's IP
ping -c 2 <container_ip>
# → 100% packet loss

# The bridge interface has NO IPv4 address
ip addr show br-<hash>
# Only shows inet6 fe80::... — NO inet 172.XX.0.1/16

Root Cause

Docker's user-defined bridge networks (created by docker compose) need a gateway IP on the host-side bridge interface (br-<hash>). This IP is normally assigned when the network is first created.

If the bridge loses its IPv4 (e.g. after docker compose restart without a full down+up, or due to a Docker daemon restart), the host has no route to the container's subnet:

ip route | grep 172.18
# → (nothing — no route)

docker-proxy still listens and accepts connections, but it forwards to the container's IP via the bridge. Without the host having an IP on that bridge, the forwarded packets can't reach the container and responses can't come back.

Fix

Preferred: Full Network Recreate

cd /opt/<project>/
docker compose down
docker compose up -d

This recreates the network fresh, which assigns the gateway IP correctly. All containers are recreated, so the network starts clean.

Quick Patch (if downtime is unacceptable)

sudo ip addr add <gateway_ip>/<mask> dev br-<hash>

Find the gateway IP from docker network inspect:

docker network inspect <name> | grep -A 5 '"Config"'
# "Gateway": "172.18.0.1"

Then:

sudo ip addr add 172.18.0.1/16 dev br-<hash>

This restores connectivity immediately but is ephemeral — it won't survive a reboot or Docker daemon restart. Use the full down && up -d for a permanent fix.

Making the Quick Patch Persistent

If you need a stopgap before the full restart, create a oneshot systemd service:

[Unit]
Description=Assign IP to Docker bridge
After=docker.service
Requires=docker.service

[Service]
Type=oneshot
ExecStartPre=/bin/bash -c "sleep 5"
ExecStart=/sbin/ip addr add <gateway_ip>/<mask> dev br-<hash> 2>/dev/null || /bin/true
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

sudo systemctl enable --now <service>

Prevention

Avoid docker compose restart <service> for critical containers on user-defined bridge networks if you suspect the network might be degraded
Use full docker compose down && docker compose up -d when something feels off with container networking
After a full machine reboot, Docker recreates everything properly — this issue typically only appears during hot-fixes or partial restarts

3.4 KiB Raw Permalink Blame History