Files
hermes-config/skills/devops/selfhosted-migration/references/cockpit-packagekit-offline.md
T
2026-07-12 10:17:17 -04:00

110 lines
4.0 KiB
Markdown

# Cockpit PackageKit "Cannot refresh cache whilst offline"
## Symptoms
- Cockpit Software Updates page shows: "Loading available updates failed — Cannot refresh cache whilst offline"
- `apt-get update` works fine from terminal
- `nm-online` times out or returns "disconnected"
## Root Cause
Three independent issues stack:
### 1. PackageKit `pk_backend_is_online()` checks NetworkManager
PackageKit's apt backend (`/usr/lib/x86_64-linux-gnu/packagekit-backend/libpk_backend_apt.so`) calls `pk_backend_is_online()` which queries NetworkManager's online state via D-Bus. On Ubuntu Server, the default network renderer is **systemd-networkd**, not NetworkManager. The ethernet interface (`enp2s0`) shows as "unmanaged" in NM:
```
nmcli dev status → enp2s0: ethernet, unmanaged
nmcli general status → STATE: disconnected, CONNECTIVITY: none
nm-online -q → times out
```
NetworkManager can't claim the interface because systemd-networkd already controls it.
### 2. PackageKit needs polkit auth for cache refresh
When Cockpit triggers a refresh, PackageKit logs:
```
uid 1000 is trying to obtain org.freedesktop.packagekit.system-sources-refresh auth
uid 1000 failed to obtain auth
```
The default polkit rules require interactive auth for system-sources-refresh.
### 3. PackageKit's refresh-cache D-Bus call fails silently
Even when auth passes (after polkit fix), the refresh finishes with "failed":
```
refresh-cache transaction /8_decaabdd from uid 1000 finished with failed after 126ms
```
The error is generic — `pk_backend_is_online()` returns FALSE, and the backend refuses to refresh.
## Diagnosis Commands
```bash
# Check NM state
nmcli general status
nm-online -q
# Check PackageKit logs
sudo journalctl -u packagekit --since "5 min ago" --no-pager
# Check APT update health
sudo apt-get update
# Trace PackageKit refresh
sudo journalctl -u packagekit -f &
dbus-send --system --dest=org.freedesktop.PackageKit \
--type=method_call /org/freedesktop/PackageKit \
org.freedesktop.PackageKit.RefreshCache boolean:false
# Check available PackageKit backends
ls /usr/lib/x86_64-linux-gnu/packagekit-backend/
dpkg -l | grep packagekit
# Check apt backend online check
strings /usr/lib/x86_64-linux-gnu/packagekit-backend/libpk_backend_apt.so | grep -i "offline"
# Output: pk_backend_is_online, Cannot refresh cache whilst offline
```
## Fix
The canonical fix — see `selfhosted-migration` skill, **Phase 6f: Cockpit + PackageKit on systemd-networkd Systems**.
### Quick fix (three steps)
```bash
# 1. Stop NM — PackageKit checks NM's state, NM says "disconnected"
# because systemd-networkd owns the interface
sudo systemctl mask NetworkManager --now
# 2. Verify PackageKit now sees online state
busctl get-property org.freedesktop.PackageKit /org/freedesktop/PackageKit \
org.freedesktop.PackageKit NetworkState
# Expect: u 2 (2 = Online)
# 3. Fix polkit so Cockpit can trigger refreshes from the web UI
sudo tee /etc/polkit-1/rules.d/50-packagekit.rules << 'POLKIT'
polkit.addRule(function(action, subject) {
if (action.id == "org.freedesktop.packagekit.system-sources-refresh" &&
subject.isInGroup("sudo")) {
return polkit.Result.YES;
}
});
POLKIT
sudo systemctl restart packagekit
sudo apt-get update
```
Then reload Cockpit at `https://<server>:9090`.
## What NOT to do
- **Don't revert to NetworkManager** — NM can't claim an interface managed by systemd-networkd. Setting `managed=true` in NM config doesn't help.
- **Don't disable systemd-networkd** — it's the default Ubuntu Server network manager and works correctly.
- **Don't set `Defaults:ray !requiretty` in sudoers** — this flag was removed in newer sudo versions (Ubuntu 26.04+). Use the Python PTY workaround instead.
- **Don't add NetworkManager connectivity check config** — disabling the NM connectivity check (`interval=0`) doesn't fix PackageKit's internal check.
- **Don't set `AssumeYes=true` in PackageKit.conf** — it controls auto-answering prompts (like "install these dependencies?"), not the offline check. The offline check is a hard D-Bus query to NetworkManager.