Files
hermes-config/skills/devops/selfhosted-migration/references/cockpit-packagekit-offline.md
T
2026-07-12 10:17:17 -04:00

4.0 KiB

Cockpit PackageKit "Cannot refresh cache whilst offline"

Symptoms

  • Cockpit Software Updates page shows: "Loading available updates failed — Cannot refresh cache whilst offline"
  • apt-get update works fine from terminal
  • nm-online times out or returns "disconnected"

Root Cause

Three independent issues stack:

1. PackageKit pk_backend_is_online() checks NetworkManager

PackageKit's apt backend (/usr/lib/x86_64-linux-gnu/packagekit-backend/libpk_backend_apt.so) calls pk_backend_is_online() which queries NetworkManager's online state via D-Bus. On Ubuntu Server, the default network renderer is systemd-networkd, not NetworkManager. The ethernet interface (enp2s0) shows as "unmanaged" in NM:

nmcli dev status → enp2s0: ethernet, unmanaged
nmcli general status → STATE: disconnected, CONNECTIVITY: none
nm-online -q → times out

NetworkManager can't claim the interface because systemd-networkd already controls it.

2. PackageKit needs polkit auth for cache refresh

When Cockpit triggers a refresh, PackageKit logs:

uid 1000 is trying to obtain org.freedesktop.packagekit.system-sources-refresh auth
uid 1000 failed to obtain auth

The default polkit rules require interactive auth for system-sources-refresh.

3. PackageKit's refresh-cache D-Bus call fails silently

Even when auth passes (after polkit fix), the refresh finishes with "failed":

refresh-cache transaction /8_decaabdd from uid 1000 finished with failed after 126ms

The error is generic — pk_backend_is_online() returns FALSE, and the backend refuses to refresh.

Diagnosis Commands

# Check NM state
nmcli general status
nm-online -q

# Check PackageKit logs
sudo journalctl -u packagekit --since "5 min ago" --no-pager

# Check APT update health
sudo apt-get update

# Trace PackageKit refresh
sudo journalctl -u packagekit -f &
dbus-send --system --dest=org.freedesktop.PackageKit \
  --type=method_call /org/freedesktop/PackageKit \
  org.freedesktop.PackageKit.RefreshCache boolean:false

# Check available PackageKit backends
ls /usr/lib/x86_64-linux-gnu/packagekit-backend/
dpkg -l | grep packagekit

# Check apt backend online check
strings /usr/lib/x86_64-linux-gnu/packagekit-backend/libpk_backend_apt.so | grep -i "offline"
# Output: pk_backend_is_online, Cannot refresh cache whilst offline

Fix

The canonical fix — see selfhosted-migration skill, Phase 6f: Cockpit + PackageKit on systemd-networkd Systems.

Quick fix (three steps)

# 1. Stop NM — PackageKit checks NM's state, NM says "disconnected"
#    because systemd-networkd owns the interface
sudo systemctl mask NetworkManager --now

# 2. Verify PackageKit now sees online state
busctl get-property org.freedesktop.PackageKit /org/freedesktop/PackageKit \
  org.freedesktop.PackageKit NetworkState
# Expect: u 2  (2 = Online)

# 3. Fix polkit so Cockpit can trigger refreshes from the web UI
sudo tee /etc/polkit-1/rules.d/50-packagekit.rules << 'POLKIT'
polkit.addRule(function(action, subject) {
    if (action.id == "org.freedesktop.packagekit.system-sources-refresh" &&
        subject.isInGroup("sudo")) {
        return polkit.Result.YES;
    }
});
POLKIT
sudo systemctl restart packagekit
sudo apt-get update

Then reload Cockpit at https://<server>:9090.

What NOT to do

  • Don't revert to NetworkManager — NM can't claim an interface managed by systemd-networkd. Setting managed=true in NM config doesn't help.
  • Don't disable systemd-networkd — it's the default Ubuntu Server network manager and works correctly.
  • Don't set Defaults:ray !requiretty in sudoers — this flag was removed in newer sudo versions (Ubuntu 26.04+). Use the Python PTY workaround instead.
  • Don't add NetworkManager connectivity check config — disabling the NM connectivity check (interval=0) doesn't fix PackageKit's internal check.
  • Don't set AssumeYes=true in PackageKit.conf — it controls auto-answering prompts (like "install these dependencies?"), not the offline check. The offline check is a hard D-Bus query to NetworkManager.