initial commit

2026-07-12 10:17:17 -04:00
commit dab5a4ebc6
1424 changed files with 330463 additions and 0 deletions
@@ -0,0 +1,109 @@
+# Cockpit PackageKit "Cannot refresh cache whilst offline"
+
+## Symptoms
+
+- Cockpit Software Updates page shows: "Loading available updates failed — Cannot refresh cache whilst offline"
+- `apt-get update` works fine from terminal
+- `nm-online` times out or returns "disconnected"
+
+## Root Cause
+
+Three independent issues stack:
+
+### 1. PackageKit `pk_backend_is_online()` checks NetworkManager
+
+PackageKit's apt backend (`/usr/lib/x86_64-linux-gnu/packagekit-backend/libpk_backend_apt.so`) calls `pk_backend_is_online()` which queries NetworkManager's online state via D-Bus. On Ubuntu Server, the default network renderer is **systemd-networkd**, not NetworkManager. The ethernet interface (`enp2s0`) shows as "unmanaged" in NM:
+
+```
+nmcli dev status → enp2s0: ethernet, unmanaged
+nmcli general status → STATE: disconnected, CONNECTIVITY: none
+nm-online -q → times out
+```
+
+NetworkManager can't claim the interface because systemd-networkd already controls it.
+
+### 2. PackageKit needs polkit auth for cache refresh
+
+When Cockpit triggers a refresh, PackageKit logs:
+```
+uid 1000 is trying to obtain org.freedesktop.packagekit.system-sources-refresh auth
+uid 1000 failed to obtain auth
+```
+
+The default polkit rules require interactive auth for system-sources-refresh.
+
+### 3. PackageKit's refresh-cache D-Bus call fails silently
+
+Even when auth passes (after polkit fix), the refresh finishes with "failed":
+```
+refresh-cache transaction /8_decaabdd from uid 1000 finished with failed after 126ms
+```
+
+The error is generic — `pk_backend_is_online()` returns FALSE, and the backend refuses to refresh.
+
+## Diagnosis Commands
+
+```bash
+# Check NM state
+nmcli general status
+nm-online -q
+
+# Check PackageKit logs
+sudo journalctl -u packagekit --since "5 min ago" --no-pager
+
+# Check APT update health
+sudo apt-get update
+
+# Trace PackageKit refresh
+sudo journalctl -u packagekit -f &
+dbus-send --system --dest=org.freedesktop.PackageKit \
+  --type=method_call /org/freedesktop/PackageKit \
+  org.freedesktop.PackageKit.RefreshCache boolean:false
+
+# Check available PackageKit backends
+ls /usr/lib/x86_64-linux-gnu/packagekit-backend/
+dpkg -l | grep packagekit
+
+# Check apt backend online check
+strings /usr/lib/x86_64-linux-gnu/packagekit-backend/libpk_backend_apt.so | grep -i "offline"
+# Output: pk_backend_is_online, Cannot refresh cache whilst offline
+```
+
+## Fix
+
+The canonical fix — see `selfhosted-migration` skill, **Phase 6f: Cockpit + PackageKit on systemd-networkd Systems**.
+
+### Quick fix (three steps)
+
+```bash
+# 1. Stop NM — PackageKit checks NM's state, NM says "disconnected"
+#    because systemd-networkd owns the interface
+sudo systemctl mask NetworkManager --now
+
+# 2. Verify PackageKit now sees online state
+busctl get-property org.freedesktop.PackageKit /org/freedesktop/PackageKit \
+  org.freedesktop.PackageKit NetworkState
+# Expect: u 2  (2 = Online)
+
+# 3. Fix polkit so Cockpit can trigger refreshes from the web UI
+sudo tee /etc/polkit-1/rules.d/50-packagekit.rules << 'POLKIT'
+polkit.addRule(function(action, subject) {
+    if (action.id == "org.freedesktop.packagekit.system-sources-refresh" &&
+        subject.isInGroup("sudo")) {
+        return polkit.Result.YES;
+    }
+});
+POLKIT
+sudo systemctl restart packagekit
+sudo apt-get update
+```
+
+Then reload Cockpit at `https://<server>:9090`.
+
+## What NOT to do
+
+- **Don't revert to NetworkManager** — NM can't claim an interface managed by systemd-networkd. Setting `managed=true` in NM config doesn't help.
+- **Don't disable systemd-networkd** — it's the default Ubuntu Server network manager and works correctly.
+- **Don't set `Defaults:ray !requiretty` in sudoers** — this flag was removed in newer sudo versions (Ubuntu 26.04+). Use the Python PTY workaround instead.
+- **Don't add NetworkManager connectivity check config** — disabling the NM connectivity check (`interval=0`) doesn't fix PackageKit's internal check.
+- **Don't set `AssumeYes=true` in PackageKit.conf** — it controls auto-answering prompts (like "install these dependencies?"), not the offline check. The offline check is a hard D-Bus query to NetworkManager.