# Cockpit PackageKit "Cannot refresh cache whilst offline"

## Symptoms

- Cockpit Software Updates page shows: "Loading available updates failed — Cannot refresh cache whilst offline"
- `apt-get update` works fine from terminal
- `nm-online` times out or returns "disconnected"

## Root Cause

Three independent issues stack:

### 1. PackageKit `pk_backend_is_online()` checks NetworkManager

PackageKit's apt backend (`/usr/lib/x86_64-linux-gnu/packagekit-backend/libpk_backend_apt.so`) calls `pk_backend_is_online()` which queries NetworkManager's online state via D-Bus. On Ubuntu Server, the default network renderer is **systemd-networkd**, not NetworkManager. The ethernet interface (`enp2s0`) shows as "unmanaged" in NM:

```
nmcli dev status → enp2s0: ethernet, unmanaged
nmcli general status → STATE: disconnected, CONNECTIVITY: none
nm-online -q → times out
```

NetworkManager can't claim the interface because systemd-networkd already controls it.

### 2. PackageKit needs polkit auth for cache refresh

When Cockpit triggers a refresh, PackageKit logs:
```
uid 1000 is trying to obtain org.freedesktop.packagekit.system-sources-refresh auth
uid 1000 failed to obtain auth
```

The default polkit rules require interactive auth for system-sources-refresh.

### 3. PackageKit's refresh-cache D-Bus call fails silently

Even when auth passes (after polkit fix), the refresh finishes with "failed":
```
refresh-cache transaction /8_decaabdd from uid 1000 finished with failed after 126ms
```

The error is generic — `pk_backend_is_online()` returns FALSE, and the backend refuses to refresh.

## Diagnosis Commands

```bash
# Check NM state
nmcli general status
nm-online -q

# Check PackageKit logs
sudo journalctl -u packagekit --since "5 min ago" --no-pager

# Check APT update health
sudo apt-get update

# Trace PackageKit refresh
sudo journalctl -u packagekit -f &
dbus-send --system --dest=org.freedesktop.PackageKit \
  --type=method_call /org/freedesktop/PackageKit \
  org.freedesktop.PackageKit.RefreshCache boolean:false

# Check available PackageKit backends
ls /usr/lib/x86_64-linux-gnu/packagekit-backend/
dpkg -l | grep packagekit

# Check apt backend online check
strings /usr/lib/x86_64-linux-gnu/packagekit-backend/libpk_backend_apt.so | grep -i "offline"
# Output: pk_backend_is_online, Cannot refresh cache whilst offline
```

## Fix

The canonical fix — see `selfhosted-migration` skill, **Phase 6f: Cockpit + PackageKit on systemd-networkd Systems**.

### Quick fix (three steps)

```bash
# 1. Stop NM — PackageKit checks NM's state, NM says "disconnected"
#    because systemd-networkd owns the interface
sudo systemctl mask NetworkManager --now

# 2. Verify PackageKit now sees online state
busctl get-property org.freedesktop.PackageKit /org/freedesktop/PackageKit \
  org.freedesktop.PackageKit NetworkState
# Expect: u 2  (2 = Online)

# 3. Fix polkit so Cockpit can trigger refreshes from the web UI
sudo tee /etc/polkit-1/rules.d/50-packagekit.rules << 'POLKIT'
polkit.addRule(function(action, subject) {
    if (action.id == "org.freedesktop.packagekit.system-sources-refresh" &&
        subject.isInGroup("sudo")) {
        return polkit.Result.YES;
    }
});
POLKIT
sudo systemctl restart packagekit
sudo apt-get update
```

Then reload Cockpit at `https://<server>:9090`.

## What NOT to do

- **Don't revert to NetworkManager** — NM can't claim an interface managed by systemd-networkd. Setting `managed=true` in NM config doesn't help.
- **Don't disable systemd-networkd** — it's the default Ubuntu Server network manager and works correctly.
- **Don't set `Defaults:ray !requiretty` in sudoers** — this flag was removed in newer sudo versions (Ubuntu 26.04+). Use the Python PTY workaround instead.
- **Don't add NetworkManager connectivity check config** — disabling the NM connectivity check (`interval=0`) doesn't fix PackageKit's internal check.
- **Don't set `AssumeYes=true` in PackageKit.conf** — it controls auto-answering prompts (like "install these dependencies?"), not the offline check. The offline check is a hard D-Bus query to NetworkManager.