The Dual YubiKey Conundrum
Carrying two hardware security keys, like YubiKeys, is standard practice for anyone serious about security. One serves as a primary, the other as a readily accessible backup. This approach, while sound, can introduce subtle complexities. The author of this piece discovered such an issue: intermittent failures when using `git push` that required a PIN entry. The problem wasn't the PIN itself, but which YubiKey happened to be plugged into the machine at the time. This seemingly minor annoyance, occurring perhaps fifteen times a day, pointed to a deeper conflict within the SSH agent configuration.
Both keys in question were configured as `sk-ssh-ed25519` credentials. These are designed for FIDO2/U2F authentication and are often integrated with system agents like `gnome-keyring` for convenience. The expected behavior is that when an SSH operation requires authentication, the agent would prompt for the PIN associated with the loaded key, and upon successful entry, authenticate the operation. However, the reality was inconsistent. Some days, entering the PIN on the first prompt worked. Other days, the first prompt would fail, only for a second prompt to appear and succeed with the exact same PIN.
Unpacking the SSH Agent Behavior
The root cause lies in how SSH agents handle multiple hardware security keys that present identical credential types. When both YubiKeys were inserted and loaded into `gnome-keyring`, the `ssh-add -l` command would list both keys, each with a unique SHA256 fingerprint. However, the SSH protocol and the agent itself might not have a robust mechanism to differentiate between two keys of the same type (`sk-ssh-ed25519`) when an operation requires a PIN prompt. The agent likely presents the prompt, but the underlying system might struggle to correctly associate the PIN entry with the specific key being queried by the SSH client.
Consider this scenario: You attempt a `git push`. The SSH client needs to authenticate. It queries the SSH agent. The agent finds two `sk-ssh-ed25519` keys. It picks one (perhaps based on insertion order, or an internal cache) and asks for its PIN. If the system correctly routes the PIN to that specific key, authentication succeeds. If it fails to correctly associate the PIN with the queried key, the prompt fails. The agent, seeing a failure, might then try the *other* `sk-ssh-ed25519` key, which then prompts for and successfully validates the PIN. This explains the observed behavior: sometimes the first try works, other times it takes a second attempt with the same PIN.
The problem is exacerbated by the fact that `sk-ssh-ed25519` keys often require a PIN because they are designed to protect a private key that never leaves the hardware. The PIN is the gatekeeper to accessing that private key for signing operations. When the system cannot reliably determine *which* hardware key the PIN is intended for, the entire authentication flow breaks down intermittently.
Mitigation Strategies and Future Considerations
The immediate workaround identified by the author is simple: if the first PIN prompt fails, wait for the second one. This acknowledges the problem without fully solving it. A more robust solution involves ensuring that the SSH agent and the underlying operating system can uniquely identify and manage multiple hardware keys of the same type.
One potential long-term fix could involve using different types of hardware keys if possible, though this is often not practical or desirable. A more likely solution would be for SSH agent implementations, or the operating system's FIDO2 integration, to improve their ability to disambiguate between multiple identical hardware tokens. This might involve more sophisticated handling of USB device identifiers or specific FIDO attestations to ensure the correct key is always targeted.
For developers and security professionals relying on multiple hardware keys for critical operations like Git commits, this highlights a gap in seamless multi-key management. While the security of having redundant keys is paramount, the usability friction introduced by such conflicts cannot be ignored. The underlying issue is that the system treats two distinct hardware security devices as if they were interchangeable instances of the same credential type, leading to authentication confusion. This is less a flaw in the YubiKeys themselves and more a subtle interaction problem between the keys, the SSH agent, and the operating system's device management.
What remains unaddressed is how widespread this issue is across different operating systems and SSH agent versions. While `gnome-keyring` was implicated here, similar issues could arise with other agents or platforms. Developers need clarity on which configurations are stable for multi-key setups to avoid unexpected authentication failures during crucial development workflows.