Using ssh-keygen to Create Resident FIDO2/SK Keys for Access to OpenSSH Linux Servers

The article explains the -resident option of the ssh-keygen command (e.g. -O resident) to create FIDO2/YubiKey keys to log in to a Linux account via ZOC Terminal or OpenSSH.
 

Table of Contents

 

Introduction

Since OpenSSH version 8.3 SSH servers support public/private key logins based on FIDO2 hardware keys like the YubiKey 5-NFC or YubiKey BIO (FIDO) or other brands.

This article will show how to use the ssh-keygen command to create a FIDO2/YubiKey based key pair that is stored on the key and which can be recreated if the associated disk based key-file gets lost. It will also discuss platform issues that may arise with the use of the ssh-keygen command on macOS and Windows.


Prerequisites

Obviously you will need a FIDO2 compatible hardware token and the token will need to be enabled for use with the FIDO protocol. We will focus on the popular YubiKeys as an example, but the procedure will be similar for other brands.

If you have not decided on a brand yet, a YubiKey 5 NFC is a good starting point. Choose the USB-A or USB-C model, depending on which type of port is most readily available and within easy reach on your computer. Please consider that you will need to physically see and touch the key often. Thus, plugging it into the back of a PC under your desk will not be a good solution.

If you have a YubiKey, but have not initialized it already, you need to prepare the YubiKey for use with the FIDO2 protocol. In most cases it will require you to just configure a PIN that protects the key, but for some models, like the YubiKey BIO you will also register your fingerprint:

  • Download Yubico Authenticator and install it.
  • Start Yubico Authenticator or YubiKey Manager (under Windows you will need to start it via right-click→As Administrator).
  • Depending on the type of key select Applications→FIDO and set a PIN (or even a fingerprint).

The YubiKey Authenticator app is also handy to see which keys are stored on the device.


Command-line Tools and Platform Issues

The standard way to create a key for a Linux login is to use the OpenSSH command-line tool ssh-keygen.

However, depending on the operating system, use of this command has actually turned into a little mess around the question of whether the program actually supports FIDO keys (or in OpenSSH parlance, if it has sk-support).

macOS

macOS has a ssh-keygen tool, but the ssh-keygen tool that comes preinstalled with macOS does not support FIDO2 keys. This means that you will need a special build. You can do a Homebrew build for OpenSSH and use the ssh-keygen from that (see this article). Alternatively, if you have ZOC Terminal installed, ZOC v9 offers a version of ssh-keygen which includes sk-support. In that case use /Applications/zoc9.app/Contents/MacOS/zoc-ssh-keygen instead of ssh-keygen.

Windows

Under Windows, the FIDO2 (Windows Hello) system, does not support some of the functions that are required to fully and properly run ssh-keygen with resident keys. For example, one of the main reasons to use resident keys is the ssh-keygen -K command, which rebuilds resident keys. This will not work under Windows (see below). Likewise, when creating a resident key, ssh-keygen under Windows cannot check for its existence in order to avoid accidentally overwriting the key.

The Microsoft implementation of ssh-keygen under Windows 10/11 tried to work around some of these limitations in a half-baked way, which creates some unexpected results. For example, when creating resident keys, you will see a warning saying that the key already exists, even in cases where it doesn't. If you have ZOC Terminal installed, ZOC offers a ssh-keygen version that tries to work around the Windows-Hello limitations in a more obvious way (it will ask you to use the Yubico-Authenticator to make sure you are not accidentally overwriting an existing key). To use the ZOC version, type "C:\Program Files\ZOC9\zoc-ssh-keygen.exe" instead of just ssh-keygen (assuming you have ZOC Terminal installed).

Linux

Under Linux you will need an OpenSSH build that includes sk-support, which pretty much includes all major Linux distributions.


Resident vs. non-Resident FIDO2/SK Keys

The -O resident flag in OpenSSH's ssh-keygen command creates what are known as "resident keys" or "discoverable credentials" when generating FIDO2 security key-based SSH keys. When this option is specified, the private key material is stored directly on the FIDO2 hardware token itself, rather than being stored in a file on the local filesystem. Resident keys still require a file-based key to use them in an OpenSSH context, but with resident keys, this file can be recreated from the hardware token.

However, resident keys come with trade-offs. They depend on the authenticator's storage capacity, which may limit the number of keys that can be stored. For example, older YubiKey models only offer 25 storage slots. Also, with resident keys, if a third party gets access to the hardware token and PIN, this third party can use the key, even if they do not have the file which is associated with the key. This however turns into a benefit if the legitimate user accesses their server from multiple computers, eliminating the need to copy the associated disk files around.


Creating an OpenSSH SK/FIDO2 Key using Command-line Tools

The simplest form of creating an OpenSSH key file that resides on the FIDO2 token and/or YubiKey is:

ssh-keygen -t ecdsa-sk -O resident

or alternatively use the ZOC variant mentioned above:

macOS: /Applications/zoc9.app/Contents/MacOS/zoc-ssh-keygen -t ecdsa-sk -O resident

Windows: "C:\Program Files\ZOC9\zoc-ssh-keygen.exe" -t ecdsa-sk -O resident

This will generate a resident, non-verify-required key with the application ssh: and a null user ID (see our other article for more details about these options).

Platform Considerations

Under macOS and Linux, if a key with the same user ID and application already exists on the device you will receive a warning about overwriting the key. Under Windows however, since the FIDO driver does not allow to check for or read resident keys, the ssh-keygen command can not check for the existence of the key, so you will either always be warned about overwriting the key (even if it does not exist) or you will be asked to use the Yubico-Authenticator to check the list of resident keys and confirm you agree to create it.

Command-line Parameters for user ID and Application

You can also apply additional parameters:

  • -O application=ssh:<application>: Supply a string to indicate what the key is used for and discern it from other keys (especially resident keys).
  • -O user=<user_id>: Supply a user_id associated with the key, instead of the empty or null userid ("openssh" default). The user ID has only indicative character and is not tied to the user ID that is used to log in to the system.
  • -C "Your Comment": Add a comment to the public-key file.

For example, to create a key that resides on the token and requires PIN-entry and touch when used, the command would be:

ssh-keygen -t ecdsa-sk -O resident -O application=ssh:myserver -C "FIDO key for myserver"

On macOS and Linux the generated key-files will be placed in your ~/.ssh folder. Under Windows, you may have to copy them to the SSH folder of your SSH client application. For example, in case of ZOC Terminal, you need to copy them to your Documents → ZOC9 Files → ssh folder.


(Re-)Creating the OpenSSH Public and Private Key Files

When generating the resident key, a typical key pair like ecdsa_sk and ecdsa_sk.pub will be created. However, as outlined above, the private key file will not actually contain the private key itself, but only a reference to the hardware token and the key on the device.

This means that these files can be created by anyone who has access to the YubiKey (or other brand) token and the associated PIN.

ssh-keygen -K

or alternatively use the ZOC variant mentioned above:

macOS: /Applications/zoc9.app/Contents/MacOS/zoc-ssh-keygen -K

Windows: The Windows FIDO System (Windows Hello) does not allow applications to discover the keys, hence none of the ssh-keygen builds can recreate the files. You can plug the YubiKey into a Mac or Linux system however and run the command there and then copy the OpenSSH files back to Windows, although this obviously defies the purpose of using resident keys in order to not need to copy the associated files around.


Resident Keys and SSH Agent

Similar to the ssh-keygen -K command, which checks the hardware token for resident keys and creates the associated disk files, there is an ssh-add -K command that will add all keys from the token to the ssh-agent.

However, this also runs into platform caveats:

macOS: Since macOS's OpenSSH is built without support for sk-keys, the standard ssh-agent will not accept sk-keys. If you have a third-party agent (e.g. from a homebrew build) you can use ZOC's variant of ssh-add to add the keys from the token to the agent: /Applications/zoc9.app/Contents/MacOS/zoc-ssh-add -K

Windows: Windows's OpenSSH agent or ZOC's built-in agent does support sk-keys. However, this only works with disk based files, because Windows FIDO2 (Windows Hello) system does not allow discovery of the resident keys. Thus, while ZOC's zoc-ssh-add.exe command would theoretically support the -K option, actually using it will result in an error from the Windows Hello system.


Managing and Deleting Resident Keys

The ssh-keygen command has no options to delete keys that reside on the hardware. Use software from the key manufacturer, e.g. the Yubico-Authenticator to list and delete existing keys (go to the Passkeys section there).


Why ZOC Terminal is an Excellent Alternative for these Tasks Compared to PuTTY

Trying to perform these steps with PuTTY will not work, because the puttygen.exe key generator does not support SK keys. As you saw, ZOC Terminal instead provides enhanced versions of ssh-keygen that even work under difficult circumstances. Using OpenSSH from the commandline is also a choice, but lacks the ease of use that ZOC offers through its host directory, key remapping or the variety of available emulations.

And while PuTTY is a reasonable choice to access Linux shells every now and then, ZOC Terminal is designed for ease of use and effectiveness day in and day out. It offers impressive features that save you a lot of time when performing routine tasks.

For example, as you see in another article (SSH Key-Generation Window), creating the key-pair and obtaining the public key takes only a matter of seconds (about 10 mouse clicks).

Connecting to your Linux system through the Quick Connect window is also very easy. If you log off and then realize you forgot something and want to connect again, you can simply press Alt+R in the disconnected tab. Alternatively, go to the Quick Connection window again and click Connect, so you are reconnected with just two mouse clicks.

For file uploads, you can upload files to your server merely by dragging them from the desktop into the ZOC terminal area.

If you are building multiple instances, you can save them from the Quick Connection window to the Host Directory. From there, you can even connect to multiple instances at once (each will then open in a separate tab).

These are just a few examples that demonstrate how using a professional SSH client like ZOC can save valuable time compared to free tools like PuTTY (which admittedly does cover the essentials quite well).

Conclusion: Using ZOC Terminal to perform routine terminal tasks with ease saves time and it also is more fun!