Skip to main content
OCLC Support

Upload ongoing patron data files using passwordless SFTP

Discover how to send and retrieve files using public/private key pair authentication for passwordless SFTP with Linux.

You can send and retrieve files using public/private key pair authentication for passwordless SFTP. This automates delivery of files to OCLC and eliminates the need to manually enter a password when using an OCLC file exchange account. It also allows your institution to change the key on a regular basis for added security.

For security reasons, OCLC does not support passing of the SFTP password via a script, but offers the public/private key pair solution instead to allow fully automated scripts to be written for uploading and downloading to our server.

Linux

This procedure is for Linux systems. Although similar methods would be used for other systems, OCLC has tested this method using Linux as the SCP and SFTP client machine. A password is requested while uploading the public key, but once replaced, future logins and file transfers do not require a password.

Generate and upload your public key

To generate and upload your public key to your OCLC file exchange account:

  1. Generate a private/public key pair on your client machine and put it in a file called /home/user1/.SSH/id_rsa.pub on your local machine.
     Note:  If you are unsure how to generate a private/public key pair, you can use the method described below, which puts the private/public key file in the correct place on your local machine automatically.
  2. Upload your public key to .SSH/authorized_keys in your OCLC file exchange account.

 Note: Do not delete the existing .SSH/authorized_keys file before uploading the new one. The new file you upload automatically replaces the existing file while retaining its permissions and ownership properties. If you delete the existing file before uploading the new file, you will have insufficient rights on the system to set the necessary group ownership for .SSH/authorized_keys and therefore will continue to be asked for a password.

Automate an SFTP transfer between two servers

SSH configuration

Most of this relies on configuration of SSHd, the SSH (Secure SHell) daemon running on the server you’re attempting to connect to. This server will be referred to as "server2.com" for demonstration purposes.

Check the “SSHd_config” on that server, typically in /etc/SSH. In some cases, these settings are not always present or set the way you need:

RSAAuthentication yes
PubkeyAuthentication yes

This enables the public/private key authentication mechanism you will use.

Public/Private key generation

Generate the keypair on the Linux box that you want to connect from. This server will be referred to as “server1.com” for demonstration purposes. It’s that box you plan to run SSH, SFTP, or Rsync.

SSH-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user1/.SSH/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in .SSH/id_rsa.
Your public key has been saved in .SSH/id_rsa.pub.
The key fingerprint is:
c1:21:e 3:01:26:0d:f7:ec:52:0e:0c:90:9b:6e:d8:47 user1@server1.com

The command above generates a public/private key pair. Respond to each prompt by pressing <Enter>. Possession of the private key is sufficient to gain access to what ever resources into which you’ve placed the corresponding public key.

You must not enter a passphrase for the private key to configure an automated solution. If you do enter a passphrase, you'll need to enter it to use the private key. 

 Note: By not placing a passphrase on your private key, the security implication is that mere possession of the private key is sufficient to gain access to whatever resources into which you’ve placed the corresponding public key. Safeguard your private key.

The private key was placed in /home/user1/.SSH/id_rsa. This needs to be kept secure because of the security implication above, but it also needs to be available to the process attempting to make an SSH, SFTP, or Rsync connection. If these tools are run under the ‘user1’ account, the tools will automatically look in the “.SSH” directory, and Iyou won’t need to specify the private key location. Otherwise, command line options must point to the right place and key.

The public key is in /home/user1/.SSH/id_rsa.pub. This is the key that gets distributed to those places that want to grant you access.

Plant the public key

  1. On the remote server, server2.com, pick any account that you want to connect to.
  2. In that account’s home directory, create a .ssh subdirectory and create a new text file named authorized_keys in that directory. If a text file by this name already exists, use the existing file.
  3. On the remote server, server2.com, pick any account that you want to connect to. In that account’s home directory, create a .ssh subdirectory and create a new text file named authorized_keys in that directory. If a text file by this name already exists, use the existing file.

     Note: If you create the file and/or directory, it is recommended that the directory be chmod 700 and the file 600. This way, only the owner can access the directory and the file within it.

  4. Add to that file the contents of the id_rsa.pub file created above. That would be a *single line* that looks something like this:
    ssh-rsa <lots of characters> user1@server1.com
    
  5. Once saved, anyone possessing the private key that matches this public key can now login as this account.

SFTP

The public key was planted in the account user2 on server2.com. So now, server1.com logged in as user1, and where the private key is stored as described above, an sftp session looks like this:

sftp user2@server2.com

“user2” specifies the remote account on server2.com to login as.

Authentication happens because the private key is matched to the public key, which indicates you are authorized to login to that account. An SFTP session is opened with no interactivity required.

 Note: If you did enter a passphrase on the private key, you would have been prompted to enter it here. This is the passphrase to unlock the private key, which is local. It has nothing to do with any passwords on the remote site

Rsync

You can use Rsync for file copy operations, which can handle intelligent compression, copy only if needed, recursion, compression attribute retention, date/time stamp, and many other operations.

If all the keys are set up as above, this Rsync command copies a file from the local machine to the remote:

rsync -e ssh file user2@server2.com:/home/user2/

Local file “file” is copied to the remote /home/user2/file after logging in as “user2” using ssh as the transport (hence the “-e ssh” option), and with that, using the private/public key pair you created for authentication with no interactivity required.

SSH

SSH works as well to open up a remote shell once the keys are in place. If all keys are set up as above, this SSH command opens a remote shell on server2, logged in as user2.

ssh user2@server2.com

Example

 Note: The examples below indicate more than one space to illustrate where a space is needed. When entering your commands, please use only one space.

If using a Linux-based system, you can use either of these commands to upload the public key file to the correct location. Before using either of these commands, you must generate a private/public key.

Method Steps
SFTP
  1. Enter SFTP [OCLC file exchange account username]@[data center].oclc.org
  2. Enter put /home/[username for local machine]/.SSH/id_rsa.pub .SSH/authorized_keys
  3. Enter exit
Secure Copy
  1. Enter scp /home/[username for local machine]/.SSH/id_rsa.pub [OCLC file exchange account username]@[data center]:.SSH/authorized_keys

If using an SFTP client such as FileZilla, take the following steps to upload the public key file to the correct location. Before using this command, you must generate a private/public key.

Method Steps
SFTP
  1. On your local machine, rename the id_rsa.pub file to authorized_keys.
  2. Login to your OCLC file exchange account. You will be in the home directory, which corresponds to your username.
  3. From your home directory, click /.SSH. You will see an existing authorized_keys file.
  4. Upload the authorized_keys file from your local machine to the /.SSH directory.  You will receive a message indicating that the target file already exists; choose the action to overwrite the target file.