Date: Thu, 21 Nov 2013 13:08:05 +0100
From: Martin Prikryl
Subject: RSA signature issue when connecting to ProFTPD
I got several reports that WinSCP/PuTTY occasionally fails public key
authentication against ProFTPD server with "Server refused public-key
signature despite accepting key!"
I found out that this happens when RSA signature has different length
than RSA key. This can happen, when the last (or last couple of) octet
of signature is zero. Then rsa2_sign() outputs only n - 1 (or more) bytes.
ProFTPSD upon receiving the signature, passes it to RSA_verify (from
OpenSSL library). As the first thing, it tests that signature size
matches key size. When it does not it fails the authentication.
I'm not sure who is wrong here. In either case, it would be nice to make
PuTTY workaround this (by padding the signature with zeros?)