[ssh_x509] ssh handshake failed in FIPS mode

ssh_x509 at roumenpetrov.info ssh_x509 at roumenpetrov.info
Mon May 6 08:57:00 EEST 2019


Hi

We are using openssh and applied pkixssh patch. ( Openssh version is
7.5p1 and applied openssh-7.5p1+x509-10.1.1.diff   ). Our Openssl was
based on 1.0.2n . We were using ssh for creating tunnels between
servers and every thing were working fine ( in FIPS as well as NonFIPS
mode ).
Recently we have upgraded openssl to 1.0.2q  and handshake started
failing. I have not clue on why it fails.  Following are the
parameters set in the configuration files and output from debug log
are also provided below.

>>> sshd - service side

RSAAuthentication yes
PubkeyAuthentication yes

IgnoreUserKnownHosts no
IgnoreRhosts yes

MACs hmac-sha2-512

Ciphers aes256-ctr
KexAlgorithms ecdh-sha2-nistp384

PubkeyAlgorithms x509v3-sign-rsa
X509KeyAlgorithm x509v3-sign-rsa,rsa-sha1
<<<<

>>>>ssh - client side

MACs hmac-sha2-512
Ciphers aes256-ctr

KexAlgorithms ecdh-sha2-nistp384

PubkeyAlgorithms x509v3-sign-rsa
X509KeyAlgorithm x509v3-sign-rsa,rsa-sha1
<<<<<

Apart from the above there are other parameters also are there which I
think may not be much relevant here !

We use this for creating tunnels and take x509 cert based
authentication. We have authorisedkey file and known host file
populated properly.

When I try to connect from client to server - handshake fails ?
following is the last part of the client debug output.
>>>>
...

debug2: ciphers stoc: aes256-ctr
debug2: MACs ctos: hmac-sha2-512
debug2: MACs stoc: hmac-sha2-512
debug2: compression ctos: none
debug2: compression stoc: none
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: ecdh-sha2-nistp384
debug1: kex: host key algorithm: x509v3-sign-rsa
debug1: kex: server->client cipher: aes256-ctr MAC: hmac-sha2-512
compression: none
debug1: kex: client->server cipher: aes256-ctr MAC: hmac-sha2-512
compression: none
debug3: send packet: type 30
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY

<<<<
In server side
>>>>

debug2: ciphers ctos: aes256-ctr [preauth]
debug2: ciphers stoc: aes256-ctr [preauth]
debug2: MACs ctos: hmac-sha2-512 [preauth]
debug2: MACs stoc: hmac-sha2-512 [preauth]
debug2: compression ctos: none,zlib at openssh.com,zlib [preauth]
debug2: compression stoc: none,zlib at openssh.com,zlib [preauth]
debug2: languages ctos:  [preauth]
debug2: languages stoc:  [preauth]
debug2: first_kex_follows 0  [preauth]
debug2: reserved 0  [preauth]
debug1: kex: algorithm: ecdh-sha2-nistp384 [preauth]
debug1: kex: host key algorithm: x509v3-sign-rsa [preauth]
debug1: kex: client->server cipher: aes256-ctr MAC: hmac-sha2-512
compression: none [preauth]
debug1: kex: server->client cipher: aes256-ctr MAC: hmac-sha2-512
compression: none [preauth]
debug1: expecting SSH2_MSG_KEX_ECDH_INIT [preauth]
debug3: receive packet: type 30 [preauth]
debug3: mm_xkey_sign entering [preauth]
debug3: mm_request_send entering: type 6 [preauth]
debug3: mm_request_receive entering
debug3: monitor_read: checking request 6
debug3: mm_answer_sign
debug3: ssh_x509_sign: key alg/type/name:
x509v3-sign-rsa/RSA+cert/x509v3-sign-rsa
debug3: ssh_x509_sign: compatibility: { 0x00000000, 0x00000000 }
debug3: ssh_x509_sign: alg=x509v3-sign-rsa, md=rsa-sha1
ssh_x509_EVP_PKEY_sign: EVP_SignInit_ex fail with
errormsg='error:060B5098:lib(6):func(181):reason(152)'
debug3: ssh_x509_sign: return -22
mm_answer_sign: Xkey_sign failed: error in libcrypto
debug1: do_cleanup
debug1: Killing privsep child 28609

<<<<

What could be the problem. Any pointers on this would be of great help for me.
Thanks and regards
gas



More information about the ssh_x509 mailing list