Commit 89dde74b69 for openssl.org

commit 89dde74b69debbf0c4d0a0ee925de87638bbfe16
Author: Nikola Pajkovsky <nikolap@openssl.org>
Date:   Thu Mar 19 12:16:08 2026 +0100

    rsa_kem: validate RSA_public_encrypt() result in RSASVE

    RSA_public_encrypt() returns the number of bytes written on success and
    -1 on failure. With the existing `if (ret)` check, a provider-side RSA KEM
    encapsulation can incorrectly succeed when the underlying RSA public
    encrypt operation fails. In that case the code reports success, returns
    lengths as if encapsulation completed normally, and leaves the freshly
    generated secret available instead of discarding it.

    Tighten the success condition so RSASVE only succeeds when
    RSA_public_encrypt() returns a positive value equal to the modulus-sized
    output expected for RSA_NO_PADDING. Any other return value is treated as
    failure, and the generated secret is cleansed before returning.

    Fixes CVE-2026-31790
    Signed-off-by: Nikola Pajkovsky <nikolap@openssl.org>

    Reviewed-by: Matt Caswell <matt@openssl.foundation>
    Reviewed-by: Tomas Mraz <tomas@openssl.foundation>
    MergeDate: Mon Apr  6 19:45:38 2026

diff --git a/providers/implementations/kem/rsa_kem.c b/providers/implementations/kem/rsa_kem.c
index 78124ba90b..ab28a3a1a2 100644
--- a/providers/implementations/kem/rsa_kem.c
+++ b/providers/implementations/kem/rsa_kem.c
@@ -315,16 +315,17 @@ static int rsasve_generate(PROV_RSA_CTX *prsactx,
     /* Step(3): out = RSAEP((n,e), z) */
     ret = RSA_public_encrypt((int)nlen, secret, out, prsactx->rsa,
         RSA_NO_PADDING);
-    if (ret) {
-        ret = 1;
-        if (outlen != NULL)
-            *outlen = nlen;
-        if (secretlen != NULL)
-            *secretlen = nlen;
-    } else {
+    if (ret <= 0 || ret != (int)nlen) {
         OPENSSL_cleanse(secret, nlen);
+        return 0;
     }
-    return ret;
+
+    if (outlen != NULL)
+        *outlen = nlen;
+    if (secretlen != NULL)
+        *secretlen = nlen;
+
+    return 1;
 }

 /**