Skip to content

[OpenSSL 4 compatibility] Use accessors for opaque ASN1_STRING#136

Open
karljs wants to merge 3 commits into
cisco:mainfrom
karljs:openssl4-compat
Open

[OpenSSL 4 compatibility] Use accessors for opaque ASN1_STRING#136
karljs wants to merge 3 commits into
cisco:mainfrom
karljs:openssl4-compat

Conversation

@karljs

@karljs karljs commented Jul 22, 2026

Copy link
Copy Markdown

OpenSSL 4.0 makes ASN1_STRING (and its typedefs ASN1_OCTET_STRING / ASN1_BIT_STRING) opaque, so direct data and length field access no longer compiles.

This adds the accessors instead, while keeping direct access under HAVE_OLD_OPENSSL in keeping with the rest of the codebase.

@karljs
karljs marked this pull request as draft July 22, 2026 22:30
@karljs

karljs commented Jul 22, 2026

Copy link
Copy Markdown
Author

Unfortunately, it looks like this is only a fix for the compilation, but not the linking. I've converted to draft status until I get the complete fix.

karljs added 3 commits July 24, 2026 12:56
OpenSSL 4.0 makes ASN1_STRING (and its typedefs ASN1_OCTET_STRING /
ASN1_BIT_STRING) opaque. This uses the ASN1_STRING_get0_data() and
ASN1_STRING_length() accessors instead, while keeping the direct access
under HAVE_OLD_OPENSSL.
OpenSSL 4.0 removes the ENGINE API, so the est_apps_startup() /
est_apps_shutdown() macros in est.h no longer link: they call
ENGINE_load_builtin_engines() / ENGINE_cleanup() and friends.

Add OpenSSL >= 4.0 variants of both macros that load the default
provider instead, keeping the existing variants for older versions,
matching the HAVE_OLD_OPENSSL pattern already used in this header.
Both ossl_srv.c copies (example/server, test/util) fail to build
against OpenSSL 4.0 due to relying on removed functionality.

Fixes:
- Add a shim mapping ASN1_STRING_get0_data() to ASN1_STRING_data() on
  OpenSSL < 1.1.0, then use the ASN1_STRING_type/length/get0_data
  accessors unconditionally. These files do not include config.h, so
  HAVE_OLD_OPENSSL is never defined here; keying on
  OPENSSL_VERSION_NUMBER is what actually keeps 1.0.x building.
- Retag DN entries in the msie_hack path with
  X509_NAME_ENTRY_set_data() instead of writing str->type, which has
  no setter once the type is opaque.
- Replace BN_pseudo_rand() with BN_rand(), available on all versions.
- Ifdef out the FORMAT_ENGINE branch on OpenSSL >= 4.0.
@karljs
karljs force-pushed the openssl4-compat branch from 437617b to 8888a06 Compare July 24, 2026 19:57
@karljs

karljs commented Jul 24, 2026

Copy link
Copy Markdown
Author

This was more invasive than I was hoping, but I was able to build all the example programs and whatnot locally.

@karljs
karljs marked this pull request as ready for review July 24, 2026 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant