Crypto Backend Redesign

Overview

This document defines the target architecture for the Trustpoint cryptography redesign.

It is intentionally opinionated and does not preserve the current credential-centric crypto design. The goal is to replace the current mixed software/PKCS#11 implementation with a maintainable architecture that gives Trustpoint one application-facing crypto backend while keeping PKCS#11 concerns fully contained.

This document is the target-state design for future work. It does not describe the current implementation. For the current credential architecture, see Credentials - Architecture.

Status

This is a design-and-implementation blueprint for the planned redesign.

Goals

• Trustpoint application code uses one crypto backend interface.

• PKCS#11 is the custody and provider model for persistent server-held keys.

• Trustpoint business code does not know whether the provider is SoftHSM, a simulator, or a hardware HSM.

• Low-level PKCS#11 concerns stay in one adapter layer.

• Long-lived private keys are not stored in the Trustpoint database.

• One-time downloadable credentials remain supported through backend-controlled export flows.

• Vendor-specific PKCS#11 library differences are handled centrally.

• Bootstrap, restore, concurrency, error mapping, and observability are part of the design rather than bolt-ons.

Non-Goals

• Redesigning Trustpoint ingress TLS termination.

  That remains out of scope for this crypto redesign.

• Preserving the current CredentialModel crypto abstraction.

• Preserving the current KEK/DEK/backup-password design around PKCS11Token.

• Allowing PKCS#11 concepts to appear in forms, views, request processors, or business services.

Repository Findings That Drive This Design

The redesign is grounded in the current repository structure and pain points:

• pki.models.credential.CredentialModel currently mixes certificate persistence, software key storage, PKCS#11 key references, HSM import/generation, and runtime key reconstruction.

• management.pkcs11_util makes PKCS#11 keys imitate cryptography private key interfaces, which leaks provider and mechanism details into the rest of the application.

• management.models.pkcs11.PKCS11Token currently mixes token metadata, PIN retrieval, session handling, KEK/DEK management, backup-password recovery, and runtime crypto state.

• Request and business flows sign directly with raw key objects in modules such as:

  • devices.issuer

  • request.operation_processor.issue_cert

  • request.operation_processor.sign

  • request.operation_processor.csr_sign

  • pki.util.crl

  • request.message_builder.cmp

  • request.message_responder.cmp

• Export/download flows in devices.views.download assume a generic stored credential can always be turned back into PKCS#12 or PEM.

• Global storage toggles in management.models.key_storage.KeyStorageConfig force application code to think in terms of software-vs-HSM storage instead of use-case policy.

Core Design Decision

Trustpoint will expose one application-facing backend:

• CryptoBackend is the only crypto boundary used by application code.

• Trustpoint does not branch on software vs HSM vs SoftHSM.

• The implementation family behind that backend is PKCS#11-based.

• Development and demo profiles use SoftHSM or an HSM simulator through the same backend contract.

The application sees one backend. Provider differences are handled by backend configuration and capability probing, not by application branching.

Use-Case Model

The design separates crypto use cases by lifecycle, not by provider type.

Managed Keys

Managed keys are long-lived keys that Trustpoint keeps using after creation.

Examples:

• local root CA keys

• local issuing CA keys

• signer keys

• CMP/EST response signing keys

• other persistent server-held private keys

Properties:

• stored and used through PKCS#11

• referenced by stable backend identifiers

• non-exportable by default

• restored by rebinding to the provider and verifying that the key still exists

Export Bundles

Export bundles are one-time delivery artifacts produced for downstream devices.

Examples:

• downloadable PKCS#12 packages

• password-protected PEM ZIP/TAR bundles

• temporary server-generated credentials that are shipped once and then used outside Trustpoint

Properties:

• created through the same CryptoBackend

• not treated as Trustpoint-managed long-lived keys

• may use provider-native exportable flows or backend-owned transient generation, depending on provider capabilities

• persisted only as metadata and audit information, not as reusable long-lived private keys in the DB

This separation is critical. It avoids forcing one-time delivery credentials into the same lifecycle as persistent CA and signer keys.

Signing Authority Policy

Trustpoint signing-authority keys must be backend-managed and non-exportable. This includes local root CA keys, local issuing CA keys, signer keys, and other long-lived server-held keys that Trustpoint uses to issue or sign credentials.

Endpoint and delivery credentials have a different lifecycle. Bootstrap TLS credentials, imported TLS server credentials, device or endpoint credentials, and owner/enrollment credentials may remain local, imported, or delivery-oriented workflow material unless a feature explicitly requires managed-key custody. Those keys are not treated as Trustpoint signing-authority keys.

Top-Level Architecture

Layer Responsibilities

Application / Business Layer

Application code may:

• request a key by alias or role

• ask for signing, certificate issuance, verification, or export bundle generation

• receive stable domain-level objects such as ManagedKeyRef or ExportBundleRef

Application code may not:

• import pkcs11

• open sessions

• choose mechanisms

• track object handles

• load PINs

• branch on provider type

Crypto Application Layer

This layer exposes the stable backend contract and orchestrates use cases:

• KeyManager: generate, import, lookup, public-key retrieval, existence checks

• CertificateService: certificate issuance, CSR creation, CRL issuance

• SigningService: sign, verify, hash, MAC

• BundleExportService: one-time exportable credential bundles

• SecretProtectionService: application secret protection for values such as onboarding secrets

Provider Layer

The provider layer contains the implementation family used by Trustpoint:

• Pkcs11Backend

• provider profile loading

• capability probing

• vendor-specific overrides

Trustpoint may support multiple PKCS#11 provider profiles over time, but the application-facing contract remains one.

PKCS#11 Adapter Layer

This is the only place that knows about:

• library loading

• slot/token selection

• login and relogin

• session pooling

• object lookup by CKA_ID and label

• mechanism selection

• provider-specific quirks

• error normalization

Target Package Structure

The target Python package structure should look roughly like this:

trustpoint/crypto/
  domain/
    algorithms.py
    errors.py
    policies.py
    refs.py
    specs.py
  application/
    backend.py
    keys.py
    certificates.py
    signing.py
    bundles.py
    secrets.py
  adapters/
    pkcs11/
      backend.py
      config.py
      capability_probe.py
      session_pool.py
      locator.py
      mechanisms.py
      error_map.py
      vendor_overrides.py
  django/
    models.py
    repositories.py

Backend Contract

The application-facing backend should be small and operation-oriented.

class CryptoBackend(Protocol):
    def ensure_managed_key(self, alias: str, spec: KeySpec, policy: KeyPolicy) -> ManagedKeyRef: ...
    def import_managed_key(self, alias: str, pkcs8_pem: bytes, policy: KeyPolicy) -> ManagedKeyRef: ...
    def get_managed_key(self, alias: str) -> ManagedKeyRef: ...
    def public_key(self, key: ManagedKeyRef) -> PublicKey: ...
    def sign(self, key: ManagedKeyRef, data: bytes, request: SignRequest) -> bytes: ...
    def issue_certificate(self, issuer: CredentialRef, subject: PublicKey | ManagedKeyRef, spec: CertificateSpec) -> IssuedCertificate: ...
    def create_csr(self, key: ManagedKeyRef, spec: CsrSpec) -> CertificateSigningRequest: ...
    def issue_crl(self, issuer: CredentialRef, spec: CrlSpec) -> CertificateRevocationList: ...
    def create_export_bundle(self, request: ExportBundleRequest) -> ExportBundleRef: ...
    def read_export_bundle(self, bundle: ExportBundleRef) -> ExportBundlePayload: ...
    def protect_secret(self, plaintext: bytes, purpose: SecretPurpose) -> ProtectedSecret: ...
    def unprotect_secret(self, protected: ProtectedSecret) -> bytes: ...

Key Referencing Strategy

The redesign must stop relying on label-only PKCS#11 references.

Managed keys should be referenced by:

• internal Trustpoint UUID

• stable application alias

• provider profile id

• PKCS#11 object identity based primarily on CKA_ID

• optional human-readable label

• stored public-key fingerprint

• algorithm and capability metadata

Object handles are never persisted. They are session-local runtime details inside the PKCS#11 adapter.

Provider Profiles

The current KeyStorageConfig and PKCS11Token models should be replaced by an explicit provider profile model.

Provider profile fields should include:

• profile name

• PKCS#11 module library path

• token selector:

  • serial number preferred

  • label optional

  • slot index only as fallback

• authentication source:

  • environment variable

  • file path / secret file

  • external secret provider hook

• optional mechanism overrides

• optional vendor name / driver family

• active flag

• last capability probe result

This makes different HSM vendor libraries a configuration problem instead of an architectural fork.

Capability Probing

At startup and whenever a provider profile changes, the backend should probe and cache capabilities such as:

• supported key generation mechanisms

• supported sign mechanisms

• supported curves

• supported RSA sizes

• wrap/unwrap support

• object copy/import constraints

• login/session quirks

The rest of the backend uses a mechanism policy that selects from supported options instead of hardcoding SoftHSM assumptions.

Managed Key Flow

Export Bundle Flow

Export bundles are intentionally separate from managed keys.

The backend may implement this using:

• exportable PKCS#11 objects where the provider supports it, or

• backend-owned transient key generation for delivery-only credentials

The application does not care which path was used.

Sessions, Login, and Concurrency

PKCS#11 session management must be centralized.

Rules:

• load each PKCS#11 library once per process

• keep one provider object per active profile

• use a bounded session pool per provider profile

• login when a session is created or borrowed, depending on provider behavior

• never share raw session objects with business code

• always reacquire object handles per session

• treat handles as ephemeral

Expected session-pool behavior:

• borrow/release for each operation

• retry once on invalid-session or provider-reset errors

• surface clear degraded-mode errors if the provider is unavailable

Certificates and Credential Records

Certificates and chains remain database records. Private key custody does not.

The future replacement for the current credential records should store:

• leaf certificate id

• chain certificate ids

• managed key reference for persistent keys, if applicable

• export bundle reference for delivery-only credentials, if applicable

• role / usage classification

• certificate status metadata

The DB stores certificate-related state and key references. It does not store reusable private key material for managed keys.

Secrets Protection

The current encrypted-field design should be replaced by a dedicated secret protection service.

It must not:

• call PKCS11Token.objects.first() from a model field

• depend on application startup cache state

• mix field serialization with provider login/session logic

It should:

• use the same CryptoBackend boundary

• have a dedicated purpose model

• rotate or rebind cleanly during restore

• keep secret protection independent from credential issuance flows

Bootstrap and Restore

Bootstrap and restore must be explicit backend workflows.

Bootstrap responsibilities:

• configure active provider profile

• validate module path and token selection

• authenticate successfully

• probe capabilities

• persist provider profile metadata

• create managed keys lazily or explicitly, depending on feature policy

Restore responsibilities:

• restore DB state

• rebind to the configured provider profile

• verify that referenced managed keys still exist

• verify that stored public-key fingerprints still match

• mark missing keys as degraded state instead of failing silently

Error Handling

The backend must expose a small, domain-oriented error model.

Recommended error categories:

• ProviderUnavailableError

• ProviderAuthError

• ManagedKeyNotFoundError

• MechanismUnsupportedError

• KeyPolicyViolationError

• ExportNotAllowedError

• BundleExpiredError

• SecretProtectionError

• TransientProviderError

All low-level PKCS#11 exceptions are mapped centrally.

Observability

Every backend operation should emit:

• operation name

• provider profile id

• algorithm

• mechanism chosen

• latency

• success/failure

• normalized error type

Logs must never contain:

• PIN values

• plaintext secrets

• private key material

• full sensitive payloads

What Should Be Removed

The redesign explicitly replaces the following current architectural elements:

• pki.models.credential.CredentialModel as the main crypto abstraction

• management.models.pkcs11.PKCS11Token

• management.models.key_storage.KeyStorageConfig as a global software-vs-HSM switch

• management.pkcs11_util as a public crypto layer

• PKCS#11 usage in forms, views, request processors, and Django models

• DB storage of long-lived managed private keys

Out-of-Scope but Related

• ingress TLS termination integration with Nginx/OpenSSL

• UI redesign for crypto/provider administration

• multi-tenant HSM partitioning

Summary

The target architecture is:

• one application-facing crypto backend

• PKCS#11-backed persistent key custody

• SoftHSM/simulator for dev and demo

• explicit provider profiles for vendor library support

• managed keys separated from one-time export bundles

• no PKCS#11 leakage into Trustpoint business code

• no DB-stored long-lived private keys