keycloak · tsaarni · Sep 10, 2024 · Oct 2, 2024 · Oct 3, 2024
diff --git a/design/secret-encryption.md b/design/secret-encryption.md
@@ -0,0 +1,163 @@
+# Encrypt / decrypt support (part II)
+
+* **Status**: Draft #1
+* **Github**: TODO
+
+## Motivation
+
+This document continues from [secure-credentials-store.md](secure-credentials-store.md) and presents the design proposal for encrypting secrets while stored in the Keycloak database.
+
+Current Vault SPI can be used to protect a subset of secrets by delegating at-rest storage to an external secret storage.
+The solution has some limitations:
+
+* Only read-only access to the vault is supported, which makes the solution suitable only for managing secrets provisioned by the administrator.
+If a secret is generated by Keycloak at runtime, like client secrets, it cannot be stored to an external vault using the current Vault SPI.
+* Using Vault SPI is not transparent, it adds complexity for the administrator since configuration of Keycloak is split into two separate steps:
+provisioning the secret into external storage and configuring a reference to Keycloak.
+
+The initial plan included part 2 to introduce encryption and decryption support, which is now proposed in this document.
+In addition to addressing the limitations of the current Vault SPI, this proposal aims to better comply with OWASP Top 10 Proactive Controls [Use Cryptography](https://top10proactive.owasp.org/the-top-10/c2-crypto/) and related regulatory requirements.
+
+
+## Overview
+
+This document outlines a design proposal for enabling Keycloak to optionally encrypt and decrypt secrets stored in the database.
+From administrator's perspective, the solution is transparent — administrator configures secrets in Keycloak as usual, and Keycloak handles the encryption before storing them in the database.
+
+
+## Scope
+
+Following fields are proposed to be encrypted.
+
+* Client secret generated by Keycloak.
+* Client secret configured by administrator for identity brokering.
+* External tokens stored in the database by Keycloak when "Store tokens" is configured for identity brokering.
+* Realm keys generated by Keycloak.
+* OTP shared secrets generated by Keycloak.
+* LDAP credentials configured by administrator.
+* SMTP credentials configured by administrator.
+
+Fields that need to remain searchable in the database cannot be encrypted.
+This can include fields like username, email, or other fields that can be considered as privacy-sensitive but need to be searchable.
+
+
+## Use cases
+
+Following use cases exists:
+
+* Encryption is enabled in a new installation and there are no plaintext secrets in the database.
+  Secrets are encrypted as they are created.
+* Encryption is enabled in an existing installation and there are plaintext secrets in the database.
+  Secrets are encrypted as a batch operation.
+* New secret type is introduced in an existing installation where said secret type was not encrypted before.
+  The existing plaintext secrets are encrypted as a batch operation.
+* KEK is rotated.
+  New seed is generated and encrypted with the new KEK.
+  All secrets are re-encrypted as a batch operation.
+* Encryption is disabled.
+  Secrets are decrypted as a batch operation.
+  This could be used to migrate data back to plaintext or to move to another encryption provider.
+  It could be used to recover from unseen issues.
+
+
+## Design
+
+### Encryption
+
+#### Overview
+
+The high level approach is as follows:
+
+* Each secret is encrypted locally using a data encryption key (DEK).
+* A new DEK is generated locally each time a secret is encrypted.
+* DEK generation is based on a key derivation algorithm (KDF) that uses a seed and random nonce generated for each secret.
+* A single seed is used to generate all DEKs, and this seed is encrypted using a key encryption key (KEK).
+* KEK can be stored externally.
+
+The encrypted seed is stored locally in the database, decrypted when  needed for the first time, and kept in memory thereafter.
+This approach aims to minimize the use of the (remote) KEK for performance reasons.
+
+The data needed to derive the same DEK again when the secret is decrypted is stored in plaintext with the encrypted data.
+
+Reference: The approach is following a solution that is used by Kubernetes: [Kubernetes KMS v2 improvements](https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/3299-kms-v2-improvements).
+
+#### Cryptography details
+
+> **TODO**
+
+
+### Data storage
+
+Secrets are stored in the database within a wrapper that includes metadata and the encrypted secret.
+
+```
+${enc:<key-id>:<nonce>:<encrypted-data>}
+```
+
+* `enc:` is a prefix that indicates the value contains encrypted data.
+* `<key-id>` is the identifier of the key used to encrypt the seed.
+* `<nonce>` is a public random value combined with the secret seed to derive the DEK.
+* `<encrypted-data>` is the encrypted secret in base64 encoding.
+
+> **TODO**:
+>
+> Q: Should data be stored in the column where plaintext data was stored or in a separate table?
+> If data is stored in separate table, how is the referential integrity maintained?
+>
+> Q: How can secrets be located in the database?
+> See chapter "Use cases".
+> For batch operations, a mechanism need to be implemented to locate secrets in the database.
+
+
+### Service Provider Interface
+
+The encryption logic can be integrated into the core of Keycloak.
+However, the remote KEK management is implemented via a separate SPI.
+This SPI handles the encryption and decryption of DEK seeds using a (potentially) remote service.
+By default, Keycloak could provide an implementation for [OpenBao](https://openbao.org/), an open-source fork of HashiCorp Vault.
+
+
+## Comparison to other solutions
+
+### Data partition encryption
+
+Data partition encryption is managed by the operating system, either at the file system level or the block device level.
+This approach is transparent to both Keycloak and the database, which do not need to be aware of the encryption.
+
+However, there are some drawbacks:
+
+* Administrators may not have control over whether the underlying file system supports encryption.
+* Once the partition is mounted, the data is accessible in plaintext.
+  Therefore, it mitigates threats such as the theft of the physical storage device, but does not protect against threats where the attacker has access to the running operating system, can execute SQL queries, or access exported database backups.
+
+Due to these limitations, users may prefer to implement encryption by other means.
+
+
+### Database level encryption
+
+Certain databases offer transparent data encryption (TDE), which encrypts data before writing it to disk and decrypts it when reading it back.
+
+The status of TDE support in open-source databases is as follows:
+
+* PostgreSQL does not support TDE.
+  There has been long-standing [discussion]((https://wiki.postgresql.org/wiki/Transparent_Data_Encryption)) in community.
+  The latest effort is by Percona who is [developing](https://www.percona.com/blog/the-making-of-an-open-source-postgresql-tde-extension/) a [TDE extension](https://github.com/Percona-Lab/postgresql-tde) for PostgreSQL.
+* MariaDB supports [Transparent data encryption](https://mariadb.com/kb/en/data-at-rest-encryption-overview/) with some [limitations](https://mariadb.com/kb/en/data-at-rest-encryption-overview/#limitations).
+
+TDE mitigates threats like the theft of the physical storage device but does not protect against threats where the attacker can execute SQL queries.
+Implementation-specific limitations exist e.g. regarding which parts of the data are encrypted and how backup utilities interact with TDE.
+
+
+### SQL extensions for encryption
+
+Databases may provide extensions to the SQL language that allow encryption and decryption of data while executing SQL queries.
+These are not transparent to the application, as the application must explicitly include the function calls in the SQL queries.
+
+Although it is theoretically possible to build an encryption solution using low-level SQL encryption primitives, creating a production-ready solution with features like key rotation would be challenging.
+Achieving good performance could be difficult due to the need to include the encryption key in the query.
+Additionally, the solution would be database-specific, requiring modifications to Keycloak to generate SQL queries tailored to each database.
+
+Example of SQL extensions include:
+
+* PostgreSQL [PGCrypto](https://www.postgresql.org/docs/current/pgcrypto.html) extension.
+* MariaDB [AES_ENCRYPT and AES_DECRYPT](https://mariadb.com/kb/en/aes_encrypt/) functions.