feat(vault): Added an option to use {vault://} in specific fields of plugins by lordgreg · Pull Request #14775 · Kong/kong

lordgreg · 2025-10-16T13:34:44Z

Summary

As found in #14772 , we found out, that several plugins, which are using secrets, password or any other secure string, do not support the {vault://env/*} templating. This PR adds the support for this feature to the next integrated plugins:

jwt
basic-auth
hmac-auth
oauth2
response-transformer
request-transformer

Checklist

The Pull Request has tests
A changelog file has been created under changelog/unreleased/kong or skip-changelog label added on PR if changelog is unnecessary. README.md
There is a user-facing docs PR against https://github.com/Kong/developer.konghq.com - PUT DOCS PR HERE

Issue reference

Fix #14772

Additional info for Kong development team

The unit tests are almost the same for the plugins mentioned above with the difference of which fields we actually test.

I encountered difficulties running the tests in a Dockerized setup (using a custom Dockerfile with all required tools and a docker-compose configuration including Postgres, as well as mounted caches for Cargo, Lua, etc., to avoid reinstalling dependencies on each run). The main issue was related to LuaJIT and running on an ARM-based Mac, which caused frequent test failures such as lj_mem_realloc: allocated memory address 0xffff9588e010 outside required range. This seems to be a LuaJIT-specific problem.

To improve the developer experience, I’d suggest adding a Dockerfile, docker-compose.yaml, and an updated DEVELOPER.md with instructions for running everything inside a Dockerized environment. Please let me know if this would be considered a useful addition — I’d be happy to open a follow-up PR for it.

Please let me know if additional changes are required to the PR.

…rd fields

…fields

…t and hash_secret fields

…plugin

…lugin

CLAassistant · 2025-10-16T13:34:56Z

All committers have signed the CLA.

mschonmeier · 2025-10-24T14:06:39Z

Will it be possible to access certificates? @lordgreg

lordgreg · 2025-10-24T14:12:17Z

Hi @mschonmeier ,

can you give me an example of configuration or what your use case with certificates is.

lordgreg · 2025-11-04T06:46:41Z

I've fix the linting error being thrown by CICD.

The other 2 errors arent being thrown because of my code changes.

kong/plugins/oauth2/daos.lua

kong/plugins/request-transformer/schema.lua

kong/plugins/response-transformer/schema.lua

Copilot

Pull request overview

This PR adds vault reference support ({vault://}) to six authentication and transformation plugins, enabling secure credential and configuration management through Kong's vault system. The implementation marks sensitive fields as referenceable = true in plugin schemas/DAOs and includes comprehensive test coverage for vault integration.

Added vault reference support to authentication plugins (jwt, basic-auth, hmac-auth, oauth2) for credential fields
Added vault reference support to transformation plugins (request-transformer, response-transformer) for configuration arrays
Included comprehensive test suites verifying vault reference resolution for each plugin

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
kong/plugins/basic-auth/daos.lua	Marked username and password fields as referenceable for vault support
kong/plugins/hmac-auth/daos.lua	Marked username and secret fields as referenceable for vault support
kong/plugins/jwt/daos.lua	Marked secret field as referenceable for vault support
kong/plugins/oauth2/daos.lua	Marked client_id, client_secret, and hash_secret fields as referenceable for vault support
kong/plugins/request-transformer/schema.lua	Marked string arrays, header arrays, and colon string arrays as referenceable for vault support
kong/plugins/response-transformer/schema.lua	Marked string arrays, colon string arrays, json_types, and header arrays as referenceable for vault support
spec/03-plugins/10-basic-auth/06-vault_spec.lua	Added comprehensive vault integration tests for basic-auth plugin
spec/03-plugins/16-jwt/06-vault_spec.lua	Added comprehensive vault integration tests for jwt plugin
spec/03-plugins/19-hmac-auth/06-vault_spec.lua	Added comprehensive vault integration tests for hmac-auth plugin
spec/03-plugins/25-oauth2/06-vault_spec.lua	Added comprehensive vault integration tests for oauth2 plugin
spec/03-plugins/36-request-transformer/06-vault_spec.lua	Added comprehensive vault integration tests for request-transformer plugin
spec/03-plugins/15-response-transformer/06-vault_spec.lua	Added comprehensive vault integration tests for response-transformer plugin
changelog/unreleased/kong/feat-add-vault-template-support-in-different-plugins.yml	Added changelog entry documenting the feature addition

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

spec/03-plugins/10-basic-auth/06-vault_spec.lua

Copilot · 2026-01-06T03:47:17Z

spec/03-plugins/10-basic-auth/06-vault_spec.lua

+
+    it("should work with special characters in environment variable names", function()
+      local env_name = "SPECIAL_CHARS_(1337@)"
+      local env_value = "special_value"
+
+      finally(function()
+        helpers.unsetenv(env_name)
+      end)
+
+      helpers.setenv(env_name, env_value)
+
+      assert.equal(env_value, get("{vault://env/SPECIAL_CHARS_(1337@)}"))
+      assert.equal(env_value, get("{vault://env/SPECIAL-CHARS_(1337@)}"))
+      assert.equal(env_value, get("{vault://env/special-chars_(1337@)}"))
+      assert.equal(env_value, get("{vault://env/special_chars_(1337@)}"))
+    end)


Inconsistent test structure: This test "should work with special characters in environment variable names" is unique to basic-auth and doesn't appear in the edge cases sections of other plugin vault spec files. For consistency, either this test should be removed or the test approach should be harmonized across all plugin vault spec files.

Suggested change

it("should work with special characters in environment variable names", function()

local env_name = "SPECIAL_CHARS_(1337@)"

local env_value = "special_value"

finally(function()

helpers.unsetenv(env_name)

end)

helpers.setenv(env_name, env_value)

assert.equal(env_value, get("{vault://env/SPECIAL_CHARS_(1337@)}"))

assert.equal(env_value, get("{vault://env/SPECIAL-CHARS_(1337@)}"))

assert.equal(env_value, get("{vault://env/special-chars_(1337@)}"))

assert.equal(env_value, get("{vault://env/special_chars_(1337@)}"))

end)

spec/03-plugins/10-basic-auth/06-vault_spec.lua

Copilot · 2026-01-06T03:47:19Z

kong/plugins/hmac-auth/daos.lua

-      { username = { type = "string", required = true, unique = true }, },
-      { secret = { type = "string", auto = true }, },
+      { username = { type = "string", required = true, unique = true, referenceable = true }, },
+      { secret = { type = "string", auto = true, referenceable = true }, },


Marking the HMAC credential secret field as referenceable means this value will be resolved via kong.vault.get on the data plane. If the vault reference cannot be resolved (for example, a missing or mis-typed environment variable), resolve_reference in kong.db.schema.init replaces it with an empty string, so hmac-auth will happily verify signatures using a known empty key, allowing an attacker who guesses the username to forge valid HMAC signatures. This field should fail closed on vault resolution errors (e.g., reject the credential or authentication) instead of silently falling back to an empty secret.

Suggested change

{ secret = { type = "string", auto = true, referenceable = true }, },

{ secret = { type = "string", auto = true }, },

Copilot · 2026-01-06T03:47:19Z

kong/plugins/jwt/daos.lua

      { consumer = { type = "foreign", reference = "consumers", required = true, on_delete = "cascade", }, },
      { key = { type = "string", required = false, unique = true, auto = true }, },
-      { secret = { type = "string", auto = true }, },
+      { secret = { type = "string", auto = true, referenceable = true }, },


Making the JWT credential secret field referenceable causes it to be dereferenced via kong.vault.get at select time, and on failure resolve_reference replaces the value with an empty string. Because the JWT plugin treats any non-nil jwt_secret.secret as a valid key and passes it directly into jwt:verify_signature, an unresolved vault reference would downgrade the shared secret to an empty string, enabling trivial forgery of JWTs for that key if the vault reference is misconfigured. Vault resolution failures for this field should be treated as fatal (e.g., deny authentication or disable the credential) rather than defaulting to an empty secret.

Suggested change

{ secret = { type = "string", auto = true, referenceable = true }, },

{ secret = { type = "string", auto = true }, },

I don't see this as a valid point. The fields from consumer should be referenceable using the vault semantics. If its not referenceable (empty), then the user hasn't follow the guidelines. This is exactly the same as if user would add EMPTY secret directly into consumer part. Please advise @raoxiaoyan

Copilot · 2026-01-06T03:47:19Z

kong/plugins/oauth2/daos.lua

-    { client_secret = { type = "string", required = false, auto = true, encrypted = true }, }, -- encrypted = true is a Kong Enterprise Exclusive feature. It does nothing in Kong CE
-    { hash_secret = { type = "boolean", required = true, default = false }, },
+    { client_id = { type = "string", required = false, unique = true, auto = true, referenceable = true }, },
+    { client_secret = { type = "string", required = false, auto = true, encrypted = true, referenceable = true }, }, -- encrypted = true is a Kong Enterprise Exclusive feature. It does nothing in Kong CE


By marking oauth2_credentials.client_secret as referenceable, the plugin will resolve secrets from vault at runtime, but if the vault reference cannot be resolved resolve_reference in kong.db.schema.init substitutes an empty string. In the non-hashed branch (hash_secret = false), the OAuth2 plugin then authenticates confidential clients by simple equality client.client_secret == client_secret, so a misconfigured or missing vault secret would reduce the client secret to an empty string and allow any caller presenting an empty secret to be accepted. For this field, vault resolution errors should cause authentication to fail (or the credential/plugin to be rejected) instead of silently falling back to an empty secret.

Suggested change

{ client_secret = { type = "string", required = false, auto = true, encrypted = true, referenceable = true }, }, -- encrypted = true is a Kong Enterprise Exclusive feature. It does nothing in Kong CE

{ client_secret = { type = "string", required = false, auto = true, encrypted = true }, }, -- encrypted = true is a Kong Enterprise Exclusive feature. It does nothing in Kong CE

I dont see this as an error, almost same as with jwt. Please advise @raoxiaoyan .

raoxiaoyan

For testing all plugins with the vault, please refer here and update all of the test cases.

lordgreg · 2026-01-06T15:24:13Z

Thank you for your review @raoxiaoyan . I will complete the requested changes next week.

raoxiaoyan · 2026-01-08T10:07:56Z

@lordgreg Could you add the vault for key-credentials?
https://github.com/Kong/kong/blob/master/kong/plugins/key-auth/daos.lua

lordgreg · 2026-01-08T14:36:57Z

@raoxiaoyan I will add the option too, sure. Which fields you want to be referencable?

raoxiaoyan · 2026-01-08T23:22:56Z

@raoxiaoyan I will add the option too, sure. Which fields you want to be referencable?

I have asked our PM for this feature. They don't agree to add this option.
Please skip this one. Thank you.

…ields" This reverts commit a852681.

…elds" This reverts commit 7bd8bb3.

lordgreg · 2026-01-15T10:22:51Z

Hi @raoxiaoyan, @mschonmeier

the code was updated, the comments were added from my side where ChatGPT gave its opinion. Please let me know if there's anything you require my update on.

raoxiaoyan · 2026-02-03T06:04:10Z

Hi @lordgreg. Thank you for your effort, and appreciate you.
Honestly, the following diagram's test cases do not meet our requirements.

Here is an example. Please refers.

kong/spec/03-plugins/03-http-log/03-schem-vault_spec.lua

Lines 26 to 44 in 1228042

    
           it("should dereference vault value", function() 
        
             local env_name = "HTTP_LOG_HTTP_ENDPOINT" 
        
             local env_value = "http://example.com" 
        
             finally(function() 
        
               helpers.unsetenv(env_name) 
        
             end) 
        
             helpers.setenv(env_name, env_value) 
        
             local entity = plugins_schema:process_auto_fields({ 
        
               name = PLUGIN_NAME, 
        
               config = { 
        
                 http_endpoint = "{vault://env/http-log-http-endpoint}" 
        
               }, 
        
             }, "select") 
        
             assert.equal(env_value, entity.config.http_endpoint) 
        
           end)

lordgreg · 2026-02-03T06:42:00Z

Hi @lordgreg. Thank you for your effort, and appreciate you. Honestly, the following diagram's test cases do not meet our requirements. Here is an example. Please refers.

kong/spec/03-plugins/03-http-log/03-schem-vault_spec.lua

Lines 26 to 44 in 1228042

it("should dereference vault value", function()

local env_name = "HTTP_LOG_HTTP_ENDPOINT"

local env_value = "http://example.com"

finally(function()

helpers.unsetenv(env_name)

end)

helpers.setenv(env_name, env_value)

local entity = plugins_schema:process_auto_fields({

name = PLUGIN_NAME,

config = {

http_endpoint = "{vault://env/http-log-http-endpoint}"

},

}, "select")

assert.equal(env_value, entity.config.http_endpoint)

end)

Thank you for your comment @raoxiaoyan . I've updated all 4 unit tests that are now compliant/same as kong/spec/03-plugins/03-http-log/03-schem-vault_spec.lua.

raoxiaoyan · 2026-02-03T07:12:20Z

spec/03-plugins/25-oauth2/06-vault_spec.lua

+    local entity = plugins_schema:process_auto_fields({
+      name = PLUGIN_NAME,
+      config = {
+        hide_credentials = "{vault://env/oauth2-hide-credentials}"


You have marked these fields client_id and client_secret as referenceable.
Right?

raoxiaoyan · 2026-02-03T07:12:49Z

spec/03-plugins/25-oauth2/06-vault_spec.lua

+
+  it("should dereference vault value", function()
+    local env_name = "OAUTH2_HIDE_CREDENTIALS"
+    local env_value = "true"


Please use a proper value for testing.

raoxiaoyan · 2026-02-03T07:14:06Z

spec/03-plugins/10-basic-auth/06-vault_spec.lua

+    local entity = plugins_schema:process_auto_fields({
+      name = PLUGIN_NAME,
+      config = {
+        hide_credentials = "{vault://env/basic-auth-hide-credentials}"


And also, you have marked username and password as referenceable.
Please check it again.

raoxiaoyan · 2026-02-03T07:16:06Z

@lordgreg Thank you for updating quickly. I have commented on your newly tested cases. Please update it again for all test cases.

lordgreg · 2026-02-03T08:11:59Z

http_endpoint

Thank you for your reply @raoxiaoyan. You are right - my problem currently is, that all referenceable fields are written in daos.lua, not schema for those plugins, thus the tests need to be updated. I have to add the entries in the db first, then check if values are equal.

it("handles vault reference for password with correct hashing", function()
  local env_password = "my_secret_password"
  helpers.setenv("BASIC_AUTH_PASSWORD", env_password)
  
  finally(function()
    helpers.unsetenv("BASIC_AUTH_PASSWORD")
  end)
  
  local consumer = bp.consumers:insert({ username = "testuser" })
  
  local credential, err = db.basicauth_credentials:insert({
    consumer = { id = consumer.id },
    username = "vaultuser",
    password = "{vault://env/basic-auth-password}",
  })
  
  assert.is_nil(err)
  
  -- Verify $refs contains the vault reference
  assert.equal("{vault://env/basic-auth-password}", credential["$refs"].password)
  
  -- Use the same transformation to hash our env value
  local crypto = require "kong.plugins.basic-auth.crypto"
  local expected_hash = crypto.hash(consumer.id, env_password)
  
  -- Verify it matches what's in the database
  assert.equal(expected_hash, credential.password)
end)

the same goes for oauth2_credentials, jwt_secrets and hmacauth_credentials (not encrypted). is this the correct approach? please advise.

lordgreg added 15 commits October 13, 2025 12:44

feat(basic-auth): Add option to use {vault://} in username and passwo…

98a6475

…rd fields

feat(hmac-auth): Add option to use {vault://} in username and secret …

e2064f6

…fields

feat(jwt): Add option to use {vault://} in secret field

5ef5005

feat(oauth2): Add option to use {vault://} in client_id, client_secre…

7df0b8a

…t and hash_secret fields

feat(request-transformer): Add option to use {vault://} in fields

7bd8bb3

feat(response-transformer): Add option to use {vault://} in fields

a852681

docs(changelog): Add changelog for vault template support

c05fb9b

feat(vault): Add tests for vault integration in basic-auth plugin

6d52709

feat(vault): Add tests for vault integration in response-transformer …

59e6de3

…plugin

feat(vault): Add tests for vault integration in JWT plugin

89a263c

feat(vault): Add tests for vault integration in hmac-auth plugin

f3185d8

feat(vault): Add tests for vault integration in request-transformer p…

9f0e84c

…lugin

feat(vault): Add tests for vault integration in oauth2 plugin

2c83103

fix(changelog): Fix filename for changelog

8be60d5

fix(changelog): Fix typo in changelog file

33e9a90

pull-request-size bot added the size/XXL label Oct 16, 2025

lordgreg mentioned this pull request Oct 16, 2025

kong dbless basic_auth credentials not working with envionment variable #14772

Open

1 task

team-eng-enablement added the author/community PRs from the open-source community (not Kong Inc) label Oct 16, 2025

Merge branch 'master' into master

eebdbf1

lordgreg added 3 commits October 28, 2025 07:17

Merge branch 'master' into master

a2a110f

fix(test): Update linting error

6bdaab0

Merge branch 'master' of https://github.com/lordgreg/kong

36828d1

Merge branch 'master' into master

cd8a648

raoxiaoyan reviewed Jan 6, 2026

View reviewed changes

kong/plugins/oauth2/daos.lua Outdated Show resolved Hide resolved

raoxiaoyan reviewed Jan 6, 2026

View reviewed changes

kong/plugins/request-transformer/schema.lua Show resolved Hide resolved

raoxiaoyan reviewed Jan 6, 2026

View reviewed changes

kong/plugins/response-transformer/schema.lua Outdated Show resolved Hide resolved

Copilot started reviewing on behalf of raoxiaoyan January 6, 2026 03:38 View session

Copilot AI reviewed Jan 6, 2026

View reviewed changes

raoxiaoyan requested changes Jan 6, 2026

View reviewed changes

lordgreg added 7 commits January 14, 2026 09:55

Revert "feat(response-transformer): Add option to use {vault://} in f…

f6ae29d

…ields" This reverts commit a852681.

Revert "feat(request-transformer): Add option to use {vault://} in fi…

b8e188f

…elds" This reverts commit 7bd8bb3.

feat(vault): Remove referenceable from hash_secret in oauth2 dao

8aa9286

fix(test): Correct filename for vault spec

c190c95

fix(test): remove vault specs

ad160aa

tests(jwt): update vault tests to match test semantics

b671a5e

tests(basic-auth): update vault tests to match test semantics

17c9ac2

pull-request-size bot added size/XL and removed size/XXL labels Jan 15, 2026

Merge branch 'master' into master

f89c5bb

lordgreg requested a review from raoxiaoyan February 3, 2026 05:49

lordgreg added 2 commits February 3, 2026 07:41

fix(test): Standardize vault-referenced tests

0877600

Merge branch 'master' of https://github.com/lordgreg/kong

cb41634

pull-request-size bot added size/L and removed size/XL labels Feb 3, 2026

raoxiaoyan reviewed Feb 3, 2026

View reviewed changes

	{ secret = { type = "string", auto = true, referenceable = true }, },
	{ secret = { type = "string", auto = true }, },

	{ client_secret = { type = "string", required = false, auto = true, encrypted = true, referenceable = true }, }, -- encrypted = true is a Kong Enterprise Exclusive feature. It does nothing in Kong CE
	{ client_secret = { type = "string", required = false, auto = true, encrypted = true }, }, -- encrypted = true is a Kong Enterprise Exclusive feature. It does nothing in Kong CE

Conversation

lordgreg commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Checklist

Issue reference

Additional info for Kong development team

Uh oh!

CLAassistant commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mschonmeier commented Oct 24, 2025

Uh oh!

lordgreg commented Oct 24, 2025

Uh oh!

lordgreg commented Nov 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

lordgreg Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

lordgreg Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

raoxiaoyan left a comment

Choose a reason for hiding this comment

Uh oh!

lordgreg commented Jan 6, 2026

Uh oh!

raoxiaoyan commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lordgreg commented Jan 8, 2026

Uh oh!

raoxiaoyan commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lordgreg commented Jan 15, 2026

Uh oh!

raoxiaoyan commented Feb 3, 2026

Uh oh!

lordgreg commented Feb 3, 2026

Uh oh!

raoxiaoyan Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

raoxiaoyan Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

raoxiaoyan Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

raoxiaoyan commented Feb 3, 2026

Uh oh!

lordgreg commented Feb 3, 2026

lordgreg commented Oct 16, 2025 •

edited

Loading

CLAassistant commented Oct 16, 2025 •

edited

Loading

raoxiaoyan commented Jan 8, 2026 •

edited

Loading

raoxiaoyan commented Jan 8, 2026 •

edited

Loading