GODRIVER-3486 Support auto encryption in unified tests. #2240

qingyang-hu · 2025-11-12T23:28:29Z

Summary

Support auto encryption in unified tests.

Background & Motivation

mongodb-drivers-pr-bot · 2025-11-12T23:39:38Z

🧪 Performance Results

Commit SHA: 7c5d45b

The following benchmark tests for version 693c4f48d62d40000701bda1 had statistically significant changes (i.e., |z-score| > 1.96):

Benchmark	Measurement	% Change	Patch Value	Stable Region	H-Score	Z-Score
BenchmarkMultiInsertLargeDocument	allocated_bytes_per_op	-5.0588	26513049.0000	Avg: 27925749.9594 Med: 27920409.0000 Stdev: 663917.7120	0.7436	-2.1278
BenchmarkBSONFlatDocumentEncoding	ops_per_second_max	-4.4843	82399.4726	Avg: 86268.0249 Med: 86325.9984 Stdev: 1837.1814	0.7539	-2.1057
BenchmarkMultiFindMany	ops_per_second_max	-3.7500	4065040.6504	Avg: 4223417.5755 Med: 4237288.1356 Stdev: 54030.4214	0.8210	-2.9313
BenchmarkMultiFindMany	ops_per_second_med	-3.2065	3610108.3032	Avg: 3729701.6649 Med: 3731343.2836 Stdev: 45311.6931	0.7918	-2.6393

For a comprehensive view of all microbenchmark results for this PR's commit, please check out the Evergreen perf task for this patch.

mongodb-drivers-pr-bot · 2025-11-12T23:40:40Z

API Change Report

No changes found!

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-06T23:56:19Z

internal/integration/unified/entity.go

 }

 // getKmsCredential processes a value of an input KMS provider credential.
 // An empty document returns from the environment.


The comment "An empty document returns from the environment" is unclear and grammatically incorrect. It should be clarified to something like "A placeholder document ($$placeholder) causes the value to be retrieved from the environment variable or default value."

Suggested change

// An empty document returns from the environment.

// If a placeholder document ($$placeholder) is provided, the value is retrieved from the environment variable or default value.

Copilot · 2025-12-06T23:56:20Z

internal/integration/unified/collection_data.go

-	// If neither documents nor options are provided, still create the collection with write concern "majority".
-	if len(c.Documents) == 0 && c.Options == nil {
+	} else {
+		// If options are provided, still create the collection with write concern "majority".


The comment states "If options are provided" but this is in the else block where c.Options is nil. The comment should say "If no options are provided" to accurately describe this code path.

Suggested change

// If options are provided, still create the collection with write concern "majority".

// If no options are provided, still create the collection with write concern "majority".

This seems relevant.

Copilot · 2025-12-06T23:56:20Z

internal/integration/unified/entity.go

+			return nil, err
+		}
+		if v == "$$placeholder" {
+			provider["sessionToken"] = os.Getenv("CSFLE_AWS_TEMP_SESSION_TOKEN")


When the sessionToken placeholder is detected, the code directly assigns os.Getenv("CSFLE_AWS_TEMP_SESSION_TOKEN") without checking if the environment variable exists or is empty. If the environment variable is not set, this will assign an empty string to the provider's sessionToken. Consider using getKmsCredential or adding a check to ensure the environment variable is set before assigning it.

Suggested change

provider["sessionToken"] = os.Getenv("CSFLE_AWS_TEMP_SESSION_TOKEN")

sessionToken, err := getKmsCredential(opt, "sessionToken", "CSFLE_AWS_TEMP_SESSION_TOKEN", "")

if err != nil {

return nil, err

}

if sessionToken != nil {

provider["sessionToken"] = sessionToken

}

prestonvasquez · 2025-12-10T16:56:17Z

internal/integration/unified/entity.go

+		}
+		for _, coll := range colls {
+			if re.MatchString(coll) {
+				_, err = db.Collection(coll).DeleteMany(ctx, bson.D{})


[question] Why don't we just drop these collections?

prestonvasquez · 2025-12-10T16:58:37Z

internal/integration/unified/client_entity.go

 	}
 }

+func createAutoEncryptionOptions(opts bson.Raw) (*options.AutoEncryptionOptions, error) {


[nit] Could we add a TODO(GODRIVER-3726) here to investigate moving this logic to an Unmarshal method?

prestonvasquez · 2025-12-10T16:59:20Z

internal/integration/unified/collection_data.go

-	// If neither documents nor options are provided, still create the collection with write concern "majority".
-	if len(c.Documents) == 0 && c.Options == nil {
+	} else {
+		// If options are provided, still create the collection with write concern "majority".


This seems relevant.

prestonvasquez · 2025-12-10T17:00:50Z

internal/integration/unified/entity.go

+			{"accessKeyId", accessKeyID},
+			{"secretAccessKey", secretAccessKey},


[nit] Suggest keying the field values for readability and breaking parameterization.

prestonvasquez · 2025-12-10T17:01:21Z

internal/integration/unified/entity.go

+			{"tenantId", "FLE_AZURE_TENANTID"},
+			{"clientId", "FLE_AZURE_CLIENTID"},
+			{"clientSecret", "FLE_AZURE_CLIENTSECRET"},


[nit] Suggest keying the field values for readability and breaking parameterization.

prestonvasquez · 2025-12-10T17:02:07Z

internal/integration/unified/entity.go

+			{"email", "FLE_GCP_EMAIL"},
+			{"privateKey", "FLE_GCP_PRIVATEKEY"},


[nit] Suggest keying the field values for readability and breaking parameterization.

prestonvasquez · 2025-12-10T17:02:42Z

internal/integration/unified/entity.go

-
-		azureClientSecret, err := getKmsCredential(azure, "clientSecret", "FLE_AZURE_CLIENTSECRET", "")
+	case "local", "local:name2":
+		defaultLocalKeyBase64 := "Mng0NCt4ZHVUYUJCa1kxNkVyNUR1QURhZ2h2UzR2d2RrZzh0cFBwM3R6NmdWMDFBMUN3YkQ5aXRRMkhGRGdQV09wOGVNYUMxT2k3NjZKelhaQmRCZGJkTXVyZG9uSjFk"


[nit] Consider constantizing.

prestonvasquez · 2025-12-10T17:04:10Z

internal/integration/unified/entity.go

-		kmipEndpoint, err := getKmsCredential(kmip, "endpoint", "", "localhost:5698")
+func (em *EntityMap) addClientEncryptionEntity(entityOptions *entityOptions) error {
+	// Construct KMS providers.
+	kmsProviders := make(map[string]map[string]any)


[blocking] Can we merge the KMS logic here with createAutoEncryptionOptions? Same with the key vault namespace? From the specs:

kmsProviders: The same as in clientEncryption.
keyVaultNamespace: The same as in clientEncryption.

It will be easier to merge the logic in GODRIVER-3726 in my idea

prestonvasquez · 2025-12-10T17:15:18Z

internal/integration/unified/client_entity.go

 	}
 }

+func createAutoEncryptionOptions(opts bson.Raw) (*options.AutoEncryptionOptions, error) {


[question] Can the analogue in integration/json_helpers_test.go be removed? Can we remove all support for auto encryption in the legacy runner?

We can keep it until the legacy tests are completely eliminated to avoid handling logic fractions.

matthewdale · 2025-12-12T06:51:38Z

internal/integration/unified/entity.go

 	}

+	// Clear automatically created collections used for queryable encryption
+	re := regexp.MustCompile("^enxcol_.*.e(sc|coc)$")


(Optional) regexp.MustCompile panics if the regex can't be compiled. Consider moving this to a package-level initialization so any issue can be discovered at package initialization, not when the test is run.

E.g.

// qeCollectionPattern matches collections automatically created for // queryable encryption. var qeCollectionPattern = regexp.MustCompile("^enxcol_.*.e(sc|coc)$")

matthewdale · 2025-12-12T06:58:56Z

internal/integration/unified/entity.go

 // An empty document returns from the environment.
 // A string is returned as-is.
-func getKmsCredential(kmsDocument bson.Raw, credentialName string, envVar string, defaultValue string) (string, error) {
+func getKmsCredential(kmsDocument bson.Raw, credentialName string, envVar string, defaultValue string) (any, error) {


Why change the return type from string to any? All of the returned values seem to still be string.

The main motivation of the change is to distinguish nonexistent value from empty. It is primarily used in line #647, where we set "sessionToken" only when it exists.

matthewdale · 2025-12-12T07:07:30Z

internal/integration/unified/entity.go

 		}
-
-		if tlsClientCertificateKeyFile != "" && tlsCAFile != "" {
+		if provider == nil {


(Optional) For maps, it's conventional to check for len(provider) == 0 because a nil and empty map both report length 0. Consider updating the check here to if len(provider) == 0 and removing the redundant if len(provider) == 0 from getKmsProvider.

qingyang-hu · 2025-12-12T20:22:27Z

internal/integration/mtest/options.go

 	embRawValue := bson.RawValue{Type: bson.TypeEmbeddedDocument, Value: data}
 	if err := embRawValue.Unmarshal(&csfle.Options); err == nil {
-		csfle.Boolean = false
+		csfle.Boolean = true


The csfle logically is enabled when it contains a document of properties.

qingyang-hu added the enhancement label Nov 12, 2025

qingyang-hu force-pushed the godriver3486 branch from 4e58f2a to 692fc7c Compare November 12, 2025 23:35

qingyang-hu marked this pull request as ready for review November 13, 2025 14:14

qingyang-hu requested a review from a team as a code owner November 13, 2025 14:14

qingyang-hu requested review from Copilot and matthewdale November 13, 2025 14:14

Copilot started reviewing on behalf of qingyang-hu November 13, 2025 14:15 View session

Copilot finished reviewing on behalf of qingyang-hu November 13, 2025 14:17

This comment was marked as outdated.

Sign in to view

qingyang-hu marked this pull request as draft November 13, 2025 14:37

qingyang-hu force-pushed the godriver3486 branch 4 times, most recently from b1f3ca6 to 6bf8480 Compare November 15, 2025 02:55

qingyang-hu removed the request for review from matthewdale December 4, 2025 19:49

qingyang-hu force-pushed the godriver3486 branch 13 times, most recently from 666c2d5 to b000132 Compare December 6, 2025 04:44

qingyang-hu force-pushed the godriver3486 branch 6 times, most recently from e036710 to a871c58 Compare December 6, 2025 20:11

update unified integration

f2113d4

qingyang-hu force-pushed the godriver3486 branch from a871c58 to f2113d4 Compare December 6, 2025 23:50

mongodb deleted a comment from Copilot AI Dec 6, 2025

qingyang-hu requested a review from Copilot December 6, 2025 23:53

Copilot started reviewing on behalf of qingyang-hu December 6, 2025 23:53 View session

Copilot AI reviewed Dec 6, 2025

View reviewed changes

qingyang-hu marked this pull request as ready for review December 8, 2025 14:18

mongodb deleted a comment from evergreen-ci-prod bot Dec 8, 2025

qingyang-hu requested review from matthewdale and prestonvasquez December 8, 2025 14:20

prestonvasquez requested changes Dec 10, 2025

View reviewed changes

github-actions bot added the review-priority-normal Medium Priority PR for Review: within 1 business day label Dec 11, 2025

minor updates

71042e1

qingyang-hu force-pushed the godriver3486 branch from 5b2b04f to 71042e1 Compare December 11, 2025 22:29

Merge branch 'master' into godriver3486

29c4d0b

matthewdale reviewed Dec 12, 2025

View reviewed changes

minor updates

7c5d45b

qingyang-hu commented Dec 12, 2025

View reviewed changes

qingyang-hu requested review from matthewdale and prestonvasquez December 12, 2025 20:23

	// An empty document returns from the environment.
	// If a placeholder document ($$placeholder) is provided, the value is retrieved from the environment variable or default value.

	// If options are provided, still create the collection with write concern "majority".
	// If no options are provided, still create the collection with write concern "majority".

-			provider["sessionToken"] = os.Getenv("CSFLE_AWS_TEMP_SESSION_TOKEN")
+			sessionToken, err := getKmsCredential(opt, "sessionToken", "CSFLE_AWS_TEMP_SESSION_TOKEN", "")
+			if err != nil {
+				return nil, err
+			}
+			if sessionToken != nil {
+				provider["sessionToken"] = sessionToken
+			}

		{"accessKeyId", accessKeyID},
		{"secretAccessKey", secretAccessKey},

		{"email", "FLE_GCP_EMAIL"},
		{"privateKey", "FLE_GCP_PRIVATEKEY"},

GODRIVER-3486 Support auto encryption in unified tests. #2240

Are you sure you want to change the base?

GODRIVER-3486 Support auto encryption in unified tests. #2240

Conversation

qingyang-hu commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Background & Motivation

Uh oh!

mongodb-drivers-pr-bot bot commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧪 Performance Results

Uh oh!

mongodb-drivers-pr-bot bot commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

API Change Report

Uh oh!

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Dec 6, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 6, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 6, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

qingyang-hu commented Nov 12, 2025 •

edited

Loading

mongodb-drivers-pr-bot bot commented Nov 12, 2025 •

edited

Loading

mongodb-drivers-pr-bot bot commented Nov 12, 2025 •

edited

Loading