WARNING: THIS SITE IS A MIRROR OF GITHUB.COM / IT CANNOT LOGIN OR REGISTER ACCOUNTS / THE CONTENTS ARE PROVIDED AS-IS / THIS SITE ASSUMES NO RESPONSIBILITY FOR ANY DISPLAYED CONTENT OR LINKS / IF YOU FOUND SOMETHING MAY NOT GOOD FOR EVERYONE, CONTACT ADMIN AT ilovescratch@foxmail.com
Skip to content

Commit 01a2505

Browse files
ni-bhandariVighnesh ShenoyVishal Guptarenovate[bot]
authored
Managed Identity Support for CosmosDB External Scaler (#86)
* Update scaler metadata Signed-off-by: Vighnesh Shenoy <[email protected]> * Update cosmos client to use credentials Signed-off-by: Vighnesh Shenoy <[email protected]> * Update docs Signed-off-by: Vighnesh Shenoy <[email protected]> * Fix serialization and update tests Signed-off-by: Vighnesh Shenoy <[email protected]> * fix MI client Signed-off-by: Nitisha Bhandari <[email protected]> * Update deployment file Signed-off-by: Nitisha Bhandari <[email protected]> * Update demo project to use MSI Signed-off-by: Nitisha Bhandari <[email protected]> * Add back constring Signed-off-by: Nitisha Bhandari <[email protected]> * minor changes Signed-off-by: Nitisha Bhandari <[email protected]> * Update demo project Signed-off-by: Nitisha Bhandari <[email protected]> * Update readme Signed-off-by: Nitisha Bhandari <[email protected]> * Address review comments Signed-off-by: Nitisha Bhandari <[email protected]> * pr feedback Signed-off-by: Nitisha Bhandari <[email protected]> * documentation updates Signed-off-by: Nitisha Bhandari <[email protected]> * chore(deps): update nuget packages (#79) Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Signed-off-by: Nitisha Bhandari <[email protected]> * fix merge Signed-off-by: Nitisha Bhandari <[email protected]> * refactor getcosmosclientwithmetadata Signed-off-by: Nitisha Bhandari <[email protected]> * add connection string validation Signed-off-by: Nitisha Bhandari <[email protected]> * fix exception Signed-off-by: Nitisha Bhandari <[email protected]> * fix appsettings Signed-off-by: Nitisha Bhandari <[email protected]> * fix appsettings Signed-off-by: Nitisha Bhandari <[email protected]> * minor readme changes Signed-off-by: Nitisha Bhandari <[email protected]> * Revert OrderGenerator to use connstrings Signed-off-by: Nitisha Bhandari <[email protected]> * minor update Signed-off-by: Nitisha Bhandari <[email protected]> * remove generator sample aks file Signed-off-by: Nitisha Bhandari <[email protected]> * Add instructions to scaler deploy manifest Signed-off-by: Nitisha Bhandari <[email protected]> * add tests for clientid Signed-off-by: Nitisha Bhandari <[email protected]> * update readme comment Signed-off-by: Nitisha Bhandari <[email protected]> * PR feedback Signed-off-by: Nitisha Bhandari <[email protected]> * update app name Signed-off-by: Nitisha Bhandari <[email protected]> * add azure identity package Signed-off-by: Nitisha Bhandari <[email protected]> --------- Signed-off-by: Vighnesh Shenoy <[email protected]> Signed-off-by: Nitisha Bhandari <[email protected]> Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Signed-off-by: ni-bhandari <[email protected]> Co-authored-by: Vighnesh Shenoy <[email protected]> Co-authored-by: Vishal Gupta <[email protected]> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
1 parent bac322b commit 01a2505

21 files changed

+587
-103
lines changed

‎README.md‎

Lines changed: 26 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ The external scaler calls Cosmos DB APIs to estimate the amount of changes pendi
5555

5656
Create `ScaledObject` resource that contains the information about your application (the scale target), the external scaler service, Cosmos DB containers, and other scaling configuration values. Check [`ScaledObject` specification](https://keda.sh/docs/concepts/scaling-deployments/) and [`External` trigger specification](https://keda.sh/docs/scalers/external/) for information on different properties supported for `ScaledObject` and their allowed values.
5757

58-
You can use file `deploy/deploy-scaledobject.yaml` as a template for creating the `ScaledObject`. The trigger metadata properties required to use the external scaler for Cosmos DB are described in [Trigger Specification](#trigger-specification) section below.
58+
You can use the files in the `deploy` folder as templates for creating `ScaledObject`s. The trigger metadata properties required to use the external scaler for Cosmos DB are described in [Trigger Specification](#trigger-specification) section below.
5959

6060
> **Note:** If you are having trouble setting up the external scaler or the listener application, the step-by-step instructions for [deploying the sample application](./src/Scaler.Demo/README.md) might help.
6161

@@ -68,31 +68,40 @@ The specification below describes the `trigger` metadata in `ScaledObject` resou
6868
- type: external
6969
metadata:
7070
scalerAddress: external-scaler-azure-cosmos-db.keda:4050 # Mandatory. Address of the external scaler service.
71-
connectionFromEnv: <env-variable-for-connection> # Mandatory. Environment variable for the connection string of Cosmos DB account with monitored container.
71+
72+
# Database & Container properties
7273
databaseId: <database-id> # Mandatory. ID of Cosmos DB database containing monitored container.
7374
containerId: <container-id> # Mandatory. ID of monitored container.
74-
leaseConnectionFromEnv: <env-variable-for-lease-connection> # Mandatory. Environment variable for the connection string of Cosmos DB account with lease container.
7575
leaseDatabaseId: <lease-database-id> # Mandatory. ID of Cosmos DB database containing lease container.
7676
leaseContainerId: <lease-container-id> # Mandatory. ID of lease container.
77+
78+
# Connection String properties.
79+
connectionFromEnv: <env-variable-for-connection> # Optional. Environment variable for the connection string of Cosmos DB account with monitored container.
80+
leaseConnectionFromEnv: <env-variable-for-lease-connection> # Optional. Environment variable for the connection string of Cosmos DB account with lease container.
81+
82+
# Managed Identity properties.
83+
endpoint: <endpoint> # Optional. Account endpoint of the CosmosDB account containing the monitored container.
84+
leaseEndpoint: <lease-endpoint> # Optional. Account endpoint of the CosmosDB account containing the lease container.
85+
clientId: <client-Id> # Optional. Client ID of the identity to be used. If this is not provided, the azure.workload.identity is used.
86+
7787
processorName: <processor-name> # Mandatory. Name of change-feed processor used by listener application.
7888
```
7989

8090
### Parameter List
8191

8292
- **`scalerAddress`** - Address of the external scaler service. This would be in format `<scaler-name>.<scaler-namespace>:<port>`. If you installed Azure Cosmos DB external scaler Helm chart in `keda` namespace and did not specify custom values, the metadata value would be `external-scaler-azure-cosmos-db.keda:4050`.
83-
84-
- **`connectionFromEnv`** - Name of the environment variable on the scale target to read the connection string of the Cosmos DB account that contains the monitored container.
85-
86-
- **`databaseId`** - ID of Cosmos DB database that contains the monitored container.
87-
88-
- **`containerId`** - ID of the monitored container.
89-
90-
- **`leaseConnectionFromEnv`** - Name of the environment variable on the scale target to read the connection string of the Cosmos DB account that contains the lease container. This can be same or different from the value of `connection` metadata.
91-
92-
- **`leaseDatabaseId`** - ID of Cosmos DB database that contains the lease container. This can be same or different from the value of `databaseId` metadata.
93-
94-
- **`leaseContainerId`** - ID of the lease container containing the change feeds.
95-
93+
- **Database & Container Properties:**
94+
- **`databaseId`** - ID of Cosmos DB database that contains the monitored container.
95+
- **`containerId`** - ID of the monitored container.
96+
- **`leaseDatabaseId`** - ID of Cosmos DB database that contains the lease container. This can be same or different from the value of `databaseId` metadata.
97+
- **`leaseContainerId`** - ID of the lease container containing the change feeds.
98+
- **Connection String Properties:**
99+
- **`connectionFromEnv`** - Name of the environment variable on the scale target to read the connection string of the Cosmos DB account that contains the monitored container. You can also opt for an identity-based connection instead, refer to the `endpoint` property.
100+
- **`leaseConnectionFromEnv`** - Name of the environment variable on the scale target to read the connection string of the Cosmos DB account that contains the lease container. This can be same or different from the value of `connection` metadata. You can also opt for an identity-based connection instead, refer to the `leaseEndpoint` property. If left null, `connection` value of the monitored container is used.
101+
- **Managed Identity Properties:**
102+
- **`endpoint`** - Account endpoint of the CosmosDB account containing the monitored container.
103+
- **`leaseEndpoint`** - Account endpoint of the CosmosDB account containing the lease container. This can be same or different from the value of `endpoint` metadata. If left null, `endpoint` value of the monitored container is used.
104+
- **`clientId`** - ClientId of the identity to be used. If this is not provided, the azure.workload.identity is used.
96105
- **`processorName`** - Name of change-feed processor used by listener application. For more information on this, you can refer to [Implementing the change feed processor](https://docs.microsoft.com/azure/cosmos-db/sql/change-feed-processor#implementing-the-change-feed-processor) section.
97106

98-
> **Note** Ideally, we would have created `TriggerAuthentication` resource that would have prevented us from adding the connection strings in plain text in the `ScaledObject` trigger metadata. However, this is not possible since at the moment, the triggers of `external` type do not support referencing a `TriggerAuthentication` resource ([link](https://keda.sh/docs/scalers/external/#authentication-parameters)).
107+
> **Note** Ideally, we would have created `TriggerAuthentication` resource that would have prevented us from adding the connection strings in plain text in the `ScaledObject` trigger metadata. However, this is not possible since at the moment, the triggers of `external` type do not support referencing a `TriggerAuthentication` resource ([link](https://keda.sh/docs/scalers/external/#authentication-parameters)).

‎deploy/deploy-scaledobject.yaml‎ renamed to ‎deploy/deploy-scaledobject-cs.yaml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Template scaled-object for using KEDA external scaler for Azure Cosmos DB.
1+
# Template scaled-object for using KEDA external scaler for Azure Cosmos DB, using connection strings.
22

33
apiVersion: keda.sh/v1alpha1
44
kind: ScaledObject
@@ -13,10 +13,10 @@ spec:
1313
- type: external
1414
metadata:
1515
scalerAddress: external-scaler-azure-cosmos-db.keda:4050
16-
connectionFromEnv: <env-variable-for-connection-string-of-monitored-container-account>
1716
databaseId: <database-id>
1817
containerId: <container-id>
19-
leaseConnectionFromEnv: <env-variable-for-connection-string-of-lease-container-account>
2018
leaseDatabaseId: <lease-database-id>
2119
leaseContainerId: <lease-container-id>
20+
connectionFromEnv: <env-variable-for-connection-string-of-monitored-container-account>
21+
leaseConnectionFromEnv: <env-variable-for-connection-string-of-lease-container-account>
2222
processorName: <processor-name>

‎deploy/deploy-scaledobject-mi.yaml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Template scaled-object for using KEDA external scaler for Azure Cosmos DB, using managed identity.
2+
3+
apiVersion: keda.sh/v1alpha1
4+
kind: ScaledObject
5+
metadata:
6+
name: <scaledobject-name>
7+
namespace: default
8+
spec:
9+
pollingInterval: 20
10+
scaleTargetRef:
11+
name: <application-deployment-name>
12+
triggers:
13+
- type: external
14+
metadata:
15+
scalerAddress: external-scaler-azure-cosmos-db.keda:4050
16+
databaseId: <database-id>
17+
containerId: <container-id>
18+
leaseDatabaseId: <lease-database-id>
19+
leaseContainerId: <lease-container-id>
20+
endpoint: <endpoint>
21+
leaseEndpoint: <lease-endpoint>
22+
clientId: <client-id>
23+
processorName: <processor-name>

‎deploy/service-account-mi.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
apiVersion: v1
2+
kind: ServiceAccount
3+
metadata:
4+
name: <service-account-name>
5+
annotations:
6+
azure.workload.identity/client-id: <client-id>
7+
azure.workload.identity/tenant-id: <tenant-id>

‎src/Scaler.Demo/OrderGenerator/Program.cs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -157,4 +157,4 @@ private static async Task TeardownAsync()
157157
Console.WriteLine("Done!");
158158
}
159159
}
160-
}
160+
}

‎src/Scaler.Demo/OrderGenerator/appsettings.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
{
22
"CosmosDbConfig": {
3-
"Connection": "<connection-string-of-monitored-container-account>",
3+
"Connection": "<connection-string-or-endpoint-of-monitored-container-account>",
44
"DatabaseId": "StoreDatabase",
55
"ContainerId": "OrderContainer",
6-
"ContainerThroughput": 11000
6+
"ContainerThroughput": 400
77
}
88
}

‎src/Scaler.Demo/OrderProcessor/Worker.cs‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ internal sealed class Worker : BackgroundService
1717

1818
private ChangeFeedProcessor _processor;
1919

20+
public static string _applicationName = "cosmosdb-order-processor";
21+
2022
public Worker(CosmosDbConfig cosmosDbConfig, ILogger<Worker> logger)
2123
{
2224
_cosmosDbConfig = cosmosDbConfig ?? throw new ArgumentNullException(nameof(cosmosDbConfig));
@@ -25,19 +27,24 @@ public Worker(CosmosDbConfig cosmosDbConfig, ILogger<Worker> logger)
2527

2628
public override async Task StartAsync(CancellationToken cancellationToken)
2729
{
28-
Database leaseDatabase = await new CosmosClient(_cosmosDbConfig.LeaseConnection)
30+
Database leaseDatabase = await DemoHelper.CreateCosmosClient(
31+
_cosmosDbConfig.Connection,
32+
!string.IsNullOrWhiteSpace(_cosmosDbConfig.MsiClientId),
33+
_cosmosDbConfig.MsiClientId, _applicationName)
2934
.CreateDatabaseIfNotExistsAsync(_cosmosDbConfig.LeaseDatabaseId, cancellationToken: cancellationToken);
3035

3136
Container leaseContainer = await leaseDatabase
3237
.CreateContainerIfNotExistsAsync(
3338
new ContainerProperties(_cosmosDbConfig.LeaseContainerId, partitionKeyPath: "/id"),
34-
throughput: 400,
3539
cancellationToken: cancellationToken);
3640

3741
// Change feed processor instance name should be unique for each container application.
3842
string instanceName = $"Instance-{Dns.GetHostName()}";
3943

40-
_processor = new CosmosClient(_cosmosDbConfig.Connection)
44+
_processor = DemoHelper.CreateCosmosClient(
45+
_cosmosDbConfig.Connection,
46+
!string.IsNullOrWhiteSpace(_cosmosDbConfig.MsiClientId),
47+
_cosmosDbConfig.MsiClientId, _applicationName)
4148
.GetContainer(_cosmosDbConfig.DatabaseId, _cosmosDbConfig.ContainerId)
4249
.GetChangeFeedProcessorBuilder<Order>(_cosmosDbConfig.ProcessorName, ProcessOrdersAsync)
4350
.WithInstanceName(instanceName)

‎src/Scaler.Demo/OrderProcessor/appsettings.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,10 @@
77
}
88
},
99
"CosmosDbConfig": {
10-
"Connection": "<connection-string-of-monitored-container-account>",
10+
"Connection": "<connection-string-or-endpoint-of-monitored-container-account>",
1111
"DatabaseId": "StoreDatabase",
1212
"ContainerId": "OrderContainer",
13-
"LeaseConnection": "<connection-string-of-lease-container-account>",
13+
"LeaseConnection": "<connection-string-or-endpoint-of-lease-container-account>",
1414
"LeaseDatabaseId": "StoreDatabase",
1515
"LeaseContainerId": "OrderProcessorLeases",
1616
"ProcessorName": "OrderProcessor"

‎src/Scaler.Demo/OrderProcessor/deploy-scaledobject.yaml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,3 +22,6 @@ spec:
2222
leaseDatabaseId: StoreDatabase
2323
leaseContainerId: OrderProcessorLeases
2424
processorName: OrderProcessor
25+
endpoint: "<cosmosdb-endpoint>"
26+
leaseEndpoint: "<cosmosdb-endpoint>"
27+
clientId: "<client-id>"

‎src/Scaler.Demo/OrderProcessor/deploy.yaml‎

Lines changed: 18 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,13 +14,27 @@ spec:
1414
metadata:
1515
labels:
1616
app: cosmosdb-order-processor
17+
azure.workload.identity/use: "true"
1718
spec:
19+
serviceAccountName: <service-account-name>
1820
containers:
1921
- name: cosmosdb-order-processor
2022
image: <docker-id>/cosmosdb-order-processor:latest
2123
imagePullPolicy: Always
2224
env:
23-
- name: CosmosDbConfig__Connection
24-
value: <connection-string-of-monitored-container>
25-
- name: CosmosDbConfig__LeaseConnection
26-
value: <connection-string-of-lease-container>
25+
- name: CosmosDbConfig__Connection # if using MSI, use account endpoint
26+
value: <connection-string-or-endpoint-of-monitored-container>
27+
- name: CosmosDbConfig__LeaseConnection # if using MSI, use lease account endpoint
28+
value: <connection-string-or-endpoint-of-lease-container>
29+
- name: CosmosDbConfig__MsiClientId
30+
value: "<client-id-if-using-mi>"
31+
- name: CosmosDbConfig__DatabaseId
32+
value: "StoreDatabase"
33+
- name: CosmosDbConfig__ContainerId
34+
value: "OrderContainer"
35+
- name: CosmosDbConfig__LeaseDatabaseId
36+
value: "StoreDatabase"
37+
- name: CosmosDbConfig__LeaseContainerId
38+
value: "OrderProcessorLeases"
39+
- name: CosmosDbConfig__ProcessorName
40+
value: "OrderProcessor"

0 commit comments

Comments
 (0)