Release 5.6.0 (2025-10-30)

Release Highlights

• SREs with Tier 0/1 won't be deploying a Nexus proxy, and can download packages directly from the internet.
• SREs now support mirroring external GitHub repositories into the internal Gitea repository accessible from workspaces.

Upgrading from 5.5.1

As part of the current release, we're offering users more control over the DNS sidecar deployment by exposing two new parameters in the SRE configuration: workload_minimum_count and workload_maximum_count. Before updating your SRE, make sure your YAML config file contains valid values for these parameters. For example:

user_services:
  dns_sidecar:
    cron_expression: "*/30 * * * *"
    replica_timeout: 600
    retry_limit: 0
    workload_minimum_count: 1
    workload_maximum_count: 2

Upload your new configuration file using dsh config upload, and then run the following command to upgrade an existing SRE:

dsh deploy sre YOURSRENAME

What's Changed

• Merge latest (v5.5.0) into develop by @cptanalatriste in #2461
• Bump stefanzweifel/git-auto-commit-action from 5.2.0 to 6.0.1 by @dependabot[bot] in #2454
• Update pulumi-azure-native by @jemrobinson in #2452
• ⬆️ Update Pulumi Docker images by @github-actions[bot] in #2453
• Update Azure SDK packages by @jemrobinson in #2463
• ⬆️ Bump the production-dependencies group across 1 directory with 26 updates by @dependabot[bot] in #2464
• Using the correct socket on the unit file for clamav-clamonacc by @cptanalatriste in #2460
• Remove unused functions by @jemrobinson in #2465
• Fix library update by @cptanalatriste in #2466
• Disabling Guacamole Login by @cptanalatriste in #2468
• Stop unneeded deployment of Nexus repositories for Tier0/1 SREs by @craddm in #2389
• Merge v5.5.1 into develop by @cptanalatriste in #2478
• ⬆️ Bump actions/setup-python from 5 to 6 by @dependabot[bot] in #2483
• ⬆️ Bump actions/checkout from 4 to 5 by @dependabot[bot] in #2473
• ⬆️ Update Pulumi Docker images by @github-actions[bot] in #2470
• Automatically ingressing code from pre-approved GitHub repositories by @cptanalatriste in #2482
• Making DSN sidecar components configurable by @cptanalatriste in #2485
• The Gitea mirror should take a new ip range for easy TRE update by @cptanalatriste in #2486
• Release v5.6.0 by @cptanalatriste in #2491

Full Changelog: v5.5.1...v5.6.0

Release 5.5.1 (2025-09-04)

Release Highlights

• Fixes the clamav-clamonacc unit file configuration, so it doesn't interrupt Ansible from configuring SRE workspaces properly.

What's Changed

• Hotfix: Using the correct socket on the unit file for clamav-clamonacc by @cptanalatriste in #2476

Upgrading from 5.5.0

Run the following command to upgrade an existing SRE:

dsh deploy sre YOURSRENAME

You might need to re-run cloud-init manually in your workspaces for the new configuration to take place.

Full Changelog: v5.5.0...v5.5.1

Release 5.5.0 (2025-07-18)

Release Highlights

• SRE users have now write privileges on /mnt/scratch, when using SRDs that support temporary storage.
• It is now possible to set up the size of Nexus persistent directory in the configuration file. And the default value has been increased from 2Gb to 10Gb.
• User services, like Gitea and HedgeDoc, sometimes change their IP address on restart, which makes them unavailable due to DNS issues. We have now a Container App Job that monitors this problem and fix it when happens.
• Documentation updates on clipboard control limitations and data egress instructions.

Upgrading from 5.4.1

Run the following command to upgrade an existing SRE:

dsh deploy sre YOURSRENAME

Changing the size of Nexus' persistent directory and running a Job for fixing container DNS will likely have an impact on the cost of running an SRE. Please use the configuration file to ensure your costs are within your budget.

What's Changed

• Merge latest (v5.4.1) into develop by @cptanalatriste in #2430
• Documentation: Deploying SREs requires Owner role by @cptanalatriste in #2433
• Enabling SRE users to write into /mnt/scratch by @cptanalatriste in #2432
• Enabling configuring the size of Nexus' persistent directory by @cptanalatriste in #2435
• ⬆️ Bump stefanzweifel/git-auto-commit-action from 5.1.0 to 5.2.0 by @dependabot[bot] in #2436
• ⬆️ Update Pulumi Docker images by @github-actions[bot] in #2437
• ⬆️ Update Pulumi Docker images by @github-actions[bot] in #2443
• Bump lycheeverse/lychee-action from 2.4.0 to 2.4.1 by @dependabot[bot] in #2444
• Pin Click to version 8.1.8 by @llewelld in #2440
• Adding a sidecar to ACI instances to fix broken DNS entries by @cptanalatriste in #2442
• Update Azure menu location to reflect changes to the egress process by @llewelld in #2455
• Update egress instructions related to IP addresses by @llewelld in #2456
• Informing DSH users about the limitations of clipboard control by @cptanalatriste in #2459

New Contributors

• @llewelld made their first contribution in #2440

Full Changelog: v5.4.1...v5.5.0

Release 5.4.1 (2025-04-01)

Release Highlights

• Fixed a josepy related issue preventing DSH installation.
• Fixed missing persistent storage for Gitea to prevent failure after container restart.

Upgrading from 5.4.1

Please back up your Gitea codebase before updating the SRE. Our testing shows that repositories created before this update won't be repaired, so you will need to re-create them once the update is completed.

Once the backup is finished, run the following command to upgrade an existing SRE.

dsh deploy sre YOURSRENAME

What's Changed

• Reduce update workflow frequency by @JimMadge in #2421
• Mounting a volume to fix Gitea after container restart. by @cptanalatriste in #2423

Full Changelog: v5.4.0...v5.4.1

Release v5.4.0 (2025-03-03)

Release Highlights

• Adds the dsh allowlist family of commands to allow easy manipulation of the package allowlists that limit access to packages on SREs where only pre-approved packages can be downloaded from CRAN/PyPi.
• Updates nexus-allowlist to handle changes to the Nexus Sonatype Repository initialization process

Upgrading from 5.3.1

Run the following command to upgrade an existing SRE

dsh deploy sre YOURSRENAME

Note that due to changes in the way the package allowlists are managed, new SREs will have no allowlist, and upgrading may remove existing allowlists. Administrators will need to manually update the allowlist as required.

What's Changed

• Bump the production-dependencies group with 5 updates by @dependabot in #2366
• Fix docker image CI script by @jemrobinson in #2367
• ⬆️ Update Pulumi Docker images by @github-actions in #2369
• Latest by @craddm in #2371
• Create GitHub Action to upload to PyPI on release by @jemrobinson in #2374
• Bump actions/setup-python from 4 to 5 by @dependabot in #2377
• ⬆️ Update Pulumi Docker images by @github-actions in #2376
• Bump the production-dependencies group with 11 updates by @dependabot in #2378
• Correct file parents by @JimMadge in #2381
• Adding commands to manipulate allowlists by @craddm in #2346
• Merge v5.3.1 into develop by @jemrobinson in #2390
• Exclude non-package files from hatch build by @jemrobinson in #2388
• ⬆️ Update Pulumi Docker images by @github-actions in #2391
• Don't upload *.pyc files to desired state by @JimMadge in #2385
• ⬆️ Bump the production-dependencies group with 6 updates by @dependabot in #2392
• Better teardown logging by @jemrobinson in #2394
• Ensure only files ending with pyc are excluded from desired state uploads by @craddm in #2396
• ⬆️ Bump lycheeverse/lychee-action from 2.2.0 to 2.3.0 by @dependabot in #2399
• ⬆️ Update Pulumi Docker images by @github-actions in #2397
• ⬆️ Bump the production-dependencies group with 7 updates by @dependabot in #2398
• ⬆️ Bump cryptography from 44.0.0 to 44.0.1 by @dependabot in #2402
• Revert "⬆️ Bump cryptography from 44.0.0 to 44.0.1" by @craddm in #2403

Full Changelog: v5.3.1...v5.4.0

Release 5.3.1 (2025-01-28)

Release Highlights

• Fixes issue with expiring SSL certificate
• Updates Nexus image to fix an initialisation problem

Upgrading from 5.3.0

Run the following command to upgrade an existing SRE

dsh deploy sre YOURSRENAME

What's Changed

• Hotfix: Renew SSL certificate in Pulumi #2380
• Hotfix: update Nexus image by @jemrobinson in #2387

Full Changelog: v5.3.0...v5.3.1

Release 5.3.0 (2025-01-20)

Release Highlights

• Adds/fixes support for Tier 0 and Tier 1 SREs
• Adds a reference section for the command line interface to the documentation

Upgrading from 5.2.1

Run the following command to upgrade an existing SRE

dsh deploy sre YOURSRENAME

What's Changed

• Bump ansible-core from 2.18.0 to 2.18.1 in /.hatch by @dependabot in #2329
• Add command reference to documentation by @craddm in #2238
• Bump the production-dependencies group with 6 updates by @dependabot in #2332
• Remove support for Internet Service Tag for Data Provider IP addresses by @craddm in #2331
• Bump peter-evans/create-pull-request from 7.0.5 to 7.0.6 by @dependabot in #2342
• Bump the production-dependencies group across 1 directory with 14 updates by @dependabot in #2344
• Bump lycheeverse/lychee-action from 2.1.0 to 2.2.0 by @dependabot in #2339
• Retrieve SRE sub name and use that when connecting to guac database by @craddm in #2351
• Merge latest (v5.2.0) into develop by @jemrobinson in #2353
• Bump stefanzweifel/git-auto-commit-action from 5.0.1 to 5.1.0 by @dependabot in #2358
• Bump the production-dependencies group with 4 updates by @dependabot in #2357
• Merge 5.2.1 changes into develop by @JimMadge in #2359
• Bump supported version to latest release by @jemrobinson in #2360
• Modifying Firewall rules to provide Internet Access to T0/T1 by @cptanalatriste in #2327
• Release v5.3.0 by @JimMadge in #2364

Full Changelog: v5.2.1...v5.3.0

Release 5.2.1 (2025-01-13)

Release Highlights

• Fixes guacamole-user-sync crash which was limiting SREs to a maximum of 10 users
• Fixes problem with listing users when SRE and SHM are deployed to different subscriptions

Upgrading from 5.2.0

Run the following command to upgrade an existing SRE

dsh deploy sre YOURSRENAME

What's Changed

• Guacamole user synchronisation problems by @jemrobinson in #2352
• Retrieve SRE sub name and use that when connecting to guac database by @craddm in #2354

Full Changelog: v5.2.0...v5.2.1

Release 5.2.0 (2024-12-05)

Release Highlights

• More logs collected in the log analytics workspace
  • Storage
    • Ingress and egress stores
    • Desired state files
    • Users' home directories
    • Container configuration and persistent state
  • Container services
  • Firewall
• Better CLI feedback and error messages
• Documentation improvements

Known issues

Backup is not functional. Following the notice in the documentation will not enable backup.

Upgrading from 5.1.0

In order to upgrade, you will need to carry out the following steps.

⚠️ Some manual interventions are needed. Please ensure that your data is appropriately backed-up before starting. ⚠️

Step-by-step upgrade instructions

N.B. throughout the instructions below, replace YOURSRENAME with the lower-case name of your SRE

Create an upgrade JSON file with the following contents

{
  "nameTable": {
    "sre_data_component": "urn:pulumi:shm-blue-sre-YOURSRENAME::data-safe-haven::dsh:sre:DataComponent::sre_data",
    "sre_desired_state_component": "urn:pulumi:shm-blue-sre-YOURSRENAME::data-safe-haven::dsh:sre:DesiredStateComponent::sre_desired_state"
  },
  "resources": [
    {
      "type": "dsh:sre:NFSV3StorageAccountComponent",
      "name": "sre_data_storage_account_data_private_sensitive",
      "component": true,
      "parent": "sre_data_component"
    },
    {
      "type": "dsh:sre:NFSV3StorageAccountComponent",
      "name": "sre_desired_state_storage_account",
      "component": true,
      "parent": "sre_desired_state_component"
    }
  ]
}

Apply the upgrade JSON as follows

dsh pulumi run YOURSRENAME 'import --file /full/path/to/your/upgrade.json --yes'
dsh pulumi run YOURSRENAME 'state unprotect --all'

Note that the first command might fail - the import should still have succeeded though.

Download the Pulumi state file

dsh pulumi run YOURSRENAME 'stack export --file /full/path/to/a/local/file.json'

Open the Pulumi state file in an editor and find-and-replace the following strings

From	To
dsh:sre:DataComponent$azure-native:storage:StorageAccount::sre_data_storage_account_data_private_sensitive	dsh:sre:DataComponent$dsh:sre:NFSV3StorageAccountComponent$azure-native:storage:StorageAccount::sre_data_storage_account_data_private_sensitive
dsh:sre:DataComponent$azure-native:storage:StorageAccount$azure-native:network:PrivateEndpoint::sre_data_storage_account_data_private_sensitive	dsh:sre:DataComponent$dsh:sre:NFSV3StorageAccountComponent$azure-native:storage:StorageAccount$azure-native:network:PrivateEndpoint::sre_data_storage_account_data_private_sensitive
dsh:sre:DataComponent$azure-native:storage:StorageAccount$azure-native:storage:BlobContainer	dsh:sre:DataComponent$dsh:sre:NFSV3StorageAccountComponent$azure-native:storage:StorageAccount$azure-native:storage:BlobContainer
dsh:sre:DataComponent$azure-native:storage:StorageAccount$pulumi-python:dynamic:Resource	dsh:sre:DataComponent$dsh:sre:NFSV3StorageAccountComponent$azure-native:storage:StorageAccount$pulumi-python:dynamic:Resource
dsh:sre:DataComponent$azure-native:storage:StorageAccount$azure-native:network:PrivateDnsZoneGroup::sre_data_storage_account_data_private_sensitive	dsh:sre:DataComponent$dsh:sre:NFSV3StorageAccountComponent$azure-native:storage:StorageAccount$azure-native:network:PrivateDnsZoneGroup::sre_data_storage_account_data_private_sensitive
dsh:sre:DesiredStateComponent$azure-native:storage:StorageAccount	dsh:sre:DesiredStateComponent$dsh:sre:NFSV3StorageAccountComponent$azure-native:storage:StorageAccount

Upload the edited Pulumi state file

dsh pulumi run YOURSRENAME 'stack import --file /full/path/to/a/local/file.json'

Deploy using v5.2.0 which will complete the rest of the upgrade

dsh sre deploy YOURSRENAME

What's Changed

• Cleaner exit when user credentials are incorrect by @craddm in #2296
• Print SRE FQDN when deployment finishes by @craddm in #2297
• Add logging for container instances by @JimMadge in #2295
• Merge latest (v5.1.0) into develop by @craddm in #2304
• Bump the production-dependencies group with 8 updates by @dependabot in #2306
• Add firewall logs by @JimMadge in #2308
• Update release checklist by @JimMadge in #2305
• Add workspace log docs by @craddm in #2312
• Ingest logs for blob containers by @JimMadge in #2310
• Add logging for file shares by @JimMadge in #2319
• Bump karancode/yamllint-github-action from 2.1.1 to 3.0.0 by @dependabot in #2324
• Bump the production-dependencies group with 9 updates by @dependabot in #2323
• Correct T2/3 PyPI/CRAN proxy information by @JimMadge in #2317
• Check that a user belongs to the correct SHM domain when registering with an SRE by @craddm in #2292
• [WIP] Add downloadable template security checklist by @craddm in #2328
• Release v5.2.0 by @JimMadge in #2326

Full Changelog: v5.1.0...v5.2.0

Release 5.1.0 (2024-11-21)

Release Highlights

• Logs from workspaces are now collected in a centralised log analytics workspace
• Research user IP address fields in the SRE configuration can now be set to Internet, rather than a specific IP address
• Bug fixes and documentation improvements

Upgrading from 5.0.1

🚨 Please update your SHM and all associated SREs! 🚨

In order to upgrade, you will need to carry out the following steps.

⚠️ Some manual interventions are needed. Please ensure that your data is appropriately backed-up before starting. ⚠️

Step-by-step upgrade instructions

N.B. throughout the instructions below, replace YOURSRENAME with the lower-case name of your SRE, YOURSHMNAME with the lower-case name of your SHM and YOURSHMFQDN with the fully-qualified domain name of your SHM (which you can find by running dsh config show-shm and looking for the key shm.fqdn).

For each SRE do the following

Delete the Hedgedoc, Identity, Gitea, and remote desktop container groups

The groups can be deleted via the portal or using Azure CLI.

In the portal, you will find the container groups in the SRE resource group, shm-YOURSHMNAME-sre-YOURSRENAME-rg. The name of the container groups follow the format shm-YOURSHMNAME-sre-YOURSRENAME-container-group-X, where X is the software within the group.

az container delete --name shm-YOURSHMNAME-sre-YOURSRENAME-container-group-hedgedoc --resource-group shm-YOURSHMNAME-sre-YOURSRENAME-rg
az container delete --name shm-YOURSHMNAME-sre-YOURSRENAME-container-group-identity --resource-group shm-YOURSHMNAME-sre-YOURSRENAME-rg
az container delete --name shm-YOURSHMNAME-sre-YOURSRENAME-container-group-gitea --resource-group shm-YOURSHMNAME-sre-YOURSRENAME-rg
az container delete --name shm-YOURSHMNAME-sre-YOURSRENAME-container-group-remote-desktop --resource-group shm-YOURSHMNAME-sre-YOURSRENAME-rg

Remove the DNS records for the deleted container groups

The CNAME and A records for the Hedgedoc, Identity, and Gitea resources need to be deleted from the public and private DNS zones.

This can be done in the portal, looking in the public DNS Zone for your SRE - YOURSRENAME.fqdn - for CNAME records, and the private DNS Zone - privatelink.YOURSRENAME.fqdn- for the A records.

Alternatively, use the Azure CLI, as below.

az network dns record-set cname delete --resource-group shm-YOURSHMNAME-sre-YOURSRENAME-rg --zone YOURSRENAME.YOURSHMFQDN --name identity
az network dns record-set cname delete --resource-group shm-YOURSHMNAME-sre-YOURSRENAME-rg --zone YOURSRENAME.YOURSHMFQDN --name gitea
az network dns record-set cname delete --resource-group shm-YOURSHMNAME-sre-YOURSRENAME-rg --zone YOURSRENAME.YOURSHMFQDN --name hedgedoc

az network private-dns record-set a delete --resource-group shm-YOURSHMNAME-sre-YOURSRENAME-rg --zone privatelink.YOURSRENAME.YOURSHMFQDN --name identity
az network private-dns record-set a delete --resource-group shm-YOURSHMNAME-sre-YOURSRENAME-rg --zone privatelink.YOURSRENAME.YOURSHMFQDN --name gitea
az network private-dns record-set a delete --resource-group shm-YOURSHMNAME-sre-YOURSRENAME-rg --zone privatelink.YOURSRENAME.YOURSHMFQDN --name hedgedoc

Delete the manually deleted resources from the Pulumi state

Run the following DSH CLI commands, ensuring that you have replaced the placeholders with the appropriate SHM and SRE names.

dsh pulumi run YOURSRENAME 'state delete urn:pulumi:shm-YOURSHMNAME-sre-YOURSRENAME::data-safe-haven::dsh:sre:IdentityComponent$pulumi-python:dynamic:Resource::sre_identity_entra_application --target-dependents'
dsh pulumi run YOURSRENAME 'state delete urn:pulumi:shm-YOURSHMNAME-sre-YOURSRENAME::data-safe-haven::dsh:sre:RemoteDesktopComponent$pulumi-python:dynamic:Resource::sre_remote_desktop_entra_application --target-dependents'

N.B. The $ character in the URN above may need to be escaped appropriately for your operating system. As written above, the command will work appropriately on Unix-based systems.

Delete pulumi_vars.yaml from blob storage

The pulumi_vars.yaml file needs to be deleted from blob storage.

• In the Azure portal navigate to the SRE resource group
• Find the "desired state" storage account (which will contain desiredstate in the middle of its name)
• In this storage account, open the desiredstate blob container.
• In the vars folder, delete the file pulumi_vars.yaml.

Delete the Entra groups and applications

Delete the Microsoft Entra groups and applications previously created by dsh.
These are now managed by Pulumi, which will not be able to run correctly if resources with identical names already exist.

The groups to be deleted are:

Data Safe Haven SRE YOURSRENAME Administrators
Data Safe Haven SRE YOURSRENAME Privileged Users
Data Safe Haven SRE YOURSRENAME Users

The applications to be deleted are:

Data Safe Haven (YOURSHMNAME) Service Principal
sre-YOURSRENAME-guacamole
sre-YOURSRENAME-apricot

SRE config files

The method of sanitising SRE names when creating remote configuration files has changed. Previously, hyphens or underscores in the SRE name were removed from the name used for the remote configuration file. If you have an SRE with a hyphen or underscore, you should download the configuration file at this point. Upload the configuration again once you have upgraded to release 5.1.0.

Redeploy the SHM and SRE

Finally, redeploy the SHM and SRE from release 5.1.0

dsh shm deploy
dsh sre deploy YOURSRENAME

What's Changed

• Bump the production-dependencies group with 13 updates by @dependabot in #2244
• Update all contributors by @JimMadge in #2257
• Merge release v5.0.1 into develop by @JimMadge in #2258
• Bump the production-dependencies group with 5 updates by @dependabot in #2259
• Update contributors names by @jemrobinson in #2260
• Bump ruff from 0.7.0 to 0.7.1 in the production-dependencies group by @dependabot in #2264
• Use Pulumi to create Entra applications by @jemrobinson in #2248
• Add confirmation checks and check for deployed SREs before teardown operations by @craddm in #2266
• Add additional documentation about the configuration of copy and paste by @craddm in #2265
• Enable monitoring agent to transmit to log analytics workspace by @craddm in #2279
• Bump lycheeverse/lychee-action from 2.0.2 to 2.1.0 by @dependabot in #2286
• Bump the production-dependencies group across 1 directory with 9 updates by @dependabot in #2287
• Allow 'Internet' for data providers IP by @JimMadge in #2247
• Change method of sanitising SRE names by @craddm in #2284
• [Documentation] Changing suggested SKU to Standard_D8s_v5 by @cptanalatriste in #2290
• docs: update @cptanalatriste as a contributor by @JimMadge in #2293
• Add documentation on updating SRE configurations by @craddm in #2291
• Bump the production-dependencies group with 8 updates by @dependabot in #2298

New Contributors

• @cptanalatriste made their first contribution in #2290

Full Changelog: v5.0.1...v5.1.0