CTSM5.4: CMIP7 Datasets. CRUJRA2024 forcing, Namelist default changes for clm6_0 physics

What's new in CTSM 5.4 (tag ctsm5.4.002)

Purpose and description of changes since CTSM 5.3 (tag ctsm5.3.021)

New features

• New surface datasets from CMIP7 data including PFT and urban distributions, land use transitions, population density, and atmospheric C isotopes. These data are only available through the historical record (1850-2023), and

  • are not available for future periods (presently known as SSP),
  • for future periods and N deposition we continue to use CMIP6 data from CESM2.

• Option to use CRUJRA2024 atmospheric driver data with clm6 and clm5 physics options (PR #2956), this is the default data-atmosphere (DATM) for clm6. This CRUJRA dataset covers 1901-2023, whereas previous GSWP3 only covers 1901-2014.

• Capability to run single-point PLUMBER tower sites, similar to the NEON tower capability (issue #1487). Initial conditions are not provided for PLUMBER sites.

• New CLM_CMIP_ERA flag in env_run.xml. Valid options are cmip7 and cmip6. Defaults to cmip7 except in compsets containing SSP for which it defaults to cmip6 because there are no future-period datasets yet available for CMIP7.

• Automatic, more flexible use of anomaly forcings for CMIP6 ISSP cases, which also use the cmip6 CLM_CMIP_ERA flag: Documentation

• Unsupported script that checks for spinup equilibrium in tools/contrib/ for spectral element grids (PR #2991).

• New paramfile tools that allow users to query and modify CLM parameter files (documentation)

• Optional time-evolving `leafcn_target`. More under “Additional detail” below.

• New vertical movement scheme for soil nitrate, which is off by default (PR #2992).

• Documentation improvements and new URL: https://escomp.github.io/CTSM/index.html.

• FATES:

  • Grazing (sci.1.81.0_api.37.1.0).
  • Johnson and Berry 2021 electron transport model (sci.1.85.0_api.40.0.0).
  • Managed Fire (sci.1.87.0_api.41.0.0).

Answer changes

Changes to defaults for `clm6` physics:

• New CMIP7 surface and landuse timeseries datasets (see in Additional Details below).
• New namelist variables `snow_thermal_cond_glc_method` and `snow_thermal_cond_lake_method` (PR #3072). Snow thermal conductivity uses Jordan1991 over glaciers to reduce Greenland melt rates by default and Sturm over land and lake land units.
• Bytnerowicz is now the default nfix_method for clm6 (#2972) which revises the temperature function for nitrogen fixation, replacing the Houlton et al. function.
• Updates to MEGAN for BVOCs (#3065 #3309). Removes dependence on soil moisture from clm6 physics.
• New model parameter values that were calibrated to improve carbon cycle representation with CRUJRA.
• New model parameter values that were calibrated to improve the fire model. Now using li2024 fire code.
• New initial conditions files for f09 ("1-degree" 1850, 2000), f19 (“2-degree” 1850), and ne30 (1850, 1979, 2000) resolutions.
• Change default for glcmec_downscale_longwave to FALSE for clm6 physics as turning off the LW downscaling improves the melt and runoff biases.
• See “Changes to FATES and the FATES parameter file” below.
• Namelist defaults change so that
  • use_c13/use_c14 are on only for HistClm60Bgc compsets with CRUJRA2024 or CAM7 forcing; examples of when use_c13/use_c14 are now off include SSP and single-point compsets, as well as cases using older forcings, such as CAM6, GSWP3v1, Qian, and CRUv7
  • when use_c13 or use_c14 is on, turn on the corresponding time series file (responding to the CLM_CMIP_ERA flag)
  • C13/C14 CMIP7 data is done using streams with new namelist variables (stream_atm_c13, stream_atm_c14)
  • irrigation is on for transient cases (1850-2000, 1850-2100, but not for clm4_5).

Changes for all physics versions:

• Parameters updated: Added MIMICS parameter `mimics_fi` (fraction of litter inputs that bypass litter pools, directly contributing to SOM) and updated other MIMICS parameters (#2365) to remove NPP control on turnover, fix density dependent control on turnover, add litterfall fluxes that bypass litter pools and contribute directly to soil organic matter.
• FATES parameter file updated: (PR #2965, PR #2904, PR #1344, PR #3087). See “FATES parameter file” section below for details.
• New surface datasets and landuse timeseries files (see “surface datasets” section below).
• CMIP7 C13/C14 atmospheric timeseries data

Heads up

• History tapes now split into two files from hX to hXi and hXa, where X is the tape number (e.g. h0i/h0a) and where "i" stands for history file containing instantaneous fields, while "a" stands for history file containing non-instantaneous fields. Details in the “history files” section below and in the PRs #2445 ESCOMP/MOSART#117 ESCOMP/RTM#61 and the corresponding issues.
• Adding time to 1d weighting fields in transient simulations PR #3328
• Regarding CMIP7 vs. CMIP6 inputs:
  • C13/C14 isotope datasets are the new CMIP7 datasets using streams, when CLM_CMIP_ERA==cmip6, the older cmip6 files are used
  • We supply only CMIP7 population density with clm6 physics in non-SSP cases, because the fire model is calibrated to that; conversely, we supply only CMIP6 population density for pre-clm6 physics and for SSP cases.
  • We supply only CESM2 nitrogen deposition (ndep), so this gets used regardless of CLM_CMIP_ERA setting.
  • For DATM we supply only CMIP6 aerosols.
  • For DATM we supply only CMIP6 CO2.
• Issue with DOUT_S_SAVE_INTERIM_REST #3351 was fixed.
• As of ctsm5.3.040, the new ctsm_pylib conda environment is incompatible with our tools from before ctsm5.3.040 and vice versa. More under “Additional detail” below.

Additional detail

Changes related to history files

(Note 1: The same information in this section applies to MOSART and RTM.
Note 2: The gist of the information in this section also appears in the CTSM User’s Guide).

Following ctsm5.3.018 "Change history time to be the middle of the time bounds" and keeping CLM history consistent with CAM history, the CTSM5.4 change intends to prevent confusion associated with the time corresponding to instantaneous history fields by putting them on separate files than non-instantaneous fields.

The now separate instantaneous history files represent the exact time step when they were written and do not include a time_bounds variable. Conversely, non-instantaneous history files represent the period of their time_bounds variable. As a result, time data on non-instantaneous history files are now read correctly during post processing (e.g. by xarray). Special handling may still be needed for instantaneous history files, whose timestamps represent the date and time at the END of the history timestep. So, e.g., an instantaneous variable saved at the end of year 2023 will get the timestamp 2024-01-01 00:00:00.

Users will now see:

1) Two history files per clm, mosart, and rtm history tape:
tape h0 becomes h0a and h0i
tape h1 becomes h1a and h1i
...
tape hX becomes hXa and hXi

2) Two history-restart files per history restart tape:
rh0 becomes rh0a and rh0i
rh1 becomes rh1a and rh1i
...
rhX becomes rhXa and rhXi

The CLM handles empty history (and corresponding history-restart) files by not generating them, while rtm and mosart give an error. Instead of refactoring rtm and mosart to behave like the clm (considered out of scope), we have introduced one active instantaneous field in mosart and one in rtm to bypass the "empty file" error.

New surface datasets and landuse timeseries files (#3482)

• Transient landuse timeseries files going back to 1700 made for f09 and 360x720 grids.
• New resolutions now supported: ne3np4.pg3, mpasa30, ne0np4.NATL.ne30x8 (#3482)
• Updates to input datasets (also referred to as raw datasets):
  • PFT/LAI/soil-color raw datasets; now from the CMIP7 timeseries that ends in 2023 (Issue #2851).
  • Two fire datasets: crop fire peak month and population density (#2701 #3302).
  • Transient (historical) urban datasets are now based on CMIP7 urban data, partitioned into TBD, HD, and MD classes in proportion to GaoOneill present day classification.

Changes to FATES and the FATES parameter file

• See HLM-FATES compatibility table in the FATES user’s guide...

CTSM 5.3: New surface datasets, git-fleximod intro, Leung dust, explicit AC, Matrix-CN, FATES transient LU, time/history metadata changes

Purpose and description of changes since CTSM 5.2 (tag ctsm5.2.005)

New features

• manage_externals replaced by git-fleximod. (PR #2559)
• No longer runs the 0th time step in first segment of startup and hybrid runs; branch and continue runs never had this 0th time step. (PR #2084)
• New CN Matrix method speeds up spinup for the BGC model. (PR #640; Liao et al. 2023).
• New Leung_2023 dust emissions. (PR #1897; Leung et al 2023, Leung et al. 2024)
• Explicit air conditioning for the urban model. (PR #2275; Li et al. 2024)
• FATES compsets can now be run with transient land use; off by default. (PR #2507)
• Ability to handle CAM7 in LND_TUNING_MODE. (PR #2632)

Answer changes

Changes to defaults for clm6_0 physics:

• Urban explicit A/C turned on (links above).
• Snow thermal conductivity method is now Sturm1997. (PR #2348; see also discussion #1960)
• New initial conditions files for f09 ("1-degree" 1850, 2000), f19 (“2-degree” 1850), and ne30 (1850, 1979, 2000) resolutions.
• New crop calendars. (PR #2664; informed by Rabin et al., 2023)
• Dust emissions method is now Leung_2023 (links above).
• Excess ice is turned on. (PR #1787)
• Updates to MEGAN for BVOCs. (PR #2588)
• New BGC fire method li2024crujra: Avoid crop fires during growing season; allow lightning ignitions in tropical closed forests; add effect of landscape fragmentation on ignitions and duration; recalibrate against GFED5 burned area and with CRU-JRA climate. (PR #2684, PR #2711, PR #2715)

Changes for all physics versions:

• Parameters updated for CRU-JRA forcing: PPE-based modifications were made to the parameters leafcn, slatop, froot_leaf, medlynslope, and kmax. Lowers LAI and biomass in boreal and tropical forests, without reducing latent heat in the tropics. Affected PFTs: NET temperate, NET boreal, BET tropical, BDS boreal, C3 arctic grass. (PR #2500)
• FATES parameter file updated (see section below).
• Pass active glacier (CISM) runoff directly to river model (MOSART) (MOSART PR #94)
• New surface datasets and landuse timeseries files (see section below).
• CNMatrix is new default spinup method (links above).

Heads up

• Small glacier changes mean that you can’t use a 5.3 surface dataset with pre-5.3 code and vice versa anymore. (Merged with PR #2500)
• Updates the definition of history variable “time” from end of time_bounds to middle of time_bounds. (PR #2838; see section below)
• Standardizes history variable attributes and a history dimension name. (PR #2052; see section below)

Additional detail

Changes related to time and history files

(Note that the same information in this section applies to MOSART and RTM.)

Startup and hybrid runs no longer run the 0th time step, consistent with the same change in CAM. (Branch and continue runs never had this 0th time step.) This means you will not get an extraneous initial history file anymore. In some circumstances this may also affect the names of history files.

In most cases, the history time variable is now defined as the middle of a history file’s time_bounds instead of the end, for consistency with the same change in CAM. The exception is if you specify hist_avgflag_pertape = 'I' for that file, in which case it will be treated as an “instantaneous” file. Instantaneous history files (a) have their time coordinate set to the end of the last timestep (as did all history files before this tag) and (b) do not include time_bounds.

The history dimension name hist_interval (of output variable time_bounds) is standardized to be nbnd. History variables time_bounds, mcdate, mcsec, mdcur, and mscur are standardized to include the calendar attribute.

New surface datasets and landuse timeseries files (PR #2500)

• Transient landuse timeseries files going back to 1700 now possible (and made for f09).
• Fix an important bug on soil fields that was there since ctsm5.2.0. This has the side effect of mksurfdata_esmf now giving identical answers with a change in number of processors, as it should. (Issue #2744)
• Surface datasets now provided for the ne0np4.POLARCAP.ne30x4 grid. (PR #2716, issue #2720)
• Surface datasets now have their version number embedded to prevent mismatch of surface dataset and CTSM version. (Issue #2723)
• Remove outdated hydrology VIC (Variable Infiltration Capacity Hydrology model) fields from surface datasets.
• Updates to input datasets:
  • PFT/LAI/soil-color raw datasets; now from the TRENDY2024 timeseries that ends in 2023. (Issues #2570 and #2452)
  • Two fire datasets: crop fire peak month and peatland fraction. (Issue #2618)
  • Glacier behavior dataset (related to how non-Greenland glaciers are handled). (Issue #423)

Changes to FATES parameter file

• PR #2507 (ctsm5.2.013) / FATES PR #1116 (sci.1.77.1_api.36.0.0)
  • Adds new parameters for new land use harvest mode and land use by PFT capabilities
• PR #2700 (ctsm5.3.003) / FATES PR #1255 (sci.1.78.3_api.36.1.0)
  • Adds two arctic shrub PFTs, increasing the number of default PFTs to 14 and update arctic grass parameters (FATES PR#1236)
  • Splits fates_turnover_leaf parameter into canopy and understory specific turnover rates and provides new values for both (FATES PR#1136)
  • Updates grass allometry parameters (FATES PR#1206)
  • Changes the prescribe nutrient uptake defaults from 1 to 0 for all PFTs (only relevant to ELM-FATES)

Changes to rpointer files

The rpointer files are simple text files that CESM uses to keep track of how far simulations have progressed, pointing to the filename of the latest restart file for that component. There is one such file for each component, so for CTSM I cases that's lnd, cpl, and atm (and rof if it's active). Normally, when the user is just extending the length of simulations, there’s no need to worry about these files.

However, if there was a problem when a simulation shut down, it's possible that different components will have mismatched restarts and rpointer files. In the past, this meant figuring out what restart file should be pointed to in each component rpointer file and correcting it by hand in an editor. There was only the final set of rpointer files that was kept for a case.

Now, with this update, the lnd, cpl, and atm rpointer files have the simulation date in the filenames, so it's easy to spot if the restarts are mismatched for one of the components. Also, since there are matching rpointer files for each time restarts are created, it's now easier to (a) make sure restarts and rpointer files are all correctly matched and (b) for a user to take a set of restarts and matching rpointer files to start up from for any part of an existing simulation. This means you don't have to hand-edit the rpointer files, making sure you don't make a mistake when you do.

Old rpointer filenames:

• rpointer.atm
• rpointer.cpl
• rpointer.lnd

New names:

• rpointer.atm.YYYY-MM-DD-SSSSS
• rpointer.cpl.YYYY-MM-DD-SSSSS
• rpointer.lnd.YYYY-MM-DD-SSSSS

Where YYYY-MM-DD-SSSSS is the year, month, day, and seconds into the day for the simulation timestamp. For example, rpointer.lnd.2000-01-01-00000 for a rpointer file for starting at midnight (beginning of the day) on January 1st, 2000.

Note that this is backwards-compatible, so for all the components you can use either the new or old format for the rpointer filenames. Thus, if you are restarting from an existing case before ctsm5.3.016, you CAN use the rpointer filenames that don't have the timestamps in the name.

Simulations supportin...

CTSM 5.2: New surface datasets and mksurfdata_esmf tool to create them

Summary

• All new surface datasets, with updated input datasets.
• New mksurfdata_esmf tool to make global surface datasets.
• New tools to create inputs for regional simulations.
• ne0np4 grid: New 1979 surface dataset and 1979-2026 land use.
• Transient urban and lake by default turned on for transient cases.
• Ocean is run as baresoil rather than wetland (for clm6_0).
• The urban streams file was also updated.
• Update the README files.
• New FATES parameter file: Tree PFT allometry, allometric mode options, leaf maintenance scaling coefficients.

New surface datasets

The new surface datasets are incompatible with previous versions (for example the ctsm5.1 series)—ctsm5.2.0 and following versions can NOT use the previous ctsm5.1 datasets, and vice versa.

See the section below about the new datasets used in their creation. Improvements in how landunits on coastal areas were also made.

Fields added to the surface datasets in ctsm5.2:

• ORGC, BULK, CFRAG, PHAQ (soil data) (currently NOT used by CTSM)
• mapunits (map units from the soil dataset)
• LANDFRAC_MKSURFDATA (for reference NOT used by CTSM)
• PCT_OCEAN (previously PCT_WETLAND was used)

Fields removed from the surface datasets in ctsm5.2:

• AREA
• PFTDATA_MASK

New input data used for making surface datasets

• New soil dataset: ISRIC/WISE dataset (Batjes, 2016; https://doi.org/10.1016/j.geoderma.2016.01.034)
• New PFT, soil-color, LAI datasets: Created by Peter J. Lawrence (2022)
• New Glacier datasets: Glacier outlines from RGI version 6 (Arendt et al., 2017).
  • vector data for GrIS and AIS retrieved from BedMachine version 4 and version 2 (Morlighem et al., 2017, 2020), respectively.
  • 30-arcsec topography/land mask retrieved GMTED2010 (Danielson and Gesch, 2011).
• New urban datasets: Gao and O'Neill (2021) and Gao and Pesaresi (2022), Oleson and Feddema (2020)
• New lake datasets: HydroLake: Messager et. al. (2016)

New mksurfdata_esmf Tool

mksurfdata_esmf is a parallel version of mksurfdata that uses ESMF regridding directly so that offline mapping files don't have to be created as a separate step. This allows surface datasets to be created at much higher resolutions.

The build for the tool is based on the CESM/CIME build system and uses cmake. This allows the build to be kept up with changes in CESM. Currently it's only setup and working on Derecho, but this design will enable it to be built and run on any CESM-supported machine (or a machine that a user ports to).

Any input grid from ccs_config can be used, or the user can supply their own mesh file to define the output grid. The user no longer has to add to the list of valid resolutions (as in the now-deprecated mksurfdata_map).

Creation of supported single point datasets. These datasets are created through the use of subset_data.

Test datasets for dynUrban, dynLake, and dynPFT is done with a simple NCO script.

All datasets can be easily made by running make all in the tools/mksurfdata_esmf/ directory.

For detailed instructions, see tools/mksurfdata_esmf/README.md.

release-clm5.0.37

Synopsis

Changes needed to run on Derecho. Flip the test list from Cheyenne to Derecho.
Add working PE layouts. Remove tools and tools testing. Update readme files.

What's Changed

• add smyle L83 compset and SSP3-7.0 extension by @jedwards4b in #1614
• Fix issues #1769, #1607, #1389 in release documentation by @olyson in #1781
• Fix issue #1864 in release documentation by @olyson in #1916

release-cesm2.2.04: Update PElayout and tests to work on Derecho

Get working PE-layouts for Derecho. Move test list over to Derecho from
Cheyenne.
Remove all of the CTSM tools, users should use the latest.
Some fixes to various tests to get them working.

NEON release: Some NEON updates fixing AG sites, update MOSART, small fixes

release tag for use with NEON:

Minor changes to python scripts and usermod_dirs for NEON cases. Also update the lightning mesh file so that it goes with the
smaller lightning file. Have NEON use new use-cases for 2018 and 2018-PD conditions for CLM. Have NEON
Agricultural sites run with prognostic crop. Simple fix for warning about NaN's in import/export data from/to coupler.

Get NEON tests working on izumi, add --inputdata-dir to subset_data and modify_singlept_site_neon.py so they aren't tied
to only running on cheyenne.

Also update MOSART with fixed for direct_to_outlet option.

Add error checking in ParitionWoodFluxes. Fix value of albgrd_col in SurfaceAlbefdoType.F90.
Previously, the wrong value (albgri_col) was being set in InitHistory.

What's Changed

• Neon updates by @wwieder in #1872
• Fix albgrd_col value in SurfaceAlbedoType by @ka7eh in #1810
• Add error checking in ParitionWoodFluxes by @ka7eh in #1814

New Contributors

• @ka7eh made their first contribution in #1810

Full Changelog: ctsm5.1.dev113...ctsm5.1.dev114

Add SSP3-7.0 extension, and SMYLE 1850 and transient, compsets and use-cases

Update externals for cism and cime to cesm2.1.4-rc-10. cime update was needed to run on izumi.
iAdds a compset for the smyle L83 model (both 1850 and transient). BWsc1850smyle (jedwards4b)
Add SSP3-7.0 extension compset and use-case, that runs for fixed 2100 conditions with 2100 landuse
and 2100 harvest and fertilizer.

Update Nitrogen-deposition datasets with files created from WACCM simulations for SSP scenarios (SSP3-7.0,
SSP5-3.4, and SSP5-8.5).

Fix several issues.

Add LILAC

Add LILAC: The Lightweight Infrastructure for Land-Atmosphere Coupling. This
infrastructure consists of two major pieces:

(1) A lightweight coupling infrastructure built on top of ESMF that makes it easier for
atmosphere models to call CTSM directly, rather than using the hub-and-spoke architecture
that is used by CESM.

(2) A set of python-based tools for building CTSM and creating its runtime inputs when
running in an atmosphere model via LILAC. Although these tools are built on top of cime,
details of the create_newcase / case.setup / case.build process are hidden from the user,
because many of the aspects of this workflow don't make sense in the LILAC context.

So far we have used LILAC to couple CTSM to WRF. There are plans to use the same
infrastructure to couple CTSM to other regional atmosphere models.

Documentation of LILAC is provided in
https://escomp.github.io/ctsm-docs/versions/master/html/lilac/index.html (though there are
still some missing sections), as well as in various presentations on the wiki
(https://github.com/ESCOMP/CTSM/wiki/Presentations).

There have been many contributors to the development, testing and documentation of LILAC;
chief among them being Mariana Vertenstein, Negin Sobhani, Bill Sacks, Joe Hamman, Sam
Levis, Mike Barlage and Dave Lawrence.

Update documentation for release-clm5.0 branch, and fix issues with no-anthro surface dataset creation

Fix the issues with the no-anthro surface dataset creation (the PtVg simulation year). This
makes sure GDP and urban is set to zero and abm to missing for this case. Also bring in the python3
fix that was intended for the last release tag.

Turn irrigation on for 2300 SSP extensions

Irrigation was turned off for the 2300 extensions. This turns it on. Also add SSP2-4.5 to the
anomaly forcing script.