Sitecore Experience Commerce: Working with Environment Variables

In this article, we will review how environment variables are utilised in the Commerce Engine solution. These are findings that provide some more context on top of Commerce Developer Reference: Configuring the Commerce Engine using environment variables, which will allow us to work with them more effectively.

Introduction

Environment variables are primarily utilised in 3 areas of the Commerce Engine and each area has nuances that we need to be aware of. The first area is in the Commerce Engine configuration file, config.json. The second is the global environment configuration, global.json. The third are is made up of the role environment and policy set configurations.

Environment variables will only be consumed into the Commerce Engine's configuration builder if they are prefixed with "COMMERCEENGINE_".

Environment Variable Usage in Configuration Files

The Commerce Engine Configuration File (config.json)

Using the environment variable configuration provider, the config.json is updated without the need for placeholders or data type specification.

Updating a property value simply requires adding an environment variable that represents the path of the property to be overridden.

For example, adding the environment variable, COMMERCEENGINE_Serilog__MinimumLevel__Default: Information, will override the corresponding configuration property by resolving it to the structure of the configuration.

{
  "AppSettings": { ... },
  "Serilog": {
    ...
    "MinimumLevel": {
      "Default": "Warning",
      ...
    }
  },
  ...
}

While the placeholder values aren't relevant to the overrides themselves, the Commerce Engine is distributed with 'PlaceholderFor<property>' values in config.json. Consider these flags to ensure that environment variables are not omitted. When replacing other properties in the Commerce Engine configuration, placeholders are not required.

The Global Environment Configuration File (global.json)

Similar to the config.json, the global environment configuration file, global.json, is loaded during the start up process of the Commerce Engine. However, while the Commerce Engine leverages the environment variables for this configuration, the replacement strategy differs from the environment variable configuration provider in that instead of resolving variables to the structure of the configuration file it instead resolves via placedholder matches.

For example, the environment variable COMMERCEENGINE_GlobalDatabaseUserName: sa will override the corresponding configuration property by resolving it to the placeholder.

{
  ...
  "Name": "GlobalEnvironment",
  "Policies: {
    "$type": "System.Collections.ObjectModel.ReadOnlyCollection`1[[Sitecore.Commerce.Core.Policy, Sitecore.Commerce.Core]], mscorlib",
    "$values": [
      ...
      {
        "$type": "Sitecore.Commerce.Plugin.SQL.EntityStoreSqlPolicy, Sitecore.Commerce.Plugin.SQL",
        ...
        "UserName": "PlaceholderForGlobalDatabaseUserName",
        ...
      },
      ...
    ]
  }
}

Role Environment and Policy Set Configuration Files

For environment variables used in role environment and policy set configurations, the Commerce Engine also uses the same strategy as per the global.json (placeholder matching over structure resolution), however the replacement is performed at the time of bootstrapping the Commerce Engine, not during the start up process of the Commerce Engine.

This means that while the role environment and policy set configurations will retain the 'PlaceholderFor...' values on disk, the engine startup will still load configurations from the commerce global database. Therefore, the Commerce Engine role will not resolve changes during the start up process with updated variables (e.g. running docker-compose up for containers).

Changing environment variables used in role environment and policy set configuration files, should be thought the same as making changes to the configuration files themselves, where bootstrapping the Commerce Engine is required to consume the changes.

Supported Data Types in Placeholders

In Commerce Developer Reference: Configuring the Commerce Engine using environment variables, it 3 mentions supported data types for placeholders, where string is the implicit (default) type, and boolean and integer values can be represented by appending '|bool' and '|int' to placeholder names respectively.

The reality is that under the hood all that is happening is identifying whether the value should be wrapped with quotation marks or not. This means that we can actually support other data types, such as decimals by appending either '|bool' or '|int' to the placeholder.

Data types apply to placeholders in the global environment, role environment and policy set configuration files. They do not apply to placeholders in the Commerce Engine configuration file (config.json).

PlaceholderValuesResults
"PlaceholderForMyVariable"dev.sitecore.com
true
"dev.sitecore.com"
"true"
"PlaceholderForMyVariable|bool" or
"PlaceholderForMyVariable|int"
dev.sitecore.com
true
3
1.2 (decimal)
dev.sitecore.com
true
3
1.2

Summary

The following table summarises how environment variables are utilised in the various configuration files of the Commerce Engine.

Configuration File(s)Property ResolverAllows Fallback / Default Value*AppliesApplied ByRequires Bootstrap
config.jsonStructureYesDuring start upEnvironment variable configuration provider (.Net Core)No
global.jsonPlaceholderNoDuring start upCommerce EngineNo
Environment/Policy Set ConfigurationsPlaceholderNoDuring BootstrapCommerce EngineYes
*Where environment variable not provided.

References

Sitecore: Using a Dedicated Custom Include Folder for Actual Custom Configuration Files

In this article, we will look at creating our own custom include folder in Sitecore to avoid the ugliness of the folder/file structure that comes with a Sitecore installation and all of its modules.

Include folder contents from default installation of Sitecore with Sitecore XC 9.3

Previously, I covered Managing Commerce Configuration to Align with Helix Principles, however this approach was overly complex and not really developer friendly. Since then I have concluded that the simpler approach would be leave the "custom" folder for the Sitecore integrations and create a dedicated custom folder outside of the "custom" for solution specific custom configurations.

Added benefits to this approach include:

  • When performing an upgrade, a new or updated configuration will not override your custom configurations based on load order.
  • You won't need to revisit the Layers.config file as potentially required with my previously documented approach.
  • For local development, you can comfortablely delete the dedicated custom folder and redeploy, avoiding any stale custom configurations from your current build.

Configuring Layers.config

As the order of the layer nodes in the Layers.confg defines the order in which they load at runtime, all that is required for this approach is to add the new custom folder to Layers.config after the "Custom" layer and ensure your custom solution configuration files are placed in the App_Config/Custom_Include folder within the projects.

...
<layer name="Custom" includeFolder="/App_Config/Include/">
  <loadOrder>
   <add path="Foundation" type="Folder" />
   <add path="Feature" type="Folder" />
   <add path="Project" type="Folder" />
  </loadOrder>
</layer>
<layer name="Custom_Include" includeFolder="/App_Config/Custom_Include/">
  <loadOrder>
    <add path="Foundation" type="Folder" />
    <add path="Feature" type="Folder" />
    <add path="Project" type="Folder" />
  </loadOrder>
</layer>
...

Add the custom folder after the "Custom" layer entry and include the "Foundation", "Feature", and "Project" folders to manage configuration files using Helix principles.

Custom Include folder contents segregated from the Include folder of the default installation of Sitecore with Sitecore XC 9.3

Summary

No longer will you need to fight for configuration priority over the platform's standard configuration files or soil your solution with 'zzz' prefixed patch files.

Managing Commerce Configuration to Align with Helix Principles

In this article, we will review a sample configuration of the layers.config file to better manage your solution's configuration files when working with Sitecore Experience Commerce 9 (XC9).

Deprecated Article: See Sitecore: Using a Dedicated Custom Include Folder for Actual Custom Configuration Files for a more streamline approach to this problem.

If you take a look at an XC9 website's /App_Config/Include folder, you would have noticed that there's still some chaos happening with folder and file prefixes of 'Y' and 'Z' to assign priority order to the patch configuration files. On top of this, you may have even previous worked on projects where there have been crazy levels of prefixing 'z', 'zz', 'zzzzz', etc. to folders and files (I know I have seen some crazy levels of this), leading to developers pulling their hair out trying to locate the file's origin. As chaotic as this is, we will look at how we can bring back order for our solution.

Configuring layers.config

Our intention is to ensure that our solution's patch config files are always in a position to override the platform's patch files, while adhering to the Helix layers principle.

With the following approach, the only additional consideration introduced is to review the platform's custom config folders and files during any future upgrades to update the layers.config file to ensure the intended load order remains in tact.

Note: The following way is just one of many that can be implemented to achieve the same outcome. There is no one right way, so if you manage your solution's configuration files differently there is no need to align to this.

  1. Backup the layers.config. Always important when you plan on modifying any files that are not part of your project solution (as infrequent as this should be).
  2. Copy the layers.config file out of the App_Config folder into a core project in the same location. This is because the layers.config file itself cannot be patched and we would want this file version controlled so that the file can be deployed to all environments, and modified (if required) by our colleagues.
  3. The layers.config file is then updated to specify the load order for all current folders. While this step is a bit of a pain, as any omissions will mean that any folder/files unspecified will be applied after the loadOrder , it allows us to ensure that all of our solution configuration files will be applied last, having the highest level of priority.
    <layer name="Custom" includeFolder="/App_Config/Include/">
      <loadOrder>
        <add path="Cognifide.PowerShell.config" type="File" />
        <add path="Sitecore.Commerce.Carts.config" type="File" />
        <add path="Sitecore.Commerce.Catalogs.config" type="File" />
        <add path="Sitecore.Commerce.config" type="File" />
        <add path="Sitecore.Commerce.Customers.config" type="File" />
        <add path="Sitecore.Commerce.GiftCards.config" type="File" />
        <add path="Sitecore.Commerce.Globalization.config" type="File" />
        <add path="Sitecore.Commerce.Inventory.config" type="File" />
        <add path="Sitecore.Commerce.LoyaltyPrograms.config" type="File" />
        <add path="Sitecore.Commerce.Orders.config" type="File" />
        <add path="Sitecore.Commerce.Payments.config" type="File" />
        <add path="Sitecore.Commerce.Prices.config" type="File" />
        <add path="Sitecore.Commerce.Shipping.config" type="File" />
        <add path="Sitecore.Commerce.WishLists.config" type="File" />
        <add path="z.Cognifide.PowerShell.config" type="File" />
        <add path="ContentTesting" type="Folder" />
        <add path="Examples" type="Folder" />
        <add path="Feature" type="Folder" />
        <add path="Foundation" type="Folder" />
        <add path="Foundation.Overrides" type="Folder" />
        <add path="Project" type="Folder" />
        <add path="Y.Commerce.Engine" type="Folder" />
        <add path="Z.Commerce.Engine" type="Folder" />
        <add path="z.Feature.Overrides" type="Folder" />
        <add path="Z.Foundation" type="Folder" />
        <add path="Z.Foundation.Overrides" type="Folder" />
        <add path="Z.LayoutService" type="Folder" />
      </loadOrder>
    </layer>
  4. We then create three folders, following the Helix principles for our project solution. I just chose to use 'Helix' as a prefix to drill the point in.
    1. Helix.Feature
    2. Helix.Foundation
    3. Helix.Project
  5. Add the new folders to the <loadOrder>.
    <layer name="Custom" includeFolder="/App_Config/Include/">
      <loadOrder>
        ...
        <add path="Z.LayoutService" type="Folder" />
        <add path="Helix.Foundation" type="Folder" />
        <add path="Helix.Feature" type="Folder" />
        <add path="Helix.Project" type="Folder" />
      </loadOrder>
    </layer>
  6. As you are developing your solution you may find that you need to set the load order for the individual configuration files within a certain layer. In this case, you can list out the file order required. In the sample below, we have Feature A, Feature B, and Feature C. Feature A has a configuration conflict with Feature B and needs higher priority. Feature C has no conflicts. We only need to specify the config for Feature A after the folder to ensure the correct patch order.
  7. <layer name="Custom" includeFolder="/App_Config/Include/">
      <loadOrder>
        ...
        <add path="Helix.Feature" type="Folder" />
        <add path="Helix.Feature/Feature.A.config" type="File" />
        <add path="Helix.Project" type="Folder" />
      </loadOrder>
    </layer>
  8. Deploy your solution.

Summary

While you could continue down the path of having project with 'z', 'zz', 'zzzzz', etc. prefixes, by taking the initial steps to isolate and manage your solution's patch files under managed folder and file load order, we bring the chaos back to order with meaningfully named patch files that can be found exactly where you expect them to be located.