Sitecore Experience Commerce: Working with Environment Variables

In this article, we will review how environment variables are utilised in the Commerce Engine solution. These are findings that provide some more context on top of Commerce Developer Reference: Configuring the Commerce Engine using environment variables, which will allow us to work with them more effectively.

Introduction

Environment variables are primarily utilised in 3 areas of the Commerce Engine and each area has nuances that we need to be aware of. The first area is in the Commerce Engine configuration file, config.json. The second is the global environment configuration, global.json. The third are is made up of the role environment and policy set configurations.

Environment variables will only be consumed into the Commerce Engine's configuration builder if they are prefixed with "COMMERCEENGINE_".

Environment Variable Usage in Configuration Files

The Commerce Engine Configuration File (config.json)

Using the environment variable configuration provider, the config.json is updated without the need for placeholders or data type specification.

Updating a property value simply requires adding an environment variable that represents the path of the property to be overridden.

For example, adding the environment variable, COMMERCEENGINE_Serilog__MinimumLevel__Default: Information, will override the corresponding configuration property by resolving it to the structure of the configuration.

{
  "AppSettings": { ... },
  "Serilog": {
    ...
    "MinimumLevel": {
      "Default": "Warning",
      ...
    }
  },
  ...
}

While the placeholder values aren't relevant to the overrides themselves, the Commerce Engine is distributed with 'PlaceholderFor<property>' values in config.json. Consider these flags to ensure that environment variables are not omitted. When replacing other properties in the Commerce Engine configuration, placeholders are not required.

The Global Environment Configuration File (global.json)

Similar to the config.json, the global environment configuration file, global.json, is loaded during the start up process of the Commerce Engine. However, while the Commerce Engine leverages the environment variables for this configuration, the replacement strategy differs from the environment variable configuration provider in that instead of resolving variables to the structure of the configuration file it instead resolves via placedholder matches.

For example, the environment variable COMMERCEENGINE_GlobalDatabaseUserName: sa will override the corresponding configuration property by resolving it to the placeholder.

{
  ...
  "Name": "GlobalEnvironment",
  "Policies: {
    "$type": "System.Collections.ObjectModel.ReadOnlyCollection`1[[Sitecore.Commerce.Core.Policy, Sitecore.Commerce.Core]], mscorlib",
    "$values": [
      ...
      {
        "$type": "Sitecore.Commerce.Plugin.SQL.EntityStoreSqlPolicy, Sitecore.Commerce.Plugin.SQL",
        ...
        "UserName": "PlaceholderForGlobalDatabaseUserName",
        ...
      },
      ...
    ]
  }
}

Role Environment and Policy Set Configuration Files

For environment variables used in role environment and policy set configurations, the Commerce Engine also uses the same strategy as per the global.json (placeholder matching over structure resolution), however the replacement is performed at the time of bootstrapping the Commerce Engine, not during the start up process of the Commerce Engine.

This means that while the role environment and policy set configurations will retain the 'PlaceholderFor...' values on disk, the engine startup will still load configurations from the commerce global database. Therefore, the Commerce Engine role will not resolve changes during the start up process with updated variables (e.g. running docker-compose up for containers).

Changing environment variables used in role environment and policy set configuration files, should be thought the same as making changes to the configuration files themselves, where bootstrapping the Commerce Engine is required to consume the changes.

Supported Data Types in Placeholders

In Commerce Developer Reference: Configuring the Commerce Engine using environment variables, it 3 mentions supported data types for placeholders, where string is the implicit (default) type, and boolean and integer values can be represented by appending '|bool' and '|int' to placeholder names respectively.

The reality is that under the hood all that is happening is identifying whether the value should be wrapped with quotation marks or not. This means that we can actually support other data types, such as decimals by appending either '|bool' or '|int' to the placeholder.

Data types apply to placeholders in the global environment, role environment and policy set configuration files. They do not apply to placeholders in the Commerce Engine configuration file (config.json).

PlaceholderValuesResults
"PlaceholderForMyVariable"dev.sitecore.com
true
"dev.sitecore.com"
"true"
"PlaceholderForMyVariable|bool" or
"PlaceholderForMyVariable|int"
dev.sitecore.com
true
3
1.2 (decimal)
dev.sitecore.com
true
3
1.2

Summary

The following table summarises how environment variables are utilised in the various configuration files of the Commerce Engine.

Configuration File(s)Property ResolverAllows Fallback / Default Value*AppliesApplied ByRequires Bootstrap
config.jsonStructureYesDuring start upEnvironment variable configuration provider (.Net Core)No
global.jsonPlaceholderNoDuring start upCommerce EngineNo
Environment/Policy Set ConfigurationsPlaceholderNoDuring BootstrapCommerce EngineYes
*Where environment variable not provided.

References

Sitecore Experience Commerce: Customising Commerce Content in the Sitecore Indexes for Storefront Search Results

In this article, we will look at how to control the storefront search results by customising the commerce content that is indexed in the Sitecore master and web indexes.

If you are looking for how to add new fields for faceting and other purposes, see Adding a property field to an index.

How Storefront Search Works

The SXA Search component creates a solr query based on a number of criteria, however focusing on how the search term is involved in the query, we see that the sxacontent field is our culprit for this customisation. This can be seen highlighted below in our storefront and in our search log.

From XC 9.3 onwards, the commerce portion of the Sitecore indexes were migrated over to the Commerce Engine for indexing performance improvements. There are a number of handlers utilised for populating the indexes with catalog data, however for managing the content related to the storefront search component we will look at the SxaContentFieldHandler.

See Catalog Search Field Handlers for a full list of handlers provided with XC, which are utilised when registering index fields in Plugin.Search.Solr.PolicySet-1.0.0.json.

The SxaContentFieldHandler Class

By default, the sellable item properties that are added to the sxacontent field consist of the following entries:

  • Tags
  • ProductId
  • Name
  • DisplayName
  • ItemDefinition
  • Brand

As we can see from the sitecore_web_index in Solr, a query for a specific 'Spectra' product, productid_t:6042260, shows that these properties are indexed against the sellable item.

{
  "sxacontent_txm":["39inch|4k|uhd|television|spectra",
    "6042260",
    "Habitat Spectra 39” 4K LED Ultra HD Television",
    "Habitat Spectra 39” 4K LED Ultra HD Television",
    "Product",
    "Spectra Televisions"],
  "productid_t":"6042260",
  ...
}

A common property that I have seen often requested to be included for searching is the sellable item's description, so we will use this property as well as a generic property from a custom component for demonstration purposes.

In the sample code below, we see that we inherit from the default SxaContentFieldHandler and execute the base ComposeValue method, which will populate the default fields listed above. We then simply add the properties to the sxaContent reponse object, which in this case is a list of strings.

You may have noticed that we are only adding the properties to sxacontent if a value is present. This is to minimise the bloat of empty entries and may lead to performance improvement however insignificant it may be. You may prefer to leave empty entries in the index to simplfy troubleshooting in mapping each entry back to its source property, so choose an approach that best suits you.

public class MySxaContentFieldHandler : Sitecore.Commerce.Plugin.Catalog.SxaContentFieldHandler
{
    public override object ComposeValue(object source, ConcurrentDictionary<string, object> context)
    {
        var sxaContent= base.ComposeValue(source, context) as List<string>;
        if (!(source is SellableItem) || sxaContent == null || sxaContent.Count() == 0)
        {
            return sxaContent;
        }

        var sellableItem = source as SellableItem;

        if (!string.IsNullOrWhiteSpace(sellableItem.Description))
        {
            sxaContent.Add(sellableItem.Description);
        }

        var myProperty = sellableItem.GetComponent<MyCustomComponent>().MyCustomProperty;
        if (!string.IsNullOrWhiteSpace(myProperty))
        {
            sxaContent.Add(myProperty);
        }

        return sxaContent;
    }
}

Now that we have created our modified SxaContentFieldHandler class, we register it in Plugin.Search.Solr.PolicySet-1.0.0.json for both the master and web indexes.

{
  "$type": "Sitecore.Commerce.Plugin.Search.ItemIndexablePolicy, Sitecore.Commerce.Plugin.Search",
  "IndexName": "sitecore_web_index",
  "FieldTypeMappers": [ ... ],
  "Fields": [
    {
      "$type": "Sitecore.Commerce.Plugin.Search.Solr.SolrIndexFieldConfiguration, Sitecore.Commerce.Plugin.Search.Solr",
      "Name": "sxacontent",
      "Type": "System.Collections.Generic.List`1[System.String]",
      "TypeHint": "textCollection",
      "Handler": {
        "$type": "Sitecore.Commerce.Engine.Search.MySxaContentFieldHandler, Sitecore.Commerce.Engine"
      }
    },
    ...
  ]
}

After deploying our customisation, boostrapping, and reindexing, we can see our custom entries in the sxacontent field.

{
  "sxacontent_txm":["39inch|4k|uhd|television|spectra",
    "6042260",
    "Habitat Spectra 39” 4K LED Ultra HD Television",
    "Habitat Spectra 39” 4K LED Ultra HD Television",
    "Product",
    "Spectra Televisions",
    "Enjoy incredible picture and dramatic detail with 4X the resolution of full HD. Enjoy incredible hues of color with technology that blends digital color for near-perfect representation. Experience Ultra HD picture quality with enhanced detail in every way.",
    "mycustomproperty"],
  "productid_t":"6042260",
  ...
}

And by performing a search in the storefront our search results now return our product from our custom search entries.

Development Considerations

Field Handlers are Synchronous

At this time, field handlers have been implemented synchronously and expect the information you require to be available on the provided source entity, e.g. sellable item in this case, therefore if you are looking at sourcing data externally it may be worth further consideration.

Following Helix Priniciples

Whether upgrading from a pre XC 9.3 solution or introducing existing component properties into the search index, by implementing the custom SxaContentFieldHandler you may be faced with having to break helix principles, i.e. housing your custom SxaContentFieldHandler in a feature project while referencing another feature project to retrieve the desired properties/content. As feature projects should not reference each other, I reluctantly chose to add MySxaContentFieldHandler to the Commerce Engine project in this instance.

References

Sitecore Experience Commerce: Pipeline Block Types

In this article, we will look at the foundational pipeline block types that are used to construct pipelines within the Commerce Engine plugins and custom solutions.

Asynchronous vs Synchronous Pipeline Blocks

From XC 10.0, the PipelineBlock class has been split into AsyncPipelineBlock and SyncPipelineBlock classes for improved performance and stability, with the primary implementation difference being whether to call the pipeline block asynchronously or not during pipeline execution.

A notable change to the AsyncPipelineBlock is that it uses the RunAsync method instead of the Run method to enforce that the return type is a Task<TOutput> and as an added reminder that the pipeline block is in fact asynchronously executed.

How Async and Sync Pipeline Blocks Work in the Same Pipeline

The initial questions I had when seeing the pipeline split were "how does this affect registering pipeline blocks to pipelines?" and "how does this affect the way in which pipelines are executed?". The good news here is that is doesn't affect either from an implementation standpoint.

From the registration perspective, both async and sync pipelines are registered as per XC 9.X registration mechanisms.

.AddPipeline<IMyPipeline, MyPipeline>(pipeline => pipeline
    .Add<MySyncPipelineBlock>()
    .Add<MyAsyncPipelineBlock>()
    .Add<MyAsyncConditionalPipelineBlock>()
    .Add<MyAsyncPolicyTriggerConditionalPipelineBlock>()
    .Add<MyConditionalPipelineBlock>()
    .Add<MyAsyncConditionalPipelineBlock>()
)

From the async/sync execution perspective, async pipeline blocks are awaited before continuing on to the following pipeline block, which ensures the output of a pipeline block is the input of the following pipeline block.

Specialised Pipeline Blocks

You may have seen a few types of enhanced pipeline blocks in XC 9.X+, being ConditionalPipelineBlock and PolicyTriggerConditionalPipelineBlock, which may have been better suited to your custom implementations over adding conditional statements to the begining of your Run/RunAsync method.

Conditional Pipeline Blocks

As indicated by their names, the ConditionalPipelineBlock and AsyncConditionalPipelineBlock classes will be conditionally executed. This is achieved by defining the BlockCondition predicate to be evaluated to determine if the Run/RunAsync method should be called.

Where the evaluated BlockCondition returns false, an alternate method called ContinueTask is executed instead of Run/RunAsync and is intended to simply return the arg or null.

Returning arg from ContinueTask considers the pipeline block as optional whereas returning null may consider the pipeline block as mandatory depending on how this output is handled in subsequent pipeline blocks or the pipeline itself.

Warning: Using the ContinueTask method to run alternate code logic may be a sign of a bad code smell, such as breaking the separation of concerns rule. Having 2 separate pipeline blocks with inverted predicates is the recommended approach.

public class MyConditionalPipelineBlock : ConditionalPipelineBlock<MyArgument, MyArgument, CommercePipelineExecutionContext>
{
    public MyConditionalPipelineBlock()
    {
        BlockCondition = obj => ((CommercePipelineExecutionContext)obj).CommerceContext.HasPolicy<MyCustomPolicy>();
    }

    public override MyArgument Run(MyArgument arg, CommercePipelineExecutionContext context)
    {
        Condition.Requires(arg, nameof(arg)).IsNotNull();

        // My code logic

        return arg;
    }

    public override MyArgument ContinueTask(MyArgument arg, CommercePipelineExecutionContext context)
    {
        return arg;
    }
}

Policy Trigger Conditional Pipeline Blocks

Another more specialised form of conditional pipeline blocks are the PolicyTriggerConditionalPipelineBlock and AsyncPolicyTriggerConditionalPipelineBlock. These blocks actually have somewhat inverted logic to their name as you will need to specify the name of a policy in the ShouldNotRunPolicyTrigger property which will ensure the pipeline is not executed.

When the policy name is present in the request header's PolicyKeys value the pipeline block will execute the ContinueTask method over the Run/RunAsync method (See Conditional Pipeline Blocks for more details).

A full set of existing policy keys and their usages can be found in Commerce Developer Reference: Policy keys. It is recommended that the Commerce Engine policy keys be avoided when creating custom policy trigger conditional pipeline blocks.

The BlockCondition predicate is already implemented to manage the conditional logic as mentioned above and the ContinueTask method will return the arg as policy trigger conditional pipeline blocks should be considered optional and not mandatory.

public class MyPolicyTriggerConditionalPipelineBlock : PolicyTriggerConditionalPipelineBlock<MyArgument, MyArgument, CommercePipelineExecutionContext>
{
    public override string ShouldNotRunPolicyTrigger => "IgnoreMyPipelineBlock";

    public override MyArgument Run(MyArgument arg, CommercePipelineExecutionContext context)
    {
        Condition.Requires(arg, nameof(arg)).IsNotNull();

        // My code logic

        return arg;
    }
}

References