Managing Commerce Configuration to Align with Helix Principles

In this article, we will review a sample configuration of the layers.config file to better manage your solution's configuration files when working with Sitecore Experience Commerce 9 (XC9).

Deprecated Article: See Sitecore: Using a Dedicated Custom Include Folder for Actual Custom Configuration Files for a more streamline approach to this problem.

If you take a look at an XC9 website's /App_Config/Include folder, you would have noticed that there's still some chaos happening with folder and file prefixes of 'Y' and 'Z' to assign priority order to the patch configuration files. On top of this, you may have even previous worked on projects where there have been crazy levels of prefixing 'z', 'zz', 'zzzzz', etc. to folders and files (I know I have seen some crazy levels of this), leading to developers pulling their hair out trying to locate the file's origin. As chaotic as this is, we will look at how we can bring back order for our solution.

Configuring layers.config

Our intention is to ensure that our solution's patch config files are always in a position to override the platform's patch files, while adhering to the Helix layers principle.

With the following approach, the only additional consideration introduced is to review the platform's custom config folders and files during any future upgrades to update the layers.config file to ensure the intended load order remains in tact.

Note: The following way is just one of many that can be implemented to achieve the same outcome. There is no one right way, so if you manage your solution's configuration files differently there is no need to align to this.

  1. Backup the layers.config. Always important when you plan on modifying any files that are not part of your project solution (as infrequent as this should be).
  2. Copy the layers.config file out of the App_Config folder into a core project in the same location. This is because the layers.config file itself cannot be patched and we would want this file version controlled so that the file can be deployed to all environments, and modified (if required) by our colleagues.
  3. The layers.config file is then updated to specify the load order for all current folders. While this step is a bit of a pain, as any omissions will mean that any folder/files unspecified will be applied after the loadOrder , it allows us to ensure that all of our solution configuration files will be applied last, having the highest level of priority.
    <layer name="Custom" includeFolder="/App_Config/Include/">
      <loadOrder>
        <add path="Cognifide.PowerShell.config" type="File" />
        <add path="Sitecore.Commerce.Carts.config" type="File" />
        <add path="Sitecore.Commerce.Catalogs.config" type="File" />
        <add path="Sitecore.Commerce.config" type="File" />
        <add path="Sitecore.Commerce.Customers.config" type="File" />
        <add path="Sitecore.Commerce.GiftCards.config" type="File" />
        <add path="Sitecore.Commerce.Globalization.config" type="File" />
        <add path="Sitecore.Commerce.Inventory.config" type="File" />
        <add path="Sitecore.Commerce.LoyaltyPrograms.config" type="File" />
        <add path="Sitecore.Commerce.Orders.config" type="File" />
        <add path="Sitecore.Commerce.Payments.config" type="File" />
        <add path="Sitecore.Commerce.Prices.config" type="File" />
        <add path="Sitecore.Commerce.Shipping.config" type="File" />
        <add path="Sitecore.Commerce.WishLists.config" type="File" />
        <add path="z.Cognifide.PowerShell.config" type="File" />
        <add path="ContentTesting" type="Folder" />
        <add path="Examples" type="Folder" />
        <add path="Feature" type="Folder" />
        <add path="Foundation" type="Folder" />
        <add path="Foundation.Overrides" type="Folder" />
        <add path="Project" type="Folder" />
        <add path="Y.Commerce.Engine" type="Folder" />
        <add path="Z.Commerce.Engine" type="Folder" />
        <add path="z.Feature.Overrides" type="Folder" />
        <add path="Z.Foundation" type="Folder" />
        <add path="Z.Foundation.Overrides" type="Folder" />
        <add path="Z.LayoutService" type="Folder" />
      </loadOrder>
    </layer>
  4. We then create three folders, following the Helix principles for our project solution. I just chose to use 'Helix' as a prefix to drill the point in.
    1. Helix.Feature
    2. Helix.Foundation
    3. Helix.Project
  5. Add the new folders to the <loadOrder>.
    <layer name="Custom" includeFolder="/App_Config/Include/">
      <loadOrder>
        ...
        <add path="Z.LayoutService" type="Folder" />
        <add path="Helix.Foundation" type="Folder" />
        <add path="Helix.Feature" type="Folder" />
        <add path="Helix.Project" type="Folder" />
      </loadOrder>
    </layer>
  6. As you are developing your solution you may find that you need to set the load order for the individual configuration files within a certain layer. In this case, you can list out the file order required. In the sample below, we have Feature A, Feature B, and Feature C. Feature A has a configuration conflict with Feature B and needs higher priority. Feature C has no conflicts. We only need to specify the config for Feature A after the folder to ensure the correct patch order.
  7. <layer name="Custom" includeFolder="/App_Config/Include/">
      <loadOrder>
        ...
        <add path="Helix.Feature" type="Folder" />
        <add path="Helix.Feature/Feature.A.config" type="File" />
        <add path="Helix.Project" type="Folder" />
      </loadOrder>
    </layer>
  8. Deploy your solution.

Summary

While you could continue down the path of having project with 'z', 'zz', 'zzzzz', etc. prefixes, by taking the initial steps to isolate and manage your solution's patch files under managed folder and file load order, we bring the chaos back to order with meaningfully named patch files that can be found exactly where you expect them to be located.

An Introduction to Sharding and Custom Sharding in Sitecore Experience Commerce 9

In this article, we will look at list and entity sharding that has been introduced in Sitecore Experience Commerce 9 (XC9), when we would create a custom shard, and how we can implement our own custom shard.

Sharding - What is it?

Let's take a quick step back and review the Shared Environment database from Sitecore Commerce 8.2.1. All commerce entities and lists were found under tables named CommerceEntities and CommerceLists respectively. These tables consisted of data pertaining to carts, orders, pricing and promotions.

With the introduction of XC9, additional entities and lists have been added to cater for the catalog and customer data, as part of the final phase out process of the Commerce Server.

Now it's a known fact that with any database that continues to accumulate data that inevitably there will be increasing performance degradation. This isn't the end of the world as databases can be tuned for performance, but there is a simple and logical way of managing data for optimal performance. This is where sharding comes into play.

Sharding, in XC9, is the logical partitioning of entities and lists into separate tables, to isolate and manage data growth for improved performance and data maintenance.

Sharding is managed via the ListShardingPolicy and EntityShardingPolicy policies in the Plugin.SQL.Sharding.PolicySet-x.y.z.json.
Each policy specifies the properties, RegularExpression and (database) TableName, which the Commerce Engine utilises to identify where the entity/list should be read from and stored to. When a list/entity is not matched to one of these policies regular expressions, it defaults to the CommerceEntities/CommerceLists tables.

When is Sharding  Required?

Just because a new entity is introduced in your project, this doesn't automatically mean that a new shard has to be implemented. Instead consider how this new entity will be utilised over time. If there is an expectation that over time this data will grow considerably where it should be housed separately, then sharding would be a good idea. However, if the data stored for this new entity is expected to be kept to a low volume, leaving the data in the default tables would be fine.

What is the Purpose of Sharding in the Commerce Global Database?

Simple answer. There is no purpose at this time. The script to create the databases was reusable and created the same structure. Only the CommerceEntities and CommerceLists tables are utilised for the Global database and I suspect this may be cleaned up in a future release.

How to Implement Custom Sharding

Creating Policies in the Commerce Engine

The following code snippet is a sample of a shard I am creating for a side-project for fulfillments.

{
    "$type": "Sitecore.Commerce.Plugin.SQL.ListShardingPolicy, Sitecore.Commerce.Plugin.SQL",
    "RegularExpression": "^List-Fulfillment.*?$",
    "TableName": "FulfillmentsLists"
},
{
    "$type": "Sitecore.Commerce.Plugin.SQL.ListShardingPolicy, Sitecore.Commerce.Plugin.SQL",
    "RegularExpression": "^List-DeletedFulfillmentsIndex-.*?$",
    "TableName": "FulfillmentsLists"
},
{
    "$type": "Sitecore.Commerce.Plugin.SQL.EntityShardingPolicy, Sitecore.Commerce.Plugin.SQL",
    "RegularExpression": "Entity-Fulfillment.*?$",
    "TableName": "FulfillmentsEntities"
},
{
    "$type": "Sitecore.Commerce.Plugin.SQL.EntityShardingPolicy, Sitecore.Commerce.Plugin.SQL",
    "RegularExpression": "Fulfillment-.*?$",
    "TableName": "FulfillmentsEntities"
}

Important note: Where sharding will be altered for a project, it is best to identify and implement it prior to development to avoid having to migrate data from its existing database table.

Creating a Database Table Shard

Database tables are based on either the entity table, or the list table definition. To create a new table shard, follow the process below for both table types.

In SQL Server Management Studio:-

  1. Right-clicking an existing table
  2. Selecting Script Table as > CREATE To > New Query Editor Window
  3. Updating the table name as appropriate, e.g.  [dbo].[<MyEntityType>Entities]
  4. For entity tables, update the constraint name as well, e.g. [PK_<MyEntityType>Entities]
  5. Execute the script

 

Safely Removing the Commerce Engine Default Storefront

In this article, we will go through the few configuration updates required to remove the Commerce Engine Default Storefront completely. Without these updates the website and Business Tools will throw errors, such as "InvalidShop" - "Shop 'CommerceEngineDefaultStorefront' does not exist."

Once we have created our replacement storefront, e.g. Storefront (above), we take the Item Name and update the configurations in the following places:-

  1. <Website Root>\App_Config\Include\Y.Commerce.Engine\Sitecore.Commerce.Engine.Connect.config at configuration > sitecore > commerceEngineConfiguration > defaultShopName
    In following development best practices, we don't want to modify this configuration file directly. Instead, we create a patch file in our project.
  2. <Sitecore BixFx Root>\assets\config.json under ShopName
  3. In Postman's environment configuration under ShopName

Once the above configurations have been updated to the desired storefront, the Commerce Engine Default Storefront storefront can be deleted.

Troubleshooting Changes to BixFx config.json in Chrome

In making changes to the BizFx configuration, if the Business Tools gets hung with loader displaying, check the Networking tab of the Developer Tools for the config.json to see if the request is coming from disk cache.

To resolve this:-

  1. Go to Application > Clear Storage
  2. Only the Cache > Application cache needs to be checked
  3. Clear site data
  4. Reload the page

We can now see the response for config.json has our updated ShopName.

Setting Currency Set for Catalog Product List Prices

In this article we go through the process of configuring the Currency Set that will be utilised when adding and editing the List Pricing properties of Sellable Items in the Merchandising Manager.

  1. In the Sitecore Content Editor, Go to /sitecore/Commerce/Commerce Control Panel/Shared Settings/Currency Settings/Currency Sets/.
  2. Copy the Item ID of the desired currency set. In the example below, I have created My Currency Set, which I will be using for this example.

  3. In the Commerce Authoring Environment configuration file (e.g. Habitat.CommerceAuthoring.json), locate the GlobalCurrencyPolicy and update the DefaultCurrencySet to the currency set's Item ID.
    {
        "$type": "Sitecore.Commerce.Core.GlobalCurrencyPolicy, Sitecore.Commerce.Core",
        "DefaultCurrencySet": "AC47503D-731A-4343-A156-3DBC7A5F1C8C"
    }
  4. Run the Bootstrap() command of the Commerce Engine.
  5. Confirm against any Sellable Item in the Merchandising Manager by clicking on the Add or Edit List Price actions.