Sitecore Experience Commerce: Adding Custom Properties to Search in BizFx

Reading Time: 8 minutes

In this article, we will review how we can extend the search components in Business Tools (BizFx) to support search functionality using custom properties for its results.

This article will not consider locale-support.

Introduction

As the search functionality in BizFx is fairly primitive, from a business user perspective, and a common request is to provide customisation guidance for the search functionality in the Business Tools, we will use the Search component from the Merchandising dashboard for our example in extending the search functionality.

Figure 1: Search component in the Merchandising Dashboard.

This implementation will assume the sellable items that have custom properties added via composer templates, however you will be able to adapt the solution where properties have been added programatically with components or policies.

Figure 2: A sellable item extended with a composer template.

Implementation and Configuration

Updating the Solr Schema

We first need to add our custom property to the managed schema of our Solr cores related to our search scopes. These schemas are found in the file system where Solr is deployed, under server/solr/<search scope>/conf/managed-schema.

The commerce engine uses switch on rebuild so we need to update the managed schema for both CatalogItemsScope and CatalogItemsScope-Rebuild for this instance.

    <!-- CommerceEngine Catalog -->
    <field name="displayname" type="string" indexed="true" stored="true"/>
    <field name="datecreated" type="pdate" indexed="true" stored="true"/>
    <field name="dateupdated" type="pdate" indexed="true" stored="true"/>
    <field name="artifactstoreid" type="string" indexed="true" stored="true"/>
    <field name="parentcataloglist" type="string" indexed="true" stored="true" multivalued="true"/>
    <field name="variantid" type="string" indexed="true" stored="true"/>
    <field name="variantdisplayname" type="string" indexed="true" stored="true"/>
    <field name="inventorysetids" type="string" indexed="true" stored="true"/>
    <field name="productid" type="string" indexed="true" stored="true"/>
    <field name="name" type="string" indexed="true" stored="true"/>
    <field name="yearmanufactured" type="string" indexed="true" stored="true"/>

The managed-schema that would be created with the XC instance will be organised to easily identify the commerce entity fields. While fields can be added via Solr, the managed-schema will be regenerated in a less XC friendly manner and make long-term mainentance more difficult.

With our Solr schema updated, if we are working locally, we will need to restart the Solr service, otherwise you will need to manage the deployment of the schema to your deployment environments.

Populating the Index’ Custom Fields

The commerce indexes are created and updated by both the FullIndexMinion and the IncrementalIndexMinion. It is within these minions that we can inject our custom code to populate the new custom fields with our commerce entity properties.

We’ll create a new pipeline block, called InitializeExtendedSellableItemIndexingViewBlock, which we can see the Run() method in the following snippet. We perform the necessary validation and copy our entity properties to custom index fields.

public override EntityView Run(EntityView arg, CommercePipelineExecutionContext context)
{
    // 1. Validation
    Condition.Requires(arg, nameof(arg)).IsNotNull();
    Condition.Requires(context, nameof(context)).IsNotNull();

    var argument = context.CommerceContext.GetObjects<SearchIndexMinionArgument>().FirstOrDefault();
    if (string.IsNullOrEmpty(argument?.Policy?.Name))
    {
        return arg;
    }

    // 2. Prepare Entities
    var entityItems = argument.Entities?.OfType<SellableItem>().ToList();
    if (entityItems == null || !entityItems.Any())
    {
        return arg;
    }

    // 3. Prepare custom properties
    var scopeIndexablePropertiesPolicy = IndexablePropertiesPolicy.GetPolicyByScope(context.CommerceContext, context.CommerceContext.Environment, argument.Policy.Name);
    if (scopeIndexablePropertiesPolicy?.ComposerProperties == null || !scopeIndexablePropertiesPolicy.ComposerProperties.Any())
    {
        return arg;
    }

    var searchViewNames = context.GetPolicy<KnownSearchViewsPolicy>();
    var childViews = arg.ChildViews.OfType<EntityView>().ToList();

    // 4. Iterate over each entity
    foreach (var si in argument.Entities.OfType<SellableItem>())
    {
        // 5. Get existing document entity view
        var documentView = childViews.First(v => v.EntityId.EqualsOrdinalIgnoreCase(si.Id)
            && v.Name.EqualsOrdinalIgnoreCase(searchViewNames.Document));

        // 6. Add custom fields
        AddComposerFields(si, documentView, scopeIndexablePropertiesPolicy);
    }

    return arg;
}

The highlighted section, 3. Prepare custom properties, is used for our AddComposerFields() method.

To add properties to the Solr document, we only need to register the field name and value to a new ViewProperty of the document EntityView and the platform will later translate and push this data into Solr. This would typically look like the following.

documentView.Properties.Add(new ViewProperty
{
    Name = "<Insert Field Name Here>",
    RawValue = "<Insert Field Value Here>"
});

In our AddComposerFields() implementation, we leverage our custom policy to identify and process fields from the composer templates.

protected void AddComposerFields(
    SellableItem si,
    EntityView documentView,
    IndexablePropertiesPolicy scopeIndexablePropertiesPolicy)
{
    // 1. Iterate over each composer template configuration
    foreach (var composerView in scopeIndexablePropertiesPolicy.ComposerProperties)
    {
        // 2. Get property value
        var composerEntityView = si.GetComposerViewFromName(composerView.Key);

        if (composerEntityView == null)
        {
            continue;
        }

        // 3. Iterate over each custom property to index
        foreach (var property in composerView.Value)
        {
            var value = composerEntityView.GetPropertyValue(property.PropertyName);

            if (value == null)
            {
                continue;
            }

            // 4. Add property to index document
            documentView.Properties.Add(new ViewProperty
            {
                Name = property.IndexFieldName,
                RawValue = value
            });
        }
    }
}

The implementation is made flexible by avoiding hard-coding any composer template or property names and instead using the custom IndexablePropertiesPolicy, which we append our configured policy to the SolrSearchPolicySet. In ComposerProperties we add our composer template name, e.g. "ManufacturedDetails", and then an array of composer property models, where the PropertyName represents the composer template property name and the IndexFieldName represents the Solr core field name.

{
  "$type": "Ajsuth.Sample.Catalog.Search.Engine.Policies.IndexablePropertiesPolicy, Ajsuth.Sample.Catalog.Search.Engine",
  "SearchScopeName": "CatalogItemsScope",
  "ComposerProperties": {
    "ManufacturedDetails": [
      {
        "PropertyName": "YearManfactured",
        "IndexFieldName": "yearmanufactured"
      }
    ]
  }
}

We register our pipeline block to both the full and incremental index minion pipelines.

.ConfigurePipeline<IIncrementalIndexMinionPipeline>(pipeline => pipeline
    .Add<Pipelines.Blocks.InitializeExtendedSellableItemIndexingViewBlock>()
        .After<InitializeSellableItemIndexingViewBlock>()
)

.ConfigurePipeline<IFullIndexMinionPipeline>(pipeline =>pipeline
    .Add<Pipelines.Blocks.InitializeExtendedSellableItemIndexingViewBlock>()
        .After<InitializeSellableItemIndexingViewBlock>()
)

By adding our InitializeExtendedSellableItemIndexingViewBlock after the InitializeSellableItemIndexingViewBlock rather than directly replacing and overriding the initial platform pipeline block, we may suffer a minor performance impact as we will need to interate over our entities another time, however we won’t have to copy platform code into our pipeline block, which will ultimately keep our code as clean as possible while preventing complexity for during upgrades.

We then follow our deployment process for our changes:

  1. Deploy our Solr managed-schemas and restart the Solr service to ingest the updated schemas.
  2. Deploying our Commerce Engine code to all instances.
  3. Bootstrap the Commerce Engine to ingest our IndexablePropertiesPolicy configuration.
  4. Restart the minions Commerce Engine instance to consume the updated policy set.
  5. Executing the Run FullIndex Minion – Catalog Items request from postman.
  6. Verify our Solr core now how our custom field indexed.
Figure 3: yearmanufactured property added to Solr core.

If you aren’t seeing the custom field for any indexed document check the *-Rebuild core as that may the current active core.

Searching Against Custom Fields in the Search Component

With the custom properties added to the indexes, we finally need to update the search component to include the new property as part of the search query.

Figure 4: Search does not query against the custom yearmanufactured field by default.

Using Solr Query Syntax

With no further customisation or configuration, we could use the Solr query syntax to apply to set the Search Term with the specific field query, e.g. yearmanufactured:2009, as the search component will process this as a raw query.

Figure 5: Search can query against the custom yearmanufactured field using Solr query syntax.

Including the Custom Field for SearchScope Queries

The search component queries against the Search Term properties appended to the _text_ field in Solr. If we wanted to add a custom field to default search query, then we can update the managed-schema of the Solr cores to include it. Below lists all of the fields that are used for the search queries as well as our custom field.

<copyField source="displayname" dest="_text_"/>
<copyField source="variantid" dest="_text_"/>
<copyField source="variantdisplayname" dest="_text_"/>
<copyField source="productid" dest="_text_"/>
<copyField  source="name" dest="_text_"/>
<copyField source="yearmanufactured" dest="_text_"/>

After deploying and restarting the Solr service, search component queries from the Business Tools will now query the custom fields as well in the default search.

Figure 6: Search queries against the custom yearmanufactured field by default with updated managed-schema.

Customising the Search Results

The last piece of the puzzle is to add our custom indexed fields to the Results entity view as seen below.

The Results entity view is populated with indexed commerce entity data and not raw commerce entity data.

First, we want to update the IndexablePolicy in the SearchPolicySet for our given search scope, with our custom field.

{
"$type": "Sitecore.Commerce.Plugin.Search.IndexablePolicy, Sitecore.Commerce.Plugin.Search",
"SearchScopeName": "CatalogItemsScope",
"Properties": {
  ...,
  "YearManufactured": {
    "TypeName": "System.String",
    "IsKey": false,
    "IsSearchable": true,
    "IsFilterable": true,
    "IsSortable": true,
    "IsFacetable": false,
    "IsRetrievable": true
  }
}

For our implementation purposes, we only need to be concerned about the IsRetrievable and IsSortable properties of the IndexableSettings object. The IsRetrievable property will allow us to provide a fallback property with an empty value, while the IsSortable property will configured the column to be sortable. Both follow the platform implementation pattern.

Next, I would like to tell you that we could extend the Results entity view simply by extending pipelines with additional pipeline blocks, but we will need to copy and replace the original pipeline blocks for the most consistent approach across commerce indexes. There is a ProcessDocumentSearchResultsBlock within the commerce plugins that will relate to the search scope, which we would copy into our solution. In this instance, we copy the pipeline block from the Sitecore.Commerce.Plugin.Catalog.dll.

Without re-writing the platform code, we will have to face a bit more rigid manual coding over a more automated, iterative approach to the IndexableSettings of our IndexablePolicy. The key area of the pipeline block we need to focus on is shown on line 5, where the results view properties are cleared. We need to copy our custom field from the properties before hand (1. Copy the field view property) and then repopulate the results view afterwards (3. Add desired indexed fields back into results view), which effectively strips the results view or the other indexed fields that we don’t want to render.

// 1. Copy the field view property 
var yearmanufactured = child.Properties.FirstOrDefault(p => p.Name.EqualsOrdinalIgnoreCase("YearManufactured"));

// 2. Clear the results view properties
child.Properties.Clear();

if (name != null)
{
    name.UiType = ViewsConstants.EntityLinkUiType;
}

// 3. Add desired indexed fields back into results view
child.AddViewPropertyToEntityViewOrDefault(name, retrievableProperties, CoreConstants.Name, typeof(string).FullName);
child.AddViewPropertyToEntityViewOrDefault(displayName, retrievableProperties, CoreConstants.DisplayName, typeof(string).FullName);
child.AddViewPropertyToEntityViewOrDefault(variantId, retrievableProperties, CatalogConstants.VariantId, typeof(string).FullName);
child.AddViewPropertyToEntityViewOrDefault(variantDisplayName, retrievableProperties, CatalogConstants.VariantDisplayName, typeof(string).FullName);
child.AddViewPropertyToEntityViewOrDefault(createdDate, retrievableProperties, CoreConstants.DateCreated, typeof(DateTimeOffset).FullName);
child.AddViewPropertyToEntityViewOrDefault(updatedDate, retrievableProperties, CoreConstants.DateUpdated, typeof(DateTimeOffset).FullName);
child.AddViewPropertyToEntityViewOrDefault(yearmanufactured, retrievableProperties, "YearManufactured", typeof(string).FullName);

We replace the platform pipeline block in SearchPipeline in our pipeline registrations.

.ConfigurePipeline<ISearchPipeline>(pipeline => pipeline
    .Replace<Sitecore.Commerce.Plugin.Catalog.ProcessDocumentSearchResultBlock, Pipelines.Blocks.ProcessDocumentSearchResultBlock>()
)

The platform pipeline block we are overriding is fully qualified here to to explicitly replace the correct pipeline block.

All that is left is to deploy the code, which is short looks requires:

  • Deploying our Commerce Engine code to all instances.
  • Bootstrap the Commerce Engine to ingest our updated IndexablePolicy configuration.
  • Restart the authoring Commerce Engine instance to consume the updated policy set.

Performing our search now, we will see that we have our custom field appended to the end of the Results entity view, which can be used to sort the results.

Figure 7: The Results entity view is customised with the yearmanufactured indexed field.

Summary

While the search component of BizFx was not developed with the intention of being configurable, we have identified a few ways to customise the search component, which will hopefully cover most business user scenarios and saves us from having to implement our own search component from scratch.

References

Sitecore Experience Commerce: Customising Commerce Content in the Sitecore Indexes for Storefront Search Results

Reading Time: 4 minutes

In this article, we will look at how to control the storefront search results by customising the commerce content that is indexed in the Sitecore master and web indexes.

If you are looking for how to add new fields for faceting and other purposes, see Adding a property field to an index.

How Storefront Search Works

The SXA Search component creates a solr query based on a number of criteria, however focusing on how the search term is involved in the query, we see that the sxacontent field is our culprit for this customisation. This can be seen highlighted below in our storefront and in our search log.

From XC 9.3 onwards, the commerce portion of the Sitecore indexes were migrated over to the Commerce Engine for indexing performance improvements. There are a number of handlers utilised for populating the indexes with catalog data, however for managing the content related to the storefront search component we will look at the SxaContentFieldHandler.

See Catalog Search Field Handlers for a full list of handlers provided with XC, which are utilised when registering index fields in Plugin.Search.Solr.PolicySet-1.0.0.json.

The SxaContentFieldHandler Class

By default, the sellable item properties that are added to the sxacontent field consist of the following entries:

  • Tags
  • ProductId
  • Name
  • DisplayName
  • ItemDefinition
  • Brand

As we can see from the sitecore_web_index in Solr, a query for a specific ‘Spectra’ product, productid_t:6042260, shows that these properties are indexed against the sellable item.

{
  "sxacontent_txm":["39inch|4k|uhd|television|spectra",
    "6042260",
    "Habitat Spectra 39” 4K LED Ultra HD Television",
    "Habitat Spectra 39” 4K LED Ultra HD Television",
    "Product",
    "Spectra Televisions"],
  "productid_t":"6042260",
  ...
}

A common property that I have seen often requested to be included for searching is the sellable item’s description, so we will use this property as well as a generic property from a custom component for demonstration purposes.

In the sample code below, we see that we inherit from the default SxaContentFieldHandler and execute the base ComposeValue method, which will populate the default fields listed above. We then simply add the properties to the sxaContent reponse object, which in this case is a list of strings.

You may have noticed that we are only adding the properties to sxacontent if a value is present. This is to minimise the bloat of empty entries and may lead to performance improvement however insignificant it may be. You may prefer to leave empty entries in the index to simplfy troubleshooting in mapping each entry back to its source property, so choose an approach that best suits you.

public class MySxaContentFieldHandler : Sitecore.Commerce.Plugin.Catalog.SxaContentFieldHandler
{
    public override object ComposeValue(object source, ConcurrentDictionary<string, object> context)
    {
        var sxaContent= base.ComposeValue(source, context) as List<string>;
        if (!(source is SellableItem) || sxaContent == null || sxaContent.Count() == 0)
        {
            return sxaContent;
        }

        var sellableItem = source as SellableItem;

        if (!string.IsNullOrWhiteSpace(sellableItem.Description))
        {
            sxaContent.Add(sellableItem.Description);
        }

        var myProperty = sellableItem.GetComponent<MyCustomComponent>().MyCustomProperty;
        if (!string.IsNullOrWhiteSpace(myProperty))
        {
            sxaContent.Add(myProperty);
        }

        return sxaContent;
    }
}

Now that we have created our modified SxaContentFieldHandler class, we register it in Plugin.Search.Solr.PolicySet-1.0.0.json for both the master and web indexes.

{
  "$type": "Sitecore.Commerce.Plugin.Search.ItemIndexablePolicy, Sitecore.Commerce.Plugin.Search",
  "IndexName": "sitecore_web_index",
  "FieldTypeMappers": [ ... ],
  "Fields": [
    {
      "$type": "Sitecore.Commerce.Plugin.Search.Solr.SolrIndexFieldConfiguration, Sitecore.Commerce.Plugin.Search.Solr",
      "Name": "sxacontent",
      "Type": "System.Collections.Generic.List`1[System.String]",
      "TypeHint": "textCollection",
      "Handler": {
        "$type": "Sitecore.Commerce.Engine.Search.MySxaContentFieldHandler, Sitecore.Commerce.Engine"
      }
    },
    ...
  ]
}

After deploying our customisation, boostrapping, and reindexing, we can see our custom entries in the sxacontent field.

{
  "sxacontent_txm":["39inch|4k|uhd|television|spectra",
    "6042260",
    "Habitat Spectra 39” 4K LED Ultra HD Television",
    "Habitat Spectra 39” 4K LED Ultra HD Television",
    "Product",
    "Spectra Televisions",
    "Enjoy incredible picture and dramatic detail with 4X the resolution of full HD. Enjoy incredible hues of color with technology that blends digital color for near-perfect representation. Experience Ultra HD picture quality with enhanced detail in every way.",
    "mycustomproperty"],
  "productid_t":"6042260",
  ...
}

And by performing a search in the storefront our search results now return our product from our custom search entries.

Development Considerations

Field Handlers are Synchronous

At this time, field handlers have been implemented synchronously and expect the information you require to be available on the provided source entity, e.g. sellable item in this case, therefore if you are looking at sourcing data externally it may be worth further consideration.

Following Helix Priniciples

Whether upgrading from a pre XC 9.3 solution or introducing existing component properties into the search index, by implementing the custom SxaContentFieldHandler you may be faced with having to break helix principles, i.e. housing your custom SxaContentFieldHandler in a feature project while referencing another feature project to retrieve the desired properties/content. As feature projects should not reference each other, I reluctantly chose to add MySxaContentFieldHandler to the Commerce Engine project in this instance.

References