Author: Andrew

Posted on

Transitioning from Sitecore Experience Commerce to OrderCloud: Order Workflow and Minions

Reading Time: 8 minutes

In this article, we will review and compare the order workflows between Sitecore Experience Commerce and OrderCloud to facilitate developers looking to transition from XC to OrderCloud as well as identify a path for migration of existing XC solutions to OrderCloud. We will also review minions from XC and how this these processes can be brought over into OrderCloud to facilitate order workflow automation.

We will look at a high-level comparison of architecture, functionality, and data models, where applicable, with greater focus on transitioning from Sitecore Experience Commerce, meaning identifying a close path to parity in OrderCloud, using the Habitat catalog and SXA Storefront.

Order Workflow

In XC, the default order workflow demonstrates the capabilities of post-order placement processing where order status transitions are achieved through manual intervention or automated processes (minions). It is intended to be extended and customised to cater for the unique business requirements of each project.

While figure 1 represents the default order workflow, XC can be customised to create or remove order statuses, and create or remove the triggers that transition an order between statuses, e.g. manual intervention via business users (customer service representatives) or automated processes such as minions.

Figure 1: XC’s order status workflow.

To provide clarity regarding each order status, XC’s definitions are:

  • Pending: Initial order state – order paid, no fulfillment.
  • OnHold: Prevents order from being progressed by minions. Customer service representatives may modify, line items, payments, shipments, etc.
  • WaitingForAvailability: One or more items in the order are not available. This status is expected for items on backorder or preorder.
  • Released: Intended status for indicating the order passed validation and is ready for shipping. It is also typically used in customisations to the workflow to indicate the order has been exported to an external order management system (OMS).
  • Problem: An order that has an unexpected status when processed by minions. Manual intervention by customer service representatives is available to transition the order back into a state to be picked up my the minions again, however this typically indicates a bug within custom implementations, which requires developer intervention.
  • Completed: Order has been shipped, has had entitlements generated, and has nothing left to process. Complete orders are eligible for the return mechandise authorisation (RMA) processes.
  • Cancelled: Order has been cancelled and can no longer be processed by minions or customer service representatives.

In OrderCloud, the order status workflow differs to XC’s in that order submission can include an approval process, which is typically a B2B order scenario, while the post-order placement workflow for order status transitions is minimal, as per figure 2.

The workflow is more rigid in that order statuses cannot be introduced, modified or removed, while the interactions with orders in a given status is also dictated by the platform, however we will address below in Custom Order Status in OrderCloud as to how we can still achieve a customised order workflow on top of the platform.

Figure 2: OrderCloud’s order status workflow.

OrderCloud’s order status definitions are:

  • Unsubmitted: Created but has not yet been submitted.
  • Open: Submitted but not yet fulfilled.
  • AwaitingApproval: On hold awaiting approval.
  • Declined: Submitted but has been declined.
  • Completed: Submitted and all the line items on the order have been shipped.
  • Canceled: Order can no longer be submitted.

Removing the order statuses for approval and the cart equivalent “Unsubmitted” status, we see that OrderCloud simply transitions between Open (XC’s Pending) and Completed or Canceled statuses.

Figure 3: OrderCloud’s order status workflow (XC equivalent).

Custom Order Status in OrderCloud

To achieve an XC equivalent order workflow in OrderCloud, the missing order statuses may be introduced under OrderCloud’s Open status, using xp to represent the status as psuedo-sub-status. Typically, this would be achieved by PATCHing the order’s xp, using either middleware as a proxy to support wrap OrderCloud API endpoints with custom logic, or have middleware perform validation via pre-hook webhooks to prevent invalid status transitions.

There is no restriction in modifying the workflow or introducing new statuses via xp, only considerations for ensuring that the workflow transitions from end-to-end as intended.

Figure 4: The XC workflow equivalent in OrderCloud.

Transaction and Payment Status

We previously have covered the payment concepts between XC and OrderCloud in Tax and Payments, but what we didn’t cover was the statuses that belong to these entities and how they can be utilised in the SXA storefront.

For Sitecore Experience Commerce, the federated payment components have a TransactionStatus property, which is intended on tracking the raw statuses specific to the payment provider. They are typically used only for transparency and reconciliation purposes, rather than driving business logic in XC.

Figure 5: The default workflow for transaction status of federated payments.

Sales activities have a PaymentStatus property, which also is intended on tracking the transaction status in a more agnostic fashion by reducing payment provider statuses to more generic status buckets, which can then be used to influence business logic. In Order Workflow, figure 1 shows that the payment status affecting the order status, namely by placing an order into a problem state where the payment status is set as Problem.

Figure 6 and 7 shows the workflows for payment sales activities in XC, which modifies existing sales activity for subsequent transactions, hence representing as transitions, however this is commonly customised to create a separate sales activity per transition for a complete historical view of events and reconciliation reporting.

Figure: 6: The default workflow for payment status of federated payments.
Figure 7: The default workflow for payment status of gift cards.

Translating XC’s payments over to OrderCloud payments resources, we see that transactions have the ResultCode property, which could be considered the equivalent to XC’s payment status, while XC’s transaction status could be stored on the payment’s xp. Looking at the payments resource in OrderCloud however we see that the intended architecture for transactions is to track each transaction for a payment rather than updating the its status, as the transactions can be created or deleted, but not updated.

Rather than porting XC’s architecture over to OrderCloud directly, there are some improvements we could introduce to reduce the complexity of the overall solution, which includes the business logic used in the order workflow and not just recreating the payment entities like for like. For example, if we treat transactions as a snapshot in time for a given payment, rather than a dynamic object representing only the most recent transaction, the ResultCode would probably be better can be used for the raw transaction status given by the payment provider.

Figure 8: Transaction ResultCode (status) workflow. Each status will be represented by its own transaction.

For the overall payment status, we can use the payment’s xp to maintain the most recent transaction status in a normalised form, agnostic of payment provider, which would simply be translated as part of the payment provider integration.

Figure 9: Payment status workflow example in OrderCloud.

Order Minions

Order minions are background jobs that process all orders in a given status to either perform some function on and/with the order, before transitioning it to its next status.

Pending Orders Minion

The pending orders minion is a shell for processing orders fresh from being placed. It is expected to be customised to fit each project’s requirements, and can be thought of as an extension of the create order process, delegated with performing asynchronous activities in a separate Commerce Engine runnning the minions role, which typically would run on dedicated resources. This is to mitigate any performance impacts on the Commerce Engine instance(s) running the shops role, which serves the storefront’s live traffic.

The pending order minion will usually be extended with processes such as exporting the order to an order management system (OMS), sending an order invoices via an email service provider, generating gift cards, activating subscriptions, and more.

In OrderCloud, we will review the order submit integration event and batch order processing approaches to process pending orders.

Order Submit Integration Event

The order submit integration event is a non-blocking asynchronous call to the middleware or dedicated jobs application, allowing you to process the order immediately. The downside to this approach is that unhandled exceptions and timeout issues may leave the order in a corrupt state, which may be difficult to diagnose.

Figure 10: Order submit integration event.

Batch Order Processing

An alternate approach to integration events is to process orders in batch jobs, which is more closely aligned to pending orders minion behaviour, by leveraging the custom order status in the order xp and the ElasticSearch backed premium search for orders. The benefit over using the batch approach is that, dependent on your system architecture, you may process orders in a separate custom application that is intended to process background jobs utilising separate resources from the middleware that services user traffic.

Figure 11: Batch processing pending orders.

Released Orders Minion

The released orders minion processes orders in the released state by creating shipments against the order, generating entitlements, capture payments, then transition the order into a completed state as depicted in figure 12. We don’t need to bring over fault pipeline blocks, and as discussed in Orders – Shipments, we will be able to fold the shipment generation together.

Figure 12: A comparison of XC’s released order process and an equivalent approach in OrderCloud.

To replicate the released orders minion in OrderCloud, we would need to follow the batch processing orders approach as described for the pending order minion equivalent behaviour.

While it might seem that the pending orders minion and released orders minion should be folded into a single process, the intention behind segregating these order states into separate processing jobs is due to the expected customisations, which may see manual triggering/transitioning or delayed processing between order states that will be considered against the project’s business requirements.

Figure 13: Batch processing released orders.

Waiting For Availability Orders Minion

The waiting for availability order minion simply iterates over orders in registered to the WaitingForOrders list and determines if all products on the order have availability, in which they are returned to the Pending status/list. It is intended to support back-order and preorder scenarios, however as these features aren’t completely functional, customisation is required to complete the implementation and cater for any unique business requirements.

Pre-order and backorder inventory features are not natively supported in OrderCloud and as XC’s implementation is also incomplete, we won’t cover how to port this across in detail, only to highlight that we can leverage the same batch processing approach we have already outlined as a framework for implementing this as an automated job.

References

Posted on

Transitioning from Sitecore Experience Commerce to OrderCloud: Orders

Reading Time: 9 minutes

In this article, we will review and compare orders and related features between Sitecore Experience Commerce and OrderCloud to facilitate developers looking to transition from XC to OrderCloud as well as identify a path for migration of existing XC solutions to OrderCloud.

We will look at a high-level comparison of architecture, functionality, and data models, with greater focus on transitioning from Sitecore Experience Commerce, meaning identifying a close path to parity in OrderCloud, using the Habitat catalog and SXA Storefront.

Conceptual Architecture and Features

Journalling (Auditing)

The journalling functionality in XC takes a snapshot of an entity when it is persisted as a means of effectively producing a change log against the entity for auditing and historical purposes. The entities are registered to the commerce role configuration under the EntityJournalingPolicy, and only the order entity is registered to the policy as part of the habitat role configurations by default.

While OrderCloud doesn’t keep any historical records of its entities to track previous changes or who made them, two options available to us for a custom implementation are webhooks or middleware proxy requests, however both require a custom database to store the audit snapshots.

The Webhook Approach

While webhooks may turn out to be sufficient for your auditing requirements, there are considerations to be aware of to ensure that you don’t get caught out with these gotchas.

Regardless if you choose to use pre-hooks or post-hooks, OrderCloud may touch multiple entities within a single request that could impact the completeness of the implementation, leading to the inability for complete reconciliation and may be mistaken as data corruption.

For POST and PUT requests, the post-hook webhooks will typically give you the entity in the API response, however this does not apply to all requests, especially PATCHes, which return a response code of 204 and an empty body. In any case, the API response alone is not sufficient for identifying only the properties that were changed or who changed them.

The pre-hook webhooks on the other hand, capture the request body, however creating a log at this point would be premature as the OrderCloud API will typically rollback or not commit changes until the request has been fully validated and processed. Pre-hooks are also a blocking process, which will lead to performance impacts on your solution.

The Middleware Proxy Approach

Using middleware as a proxy for all OrderCloud requests that will have auditing attached may provide the most complete approach but may also come with the most complexity and performance implications.

The proxy request in middleware doesn’t mean you are limited to only calling the OrderCloud API in question, but can provide the opportunity to retrieve the entity prior to making changes, and then retreiving the entity after changes have been made for endpoints such as any PATCH endpoint, which doesn’t return the entity in the response. While this may seem great, you will be facing performance implications, which can only be managed through asynchronous calls to an extent. It is also possible to cater for multiple entities being touched in OrderCloud with this approach, but again will have performance implications.

Entitlements

In XC, entitlements can be considered as a product classification that typically represents a sellable item as a trackable unit of ownership or license, which may change from customer or ambient activity, such as redemptions or expiry. Upon purchase of a sellable item with an entitlement classification, the Commerce Engine will create an specialised entitlement commerce entity, which can then be further customised for extended functionality. for the spei include gift cards, installations, warrranties, services and subscriptions.

Figure 1: Entitlement creation from order.

The entitlements are created with entity references to the order and the customer entities, while the order and customer entities are also updated with entity references back to the entitlements to form two-way relationships.

Figure 2: Entitlement architecture in XC.

OrderCloud does not have any native support for entitlements and leveraging the platform to port the entitlement implementation is limited due to a lack of support for custom entities. Instead, it is recommended that entitlements be treated as an external integration, which could just be the middleware working with a custom database.

Figure 3: Entitlement architecture in an OrderCloud solution.

While using the buyer user’s xp to handle entitlements may seem tempting, xp is limited to store 8000 bytes, so with repeat customers this approach will eventually see API errors when the xp has reached its limit.

Shipments

In Fulfillment to Shipping, we covered the fulfillment components and shipping estimates that are applied to orders to gather fulfillment information during the storefront checkout delivery step. Post order-placement, the actual fulfillment of the order may or may not be in alignment with representation of fulfillment in the storefront and can be controlled via customisation. For example, the shipping details may have been collected for the entire order and a flat fee charged, however business processes may see the order be fulfilled across multiple shipments. Any variance to actual shipping costs from the estimates would typically not be reconciled with the customer.

XC provides the shipment commerce entity and, as part of the Released Orders Minion, will generate a shipment for the order or for each line item, respective to whether cart-level or cart line-level fulfillments were applied to the cart during the checkout steps.

Figure 4: XC shipments for cart-level and cart line-level fulfillments.

In OrderCloud, shipment flexibility is increased as shipments are not bound to the ship estimates, like XC’s shipments to physical fulfillments. Shipments can be created against the entire order or a subset of line items with support for partial quantity fulfillment.

Figure 5: OrderCloud shipments for order-level and line-level fulfillments do not have to align to ship estimates.

Return Merchandise Authorisation (RMA)

RMAs in the Commerce Engine are another partial implementation to get faciliate development by providing the foundation. This is commonly an area of customisation to allow RMAs to fit with the business processes, e.g. restricting returns on certain items, calculating refunds, determining whether the inventory needs to be incremented on a return, etc. The RMA functionality allows customer service representatives to raise a return request, with a line item, quantity, and reason for return, and the RMA will be created with an approval process. The remaining functionality is introduced via customisation.

In OrderCloud, order returns are a recently released feature that has more complete functionality than XC’s RMA offering. A return can be created against multiple line items, which also consists of an approval workflow and custom refund calculations can be achieved via the OrderReturn integration event.

Functionality

The Storefront Checkout Order Submission Step

In XC, submitting an order is not just about converting a cart to an order, but performing necessary processes such as payment authorisation and validating the order is in a valid state prior to finalising the order. The following sections cover these processes that we would look to transfer into an OrderCloud implementation.

Turning our attention to the storefront functionality of the order confirmation checkout step from Carts to Unsubmitted Orders: Storefront User Journey, we will look at adding the email address and payment details to the cart/unsubmitted order, focusing on varying APIs of the two platforms.

Figure 6: The order submission aspect of the storefront user journey.

When submitting an order in XC, the Commerce Engine will perform validations and processes that accompany order creation, which is depicted at a high level in figure 7. The darker nodes represent functionality that is built into OrderCloud’s order submit endpoint, while the nodes in italics represent functionality we wouldn’t need to bring over to OrderCloud. This leaves the Authorise Payment and Assign Confirmation ID as the custom functionality we would need to incorporate into an OrderCloud solution.

Figure 7: A comparison of XC’s create order process and an equivalent approach in OrderCloud.
Order Validation

When submitting orders in XC, the Commerce Engine ensures the cart is in a valid state prior to converting it to an order. The validation can be tightened through customisation for business specific validation requirements, but for the default validation rules, the Commerce Engine will evaluate the following:

  • The payments total matches the cart’s grand total.
  • An email is assigned to the cart.
  • The cart has line items.
  • Applied coupons for promotions are eligible.
  • Applied promotions are eligible.
  • Inventory availability is not exceeded.

Similarly, OrderCloud’s submit order endpoint validates an order with the following:

  • The order has line items.
  • The order wasn’t already submitted.
  • The order has been calculated via integration event, if applicable.
  • All payments have been accepted.
  • Applied promotions are eligible.
  • Inventory availability is not exceeded.

The difference validate gaps between the two platforms that we will need to implement for parity:

  • The payments total matches the order’s grand total.
  • An email is assigned to the order.

For more details on the internal logic of OrderCloud’s order submit process, including validation, see Order Submit Logic.

Payment Capture
Credit Cards

The SXA storefront provides an auth-capture implementation for braintree payments where upon order submission the payment is authorised and during the released order minion the payment is captured.

Figure 8: High level credit card payment flow in XC.

In OrderCloud, the payments resource represents credit card payments and validation is independent of the order creation and validation processes. In figure 10, we see that the authorise payment is incorporated as part of the middleware’s order creation process.

Gift Cards

For the gift card implementation, the gift card amount is captured post order placement, during the pending orders minion.

XC’s gift card implementation is only intended as a sample, which would typically be replaced with an integration with a gift card provider, so we typically wouldn’t need to focus so much on the default XC implementation and instead focus on porting the custom integration.

Figure 9: High level gift card payment flow in XC.

For gift cards, in the previous article, Taxes and Payments, we identified that the gift cards were represented as spending accounts in OrderCloud.

Order Confirmation Id

The order’s OrderConfirmationId is generated as part of order creation, however this is commonly overwritten with an order number incrementor implementation for a user-friendly confirmation id or an external system (e.g. OMS) identifier for reconciliation purposes.

In OrderCloud, generating unique order IDs is a native feature. While entity IDs are generated with a unique alphanumeric characters, OrderCloud’s incrementors resource can be configured and referenced when explicitly assigning an entity ID. This is achieved by wrapping the incrementor ID in curly braces, which also supports prefixed and suffixed values for even further versatility.

// Request Body
{
  "ID": "MyPrefix-{MyOrderIncrementorID}"
}

// Response Body
{
  "ID": "MyPrefix-010010",
  ...
  "IsSubmitted": false,
  "xp": null
}

A few consideration when it comes to the order confirmation IDs:

  • In a multi-tenant solution, you can create a unique incrementor, which may create the same incrementor value, e.g. 010010, however when assigning an order ID, a unique storefront prefix can prevent duplicate IDs from being created, e.g. <Storefront 1>-{Storefront1OrderIncrementorID}.
  • If your solution cannot guarantee unique IDs for orders across the Marketplace, the ID can be stored on the order’s xp instead.
Order Submission by Middleware Proxy

We will find a significant difference from an implementation perspective when planning to achieve close parity over in OrderCloud, namely the shift in development methodology between developing within the XC platform and developing on top of the OrderCloud platform. We will generally be able to achieve close parity when it comes to the happy paths, however when it comes to the unhappy paths, this will be where you would likely utilise your middleware to perform the order submit request along with surrounding custom logic.

In OrderCloud, we have the ability to extend platform functionality prior to execution by using middleware as a proxy, wrapping the OrderCloud API endpoints, or creating pre-hook webhooks to intercept an API request. Both approaches have merit and will typically be a design and implementation decision.

For more information on extending OrderCloud, see Flexible Fullfilment – Extending OrderCloud.

In our happy path only example, figure 10 provides an OrderCloud solution equivalent of XC’s create order process, by wrapping OrderCloud’s order submit and several other related endpoints, encompassing all aspects of the order submission process.

  1. By calling the order validate endpoint, we ensure that no stale data, in particular order calculations, are being retrieved from the cache of other API calls we will make, such as getting the order worksheet.
  2. The order worksheet is retrieved to perform custom validation.
  3. Payments are not stored on the order worksheet at this time, so it is necessary to retrieve payments separate to the order.
  4. For our custom validation:
    1. We identified in Tax and Payments that XC has an order-specific email, which may not be the user’s address, so we would need to confirm that the order-specific email has been provided on the order.
    2. We also acknowledged earlier that XC validates the cart’s payments total against the grand total, so this is another custom validation we can implement in our OrderCloud solution.
  5. While XC doesn’t create a sales activity until the post-order processing stage, OrderCloud transactions are required to keep track of payment activity.
  6. Patching the order request covers:
    1. Updating the order confirmation id with our incrementor id. It is possible that the order’s ID is generated upon initial creation instead of during the order creation process, however this will see submitted orders not necessarily coming in in a linear fashion and potentially many generated IDs being discarded due to order abandonment.
    2. Applying the initial custom order status, which we will visit in a later article.
Figure 10: An example approach for XC create order equivalent in an OrderCloud solution.

References

Posted on

OrderCloud: Order Submit Logic

Reading Time: 4 minutes

In this article, we will review the business logic that Sitecore OrderCloud utilises when the order submit endpoint (POST /orders/{direction}/{orderID}/submit) is executed for a better checkout design, implementation and error handling.

Introduction

When it comes to submitting an order that is either incomplete or has become invalid, the OrderCloud API provides fairly easy to understand error codes to facilitate troubleshooting. What is not provided however, is a complete list of error codes that one may expect from an endpoint in order to be proactive in catering for such situations.

While a global catch all may be the quickest and easiest way to approach order submission error handling, the user experience of being informed that “A technical difficulty has occurred. Please contact support” in your storefront’s checkout will likely have a negative impact on the conversion rate. In addition, the admin application tends to be neglected with the love required to support business users in identifying and rectifying the issues themselves, so more often than not it is the developer to the rescue, but there are ways we can prevent these scenarios from occurring.

As OrderCloud is always utilising the latest code base, it is possible that the documented processes can change over time without warning.

Order Submission Logic

The following sections covering order submission logic, will highlight most of the platform logic, but will omit more trivial details.

When an exception is thrown by the API, denoted by the red End terminators with given error code, any pending changes to the underlying database entities are rolled back.

Order Submit

The order submit logic is essentially the entry point for validating and processing a submitted order.

Figure 1: Order submit logic.

Validate Order

The validate order process not only used in the order submit process, but is the entire validation taking place for the order validate endpoints.

In figure 2, soft errors, represented by the red boxes, allows the API to validate other areas of the order and return errors in list form, which can see your overall solution being more performant with less back and forth to resolve all order validation errors.

The Deduct inventory sub-process will validate inventory is available for the order, but does not persist inventory changes as this can be found in figure 1, under Commit inventory.

Figure 2: Validate order logic.

Validate Buyer-Seller Relationship

The buyer-seller relationship effectively validates that the user is authorised to purchase an order directly from a supplier. The relationship can be implicit, i.e. the supplier can be configured with AllBuyerCanOrder set as true or the user is an admin user, or an explicit relationship between the buyer and supplier has been defined.

Figure 3: Validate buyer-seller relationship logic.

Validate Promotions

The validate promotions logic in figure 4 shows the relevant flow for the order submit process.

Figure 4: Validate promotions logic.

Validate Promotion

For each promotion applied to the order, it is validated during the order submit to ensure it is still eligible. Any and all errors found are added to the error list, which will be returned by the API as part of the validate order logic in figure 2.

Figure 5: Validate promotion logic.

Deduct Line Item Inventory

Order inventory will be validated to ensure that there is sufficient inventory for all order lines. Any and all errors found are added to the error list, which will be returned by the API as part of the validate order logic in figure 2.

Figure 6: Deduct line item inventory logic.

Validate and Adjust Spending Account

For the sub-process Validate and adjust spending accounts in figure 1, the following flowchart represents how each spending account is evaluated as they are iterated over for all applied payments for the order.

Figure 7: Validate and adjust spending account logic.

Summary

With this insight into the order submit processes, you should now be well equipped to identify how extending OrderCloud order submit functionality through webhooks, wrapped API calls, and integration events fit in with your solutions, as well as anticipate the many errors that can occur and be more proactive with more specific error handling over a global catch all implementation.

References

Posted on

Sitecore Experience Commerce: Adding Custom Properties to Search in BizFx

Reading Time: 8 minutes

In this article, we will review how we can extend the search components in Business Tools (BizFx) to support search functionality using custom properties for its results.

This article will not consider locale-support.

Introduction

As the search functionality in BizFx is fairly primitive, from a business user perspective, and a common request is to provide customisation guidance for the search functionality in the Business Tools, we will use the Search component from the Merchandising dashboard for our example in extending the search functionality.

Figure 1: Search component in the Merchandising Dashboard.

This implementation will assume the sellable items that have custom properties added via composer templates, however you will be able to adapt the solution where properties have been added programatically with components or policies.

Figure 2: A sellable item extended with a composer template.

Implementation and Configuration

Updating the Solr Schema

We first need to add our custom property to the managed schema of our Solr cores related to our search scopes. These schemas are found in the file system where Solr is deployed, under server/solr/<search scope>/conf/managed-schema.

The commerce engine uses switch on rebuild so we need to update the managed schema for both CatalogItemsScope and CatalogItemsScope-Rebuild for this instance.

    <!-- CommerceEngine Catalog -->
    <field name="displayname" type="string" indexed="true" stored="true"/>
    <field name="datecreated" type="pdate" indexed="true" stored="true"/>
    <field name="dateupdated" type="pdate" indexed="true" stored="true"/>
    <field name="artifactstoreid" type="string" indexed="true" stored="true"/>
    <field name="parentcataloglist" type="string" indexed="true" stored="true" multivalued="true"/>
    <field name="variantid" type="string" indexed="true" stored="true"/>
    <field name="variantdisplayname" type="string" indexed="true" stored="true"/>
    <field name="inventorysetids" type="string" indexed="true" stored="true"/>
    <field name="productid" type="string" indexed="true" stored="true"/>
    <field name="name" type="string" indexed="true" stored="true"/>
    <field name="yearmanufactured" type="string" indexed="true" stored="true"/>

The managed-schema that would be created with the XC instance will be organised to easily identify the commerce entity fields. While fields can be added via Solr, the managed-schema will be regenerated in a less XC friendly manner and make long-term mainentance more difficult.

With our Solr schema updated, if we are working locally, we will need to restart the Solr service, otherwise you will need to manage the deployment of the schema to your deployment environments.

Populating the Index’ Custom Fields

The commerce indexes are created and updated by both the FullIndexMinion and the IncrementalIndexMinion. It is within these minions that we can inject our custom code to populate the new custom fields with our commerce entity properties.

We’ll create a new pipeline block, called InitializeExtendedSellableItemIndexingViewBlock, which we can see the Run() method in the following snippet. We perform the necessary validation and copy our entity properties to custom index fields.

public override EntityView Run(EntityView arg, CommercePipelineExecutionContext context)
{
    // 1. Validation
    Condition.Requires(arg, nameof(arg)).IsNotNull();
    Condition.Requires(context, nameof(context)).IsNotNull();

    var argument = context.CommerceContext.GetObjects<SearchIndexMinionArgument>().FirstOrDefault();
    if (string.IsNullOrEmpty(argument?.Policy?.Name))
    {
        return arg;
    }

    // 2. Prepare Entities
    var entityItems = argument.Entities?.OfType<SellableItem>().ToList();
    if (entityItems == null || !entityItems.Any())
    {
        return arg;
    }

    // 3. Prepare custom properties
    var scopeIndexablePropertiesPolicy = IndexablePropertiesPolicy.GetPolicyByScope(context.CommerceContext, context.CommerceContext.Environment, argument.Policy.Name);
    if (scopeIndexablePropertiesPolicy?.ComposerProperties == null || !scopeIndexablePropertiesPolicy.ComposerProperties.Any())
    {
        return arg;
    }

    var searchViewNames = context.GetPolicy<KnownSearchViewsPolicy>();
    var childViews = arg.ChildViews.OfType<EntityView>().ToList();

    // 4. Iterate over each entity
    foreach (var si in argument.Entities.OfType<SellableItem>())
    {
        // 5. Get existing document entity view
        var documentView = childViews.First(v => v.EntityId.EqualsOrdinalIgnoreCase(si.Id)
            && v.Name.EqualsOrdinalIgnoreCase(searchViewNames.Document));

        // 6. Add custom fields
        AddComposerFields(si, documentView, scopeIndexablePropertiesPolicy);
    }

    return arg;
}

The highlighted section, 3. Prepare custom properties, is used for our AddComposerFields() method.

To add properties to the Solr document, we only need to register the field name and value to a new ViewProperty of the document EntityView and the platform will later translate and push this data into Solr. This would typically look like the following.

documentView.Properties.Add(new ViewProperty
{
    Name = "<Insert Field Name Here>",
    RawValue = "<Insert Field Value Here>"
});

In our AddComposerFields() implementation, we leverage our custom policy to identify and process fields from the composer templates.

protected void AddComposerFields(
    SellableItem si,
    EntityView documentView,
    IndexablePropertiesPolicy scopeIndexablePropertiesPolicy)
{
    // 1. Iterate over each composer template configuration
    foreach (var composerView in scopeIndexablePropertiesPolicy.ComposerProperties)
    {
        // 2. Get property value
        var composerEntityView = si.GetComposerViewFromName(composerView.Key);

        if (composerEntityView == null)
        {
            continue;
        }

        // 3. Iterate over each custom property to index
        foreach (var property in composerView.Value)
        {
            var value = composerEntityView.GetPropertyValue(property.PropertyName);

            if (value == null)
            {
                continue;
            }

            // 4. Add property to index document
            documentView.Properties.Add(new ViewProperty
            {
                Name = property.IndexFieldName,
                RawValue = value
            });
        }
    }
}

The implementation is made flexible by avoiding hard-coding any composer template or property names and instead using the custom IndexablePropertiesPolicy, which we append our configured policy to the SolrSearchPolicySet. In ComposerProperties we add our composer template name, e.g. "ManufacturedDetails", and then an array of composer property models, where the PropertyName represents the composer template property name and the IndexFieldName represents the Solr core field name.

{
  "$type": "Ajsuth.Sample.Catalog.Search.Engine.Policies.IndexablePropertiesPolicy, Ajsuth.Sample.Catalog.Search.Engine",
  "SearchScopeName": "CatalogItemsScope",
  "ComposerProperties": {
    "ManufacturedDetails": [
      {
        "PropertyName": "YearManfactured",
        "IndexFieldName": "yearmanufactured"
      }
    ]
  }
}

We register our pipeline block to both the full and incremental index minion pipelines.

.ConfigurePipeline<IIncrementalIndexMinionPipeline>(pipeline => pipeline
    .Add<Pipelines.Blocks.InitializeExtendedSellableItemIndexingViewBlock>()
        .After<InitializeSellableItemIndexingViewBlock>()
)

.ConfigurePipeline<IFullIndexMinionPipeline>(pipeline =>pipeline
    .Add<Pipelines.Blocks.InitializeExtendedSellableItemIndexingViewBlock>()
        .After<InitializeSellableItemIndexingViewBlock>()
)

By adding our InitializeExtendedSellableItemIndexingViewBlock after the InitializeSellableItemIndexingViewBlock rather than directly replacing and overriding the initial platform pipeline block, we may suffer a minor performance impact as we will need to interate over our entities another time, however we won’t have to copy platform code into our pipeline block, which will ultimately keep our code as clean as possible while preventing complexity for during upgrades.

We then follow our deployment process for our changes:

  1. Deploy our Solr managed-schemas and restart the Solr service to ingest the updated schemas.
  2. Deploying our Commerce Engine code to all instances.
  3. Bootstrap the Commerce Engine to ingest our IndexablePropertiesPolicy configuration.
  4. Restart the minions Commerce Engine instance to consume the updated policy set.
  5. Executing the Run FullIndex Minion – Catalog Items request from postman.
  6. Verify our Solr core now how our custom field indexed.
Figure 3: yearmanufactured property added to Solr core.

If you aren’t seeing the custom field for any indexed document check the *-Rebuild core as that may the current active core.

Searching Against Custom Fields in the Search Component

With the custom properties added to the indexes, we finally need to update the search component to include the new property as part of the search query.

Figure 4: Search does not query against the custom yearmanufactured field by default.

Using Solr Query Syntax

With no further customisation or configuration, we could use the Solr query syntax to apply to set the Search Term with the specific field query, e.g. yearmanufactured:2009, as the search component will process this as a raw query.

Figure 5: Search can query against the custom yearmanufactured field using Solr query syntax.

Including the Custom Field for SearchScope Queries

The search component queries against the Search Term properties appended to the _text_ field in Solr. If we wanted to add a custom field to default search query, then we can update the managed-schema of the Solr cores to include it. Below lists all of the fields that are used for the search queries as well as our custom field.

<copyField source="displayname" dest="_text_"/>
<copyField source="variantid" dest="_text_"/>
<copyField source="variantdisplayname" dest="_text_"/>
<copyField source="productid" dest="_text_"/>
<copyField  source="name" dest="_text_"/>
<copyField source="yearmanufactured" dest="_text_"/>

After deploying and restarting the Solr service, search component queries from the Business Tools will now query the custom fields as well in the default search.

Figure 6: Search queries against the custom yearmanufactured field by default with updated managed-schema.

Customising the Search Results

The last piece of the puzzle is to add our custom indexed fields to the Results entity view as seen below.

The Results entity view is populated with indexed commerce entity data and not raw commerce entity data.

First, we want to update the IndexablePolicy in the SearchPolicySet for our given search scope, with our custom field.

{
"$type": "Sitecore.Commerce.Plugin.Search.IndexablePolicy, Sitecore.Commerce.Plugin.Search",
"SearchScopeName": "CatalogItemsScope",
"Properties": {
  ...,
  "YearManufactured": {
    "TypeName": "System.String",
    "IsKey": false,
    "IsSearchable": true,
    "IsFilterable": true,
    "IsSortable": true,
    "IsFacetable": false,
    "IsRetrievable": true
  }
}

For our implementation purposes, we only need to be concerned about the IsRetrievable and IsSortable properties of the IndexableSettings object. The IsRetrievable property will allow us to provide a fallback property with an empty value, while the IsSortable property will configured the column to be sortable. Both follow the platform implementation pattern.

Next, I would like to tell you that we could extend the Results entity view simply by extending pipelines with additional pipeline blocks, but we will need to copy and replace the original pipeline blocks for the most consistent approach across commerce indexes. There is a ProcessDocumentSearchResultsBlock within the commerce plugins that will relate to the search scope, which we would copy into our solution. In this instance, we copy the pipeline block from the Sitecore.Commerce.Plugin.Catalog.dll.

Without re-writing the platform code, we will have to face a bit more rigid manual coding over a more automated, iterative approach to the IndexableSettings of our IndexablePolicy. The key area of the pipeline block we need to focus on is shown on line 5, where the results view properties are cleared. We need to copy our custom field from the properties before hand (1. Copy the field view property) and then repopulate the results view afterwards (3. Add desired indexed fields back into results view), which effectively strips the results view or the other indexed fields that we don’t want to render.

// 1. Copy the field view property 
var yearmanufactured = child.Properties.FirstOrDefault(p => p.Name.EqualsOrdinalIgnoreCase("YearManufactured"));

// 2. Clear the results view properties
child.Properties.Clear();

if (name != null)
{
    name.UiType = ViewsConstants.EntityLinkUiType;
}

// 3. Add desired indexed fields back into results view
child.AddViewPropertyToEntityViewOrDefault(name, retrievableProperties, CoreConstants.Name, typeof(string).FullName);
child.AddViewPropertyToEntityViewOrDefault(displayName, retrievableProperties, CoreConstants.DisplayName, typeof(string).FullName);
child.AddViewPropertyToEntityViewOrDefault(variantId, retrievableProperties, CatalogConstants.VariantId, typeof(string).FullName);
child.AddViewPropertyToEntityViewOrDefault(variantDisplayName, retrievableProperties, CatalogConstants.VariantDisplayName, typeof(string).FullName);
child.AddViewPropertyToEntityViewOrDefault(createdDate, retrievableProperties, CoreConstants.DateCreated, typeof(DateTimeOffset).FullName);
child.AddViewPropertyToEntityViewOrDefault(updatedDate, retrievableProperties, CoreConstants.DateUpdated, typeof(DateTimeOffset).FullName);
child.AddViewPropertyToEntityViewOrDefault(yearmanufactured, retrievableProperties, "YearManufactured", typeof(string).FullName);

We replace the platform pipeline block in SearchPipeline in our pipeline registrations.

.ConfigurePipeline<ISearchPipeline>(pipeline => pipeline
    .Replace<Sitecore.Commerce.Plugin.Catalog.ProcessDocumentSearchResultBlock, Pipelines.Blocks.ProcessDocumentSearchResultBlock>()
)

The platform pipeline block we are overriding is fully qualified here to to explicitly replace the correct pipeline block.

All that is left is to deploy the code, which is short looks requires:

  • Deploying our Commerce Engine code to all instances.
  • Bootstrap the Commerce Engine to ingest our updated IndexablePolicy configuration.
  • Restart the authoring Commerce Engine instance to consume the updated policy set.

Performing our search now, we will see that we have our custom field appended to the end of the Results entity view, which can be used to sort the results.

Figure 7: The Results entity view is customised with the yearmanufactured indexed field.

Summary

While the search component of BizFx was not developed with the intention of being configurable, we have identified a few ways to customise the search component, which will hopefully cover most business user scenarios and saves us from having to implement our own search component from scratch.

References

Posted on

OrderCloud: Resource Dependency Cheat Sheet

Reading Time: 5 minutes

In this article, we will review the OrderCloud resources that we interact with directly or indirectly within the OrderCloud API to identify the dependencies that will guide the order of operations for implementations such as a data import. We will also identify a few platform nuances, or gotchas, to be aware of.

Introduction

We will start at the end, for quick reference, with the following dependency overview diagram shows OrderCloud resources, excluding the complete set of order resources, organised into priority groups. Each group number, represents the order in which data should typically be processed in an implementation to avoid any dependency conflicts in the OrderCloud API responses. For example, resources in group 1 would typically be executed before resources in group 2, group 2 before group 3, etc.

The overview diagram does not show the actual dependencies between resources, however in Resource Dependencies we break down each section (or logically grouped resources), aligned to the OrderCloud portal, to document the specific dependencies and note any nuances that should be considered when working with resources.

Figure 1: Resource dependency groups overview.

Resource Dependencies

In the following resource dependency graphs, the varying line colours are simply to differentiate the dependency references for each resource. The following legend also applies to the all graphs.

A resource reference is a reference in the resource’s url as opposed to the request body, e.g. /catalogs/{catalogID}/categories.

In order to understand the dependency graphs, we will look at the following example, where we want to import/sync supplier data via the OrderCloud API. If we were to attempt to create a supplier user where the supplier does not exist, we would encounter an EntityNotFound error in the API’s response.

Figure 2: Resource dependency example.

Using the dependency graph we can see that the supplier is a dependency and therefore if we aren’t planning on creating or updating the supplier then we should at least validate its existence before proceeding and with additional defensive programming we can proactively consider an alternate path of code logic instead.

private async Task&lt;User&gt; CreateSupplierUser(User supplierUser, string supplierID, string accessToken)
{
    var supplier = await oc.Suppliers.GetAsync(supplierID, accessToken);
    if (supplier == null)
    {
        // We implement our alternate code path here, for example:
        Logger.LogError($"Supplier '{supplierID}' should already exist. Supplier User '{supplierUser.ID}' will not be created.");
        return null;
    }

    return await oc.SupplierUsers.CreateAsync(supplierID, supplierUser, accessToken);
}

If we were importing all supplier information, using the dependency graph as a reference, we can identify the order of operations for our implementation to avoid dependency conflicts and optimise performance. For example, we know that we need to import suppliers before importing supplier users, while supplier addresses, suppliers users and supplier user groups can be safely imported asynchronously because they belong to the same dependency group.

// Group 1 dependencies
await ImportSuppliers();

// Group 2 dependencies
var supplierAddresses = ImportSupplierAddresses();
var supplierUsers = ImportSupplierUsers();
var supplierUserGroups = ImportSupplierUserGroups();
await Task.WhenAll(supplierAddresses, supplierUsers, supplierUserGroups);

// Group 3 dependencies
await ImportSupplierUserGroupAssignments();

Identifying the order of operations and employing defensive programming techniques will be critical in your solution design.

Authentication and Authorization Resources

Figure 3: Authentication and Authorization resource dependencies.

Seller

The seller resource dependencies have been broken out across two diagrams for clarity.

Figure 4: Seller resource dependencies.
Figure 5: Seller resource dependencies (cont.).

Buyers

The buyer resource has an implicit reference to the catalog resource as there is a nuance in the platform relating to buyer creation. When creating a buyer, the OrderCloud platform will create a catalog with the same ID. If a catalog with the same ID already exists, an IdExists error will be returned. However, if the catalog‘s ID is passed in as the DefaultCatalogID then the catalog will be associated to it instead, whereas if there is no catalog in existence an NotFound error will be returned.

Conditions1234
DefaultCatalogID providedNNYY
Catalog existsNYNY
Actions1234
Catalog created (201 response code)X
Catalog assigned to Buyer (201 response code)X
IdExists error (409 response code)X
NotFound error (404 response code)X
Table 1: Decision table for POST /buyers.
Figure 6: Buyer resource dependencies.

Suppliers

Figure 7: Supplier resource dependencies.

Product Catalogs

The product catalogs resource dependencies have been broken out across two diagrams for clarity.

Figure 8: Product Catalogs resource dependencies.

You will notice that the spec and spec option has a cyclical dependency. The nuance to the spec resource is that the DefaultOptionID cannot be assigned until after the spec option has been created, however it relies on the spec to exist. This chicken or egg scenario can be resolved by temporarily removing the DefaultOptionID before creating the spec and spec options then patching the value back onto the spec afterwards.

For the implicit references of Inventory > Product and Variant Inventory > Variant, the inventory objects updated on the Product and Variant input models, so they can be treated as being on the same level if applied together. However, the inventory cannot be created/updated without the existence on the Product/Variant entities.

// 1. Remove the DefaultOptionID from the spec prior to creation
var defaultOptionID = spec.DefaultOptionID;
spec.DefaultOptionID = null;
await orderCloudClient.Specs.CreateAsync(spec, accessToken);

// 2. Create the spec options
await ImportSpecOptions(specOptions);

// 3. Patch the spec's DefaultOptionID
await PatchSpecDefaultOptionID(spec, defaultOptionID);
Figure 9: Product Catalogs resource dependencies (cont.).

Orders and Fulfillment

For orders and fulfillment, we will focus on the promotion and seller approval rule resources as these a more typical of a Marketplace setup, whereasall other order resources typically represent historical order placement data, which is outside the scope of this article.

There is a seemingly cyclical dependency between the promotion and order resources, however the promotion-to-order is an implicit resource reference, which does not impact the validation logic for persistence of the promotion resource. Instead, it is used to identify the overall redemption count and user redemption count from orders placed and is used in the validation logic when applying a promotion to an order. When migrating data, excluding order data, between environments, this data inconsistency may be incorrectly identified as an environmental bug, which you can avoid troubleshooting with this knowledge.

Figure 10: Orders and Fulfillment resource dependencies.

Summary

Resource dependencies can be considered a huge gotcha when it comes to long-term maintenance. By having a sound knowledge of resource dependencies and nuances within the OrderCloud API, and employing defensive programming techniques, robust and performant ecommerce implementation can become a byproduct of good solution design.

References

Posted on

Transitioning from Sitecore Experience Commerce to OrderCloud: Tax and Payments

Reading Time: 6 minutes

In this article, we will review and compare tax and payments between Sitecore Experience Commerce and OrderCloud to facilitate developers looking to transition from XC to OrderCloud as well as identify a path for migration of existing XC solutions to OrderCloud.

We will look at a high-level comparison of architecture, functionality, and data models, with greater focus on transitioning from Sitecore Experience Commerce, meaning identifying a close path to parity in OrderCloud, using the Habitat catalog and SXA Storefront.

Conceptual Architecture and Features

Tax

The Commerce Engine’s tax plugin is considered sample code that is sufficient enough to support basic tax requirements, however tax calculations will typically be customised to support the tax provider integrations that cater to tax laws for the countries and/or regions the storefront will be servicing.

The following list contains highlights of the tax plugin:

  • Tax is calculated as a cart adjustment when a fulfillment component is applied to the cart.
  • Tax is calculated as a cart line adjustment where a fullfillment component exists on the cart line.
  • XC waits for a cart to contain a fulfillment component before calculating taxes, as the delivery address is intended to identify applicable tax laws, but not catered for without customisation.
  • Sellable item pricing does not include tax.
  • Applying the tag taxexempt will omit sellable items from tax calculations.
  • Tax is calculated as a flat rate, i.e. tax is not dynamically calculated based on country/region tax laws.

The GlobalTaxPolicy provides the configuration properties of the sample tax plugin.

{
  "$type": "Sitecore.Commerce.Plugin.Tax.GlobalTaxPolicy, Sitecore.Commerce.Plugin.Tax",
  "DefaultCartTaxRate": 0.1,
  "DefaultItemTaxRate": 0.1
  "TaxExemptTagsEnabled":true,
  "TaxExemptTag":"taxexempt",
  "TaxCalculationEnabled": true, // Not hooked up
  "PriceIncudesTax": false, // Not hooked up
  "CalculateTaxBasedOn": "ShippingAddress", // Not hooked up
  "ShippingTaxClass": "CartItems", // Not hooked up
  "RoundAtSubTotal": false // Not hooked up
}

OrderCloud relies on the middleware application to supply tax calculations, similar to the Ship Estimates covered previously, which is triggered via calculate integration event; The /calculate endpoint calls the middleware’s /ordercalculate endpoint to perform tax calculations as well as any line item or promotion cost overrides.

Complete feature parity is not something that can be achieved for the tax implementation purely because the middleware is responsible for providing tax, which could involve an integration with a tax provider or custom databases to store and retrieve tax data from. Instead, we will only suggest that by utilising the details covering XC’s tax plugin above as considerations, along with the order calculate event integration event overview in figure 1, this may serve as a starting point for designing your own tax solution in OrderCloud.

The calling system in figure 1 may be the client-side storefront or the middleware, depending on the implementation details.

Figure 1: The order calculate integration event overview.

Payments

Gift Cards vs Spending Accounts

XC provides a gift card payment type, which is integrated into the SXA storefront checkout to demonstrate end-to-end functionality of gift card payments. It is treated as a sample implementation as the functionality in the Commerce Engine is typically replaced with an integration to a gift card payment provider.

Reviewing OrderCloud’s supported payment types, spending accounts most closely resemble XC’s gift card implementation as it has the ability to track the available funds. One minor difference is that spending accounts are created under the context of a buyer and is therefore tied to the buyer users within that buyer, whereas gift cards in XC are globally accessible.

To differentiate spending accounts representing gift cards from other spending account types, xp can be leveraged to specify the type.

Figure 2: Gift card architecture comparison.

Global Payment Configuration

In an XC SXA implementation, payments are configured at a global level under sitecore/Commerce/Commerce Control Panel/Shared Settings, as depicted in figure 3. The configuration is split across Payment Option Types and Payment Options and linked via the PaymentOption Type property on the payment option item, however there really isn’t any discernable difference between the two within the SXA/Commerce Engine implementation.

Figure 3: Payment option to payment option type in the Sitecore Content Editor.

Instead of a BYO payment types approach, OrderCloud has defined three payment types, being purchase order, credit card, and spending account, with each type serving a different approach to handling order payments. For example, the credit card payment type would integrate with a payment service provider for perform transactions, while the spending account manages a balance maintained purely with OrderCloud, including adjusting the balance upon successful order submission.

Figure 4: Payment architecture overview.

As XC contains multiple Payment Option Types that could map to a single OrderCloud payment type, xp can be leveraged to represent multiple sub-types. The following table shows how XC’s Payment Option Types could map to Payment Types in OrderCloud.

Payment Option Type (XC)Payment Type (OrderCloud)xp (OrderCloud)
Card PaymentCredit Cardxp.Type: “Basic Card”
Federated PaymentCredit Cardxp.Type: “Braintree”
Gift Card PaymentSpending Accountxp.Type: “Gift Card”
Loyalty Card PaymentSpending Accountxp.Type: “Loyalty Card”

More information on payment types can be found in OrderCloud: Flexible Fulfillment > Apply Payments.

Storefront Payment Configuration

Payment options are also configured against an SXA storefront with a subset of those configured in the Shared Settings.

Figure 5: Storefront payment configuration.

While OrderCloud doesn’t have the context of storefronts, storefront-specific payment options will likely be a consideration for middleware’s logic for identifying applicable payments options.

Payment Components vs Payments

When it comes to the implementation, this is where the Commerce Engine plugins take the payment configurations and provide the functionality for the payment types. The sample Braintree plugin and the GiftCards plugin will add a FederatedPaymentComponent or a GiftCardComponent to the cart respectively, which will then be used to dictate how those payments are interacted with throughout the platform, during and post order creation.

OrderCloud uses the payments entity in a more concrete way, using the Type property to facilitate the payment behaviour. For example, selecting the spending account type will require the also specifying a spending account that is assigned to the buyer user via buyer or buyer user group assignment.

Figure 6: An order paid for using both a credit card and a gift card payment types.

Sales Activities vs Transactions

XC orders use sales activities to track the transaction activities of each order payment component while OrderCloud uses transactions for the same purpose.

Figure 7: Sales activities vs transaction architecture.

Functionality

The Storefront Checkout Billing Step

Turning our attention to the storefront functionality of the billing checkout steps from Carts to Unsubmitted Orders: Storefront User Journey, we will look at adding the email address and payment details to the cart/unsubmitted order, focusing on varying APIs of the two platforms.

Figure 8: The checkout billing aspects of the storefront user journey.

Viewing the interaction diagram in figure 9, the SXA storefront utilises the BillingDataJsonResult to provide the necessary data to render and facilitate payment option logic in the storefront. Once the user has submitted the payment details, all payment data is then sent to the Commerce Engine together to be added to the cart.

A few notes on the payment functionality:

  • Payments are only supported at the cart-level.
  • The email address is collected per order as the user may wish not to use the email address associated to their account (for registered users). This email is typically used for communications to the customer, such as email confirmation, invoicing, shipments, etc.
  • XC has logic to prevent a gift card sellable item to be purchased with a gift card payment.
Figure 9: The billing checkout step of the SXA storefront.

To replicate similar behaviour in an OrderCloud storefront implementation, figure 10 provides an approach to the billing checkout step.

  1. The /paymentoptions middleware endpoint retrieves the applicable payment options for the order, dependent on the storefront and business requirements.
  2. The email address is PATCHed to the order xp for specific order context.
  3. All payments can be added to the order via ../payments endpoint; the data sent may differ based on payment type.
Figure 10: An example approach for the delivery checkout step in an OrderCloud storefront.

Data Mapping

With the conceptual analysis above, we will now review what data mapping would look like for migration and from a comparison standpoint.

In the XC Entity/Component column, components are assumed to live on the primary XC entity being mapped.

OrderCloud IDs do not allow spaces. It is important that the IDs are parsed to remove/replace invalid characters consistently.

Spending Accounts

The following mapping shows how XC’s default implementation of gift cards can be translated into OrderCloud using spending accounts.

OC PropertyData TypeRequiredXC Entity/ComponentXC PropertyData TypeNotes
buyerIDstringYesN/AN/AN/A
IDstringNoGiftCardGiftCardCodestringUse format “GiftCard-<GiftCardCode>”
NamestringYesGiftCardNamestring
BalancedecimalYesGiftCardBalance.Amountdecimal
AllowAsPaymentMethodbooleanNoN/AN/AN/ASet to true.
RedemptionCodestringNoGiftCardGiftCardCodestring
StartDateDateTimeOffsetNoGiftCardActivationDateDateTimeOffset
EndDateDateTimeOffsetNoN/AN/AN/A
xpobjectNoN/AN/AN/A
xp.TypestringNoN/AN/AN/ASet to "GiftCard".
xp.InitialAmountdecimalNoGiftCardOriginalAmount.Amountdecimal
xp.CurrencystringNoGiftCardOriginalAmount.CurrencyCodestringRecommended for handling multi-currency validation.

References

Posted on

Transitioning from Sitecore Experience Commerce to OrderCloud: Fulfillments to Shipping

Reading Time: 5 minutes

In this article, we will review and compare Sitecore Experience Commerce fulfillments and OrderCloud shipping to facilitate developers looking to transition from XC to OrderCloud as well as identify a path for migration of existing XC solutions to OrderCloud.

We will look at a high-level comparison of architecture, functionality, and data models, with greater focus on transitioning from Sitecore Experience Commerce, meaning identifying a close path to parity in OrderCloud, using the Habitat catalog and SXA Storefront.

Conceptual Architecture and Features

Fulfillment Configuration

Global Fulfillment Configuration

In an XC SXA implementation, fulfillments are configured at a global level under sitecore/Commerce/Commerce Control Panel/Shared Settings, as depicted in figure 1. The configuration is split across Fulfillment Option Types and Fulfillment Options and linked via the Fulfillment Option Type property on the fulfillment option item.

Figure 1: Fulfillment option to fulfillment option type in the Sitecore Content Editor.

This tiered architecture can be a little confusing, so each tier can be thought of as follows:

  • Fulfillment Option Type: The high-level approach in which the order will be fulfilled, e.g. Deliver to Address, Digital Delivery, Pick Up From Store, etc.
  • Fulfillment Option: The collection of methods for the type of fulfillment, e.g. Ship items. This could also be as considered a shipping vendor, such as DHL or FedEx.
  • Fulfillment Method: The specific method of delivery, which will be assigned to the order and may be accompanied by a fee.

The fulfillment fees associated to each fulfillment method are then configured in the Commerce Engine environment configuration under GlobalPhysicalFulfillmentPolicy.FulfillmentFees, which also supports multi-currency fees per fulfillment method.

"$type": "Sitecore.Commerce.Plugin.Fulfillment.GlobalPhysicalFulfillmentPolicy, Sitecore.Commerce.Plugin.Fulfillment",
"FulfillmentFees": {
  "$type": "System.Collections.Generic.List`1[[Sitecore.Commerce.Plugin.Fulfillment.FulfillmentFee, Sitecore.Commerce.Plugin.Fulfillment]], mscorlib",
  "$values": [
    {
      "$type": "Sitecore.Commerce.Plugin.Fulfillment.FulfillmentFee, Sitecore.Commerce.Plugin.Fulfillment",
      "Fee": {
        "$type": "Sitecore.Commerce.Core.Money, Sitecore.Commerce.Core",
        "CurrencyCode": "USD",
        "Amount": 15.0
      },
      "Name": "Ground",
      "Policies": {
        "$type": "System.Collections.Generic.List`1[[Sitecore.Commerce.Core.Policy, Sitecore.Commerce.Core]], mscorlib",
        "$values": []
      }
    },
    ...
}

Bringing this together into a friendly-ish diagram view, figure 2 shows the representation of fulfillment architecture across the Sitecore and Commerce Engine platforms.

Figure 2: Fulfillment architecture overview.

In OrderCloud, there are no fulfillment object equivalents to house and support fulfillment configurations. Instead, OrderCloud relies on the middleware application to provide its business logic, integrations, calculations, etc. to address the unique shipping business requirements for the OrderCloud storefront solution, which we cover in greater detail in Ship Estimates.

For now, figure 2 may be used as a reference point when evaluating how fulfillment/shipping data could be designed if business requirements identify that the custom middleware solution should persist shipping configuration data to a custom database.

Storefront Fulfillment Configuration

One last configuration requirement for an SXA storefront is to select the fulfillment options that will be applicable for that particular storefront.

Figure 3: Storefront fulfillment configuration.

Once again, OrderCloud lets the middleware manage the storefront specific shipping logic and calculations, which is covered in Ship Estimates, so this may just be a consideration for your custom middleware implementation.

Functionality

Ship Estimates

While OrderCloud depends on the middleware application to drive shipping data and behaviour, it does not leave the implementer high and dry. The order checkout integration event sees the /estimateshipping endpoint trigger the /ShippingRates endpoint of the middleware, handing it the order worksheet for context. The middleware can then execute its custom logic, which may include integrations to shipping providers, external systems, etc., to determine ship estimates.

The calling system in figure 4 may be the client-side storefront or the middleware, depending on the implementation details.

Figure 4: The order checkout integration event interaction overview.

The resulting ShipEstimateResponse model from the middleware’s /ShippingRates endpoint will be expected to consist of a breakdown of shipping estimates, representing a subset of line items and the shipping methods and costs that are available for the order.

In comparison to the SXA storefront, the ShipEstimateResponse model is basically the equivalent to the DeliveryDataJsonResult application model, which contains the ShippingOptions at the order and line levels.

Figure 5: The high-level view of ship estimates in a ShipEstimateResponse model.

For the integration event to trigger, the order checkout integration event will need to be configured with the middleware’s publicly accessible domain assigned to the CustomImplementationUrl of the integration event, and integration event’s ID assigned to the OrderCheckoutIntegrationEventID of the API Client.

The API Client used for the integration event will likely be the same client used for the Anonymous and Registered Users.

Figure 6: OrderCheckout integration event assignment to API client example.

The Storefront Checkout Delivery Step

Turning our attention to the storefront functionality of the delivery checkout steps from Carts to Unsubmitted Orders: Storefront User Journey, we will look at adding fulfillment details to the cart/unsubmitted order, focusing on varying APIs of the two platforms.

Figure 7: The fulfillment focused view of the storefront user journey.

Viewing the interaction diagram in figure 8, the SXA storefront utilises the DeliveryDataJsonResult to provide the necessary data to render and drive fulfillment functionality. Once the user has submitted the fulfillment details, all fulfillment data is then sent to the Commerce Engine together to be added to the cart.

A few notes on the fulfillment functionality:

  • Fulfillments can be set against the cart as a whole or for each and every cart line.
  • As sellable items can be classified as either a physical or digital item, XC contains logic to tie these to varying fulfillment types, which will collect different data relevant to its classification. For example, physical items require a physical fullfillment, consisting of a physical delivery address and the fulfillment method id, while digital items require digital fulfillments, consisting of recipient email address, gifting message and the fulfillment method id.
Figure 8: The delivery checkout step of the SXA storefront.

To replicate similar behaviour in an OrderCloud storefront implementation, figure 9 provides an approach to the delivery checkout step, which intentionally deviates from the SXA storefront functionality to demonstrate a more granular approach to building up the order using OrderCloud API.

  1. The /shippingoptions middleware endpoint retrieves the available shipping options for the order, such as order-level or line-level shipping, for different data collection based on the type of fulfillment required.
  2. Shipping address(es) can be input manually or by referencing an existing address assigned to the user.
  3. The ../estimateshipping integration event can leverage the shipping addresses for destination-specific logic.
  4. The ../shipmethods sets the shipping methods for the order or line items.
Figure 9: An example approach for the delivery checkout step in an OrderCloud storefront.

References

Posted on

Transitioning from Sitecore Experience Commerce to OrderCloud: Carts to Unsubmitted Orders

Reading Time: 10 minutes

In this article, we will review and compare Sitecore Experience Commerce carts and OrderCloud unsubmitted orders to facilitate developers looking to transition from XC to OrderCloud as well as identify a path for migration of existing XC solutions to OrderCloud.

We will look at a high-level comparison of architecture, functionality, and data models, with greater focus on transitioning from Sitecore Experience Commerce, meaning identifying a close path to parity in OrderCloud, using the Habitat catalog and SXA Storefront.

Conceptual Architecture and Features

Carts vs Unsubmitted Orders

Architecture Overview

In XC, the cart entity stores information about the order to be placed. When an order is placed, the contents of a cart is copied over into a new order entity with an initial order status of Pending.

OrderCloud essentially combines the cart and order into the order object for the entire cart/order lifecycle. The initial status of the order is Unsubmitted, which is effectively the XC cart equivalent. When the order is submitted, the order will then transition its status to Open, matching the XC order with status of Pending.

It is also possible that the OrderCloud order submitted status will be set to AwaitingApproval, however this status is not relevant for the SXA storefront comparison.

Figure 1: The XC and OrderCloud representations of a shopping cart being constructed and their inital state once the order has been submitted.

Referencing figure 2, there are a few important details regarding the XC and SXA cart to order architecture in relation to OrderCloud’s architecture:

  1. SXA sees that a customer only interacts with a single cart per storefront, by using the naming convention, “Default<customer id><storefront name>“, to identify the cart id and is the only means of a relationship that the customer has with the cart in the XC database.
  2. The Commerce Engine’s /carts endpoint, which retrieves the cart, requires the cartId parameter and does not have consideration for the user context.
  3. While order ids must be unique, orders are created with randomly generated ids and are not copied from the cart’s fixed id. Orders are persisted, accomponied with a list entry, which represent the relationship between the customer and the orders they have placed.
  4. During order creation, after the cart contents has been copied to the order, the cart is emptied to allow it to be used for subsequent order placement.

In comparison to OrderCloud’s architecture, orders are tightly coupled to their owners via the FromUserID property, and as both order ids in XC and OrderCloud are generated there is no chance for Id conflicts.

  1. In managing a single order, the most recent unsubmitted order is considered the active ‘cart’ and as such does not need to rely on an order id to identify the cart as it’s relationship to the user as it’s tightly coupled to the owner via the FromUserID property.
  2. The /me/orders endpoint retrieves the order based on user context, which is extracted from the auth token, rather than the order id.
  3. The order id is also generated by default, so there’s no chance of an ID conflict when allowing the platform to generate it.
  4. There is no implementation concerns around migrating a cart to an order.
Figure 2: The SXA storefront will use a single cart per customer, which is converted into each order, whereas OrderCloud creates the orders directly allowing multiple unsubmitted orders being active simultaneously.

Storefront User Journey

In figure 3, we see a high-level user journey for the SXA storefront alongside the OrderCloud equivalent. While OrderCloud’s user journey appears to be longer, this is only due to order requiring explicit creation and XC consolidating shipping address and shipping method into fulfillment details, and billing address and payment into payment details. Ultimately, the user journey is matched and there are no sacrifices necessary.

Figure 3: Cart to order vs unsubmitted order to submitted order user journey comparison.

Viewing the Cart/Order

In XC, the GET /Carts({cartId}) endpoint is typically called with the $expand query parameter to retrieve all cart components, cart lines and their components for the full view of the cart.

In OrderCloud, the RESTful API provides individual endpoints for extracting order data at more granular levels. For example the order details can be retrieved via the GET /orders/{direction}/{orderID} endpoint, but it won’t provide the line item details, which can be retrieved in bulk using GET /orders/{direction}/{orderID}/lineitems or individually via GET /orders/{direction}/{orderID}/lineitems/{lineitemID}.

Alternately, OrderCloud also provides GET /orders/{direction}/{orderID}/worksheet, which returns the order object along with all line items and all responses from integration events. providing a more complete view, however some objects such as applied promotions will still require additional requests from the orders resource to retrieve them on an as needed basis.

Cart vs Order Totals

In the table below, we see a comparison between the cart line totals side by side with the equivalent line item totals. The notable differences are as follows:

  • XC Adjustments may include taxes, fulfillment fees, and promotion discounts, whereas OrderCloud breaks out the each adjustment classification at the order level.
  • As calculations with adjustments are calculated differently at the line-level and cart/order level, calculating the GrandTotal/Total may result in differently between the platforms.
  • The PaymentsTotal is not available on the order, however the payments total can be calculated by the middleware using OrderCloud’s GET/orders/{direction}/{orderID}/payments endpoint.
Cart (XC)CalculationOrder (OC)CalculationMatch
SubTotalSum of all line-level SubTotals.SubtotalSum of all LineItem.LineSubtotals.Yes
AdjustmentsTotalSum of all cart-level Adjustments.ShippingCostHandled by middleware via OrderCalculate integration event.No
TaxCostHandled by middleware via OrderCalculate integration event.No
PromotionDiscountSum of all line item-level promotion discount amounts applied.No
GrandTotalSubTotal + sum of all cart-level adjustments flagged with IncludeInGrandTotal being true + sum of all line-level AdjustmentsTotalsTotalSubtotal + TaxCost + ShippingCost – PromotionDiscount.No
PaymentsTotalSum of all PaymentComponent Amounts.N/A

Cart Lines vs Line Items

Adding a Cart Line vs Line Item

From an API perspective, to add a cart lines in XC, the request requires the cartId, quantity, and itemId, consisting of catalogId, sellableItemId, and variationId.

The catalogId is intended to resolve pricing and promotions for sellable items and their variants, based on the catalog associations to price books and promotion books as per the XC architecture.

POST https://authoring.engine.dev/api/AddCartLine HTTP/1.1

{
  "cartId": "{orderId}",
  "itemId": "{catalogId}|{sellableItemId}|{variantId}",
  "quantity": 1
}

Adding a line item in OrderCloud is much the same as seen in the example request below. orderId parameter replaces the cartId, while the ProductId equates to the sellableItemId. The catalogId however, is not required as pricing and promotions are not dependent on catalog assignments, and rather than passing the variantId, the individual product specs with OptionIDs are specified, ultimately resolving to variant equivalent, but also caters for specialised open text specs that OrderCloud supports.

POST https://sandboxapi.ordercloud.io/v1/orders/outgoing/{orderId}/lineitems HTTP/1.1

{
  "ProductID": "{productId}",
  "Specs": [
    {
      "SpecID": "{specId}",
      "OptionID": "{specOptionId}"
    },
    {
      "SpecID": "{specId2}",
      "OptionID": "{specOptionId2}"
    }
  ]
}

Cart Line Rollup

In the SXA storefront, when adding a sellable item/variation to the cart that has already been added, the quantity will be updated to reflect the initial cart’s quantity plus the quantity added. This is known as cart line rollup and is controlled by the RollupCartLinesPolicy.Rollup property in the Commerce Engine’s environment role configuration.

Figure 4: Add cart line functionality.

As line rollup is not a native feature of OrderCloud’s RESTful API, this functionality could be implemented in the middleware. Instead adding line items will create a new line item in the order, in the same manner that XC will with rollup disabled.

Figure 5: Create line item functionality.

Line-Level Pricing

XC’s cart lines contain the UnitListPrice and the SellPrice on the PurchaseOptionMoneyPolicy, which can be thought of as “was” and “now” pricing respectively.

The SellPrice can evaluate to the same value as the UnitListPrice, to which the SXA storefront will treat this as a standard price.

"Policies": [
    {
        "@odata.type": "#Sitecore.Commerce.Plugin.Pricing.PurchaseOptionMoneyPolicy",
        "PolicyId": "5e6c1f42c7c94bc4b79696e716b6cda5",
        "Models": [],
        "Expires": "2019-04-22T13:13:18.8117816Z",
        "SellPrice": {
            "CurrencyCode": "USD",
            "Amount": 6
        },
        "FixedSellPrice": false
    }
],
"UnitListPrice": {
    "CurrencyCode": "USD",
    "Amount": 2429.99
}

The MessagesComponent, also shows the audit trail detailing how the cart line’s unit list price and sell price were resolved, which are evaluated from ListPrices and Price Card Snapshots at the sellable item and item variation levels.

{
	"@odata.type": "#Sitecore.Commerce.Core.MessagesComponent",
	"Id": "7e6bc27965ef4f3a954e6fcdadb96fa5",
	"Name": "",
	"Comments": "",
	"Policies": [],
	"Messages": [
		{
			"Code": "Pricing",
			"Text": "SellPrice&lt;=PriceCard.Snapshot: Price=$10.00|Qty=1.0|PriceCard=Habitat_PriceCard"
		},
		{
			"Code": "Pricing",
			"Text": "ListPrice&lt;=PricingPolicy: Price=$1,919.69"
		},
		{
			"Code": "Pricing",
			"Text": "Variation.SellPrice&lt;=Variation.PriceCard.Snapshot: Price=$9.00|Qty=1.0|Variation=56042567|PriceCard=Habitat_VariantsPriceCard"
		},
		{
			"Code": "Pricing",
			"Text": "Variation.ListPrice&lt;=Variation.PricePolicy: Variation=56042567|Price=$2,429.99"
		},
		{
			"Code": "Pricing",
			"Text": "CartItem.SellPrice&lt;=PriceCard.ActiveSnapshot: Price=$6.00|Qty=5.0"
		},
		{
			"Code": "Pricing",
			"Text": "CartItem.ListPrice&lt;=SellableItem.Variation.ListPrice: Price=$2,429.99"
		}
	],
	"ChildComponents": []
}

OrderCloud currently only stores the resolved UnitPrice on the line item, so while you can trust that the UnitPrice is resolved from the correct PriceBreak and prioritises SalePrice over Price, this information is not apparent.

For orders in an unsubmitted state, the /products/ resource may be used to retrieve the product information will return the the current active snapshot, however once the order has been placed the product and pricecards resources can no longer be considered the source of truth. In the interim, you may wish to create a webhook to add the additional PriceSchedule data to the line item’s xp for historical and reconciliation purposes.

The other aspect of note is that the Currency is set at the order level, which applies to all monetary values on the order and line items.

"Order": {
  "ID": "MyOrder"
  ...
  "Currency": "USD"
  ...
},
"LineItems": [
  {
    "ID": "MyLineItem",
    ...
    "UnitPrice": 2429.99,
    ...
  }
]

Cart Line vs Line Item Totals

In the table below, we see a comparison between the cart line totals side by side with the equivalent line item totals. The notable differences are as follows:

  • XC Adjustments may include taxes, fulfillment fees, and promotion discounts, whereas OrderCloud only includes PromotionDiscount at the level.
  • XC inverts the promotion discount adjustments so a $20 discount will be represented as -$20 in XC, so calculation of the GrandTotal is valid.
  • XC’s GrandTotal effectively matches the OrderCloud’s LineTotal as taxes and fulfillment fees aren’t flagged with IncludeInGrandTotal.
CartLine (XC)CalculationLineItem (OC)CalculationMatch
SubTotalSellPrice * QuantityLineSubtotalUnitPrice * QuantityYes
AdjustmentsTotalSum of all line-level Adjustments.PromotionDiscountSum of all line item-level promotion discount amounts applied for the current line item.No
GrandTotalSubTotal + sum of line-level adjustments, where adjustments are flagged with IncludeInGrandTotal being true.LineTotalLineSubtotal – PromotionDiscountYes
PaymentsTotalPayments not considered at line-level without customisation.N/A

Extending the Cart and Cart Lines vs Order and Line Items

Those familiar with the SXA storefront and the Commerce Engine will be pleased with the significantly reduced development efforts involved in extending the order and order line items. Gone are the days of the plumbing extended strongly-typed model classes through several application layers in strongly-typed classes and dependency injection. And that’s just the storefront. The Commerce Engine would typically require creating new endpoints and replacing several more classes.

OrderCloud’s extended properties (xp) is not only a dynamically built object in OrderCloud, but the OrderCloud .NET and JavaScript SDKs also represent xp as a dynamic type, meaning you can write to a new property within the xp in one class and read from it in another class without any plumbing.

var order = new Order();
order.xp.MyNewProperty = "My New Property Value";

var lineitem = new LineItem();
line.xp.MyLineItemProperty = true;

Merge Carts vs Transfer Order

A typical ecommerce feature is retaining a cart/unsubmitted order of a guest/anonymous user as they log in or register to the storefront. In the SXA storefront, the register component will essentially transfer ownership to the newly registered user, while the login component will merge the anonymous cart with registered user’s cart, if it exists, by copying across all cart lines. If Cart Line Rollup is enabled, quantities will be updated where appropriate instead of copying over the cart line.

Figure 6: XC merges the anonymous cart with the existing customer cart on log in (rollup enabled).

OrderCloud contains similar functionality in allowing ownership an order from an anonymous user to be transferred to a profiled (registered) user. For registration, the order will need to be explicitly transferred.

Where a profiled user logs in, the difference between transferring an order and XC’s merging of carts, is that the buyer user may now have two active unsubmitted orders. The implementor would need to manage merging the orders together and either explictly deleting the second order or let OrderCloud’s unsubmitted order cleanup process delete it.

Figure 7: OrderCloud transfers an anonymous order to an authenticated registered user.

Abandoned Cart/Order Cleanup

The Commerce Engine’s minions role is responsible for cleaning up abandoned and empty carts. As seen below the PurgeCartsMinion runs daily, removing empty carts that are more than 2 days old and carts that have not been updated for more than 14 days. Both the minion and purge carts policies are configurable.

{
  "$type": "Sitecore.Commerce.Core.MinionPolicy, Sitecore.Commerce.Core",
  "WakeupInterval": "01.00:00:00",
  "ListsToWatch": [
    "Carts"
  ],
  "FullyQualifiedName": "Sitecore.Commerce.Plugin.Carts.PurgeCartsMinion, Sitecore.Commerce.Plugin.Carts",
  "ItemsPerBatch": 10,
  "SleepBetweenBatches": 500
}

{
  "$type": "Sitecore.Commerce.Plugin.Carts.PurgeCartsPolicy, Sitecore.Commerce.Plugin.Carts",
  "AbandonedCartsThreshold": 14,
  "EmptyCartsThreshold": 2
}

In OrderCloud, unsubmitted order cleanup is an internal automated process with the rules outlined below, which are not configurable.

User TypeHas Line Items?Retention Policy
AllNo24 hours
AnonymousYes7 days
ProfiledYes90 days from LastUpdated date

Data Mapping

With the conceptual analysis above, we will now review what data mapping would look like for migration and from a comparison standpoint.

In the XC Entity/Component column, components are assumed to live on the primary XC entity being mapped.

OrderCloud IDs do not allow spaces. It is important that the IDs are parsed to remove/replace invalid characters consistently.

Orders

POST /orders/{direction}

In the Architecture Overview, we note that the cart equivalent in OrderCloud is an unsubmitted order. While no direct data mapping required, the following table details the alignment with XC.

OC PropertyData TypeRequiredNotes
directionstringYesIs a resource parameter, not body property.
Set to Outgoing for buyer users.
IDstringNoAllowing the ID to be generated by the platform effectively matches XC order behaviour.
FromCompanyIDstringNoThis value will default to the buyer, which effectively matches XC.
ToCompanyIDstringNoThis value will default to the marketplace owner, which effectively matches XC.
FromUserIDstringNoDefaults to the current user context.
BillingAddressIDstringNoNot required at the time of creation.
ShippingAddressIDstringNoNot required at the time of creation.
CommentsstringNo
ShippingCostnumberNoNot required at the time of creation.
TaxCostnumberNoNot required at the time of creation.
xpobjectNoAny custom components required for the initial state can be added to xp.

Line Items

POST /orders/{direction}/{orderID}/lineitems

Adding a Cart Line vs Line Item has covered how to align the create line ltem request to align with XC. The following table covers the full view of the request.

OC PropertyData TypeRequiredXC PropertyNotes
directionstringYesIs a resource parameter, not body property.
Set to Outgoing for buyer users.
orderIDstringYesIs a resource parameter, not body property.
OrderCloud’s order.ID
IDstringNoBoth XC and OrderCloud generate this ID by default.
ProductIDstringYesItemIdOnly the SellableItemId portion of the ItemId is required.
QuantitystringYesQuantityThis value will default to the marketplace owner, which is effectively matches XC.
UnitPricestringNoWill be resolved from the PriceSchedule assignment of the current user context.
CostCenterstringNo
DateNeededstringNo
ShippingAccountstringNo
ShippingAddressIDstringNo
ShipFromAddressIDstringNo
InventoryRecordIDstringNo
SpecsobjectNoItemIdUsed over variant ID. All spec IDs and specOptionIDs that represent the variant product will need to be specified.
xpobjectNoCustom components can be added to xp as needed.

References

Posted on

OrderCloud: Debugging Integration Events and Webhooks using ngrok

Reading Time: 3 minutes

In this article, we will review how we can redirect traffic from OrderCloud’s integration events and webhooks to our local web server to assist debugging efforts in local development.

The OrderCloud Headstart application will be referenced in this example, which may differ slightly from your integration service/middleware application.

Introduction

As OrderCloud’s integration events and webhooks require public urls to be available for testing and having complete functionality available, the question quickly arises as to how developers will be able to develop and debug the respective middleware endpoints in a local environment. One solution we will review is to use ngrok, a free service, which allows us to expose a web server running on our local machine to the internet.

Installation, Configuration, and Testing

High-level steps are provided below, however steps may vary based on operating systems and command tools. See ngrok: Getting Started > Setup & Installation, and register or log in to a free account, for alternative setup steps.

Preparing ngrok

Using Powershell, install ngrok using choco.

choco install ngrok

Add the authtoken, available from ngrok: Getting Started > Your Authtoken once signed up for a free account. 

ngrok authtoken <auth token>

Exposing the Secure Web Server with ngrok

Moving over to the local imiddleware, we will need to obtain the port number used, e.g. the HTTPS url from the applicationUrl in the Local profile.

{
  "profiles": {
    "Local": {
      "commandName": "Project",
      "environmentVariables": {
        "APP_CONFIG_CONNECTION": "<app config connection>",
        "ASPNETCORE_ENVIRONMENT": "Development"
      },
      "applicationUrl": "https://localhost:5001;http://localhost:5000"
    }
  }
}

Now we can start HTTP tunnel forwarding to our local middleware.

ngrok http https://localhost:<port number>

The tunnel will then be created with a random URL, which we need to take note of for our OrderCloud configurations.

Figure 1: Randomly generated URL from grok.

Configuring OrderCloud’s Integration Events

In OrderCloud, we can create or update an integration event using the generated URL from ngrok as the CustomImplementationUrl. The HashKey will also need to be configured to match the OrderCloudSettings:WebhookHashKey appsettings in HeadStart.

As ngrok does generate a random URL each time when using free accounts, the CustomImplementationUrl will need to be patched each time you expose the local web server.

{
  "ID": "Middleware",
  "Name": "Middleware",
  "ElevatedRoles": [
    "FullAccess"
  ],
  "EventType": "OrderCheckout",
  "HashKey": "samplehash",
  "CustomImplementationUrl": "https://8f79-125-253-105-232.ngrok.io"
}

If the integration event has been newly created, don’t forget to assign it to the API client’s OrderCheckoutIntegrationEventID so that the integration event can trigger the endpoints of the middleware.

{
  "ID": <api client id>,
  "ClientSecret": null,
  "AccessTokenDuration": 600,
  "Active": true,
  "AppName": "Storefront",
  "RefreshTokenDuration": 0,
  "DefaultContextUserName": "Storefront-anonymous-user",
  "xp": {},
  "AllowAnyBuyer": true,
  "AllowAnySupplier": false,
  "AllowSeller": false,
  "IsAnonBuyer": true,
  "AssignedBuyerCount": 0,
  "AssignedSupplierCount": 0,
  "OrderCheckoutIntegrationEventID": "Middleware",
  "OrderCheckoutIntegrationEventName": "Middleware",
  "MinimumRequiredRoles": [],
  "MinimumRequiredCustomRoles": [],
  "MaximumGrantedRoles": [],
  "MaximumGrantedCustomRoles": []
}

Configuring OrderCloud’s Webhooks

When configuring webhooks, the Url will require the complete application service/middleware application endpoint to be defined, along with the HashKey, which should be configured to match the OrderCloudSettings:WebhookHashKey appsettings in HeadStart.

Webhooks also require the ApiClientIDs to be assigned on the resource itself rather that the API client and the WebhooksRoutes need to be specified in order for the OrderCloud API to trigger the webhook, which should then be redirected from ngrok through to the local web server.

{
  "ID": "MyWebhook",
  "Name": "MyWebhook",
  "Url": "https://8f79-125-253-105-232.ngrok.io/mywebhook",
  "HashKey": "samplehash",
  "ElevatedRoles": [
    "BuyerAdmin"
  ],
  "ApiClientIDs": [
    <api client id>
  ],
  "WebhookRoutes": [
    {
      "Route": "v1/buyers",
      "Verb": "POST"
    }
  ]
}

Testing the Endpoints in the Middleware

With Visual Studio’s debugger attached to the local IIS site or is being debugged directly, when an integration event or webhook endpoint is requested via an appropriate API client, ngrok will redirect the request to the local web server and the debugger will be able to capture the traffic for local debugging.

Figure 2: Visual Studio’s debugger successfully triggered from OrderCloud’s estimateshipping endpoint redirected to a localwebserver using grok.

Troubleshooting

An attempt was made to access a socket in a way forbidden by its access permissions

When attempting to run ngrok, if the response “An attempt was made to access a socket in a way forbidden by its access permissions” is received, restarting the Host Network Services Windows service should resolve this issue.

References

Posted on

Transitioning from Sitecore Experience Commerce to OrderCloud: Sellable Items To Products

Reading Time: 8 minutes

In this article, we will review and compare Sitecore Experience Commerce sellable items and OrderCloud products to facilitate developers looking to transition from XC to OrderCloud as well as identify a path for migration of existing XC solutions to OrderCloud.

We will look at a high-level comparison of architecture, functionality, and data models, with greater focus on transitioning from Sitecore Experience Commerce, meaning identifying a close path to parity in OrderCloud, using the Habitat catalog and SXA Storefront.

Conceptual Architecture and Features

Physical vs Digital Sellable Items

In XC, a sellable item is considered digital if it contains a tag that is also registered to the DigitalItemTagsPolicy. Digital products will not track inventory and will modify the delivery step of the checkout flow to collect information relevant to digital items, such as an email address and custom messaging, rather than a delivery address and shipping provider information, which is relevant for physical items.

While OrderCloud doesn’t have explicit digital product flagging and accompanying behaviour, there are some options available for us to model products as digital items. For example, we can disable inventory tracking at the product level and copy the tags across to the product’s extended properties (xp), and then have the middleware be responsible for using this information to treat the product as digital and manage the checkout process accordingly.

{
  "Name": "MyDigitalProduct",
  "Inventory": {
    "Enabled": false
  },
  "xp": {
    "tags": "subscription"
  }
}

Sellable Items vs Products

In XC, sellable items can be represented as standalone products, which are sellable items without variations, or as a product family, which consists of a number of item variations each representing an individual product that share mostly similar properties to the other item variations.

OrderCloud can also represent products as both standalone product or a product with variants.

Figure 1: Sellable items with and without item variations vs products with and without variants

Item Variations vs Variants

In XC, the item variations are created within a sellable item entity, each representing its own unique product.

Variation properties are defined in the environment role configuration under VariationPropertyPolicy and sellable item families are expected to use a subset of these properties to create unique combinations.

In OrderCloud, the architecture of product families have the same intention, however there is enforcement around creating only unique variations using specs and spec options. A spec closely relates to the VariationPropertyPolicy in that it specifies the property that will break down the product into a variant, while the spec option specifies all of the values that will be assigned to the spec. A product can have multiple specs assigned, e.g. the first spec could represent colors while the second could represent sizes.

While specs can then be assigned to more than one product, there’s more control in creating specs that are unique to products even if they do represent the same property type, such as size, as this will reduce the long term maintenance.

Figure 2: The product-spec assignment overview.

OrderCloud then provides the endpoint, POST/products/{productID}/variants/generate, to generate variants from all possible combinations of the assigned specs. Referencing figure 3 it can be noted that when migrating sellable items from XC there will be instances where not all combinations are required, however any unwanted variants can be disabled.

Figure 3: An example of a sellable item with item variations vs a product with generated variants.
Item Variation Corruption

While the SXA storefront and commerce data provider will utilise the VariationPropertyPolicy configuration for the item variation controls on the product details page in the storefront, there is no validation during item variation creation. This means that is is possible to corrupt sellable item families in the following ways:

  1. Two or more varations share the same values across all of their variation properties. In figure 4, the product family utilises variation properties color and size, however item variations C and D are both configured with same color and size. In this instance, duplication may have been unintentional or a third variation property is missing that could make the variation unique. The Style property could be added in this instance to all item variations assuming the values between item variations C and D would be different.
  2. The subset of variation properties are not all populated with values. In figure 4, the Color and Size properties are filled out for all item variations, except for item variation E which has a null value for Size.
Figure 4: Examples of how item variations can be inadvertently corrupted.

In figure 4, the Style property is null for all item variations.

Folding Sellable Items with Variations into Standalone Products

During sellable item migration there are two more scenarios that may come up, specifically in relation to sellable items with a single item variation. The first is in noting point 2 of the item variation corruption, a sellable item is flagged as corrupted as no variation properties of the item variation have values, which can be easily folded into a standalone sellable item.

The second scenario is where the single item variation does contain values for one or more variation properties. Here, is where further consideration may be required as to whether this item variation should be folded back into a standalone sellable item. The reason for this consideration is because the Commerce Engine sets a global set of variation properties, however this may not necessary apply to the sellable item in question, but was designed this way because the global configuration in the VariationPropertyPolicy.

Figure 5: Opportunities to fold sellable items with variations into standalone sellable items.

Static and Dynamic Bundles

XC has special product types called static bundles and dynamic bundles, which are configurations of sellable items that are sold together. These bundles can either represent a fixed group of sellable items or a list of sellable items with alternate sellable items, which can be substituted out by the user, and/or be an optional extra into the bundle.

OrderCloud currently does not support any type of product bundling. It is possible to create a pseudo-bundle using a product, however this would need to be workshopped separately to cover data and functional requirements, which sits outside of the scope of this analysis.

Extended Properties

In XC, a sellable item can have its properties extended via composer templates or programatically with custom components, while item variations only be extended programatically. These can be generally be translated over to OrderCloud using its eXtended Properties (xp), however considerations will be required around non-standard data types.

Images

XC leverages Sitecore’s media library for hosting images and stores the Sitecore Id of the image on the ImagesComponent of the sellable item or variant it’s associated to.

Over in OrderCloud, products don’t have an explicit property for images as it relies on external systems, such as DAMs or CDNs, to host imagery instead. In addition, the OrderCloud philosphy is that as there is no one-size-fits-all solution when it comes to working with product images, so the absence of a dedicated property provides the flexibility of allowing a bespoke data model to be added to the product’s xp to best represent the client’s business requirements. Examples of potential requirements include, a single array of images (image urls), image sets representing different views or components of a product, image sets representing varying quaility or sizing for omni-channel optimisation. etc.

Relationship Definitions

XC has a concept of relationship definitions, which allows sellable items to have a relation to another sellable item with a given context. For example, the Habitat catalog relationship definitions for associated sellable items for installation, training, warranty, and most notably related sellable items.

Figure 6: Related sellable items relationship

The Commerce Engine and BizFx application provides an interface for creating these relationships and applies validation to ensure that the related products exist and are not duplicated. The Commerce Engine also includes smarts to filter out invalid sellable items for a given context, e.g. inactive sellable items, sellable items that are not associated to the current catalog, etc.

In OrderCloud, there is no explicit product to product assignment available, however the list of associated products can be added to the product’s xp, e.g. xp.RelatedProducts, to cover the data architecture aspect of this feature gap.

Figure 7: XC related products association architecture vs. a sample approach using xp in OrderCloud.

With a suggested data architecture addressed, the remaining considerations will reside in the functional behaviour. The following functionality may need to be implemented in the middleware of the OrderCloud solution to replicate the XC behaviour:

  • Validate related product exists on association/assignment.
  • Filter out products not eligible in the current context:
    • Inactive products
    • Products not associated to catalogs
  • When deleting a product, remove product id from other products that it is assigned to in their xp.RelatedProducts.

Inventory and Pricing

The topics of inventory and pricing comparisons between XC and OrderCloud can be found in their dedicated article in Transitioning from Sitecore Experience Commerce to OrderCloud: Inventory and Pricing.

Other Considerations

Entity Versioning and Workflow

As OrderCloud has no concept of entity versioning, one approach towards migration is to only migrate the latest published versions of sellable items. In a similar manner the publishing workflow that applies to sellable items may see a project consider the latest entity version as the source of truth regardless of its published state. Considerations would need to be made on project by project basis, which may entail a level of data cleansing prior to migration.

Property Localisation

XC allows entity properties to be localisable for content that can be displayed in multiple languages. OrderCloud does not support localisation, so this may be a consideration for workshopping a solution to be handled by the custom middleware.

Data Mapping

With the conceptual analysis above, we will now review what data mapping would look like for migration and from a comparison standpoint.

In the XC Entity/Component column, components are assumed to live on the primary XC entity being mapped.

OrderCloud IDs do not allow spaces. It is important that the IDs are parsed to remove/replace invalid characters consistently.

Products

OC PropertyData TypeRequiredXC Entity/ComponentXC PropertyData TypeNotes
OwnerIDstringNoN/AN/AN/A
DefaultPriceScheduleIDstringNoSellableItemListPriceMoneyPriceSchedule will be created from the SellableItem’s ListPrice, using the Product.ID as the Price Schedule’s ID.
AutoForwardboolNoN/AN/AN/A
IDstringNoSellableItemFriendlyIdstring
NamestringYesSellableItemDisplayNamestring
DescriptionstringNoN/AN/AN/A
QuantityMultiplierintNoN/AN/AN/A
ShipWeightfloatNo[ItemSpecificationsComponent]Weightdouble
ShipHeightfloatNo[ItemSpecificationsComponent]Heightdouble
ShipWidthfloatNo[ItemSpecificationsComponent]Widthdouble
ShipLengthfloatNo[ItemSpecificationsComponent]Lengthdouble
ActivebooleanNoSellableItemPublishedbooleanThis assumes only the latest published entity version is being migrated.
ShipFromAddressIDstringNoN/AN/AN/A
Inventory.EnabledbooleanNoN/AN/AN/AVirtual products won’t have inventory in either XC or OrderCloud. This should be set to disabled if any of the XC tags contain a value representing an XC virtual product.
Inventory.NotificationPointintNoN/AN/AN/A
Inventory.VariantLevelTrackingbooleanNoN/AN/AN/A
Inventory.OrderCanExceedbooleanNoN/AN/AN/A
Inventory.QuantityAvailableintNoInventoryInformationQuantityintIf transitioning a single inventory set/ record per product.
DefaultSupplierIDstringNoN/AN/AN/A
AllSuppliersCanSellbooleanNoN/AN/AN/A
xpobjectNoN/AN/AN/AXC composer views and programatic components can be added to xp as needed.
xp.BrandstringNoSellableItemBrandstring
xp.ManufacturerstringNoSellableItemManufacturerstring
xp.TypeOfGoodstringNoSellableItemTypeOfGoodstring
xp.TagslistNoSellableItemTagslistConvert to list of string using name property only.
xp.ItemDefinitionslistNo[CatalogsComponent].[CatalogComponent]ItemDefinitionstringThe ItemDefinition of each CatalogComponent should be added to the list, excluding duplicate values.
xp.RelatedProductslistNoRelationship (Commerce List)IdstringThe relationships are stored in lists, not entities. All Ids will need to be parsed to their friendly ids.

Specs

OC PropertyData TypeRequiredXC Entity/ComponentXC PropertyData TypeNotes
IDstringNoSellableItem
ItemVariationComponent		FriendlyId
<variation property>		stringRecommended using ‘{SellableItem.FriendlyId}_<variation property name>’ to create unique name.
ListOrderintegerNoN/AN/AN/A
NamestringYesItemVariationComponent<variation property>stringThe name of the variation property, not the value of the variation property.
DefaultValuestringNoN/AN/AN/A
RequiredbooleanNoN/AN/AN/ASet to true.
AllowOpenTextbooleanNoN/AN/AN/Afalse by default.
DefaultOptionIDstringNoN/AN/AN/A
DefinesVariantbooleanNoN/AN/AN/ASet to true.
xpobjectNoN/AN/AN/A

Spec Options

OC PropertyData TypeRequiredXC Entity/ComponentXC PropertyData TypeNotes
specIDstringYesSellableItem
ItemVariationComponent		FriendlyId
<variation property>		stringSee Spec Data Mapping ID property.
IDstringNoItemVariationComponent<variation property>stringThe value of the variation property.
ValuestringYesItemVariationComponent<variation property>stringThe value of the variation property.
ListOrderintegerNoN/AN/AN/A
IsOpenTextbooleanNoN/AN/AN/Afalse by default.
PriceMarkupTypestringNoN/AN/AN/A
PriceMarkupnumberNoN/AN/AN/A
xpobjectNoN/AN/AN/A

Spec Product Assignments

OC PropertyData TypeRequiredXC Entity/ComponentXC PropertyData TypeNotes
SpecIDstringYesSellableItem
ItemVariationComponent		FriendlyId
<variation property>		stringSee Spec Data Mapping ID property.
ProductIDstringYesSellableItemFriendlyIdstring
DefaultValuestringNoN/AN/AN/A
DefaultOptionIDstringNoN/AN/AN/A

Variants

OC PropertyData TypeRequiredXC Entity/ComponentXC PropertyData TypeNotes
productIDstringYesSellableItemN/AN/A
variantIDstringYesItemVariationComponentIdstring
IDstringNoItemVariationComponentIdstring
NamestringYesItemVariationComponentDisplayNamestring
DescriptionstringNoItemVariationComponentN/AN/A
ActivebooleanNoItemVariationComponentDisabledbooleanValue to be inverted. Set to false if not a valid variant.
ShipWeightfloatNo[ItemSpecificationsComponent]Weightdouble
ShipHeightfloatNo[ItemSpecificationsComponent]Heightdouble
ShipWidthfloatNo[ItemSpecificationsComponent]Widthdouble
ShipLengthfloatNo[ItemSpecificationsComponent]Lengthdouble
Inventory.QuantityAvailableintNoInventoryInformationQuantityintIf transitioning a single inventory set/ record per product.
xpobjectNoN/AN/AN/AXC composer views and programatic components can be added to xp as needed.
xp.TagslistNoSellableItemTagslistConvert to list of string using name property only.

References