What to consider to automate your custom Sitecore release pipeline?


In the modern world, we all like automating repeatable tasks; one of them is the continuous delivery of Sitecore solutions. This post will show you a generic checklist about what to consider when building your Sitecore deployment pipeline for CM and CD servers. (Custom xConnect implementations and Solr server configurations are not in the scope of this article), decoupled from any tool like Azure DevOps, TeamCity, Jenkins, Octopus, etc. Let’s agree on the following requirements:

We have 3 environments: DEV, UAT, and PROD.

DEV:

  • This is our internal environment to test features before it goes to the client’s hands.
  • We want to be able to deploy anytime after a feature implementation is done.
  • Rollback ability is not a requirement.

UAT:

  • The client wants to test the next package of features on this environment and needs a one-week buffer to finish the tests.
  • The new package of features should be delivered every 2 weeks.
  • Rollback ability is not a requirement.

PROD:

  • The client wants to roll out the new package of features every 2 weeks; the schedule is shifted with one week of UAT deployment.
  • The new package of features should be delivered every 2 weeks.
  • Rollback to the previous version is a requirement.

When we are talking about deployment of a Sitecore solution, it means the deployment of: 

  • Base Sitecore package, which does not contain any custom implementation, only the Sitecore package with our basic configurations such as connection strings to databases, identity server, xConnect services, search engine, etc. 
  • The custom implementation consists of built assemblies, static files, configuration files, Sitecore configuration patches, Razor views, etc. 

1. Clean, base Sitecore package

My suggestion is to handle the 2 resources differently. The Sitecore base package can only be changed if the infrastructure changes or with a Sitecore upgrade, which happens infrequently. On the other hand, I suggest using parameters for configurations that differ on each staging environment, e.g., connection strings. By using parameters, you can reuse the same Sitecore package for different stage environments. I suggest using a similar scaled setup for each staging environment to have the closest similarity to PROD. For example, if you have a scaled environment with more content delivery servers on PROD, use a similar setup on DEV and UAT but with fewer resources and with fewer content delivery servers.

2. Custom implementation package

This package usually consists of the following:

  • Assemblies
  • Razor views
  • Static assets, like JavaScript and CSS
  • Sitecore configuration patches
  • Serialized items

The first three types of resources from the list above are static in the manner of deployment, means these should be deployed as it is built.

Sitecore configuration patches can be static – same values on each environment – or dynamic – using different values based on the environment. Static configuration patches are simply part of the version control, e.g., patches for custom processors and link providers. Dynamic configuration patches mean values changed by the deployment and can have different values on each server or role or environment, e.g., site configurations. To achieve that, multiple ways are available to do that:

  1. Replace default values of the existing Sitecore configuration patches with a scripting language, which is part of your release pipeline.
  2. Use Sitecore custom rules to switch between configurations, for example, based on the stage environment.

In both cases, you can decide where to store these variables of the different stages. You have the following options:

  1. In the same git repository where the custom implementation is: developers has access to read and write
  2. In a git repository separate from the custom implementation: developers has limited access to it
  3. In the release pipeline service, you use – Jenkins, TeamCity, Azure DevOps, Octopus, etc.: developers usually don’t have access to it but can have limited access

The developer access levels I mentioned above are typical limitations for these options, but the actual access level depends only on the configuration of your solution.

This package also includes serialized items that need to be synced to the Sitecore databases (core or master) when a deployment happens. The most popular available tools to serialize items are Sitecore Content Serialization (from Sitecore 10), Unicorn, and TDS. We separate them into 3 groups:

  1. Development-related items – templates, branches, renderings, layouts, custom extension items in the core database, etc.: These items we intend to deploy to the databases with each deployment and overwrite the previous versions of the items
  2. Content items – root items, start items, global setting items, global folders, etc.: These items we intend to deploy once if the item does not exist yet
  3. Test content items - test items to accelerate testing on DEV: These items we want to deploy once, only to the DEV environment

How to handle support patches?

This is an important topic to discuss because I have never seen a Sitecore project without any Sitecore Support patches. These are bug fixes provided by the Sitecore Support team, which can consist of the following files and items:

  • Compiled new assemblies, usually with names like Sitecore.Support.123456.dll
  • Compiled existing Sitecore assemblies, like Sitecore.Kernel.dll
  • Configuration patches, like /App_Config/Include/Sitecore.Support.123456.config
  • Item package containing new items
  • Item package containing modified existing items

You have two ways, how to include these changes in your release pipeline.

One option is to include it in your release pipeline, separately from the custom implementation. The advantage of this way is that the DevOps team has complete control over the Sitecore Support patches. The disadvantage, however, is that patches are not transparent for the Sitecore developer team, and locally they don’t have the same patches installed, or a custom separated deployment script is needed to deploy the patches locally.

My suggestion is to handle these in a separated project, whether you use Helix principles or not. My preferred way is to include these files and serialized items in your custom Sitecore Visual Studio solution and let MSBuild publish them. This way, the Sitecore Support patches are transparent for the developers, and they can deploy the same patches locally easily without any custom publish script.

It makes sense to mix two ways in some cases, and the team can decide where to put a Sitecore Support patch based on what it contains. For example, infrastructure-related patches can be placed in the release pipeline, but functionality-related patches in the custom implementation solution.

Database handling

As the content databases (master and web) can grow big, especially in a PROD environment,  it is also crucial to handle database backups. Massive databases need huge free disk space on the database server to create backups, but this is not an issue because the hard drive is relatively cheap. Whether you need database backups or not, you most definitely need it on a PROD environment to roll back anytime to a stable point of the database in case of database corruption.

The question is, when to do the backups?

Regular nightly backups are recommended if the content changes are frequent – hundreds of item change on a day by editors or automated synchronization. But do you need it as part of the release pipeline? I say no because creating backups can take an hour(s) with big content databases with multiple sites. Therefore, creating backups will be the bottleneck of your release pipeline and will increase the PROD deployment from minutes to hours. 

Then how to handle rollback?

It’s all about timing your PROD deployment. If you have automated nightly database backups, then schedule your PROD deployment for the morning and not the afternoon to lose less content in case of a rollback. As a rollback should happen infrequently, you should handle this case in place not to overcomplicate the deployment of your Sitecore solution.

Conclusion

Fortunately, Sitecore solution deployments can be standardized because the deployment processes are similar to each other on all Sitecore projects. Hopefully, you can find out your automatic deployment pipeline and integrate into any tool you chose with this guideline.

Are you interested in how to do a Zero-Downtime deployment using Azure Kubernetes Service?

Do you need help to automate your Sitecore release pipeline? Contact us!