“Returning from this year’s Sitecore Symposium in Orlando, I would like to share some improved/new insights that have come to me due to attending some great sessions. Some of this blog post will assume you are using a Sitecore on Azure PAAS solution. Most sessions I attended were related to continuous integration, continuous deployment, architectures, scaling, high availability and disaster recovery. The sessions from Bas Lijten, Dennis Weston and Rob Habraken inspired me most and therefore I will elaborate/reflect on most of their content."
Database deployment strategies: Do we need to have multiple web databases per region/slot? Does one web database per region fulfill all needs? In our case we are used to copy production web databases, connect them to staging slots and perform unicorn syncs. Would it be sufficient to just use one web database and perform the unicorn sync against the live web database? Point-in-time restore within SQL Azure assures rollback possibilities. Is there a need of two web databases per region, production/staging slot, add them to your publishing targets within the CM to always keep them (both) up to date with the latest content? I am not a hundred percent sure what strategy would suit your solution best but it does make sense to overlook your current strategy since all choices affect maintainability and total throughput time while deploying.
Multiple regions / multiple core databases: I have seen multiple architectures including more than one core database (per region). I do wonder whether the core databases where actively used by the CD in that region or if they were drawn in the architecture because of geo-replication/fail-over purposes. It would be great if Sitecore adds documentations/recommendations on using your core database in more than one region, since that is (besides the X-connect HA setup) one of the major SPOFs in the architecture of a lot of Sitecore solutions as I have experienced/seen.
HA/Geo-replicating X-connect: In case of regional outages within Azure you want to make sure your website doesn’t go down. A lot of architectures assure this is properly setup by using multiple content-delivery nodes with corresponding databases and a loadbalancer/trafficmanager setup to lead traffic towards healthy endpoints. When using Sitecore 9 XP (paas) scaled there is a huge set of App Services/databases and corresponding configuration for setting up X-connect. It depends on the loss you are willing to take but there are a few strategies/considerations you should look into when defining your DR/HA.
- Are all my databases geo-replicated to a backup region (using elastic pools would reduce costs). - Are all my databases configured for fail-over and are the DNS endpoints configured in all connection strings? - Are you willing maintain/pay for an additional architecture (cold standby) of X-connect in another region or is a ready to deploy arm template sufficient?
Distribution of traffic: Throughout most of the architecture drawing I have seen on Symposium the most used solution regarding the distributing of traffic involves nested traffic manager setups in relation with Application Gateways/Load balancers. While obviously I did not speak to everyone, I did not hear a lot about the Azure Front Door service. Perhaps due to the fact it is still in public preview. Nevertheless I suppose the service it worth mentioning since it has some nice feature for enhancing your solution (besides the fact that is it a global service).
Protect your Sitecore solution using a WAF: From what I understand the current Azure Application Gateway WAF is not a good match in front of your CM (CD seems to be working just fine). There are issues with OWASP protection (blocks) related to expected behavior within CM activities. We have experienced similar issues within our WAF solution, although this is not an Azure AG(WAF). Looking at the Azure AG (WAF) functionality overall I would advise companies to invest in a solid WAF solution given the basic configuration options in Azure AG (WAF). There are far more configuration possibilities within solutions from Barracuda, F5 or Fortinet. That your websites ought to be running behind a WAF seems to be a no-brainer.
Scaling your CM: It seems not to be an issue anymore to use multiple CM App Services when using the Sitecore publishing service. When you’re not using the publishing service and still want to out-scale your CM by multiplying with an additional App Service(s) it only seems to be a requirement to configure 1 CM to have the publishing role. As mentioned in the documentation: here. I suppose this is not anything new, but in my understanding there were more arguments not to have multiple CM roles. Just wanted to mention in case you had the same understanding.
Accelerate your deployments: While deploying your initial infrastructure it makes sense to configure all your connection-strings within the App Service (+slots). It also makes sense to apply additional App Settings that you would normally use within your msdeploy parameters. By striping them from your deployment it makes your vanilla deployment less complex and more maintainable. For more information check this blog. It does require you to diff your parameters when upgrading to newer versions however. When you have striped your packages it seems be efficient (time-wise compared with arm template) to deploy the binaries using the build in Azure DevOps task ‘Azure App Service Deploy’. When you have your package for release deployments you could consider combining your vanilla, modules and business scwdp packages during build. This will reduce your amount of tasks and result in faster deployments.
What really made my day was the fact that it is possible to parallel execute Azure DevOps tasks by configuring custom conditions. Check Microsoft docs for syntax’s here. The example that was given in the session was custom condition: “and(eq(variables[‘region’],‘west’), eq(variables[‘role’], ‘cd’))”, meaning the task would only execute whenever variables were given the values ‘west’ and ‘cd’, by adding this condition to multiple tasks they will execute in parallel. I have not tried it myself just yet, but since (I believe) all tasks include the section Control Options it would be quite easy to control your task sequence to tweak and control parallel executions. Instructions on how to achieve this are described in detail: here.
Another outcome of deploying to Azure App Services was that it takes less time deploying to an App Service within the Standard pricing tier vs the Premium pricing tiers. It is even faster to deploy to a S0 vs a S3. This is most likely a result of all underlying infrastructure/configuration provisioning/data replicating etc. provided when using higher pricing tiers.
Be aware and make note of the following: - While swopping slots and using sticky App Settings - settings are being applied after swop has been initialized and completed. Causing cold startup. Use ‘Swap with preview’ instead. - When using Ip restrictions on Azure App Services - while not adding 127.0.0.1 to the list of allow ip addresses. Causing cold startup. - When allowing ‘SQL Azure services’ to your Azure SQL server implicitly means allowing ‘ALL’ Azure services to access your SQL server, regardless of subscription. - Using Azure CDN in combination with Sitecore will (in average) reduce 40% load otherwise consumed/served by your platform. In potential this could save money when downgrading resources. Will almost definitely make multi-regional websites perform faster. - If you have not done already, developers: “there is a debug mode within Sitecore Experience Editor”. Go to Presentation -> Edit -> Debug. Optimize your code when possible! - Turning on ‘Dilithium’ when using Unicorn Sync will speed up the overall process. Have not tested this myself.
Tools worth looking into: - CypresUI - UI Regression - JEST - React (js testing) - build/release pipeline - Load Storm - Performance testing / Response Time / Multi-region testing - Statuscake - Check response times / Site status - Sumologic - Monitoring - BackStopJS - build/release pipeline - visual regression testing UI by comparing DOM screenshots
In addition to the tooling we already use within our pipelines:
- SonarQube/Sonarcloud - code analysis. - WhiteSource - security check solutions open source components. - Loadtests using webtest files and Azure DevOps
Conclusion: We keep improving our solutions, removing all bottlenecks. Amazing to meet peers that are in the same field of work. Really looking forward what JSS, SXA, Cortex and the acquisition of Stylelabs will bring us in the near future and how that will impact our current solutions/environments.
Last but not least: Just watch these videos and appreciate/support what this guy is doing! https://www.notimpossible.com/