Blog

  • Debugging xConnect with xConnectHelper

    Debugging xConnect with xConnectHelper

    Setting up and testing xConnect can be a challenge – especially in on premise setups. Are all the certificates set up? Are all databases up and connection strings correct?

    Initiially, tracking, processing and reporting can be somewhat of a black box because to see it working correctly i.E. to test experience profile functionality, you’ll have to first write code to identify a contact through a form submit, start a session with interactions, trigger identification and wait for your session to expire and wait for processing / reporting in order for your contact to show up in the expirence profile manager.

    To aid with this, I have ported my trusted little xDB Helper over to Sitecore 9+ / xConnect. Because of the many API and conceptual changes, this was a re-write from scratch. The source code is available on Github.

    How does the helper help?

    After installing, you can access the page through

    /sitecore%20modules/Web/xConnectHelper/xConnectHelper.aspx?key=<yourkey>
    

    On this page you can…

    • View status information on the current Tracker config and collection service connection
    • View basic contact facets
    • Set one or multiple identifiers for the current contact
    • Set basic contact facets
    • Flush current session which immediately triggers processing

    xconnect-helper-2

    Some possible use cases:

    I want to check the xConnect connection:
    If there is a connection problem, this will be immediately shown on the xConnectHelper page. Also, if the tracker is not active becaues of config or license issues.

    xconnect-helper-1

    I want to check if data is written to the collection and reporting db:
    Interact with your website and then hit flush current session. Processing of your contact and interactions will immediately start and data should be visible in the databases within seconds.

    I want to test contact search / experience profile:
    The experience profile manager only lists identified contacts. You can set an identifier for your contact, then flush session. Your contact should show in the list within a few seconds.

    How to install

    1. Clone Repo
    2. Build and publish to your site.
    3. Set a hard-to guess key in App_Config/Modules/xConnectHelper/xConnectHelper.config
    4. Access your page through /sitecore%20modules/Web/xConnectHelper/xConnectHelper.aspx?key=yourkey

    Important Note: Because this tool is made to debug current contacts and interactions in-session, it can not be used as a regular Sitecore admin page and is therefore publicly accessible (ony restricted by your access key). You are responsible for setting appropriate measures to regulate access to this page.

  • Watch your IDTables when upgrading

    Recently after a major Sitecore upgrade, we ran into issues with items that have been synced from a DAM through a data provider. These items suddenly had different Guids than before which led to many broken links.

    The reason for this took some time to figure out:

    This dataprovider was using the IDTable to maintain consistent Guids. While on the old version, the IDTable was stored in the master database, on the new CM server, it was pointing to the web database.

    <!--  ID TABLE  -->
    <IDTable type="Sitecore.Data.SqlServer.SqlServerIDTable, Sitecore.Kernel" singleInstance="true">
    <param connectionStringName="web"/>
    <param desc="cacheSize">2500KB</param>
    </IDTable>
    

    The reason for this lies in the roles-based configuration.

    <add key="role:define" value="ContentManagement" />
    

    Checking through the different configurations i find the configs as follows:

    Role IDTable points to
    Standalone master
    ContentManagement web
    Processing master
    Reporting master
    ContentDelivery web

    The config for the ContentManagement role seems like a bug to me. We patched our config accordingly to have the IDTable saved in master again for this role.

    <configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/">
      <sitecore role:require="ContentManagement">
    	<IDTable type="Sitecore.Data.$(database).$(database)IDTable, Sitecore.Kernel" singleInstance="true">
    		<param connectionStringName="web">
    			<patch:attribute name="connectionStringName">master</patch:attribute>
    		</param>
    	</IDTable>
      </sitecore>
    </configuration>
    

    Look out when you are upgrading or switching roles in conjunction with DataProviders! I hope this saves you some time.

  • 10 issues you might run into when upgrading to Sitecore 9

    In this post I’ll list 10 issues that – depending on your particular setup, you might also run into when upgrading to Sitecore 9. They are all easy to fix, so I hope this post saves you some time.

    1. Proxy Data providers

    
    Could not find configuration node: proxyDataProviders/main
    
    

    If you have configured extra databases other than core, master or web i.E. additional publish targets, your old config will need to be updated. Remove the proxyDataProvider  section from your database config node. Additionally you’ll need to define PropertyStoreProvider and eventing/eventQueueProvider. See this Stackexchange post for details

    2. Compilation errors because of ProxyDisabler()

    Anything to do with proxy items has been deprecated and removed in Sitecore 9. Any occurrence of ProxyDisabler() in your code has to be removed.

    
    using (new ProxyDisabler())
    {
    ... this will fail
    }
    
    

    Look out for third party libraries that were not compiled against the Sitecore 9 kernel. You might run into runtime errors if they include ProxyDisabler anywhere. Suggestion is to obviously only use third party libraries that are compatible with Sitecore 9.

    3. WFFM Failing Upgrade Post Steps

    If you’re upgrading WFFM from before 8.2 you might run into an exception because the upgrade post steps are not compatible with Sitecore 9 API.

    
    Executing Post Installation instructions: 'Sitecore.Forms.Core.Configuration.UpgradePostStep, Sitecore.Forms.Core'
    
    System.MissingMethodException: Method not found: 'Sitecore.Data.ID Sitecore.Marketing.Definitions.IDefinition.get_Id()'.
    
    at Sitecore.Update.Installer.Items.CustomActionInstaller.Process(IProcessingContext entry, IProcessingContext context)
    
    at Sitecore.Update.Installer.DiffInstaller.ExecutePostInstallationInstructions(String packagePath, String historyPath, InstallMode mode, MetadataView view, ILog installationProcessLogger, List`1&amp;amp;amp; entries)
    
    at Sitecore.Update.Wizard.InstallPackage.ExecutePostStep(List`1 entries, MetadataView view, IEnumerable`1 log)
    
    

    The solution is to decompile the post steps from the upgrade package, fix the code and run them separately. I’ve added a gist with an aspx page that runs the fixed code.

    4. TypeLoadException during WFFM upgrade

    After installing the Web Forms for Marketers 9.0 rev. 171003 package, your instance might fail to start with the following exception:

    
    ReflectionTypeLoadException: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.]
    
    System.Reflection.RuntimeModule.GetTypes(RuntimeModule module) +0
    
    System.Reflection.Assembly.GetTypes() +112
    
    RazorGenerator.Mvc.PrecompiledMvcEngine..ctor(Assembly assembly, String baseVirtualPath, IViewPageActivator viewPageActivator) +353
    
    Sitecore.Forms.Core.RazorGeneratorMvcStart.Start() +68
    
    

    To continue the WFFM upgrade, remove Sitecore.Forms.Core.dll and temporarily disable all WFFM configs in /App_Config/Modules to continue with the next update package.

    If this happens after you’ve installed the final WFFM update package, check that you have the correct version of Sitecore.Forms.Core.dll inside your bin folder and also check, if the WFFM configuration files are correct in /App_Config/Modules

    5. Compatibility issues with new versions of System.IdentityModel.Tokens.Jwt

    If you are using the System.IdentityModel.Tokens.Jwt library, you’re most likely using a recent one (5.1+ at the time of writing). Sitecore uses version 4.0.4 for validating the license and there are breaking changes between 4.x and 5.x in that library.

    Luckily, both assemblies can be used at the same time by adding them to a different folder than /bin and setting the following in the assemblyBinding section of web.config:

    <dependentAssembly>
            <assemblyIdentity name="System.IdentityModel.Tokens.Jwt" publicKeyToken="31bf3856ad364e35" culture="neutral" xmlns="urn:schemas-microsoft-com:asm.v1" />
            <codeBase version="5.1.5.0" href="jwt\5.1.5\System.IdentityModel.Tokens.Jwt.dll" />
            <codeBase version="4.0.40306.1554" href="jwt\4.0.4.403061554\System.IdentityModel.Tokens.Jwt.dll" />
    </dependentAssembly>
    

    6. SolrSearchIndex. No matching constructor was found on ContentDelivery

    In case the Update Installation wizard has created a Sitecore.ContentSearch.Solr.Index.Master.patch.config file during the upgrade process, this file will reference the master index which is disabled for the ContentDelivery role. Review the config changes in the .patch.config and if they are needed, apply them to the Standalone or ContentManagement role only.

    
    ConfigurationException: Could not create instance of type: Sitecore.ContentSearch.SolrProvider.SolrSearchIndex. No matching constructor was found.]
    
    Sitecore.Configuration.DefaultFactory.CreateFromTypeName(XmlNode configNode, String[] parameters, Boolean assert) +280
    
    Sitecore.Configuration.DefaultFactory.CreateObject(XmlNode configNode, String[] parameters, Boolean assert, IFactoryHelper helper) +163
    
    Sitecore.Configuration.DefaultFactory.CreateObject(XmlNode configNode, String[] parameters, Boolean assert) +72
    
    Sitecore.Configuration.DefaultFactory.GetInnerObject(XmlNode paramNode, String[] parameters, Boolean assert) +941
    
    Sitecore.Configuration.DefaultFactory.AssignProperties(XmlNode configNode, String[] parameters, Object obj, Boolean assert, Boolean deferred, IFactoryHelper helper) +527
    
    Sitecore.Configuration.DefaultFactory.CreateObject(XmlNode configNode, String[] parameters, Boolean assert, IFactoryHelper helper) +320
    
    Sitecore.Configuration.DefaultFactory.CreateObject(XmlNode configNode, String[] parameters, Boolean assert) +72
    
    Sitecore.Configuration.DefaultFactory.CreateObject(String configPath, String[] parameters, Boolean assert) +693
    
    Sitecore.ContentSearch.ContentSearchManager.get_SearchConfiguration() +305
    
    Sitecore.ContentSearch.SolrProvider.SolrContentSearchManager.Initialize() +21
    
    

    7. Query.MaxItems affecting query results

    This is a hard one to find: The default limit for items returned by Query.MaxItems is 100. In some previous versions, this had been patched to to 260 through Sitecore.ExperienceExplorer.config. If you happen to upgrade from one of those previous versions and are using queries, check your code if it still works as expected.

    In some cases your queries might only return 100 items now instead of 260 and this might result in different behaviour. Thanks to Christoph Brunner for finding this one.

    Just the usual reminder around queries: It is usually a bad idea to increase the Query.MaxItems value as this can drastically reduce performance. Always consider carefully if it is a good option to use Queries.

    8. SessionProvider issue when Tracker is enabled

    On our solution, the UpdateInstallationWizard had created a Sitecore.Analytics.Tracking.patch.config as follows:

    
    <?xml version="1.0" encoding="utf-8"?>
    <configuration xmlns:p="http://www.sitecore.net/xmlconfig/" xmlns:s="http://www.sitecore.net/xmlconfig/set/" p:p="1">
      <sitecore>
        <tracking>
          <sharedSessionState s:defaultProvider="mssql">
            <providers>
              <add name="InProc">
                <p:d />
              </add>          
              <add name="mongo">
    			<p:d />
    		  </add>
              <add name="mssql" p:after="*[1=2]" s:type="Sitecore.SessionProvider.Sql.SqlSessionStateProvider, Sitecore.SessionProvider.Sql" s:sessionType="shared" s:connectionStringName="session" s:pollingInterval="2" s:compression="true" />
            </providers>
          </sharedSessionState>
        </tracking>
      </sitecore>
    </configuration>
    
    

    This led to the following exception:

    
    Exception: System.NullReferenceException
    Message: Object reference not set to an instance of an object.
    Source: Sitecore.SessionProvider.Sql
    at Sitecore.SessionProvider.Sql.SqlSessionStateProvider.Initialize(String name, NameValueCollection config)
    at Sitecore.Configuration.DefaultFactory.GetProviders[TProvider,TCollection](List`1 nodes)
    at Sitecore.Configuration.DefaultFactory.GetProviders[TProvider,TCollection](String rootPath, TProvider&amp; defaultProvider)
    at Sitecore.Configuration.ProviderHelper`2.ReadProviders()
    at Sitecore.Configuration.ProviderHelper`2.get_Provider()
    at Sitecore.Analytics.Data.Dictionaries.DictionaryData.Session.SessionDictionaryData..ctor(SharedSessionStateConfigBase configuration)
    
    

    As we wanted to use the default SQL Session state configuration, the simple solution was to just delete this .patch.config file.

    9. Good performance but slower startup

    Overall performance of the 9+ versions is very good but the startup time of Content Editor, Experience Editor seems to be a bit longer than with previous versions. Also on our solution, WFFM forms take quite some time when loaded for the first time after an App pool restart. The subsequent loads are quick again.

    To provide the best experience for editors and visitors of your sites it is a good idea to implement a warmup of ContentEditor, Experience Editor and your forms on the ContentDelivery instances.

    Checkout Jeff L’Hereux’s Warmup Script for Habitat Home as a starting point.

    10. SOLR Index config issues

    
    Could not find add method: ExcludeTemplate (type: Sitecore.ContentSearch.SolrIndexConfiguration)
    
    

    This will most likely only happen to you if you’re upgrading from a version older than 8.2. The config structure has changed so you’ll need to update all of your custom index configs.

    There are example configs in App_config/Include/Example. Also check this great summary of changes by Radoslaw Kozlowski

     

  • Customize Solr Managed Schema in Sitecore 9+

    In Sitecore 9+ we can now populate Solr managed schema through the Sitecore control panel. It is still possible to use the legacy schema.xml files, but it is actually a lot more comfortable to be able to just deploy the schema with one click.

    Sitecore currently has some basic schema which it populates and has created the contentSearch.PopulateSolrSchema pipeline as extension point. Mikael Högberg has written a post describing this extension point and I’ve built the following simple processor based on that. It reads Managed Schema update commands as XML and applies them to the indexes you have specified (pipe-separated)

    Feel free to use and extend it.

    Caveat: Solr only accepts the add-field-type command if the field is not contained in the managed schema yet. The replace-field-type will only work if it is already there. This is a bit annoying and could be worked around by checking if the field exists in the schema in code and then changing add to replace and vice versa.

    Also, the schema update will fail silently if there are any errors in the update commands. I’ve found it helpful to first test the update commands as JSON against Solr’s API directly and then convert that JSON to XML.

    Update 05.11.2018:
    Added a add-or-replace-field-type command which is automatically replaced to “add-field-type” or “replace-field-type” depending if the field type already exists in the schema.

  • Santa’s little xDB Helper

    Sorry for the xmas pun but hey, it’s December.

    Back to topic. Debugging xDB issues on a remote environment can be very difficult. Is the MongoDB connection OK? Is tracking configured correctly? Are contacts being created and identified? Is my data pushed to the correct places, when my session ends? Why is GeoIP detection not working?
    (more…)

  • Allow lookup field queries to respect site root

    This post shows two ways of using relative lookup field queries in a multisite environment. (more…)

  • Things to look out for when setting up xDB

    This post highlights some potential pitfalls when setting up xDB on a Sitecore 8.x environment. (more…)

  • Why silent saving with a processing server is a bad idea

    This post highlights why silent saving, EventDisabler or using BulkUpdateContext can lead to inconsistent data on a multi-CM environment i.E. when using a processing server. (more…)

  • Using IP Geolocation and SSL on Azure

    I want to share some lessons learned from a recent migration of a Sitecore infrastructure to Azure. If you are running a load balanced environment over SSL and relying on IP Geolocation then you need to look out for the following gotchas:
    (more…)

  • Is there a difference between master and web?

    This question has been in the back of my mind for as long as I’ve been working with Sitecore. Is there an internal difference in the way items are stored to and retrieved from these databases? I put some time into investigating:
    (more…)