Mark's Sitecore Blog

Blog

  • PublishAgent skipping all items when CheckSecurity is activated

    We ran into an issue when trying to publish the entire site on a regular basis using PublishAgent. We have it set up to publish the entire site every hour.

    <agent type="Sitecore.Tasks.PublishAgent" method="Run" interval="01:00:00">
 <param desc="source database">master</param>
 <param desc="target database">web</param>
 <param desc="mode (full or smart or incremental)">smart</param>
 <param desc="languages">en</param>
 </agent>

    We noticed though, that new items weren’t being published. The publish log showed the following:

    ManagedPoolThread #14 13:12:13 INFO PublishAgent started (source: master, target: web, mode: Smart, languages: en)
ManagedPoolThread #14 13:12:13 INFO Asynchronous publishing started. Job name: Publish to 'web'. Languages: en
10580 13:12:13 WARN SitePublish detected. PublishContext was overridden with DisableDatabaseCaches=True.
10580 13:12:13 INFO Starting [Publishing] - ProcessQueue
10580 13:13:04 INFO Finished [Publishing] - ProcessQueue in 50986 ms
10580 13:13:04 INFO Publish Mode : Full
10580 13:13:04 INFO Created : 0
10580 13:13:04 INFO Updated : 0
10580 13:13:04 INFO Deleted : 0
10580 13:13:04 INFO Skipped : 36712

    Publishing manually was working fine.

    After searching for a while, we found out the reason for PublishAgent skipping all items was quite obvious: We have set Publishing.CheckSecurity=true to make sure users could only publish the items they were allowed to. Unfortunately PublishAgent runs as sitecore\Anonymous who of course doesn’t have sufficient rights.

    To solve this problem, you can implement a class that inherits PublishAgent and overrides the Run method. Then use UserSwitcher to switch to a user that has the appropriate rights.

  • Use System.Boolean when indexing checkbox fields

    I faced the issue that checkbox values were not being updated in the index correctly. When I unchecked a checkbox and saved, the index would still have “1” stored as value for my field. To prevent this issue, use Sytem.Boolean as type instead of System.String.

    <field fieldName=”MyCheckboxField” storageType=”YES” indexType=”TOKENIZED” vectorType=”NO” boost=”1f” type=”System.Boolean” settingType=”Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider” />

    Internally, lucene still stores “0” or “1”, but when using System.String, values of false would be considered as “nothing” when updating the index, leading to not updating the value of the field. Changing to System.Boolean made it work.

     

  • Sitecore Start Menu Shortcuts not showing

    We noticed, that it is not possible to create Application Shortcuts anymore in SC 7.2. All our newly created shortcuts wouldn’t be displayed, while old shortcuts were visible.

    The reason for this lies in the changed raw value inside of link fields. Internal links are saved as GUIDs instead of paths:

    <link linktype="internal" querystring="" target="" id="{7EADA46B-11E2-4EC1-8C44-BE75784FF105}" />

    When we change the raw value of the link back to url=”…”, the link gets displayed again:

    <link text="Content Editor" linktype="internal" url="/Applications/Content editor" anchor="" title="" class="" target="">

    Obviously, Sitecore expects urls inside shortcuts to be paths instead of GUIDs. Unfortunately it is not possible to create internal links with url=”” through the link dialog. You will have to edit the raw values manually.
    We have reported this bug to Sitecore Support.

  • View running Sitecore background jobs

    I’ve created a very simple but useful admin page recently that shows all background jobs (indexing, publishing etc.) currently running on the Sitecore instance. So if you’re ever wondering why the instance is running slow or why jobs are being queued you can take a peek at /sitecore/admin/ViewJobs.aspx

    You might want to apply some styling though 😉
    Here you go:

    By the way: Sitecore rocks lets you do the same thing too. But I found an admin page would be more handy in production.

  • Determining the Context Index when performing bucket queries

    We have faced a problem when working with multilists and custom indexes. Some of our multilists would not contain any data even though the lucene queries would have matches when querying the index directly. We found that the problem had to do with the way Sitecore determines the index to perform the bucket query on.

    Reason

    When working with custom indexes in the Backend (i.E. bucket queries, multilist with search,…), Sitecore has a method to determine which index is most appropriate to use for the base item (i.E. the bucket container) you’re querying on.

    In version 7.2, basically, Sitecore checks, which indexes are defined for the context database and then checks if the base item is included in the respective index by checking include/exclude templates. The first matching index is used. It is important to know that the root paths configured on the crawlers don’t have any affect on this. That means in some cases an index may be selected that doesn’t actually contain any information about the data i’m querying. I bug imo.

    Workaround

    To work around this you have the following options:

    a) Index config files are read in alphabetical order. You might be able to move around your custom indexes to let Sitecore select the appropriate index.
    b) Hook into the contentSearch.getContextIndex pipeline and implement your own logic.

    Since a) is pretty straighforward but might not work for all scenarios, I’ll show an example of b):

    I wrote a simple hook that allows me to read the name of a custom index from a field on the base item (i.E. the item bucket base item):This allows me to use a field named CustomSearchIndex to optionally set the index name used for querying.

    AppliedIndexChecker admin page

    To check which context index is resolved on specific items, i have implemented a simple admin page that allows me to resolve the index based on a GUID or an item path.

    Summary

    This workaround does the job, but it would of course be nicer to have Sitecore correctly resolving context indexes. Also it would be good to be able to set the index directly in the multilist / bucket configuration. I hope there will be an improvement here in future versions.

  • Unable to create document builder – crawling exception after upgrading to SC 7.2

    After upgrading to Sitecore 7.2 I ran into this exception in the crawling log when trying to index PDF documents:
    Unable to create document builder (). Please check your configuration. We will fallback to the default for now.

    The solution was to add the document builder type to my custom index configuration:

    <documentBuilderType>Sitecore.ContentSearch.LuceneProvider.LuceneDocumentBuilder, Sitecore.ContentSearch.LuceneProvider</documentBuilderType>

    After that, reading PDF content through my iFilter provider worked fine again.

  • NullReferenceException in custom Index after upgrading to Sitecore 7.2

    After upgrading to Sitecore 7.2 we got the following exception when trying to build our custom indexes:

    System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.NullReferenceException: Object reference not set to an instance of an object.
 at Sitecore.ContentSearch.LuceneProvider.LuceneIndex.Reset()
 at Sitecore.ContentSearch.LuceneProvider.LuceneIndex.PerformRebuild(IndexingOptions indexingOptions, CancellationToken cancellationToken)
 at Sitecore.ContentSearch.LuceneProvider.LuceneIndex.Rebuild()
 --- End of inner exception stack trace ---
 at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
 at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
 at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
 at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
 at (Object , Object[] )
 at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
 at Sitecore.Jobs.Job.ThreadEntry(Object state)

    The resolution to this is quite simple. Sitecore has included <initializeOnAdd> to the index configuration. If this field is missing in the configuration, building the index fails. Just make sure you have the following line to your custom index configuration:

    <myCustomIndexConfiguration type="Sitecore.ContentSearch.LuceneProvider.LuceneIndexConfiguration, Sitecore.ContentSearch.LuceneProvider">

<indexAllFields>true</indexAllFields>

<!-- Needed since SC7.2 -->
<initializeOnAdd>true</initializeOnAdd>
 ....
  • Enhance the publishing dialog

    Enhance the publishing dialog

    On large sites with many authors working concurrently it can be very annoying to wait for your turn to publish. Also, not being able to cancel a publish job is very annoying.

    Luckily, Ivan Sharamok has created a fantastic module which extends the publish dialog and lets you see which jobs are running and also cancel them if needed.

    Download the Shared Source Module here.

    Btw. I’ve tested it with Sitecore 7 too and it works.

    Looking forward to those publishing enhancements that are announced to be part of Sitecore 7.2.

  • Sitecore multi language challenges

    Sitecore multi language challenges

    We have recently completed a project for an international Site featuring a large amount of content in many languages. Not only did we have to support several base languages – it also was a requirement to  be able to customize content for specific countries. This lead us to use Sitecore’s built in mechanism of language/country pairs i.E.” en-US” or “en-GB” in combination with the Language Fallback Shared Source module to fall back to the base language i.E. “en” in case there was no specific translation.

    With LanguageEmbedding set to “always” – this way we had unique URLs for every content page in every language i.e. mydomain.com/en-US/ without using cookies for keeping track of the selected language / country.

    So far so good. But when authors started entering content, some issues started showing up. Soon we had over 140 Sitecore languages set up in the system and the content-tree quickly contained thousands of items.

    This led to…

    Issue #1:Uploads to Media Library becoming extremely slow

    By default, Media.UploadAsVersionableByDefault is set to “false” leading Sitecore to upload media items unversioned. This is usually a good thing but unfortunately in this case, Sitecore creates a version for each language defined in the database even if we don’t need them. Having media items versioned wasn’t an option because Authors would have had do upload the media in every language.
    Luckily Yogesh Patel posted a workaround to this problem which pointed us in the right direction but we couldn’t really adapt it because it was not possible to override the MediaCreator.CreateItem() method which countains a loop creating a version in every language.

    A simple trick helped us out here instead:
    We set the versionedTemplate to system/media/unversioned/… for each mediaType and set Media.UploadAsVersionableByDefault to true in Sitecore Config. This way, the media items are created with shared fields but the CreateItem() method only creates one language version for each uploaded media item. Uploads were instantly very fast again. Of course this solution prevents using versioned media items with versioned fields but we do not need them.

    Manually removing all un-needed language versions from the database tables helped us speed up things even more as millions of unused records could be removed.

    Issue #2: Large Cache sizes

    Sitecore item and prefetch caches tended to use several gigabytes of space. This was because when fetching an item in a specific language i.E. “en-US” it would add this version to the cache even if there was no specific content for that item. That way a lot of items were added to cache even if they didn’t contain any content, leading to a general slow down while editing. (Clearing items out of the cache while saving was taking a lot of time)

    What helped was a tweak on the LanguageFallback module’s GetItem() method. Whenever an item was requested in “en-US” and the corresponding fallback version in “en” was returned, our cache would save the connection and next time return the “en” version immediately without even calling GetItem() on the “en-US” version. That way we could avoid a lot of cache entries. The Item cache size decreased from several GB to just a few hundred. Of course we needed to clear our fallback cache in case an “en-US” version ever gets created by authors which was not trivial.

    Issue #3: Fix sorting in language dropdown

    Having over 140 languages in random sorting was not very good for authors to work with. Fortunately, we followed Igor Strikovs solution to implement custom sorting for languages letting us display all base languages at the top. Thanks Igor for that post.

    Conclusion

    As you see, building sites with a large amount of languages can lead to issues and currently there seems to be no out of the box solution for dealing with requirements like this. We couldn’t find any other way of solving this and still having clean “RESTful” URLs without using cookies to store the currently selected country.

    Alternatives

    An other way to achieve the above would probably be to extend Sitecore’s LinkManager and manage language fallbacks while resolving languages instead of a per-item basis. I.e. resolving “en-DE” would check if that language exists in the database and if not, it would directly switch the context language to “en”. That way we would only need to create Sitecore Languages when they were really needed to localize content. Unfortunately, extending LinkManager isn’t that trivial.

    Any thoughts on this are greatly appreciated.