This post follows on from Part 1.

The symptom:

I guess this is pretty self-explanitory, but it depends how you are arriving at this problem. You might be using real accounts, virtual users, or whatever. The effect is that your extranet user, i.e. public website user, appears to lose their session some time after logging in. They are effectively logged-out. You won’t notice this with anonymous users, only users that have logged in. This may also initially appear as a security problem, perhaps the user is suddenly unable to access content they should have access to.

The problem:

Pretty much the same as in Part 1.  If the Sitecore staging module is set to the recommended “Full” cache clearing mode, and a publishing operation occurs, all extranet users are logged off after the staging module .xml work files are processed. As explained previously, there is a short delay between the publishing operation starting and the web service actually being called from the master.

The cause:

When the Sitecore staging module is set to the recommended “Full” cache clearing mode, the slave server clears all caches simultaneously, but it also calls Context.ClientData.RemoveAll() which effectively clears the session for extranet users.  This affects the current available version of the staging module  for Sitecore 6 as of July 2009 (v5.4.0 rev 080625). In contrast, the current version for sitecore 5 has been fixed as of July 17th, and should no longer have this issue.

The solution:

The staging mode in “partial” cache clearing mode does not invoke Context.ClientData.RemoveAll(), so the simple solution is to use the “partial” cache setting. Again the problem is as stated before – this is not the recommended setting, and this is reiterated in the troubleshooting section. There is also no explanation of the scenarios where partial cache clearing is not enough, just that in the “partial setting does not clear the entire item cache”. I had previously thought that this would affect publishing operations that include template changes, but I haven’t explored that notion further.

Finally, despite this rather large bug being known, it is not documented at all other than in forum posts. Nor is it in the release notes.   A slightly similar but unrelated  issue (almost certainly less common too) is  covered in point 6.2.3  of the installation document.

For the record, and in case anyone does this see this post in that way, I’m not anti-Sitecore,  I like the product (otherwise I wouldn’t bother doing this), I like the level of support too (they must have some  patience!), I just think the staging module is pretty flawed in some areas. Finally, some of these issues are already known, but don’t appear in any list of ‘known issues’. This is immensely frustrating!

The Sitecore staging module is an ’official’ (i.e. supported) module that allows you to separate your live, public-facing servers from your CMS server that editors directly create content on. From now I will refer to the internal CMS server as  ’master’ and all public facing web servers as ‘slave’ – this is the same naming convention used in the staging module documentation.  A typical configuration will consist of one master server located locally within your office, and one or more slave servers located externally with a hosting company.

There are three issues that occur when using the staging module that I’m intending to eventually highlight, these are:

1. Cache clearing
2. Extranet users are logged-out after each publishing operation.
3. Publishing delay

Cache clearing

The symptom:

Your live website slows to a halt at regular intervals, depending on how often you publish. CPU utilization of the w3wp.exe process will often suddenly ramp up to 100% for some time (probably somewhere within 5-90 seconds)  and the SQL server process will be doing the same, or be worse.  After this time, your site returns to normal. Page requests to your site may hang for the entire duration and if they don’t close the browser, will eventually get a response.  This happens after each publishing operation + staging task (scheduled) publishing delay.

The problem:

After publishing content in a staging environment, the master server calls a webservice on each slave server to clear caches. The recommended setting for this is “Full” cache clearing rather than “partial”. You can verify if staging is the cause of all this by performing the following steps:

1. On the master server, check the folder /sitecore modules/staging/workdir/cache for the presence of any .xml files. If you haven’t published recently, this should be empty. (If there are loads of .xml files you probably have another problem).
2. Publish something from the master server to a slave server
3. Immediately check the /workdir/cache folder again (it should now contain one .xml file)
4. Wait for the .xml file to disappear (it’s now being processed)

As soon as the .xml file disappears, the master server should hit the webservice on the slave server to clear caches. If at this point, your site slows down and the CPU and SQL load jumps unacceptably, then this is very likely to be a problem caused by staging and the aggressive cache-clearing invoked by the ‘Full’ setting.

The cause:

 The documentation does state that ’full’ method is slow, however I think it underplays how much impact this can have. This setting is configured on the master server and is used in the webservice call to the slave after a publishing operation to clear the caches on the slave. The main difference is that ‘partial’ clears the front-end HTML cache (i.e. your rendered outputs) and the Data cache. Using this mode, your site will probably hardly blink unless you have very intensive renderings. The ‘full’ mode however does what it says in the documentation, and clears all caches, and Sitecore has a lot of them. You can see a list of these in /sitecore/admin/cache.aspx . It also does some other additional clearing operations and then does all this all again for the shell website (the sitecore shell at /sitecore is just another website). The whole effect is that web server and SQL server load can spike in a big big way. In my particular case, the resulting server load and time-delay is comparable to the same load and delay that occurs after recycling the application pool, or in other words, every time I publish the site feels like it’s rebooting, and that seems to be pretty much what is happening. The ‘rebooting’ analogy also lends itself to the #2 point about extranet users (post to follow). If you publish frequently, have a large number of items, or heavy load then this makes things much worse.

The solution:

The simple solution is the try the “partial” caching mode instead.  Sitecore say that in some scenarios items will not be published correctly and therefore only the “full” setting is recommended. This is the crux of the problem. You’re damned if you do, and damned if you don’t. I really think that Sitecore needs to come up with a better way to handle staging, either by fixing this module, or a different approach altogether. So far we’re stuck between a method that isn’t recommended and doesn’t always work, and a method that is recommended but can easily cripple your performance whenever you publish. 

More to follow…. in part two.

1. Duplicate the local working folder to another folder.
2. Open the new copy solution in Visual Studio
3. File > Source Control > Change Source Control
4. Check / Select all items
5. Choose [Unbind] for each project to remove any association between the copy folder, and VSS.
6. Enable options to show hidden and system files in Windows Explorer.
7. Delete all files ending with *.scc, *.vspscc, *.user, *.vssscc, *.suo
8. Add the copy folder into the new source control system.

Problems with this include loosing all history information. Going forwards, you’re starting at the current state with nothing to roll back to.

In my case most features worked, Edit buttons would invoke row editing mode, cancel button works fine, rowCommand event generally worked, but it wasRowUpdating that simply wasn’t working. Adding System.Web.UI.WebControls.GridView to <typesThatShouldNotBeExpanded> in web.config resolves this issue. your web.config would then look like :

<typesThatShouldNotBeExpanded>
<type>System.Web.UI.WebControls.Repeater</type>
<type>System.Web.UI.WebControls.GridView</type>
<type>System.Web.UI.WebControls.DataList</type>
</typesThatShouldNotBeExpanded>

This issue does now have a fleeting mention on http://sdn.sitecore.net/Scrapbook….. but requires login to view it.

While looking for that link, I also came across another blog post detailing the same issue and frustrations I’ve had with Gridview events that may have more detail.

Relating this to Sitecore CMS, you should be aware that Sitecore ships with the following line in web.config:

<!– AUTOMATIC DATA BIND
Indicates if the data bind function is run automatically
–>
<setting name=“AutomaticDataBind“ value=“false“ />

This means that you will need to either call Databind whenever you need it, or you can simply set this value to true.  This AutomaticDataBind setting will also cause problems with other types of databound controls, such as templated controls. It’s generally simpler to set it to ‘true’..

This is all fine and well except that it is permanently looking at the web database rather than master database. This can be observed by creating a new item, saving it, and then selecting preview. You get a 404 error, unless you publish it first.

Unfortunately this nice feature is 80% broken and can only be used for looking at published content.

Update: This was a registered bug in February 2009 with a workaround available from Sitecore support, but this still has not been released and I believe the bug still exists in Sitecore 6.1

Sitecore Could not load file or assembly ‘System.Data.SQLite, Version=1.0.48.0, Culture=neutral, PublicKeyToken=db937bc2d44ff139′ or one of its dependencies. An attempt was made to load a program with an incorrect format.

This is actually detailed in page 20 of the sitecore troubleshooting doc, but I missed it first time round. This is caused by the presence of System.Data.SqlLite in the bin folder when running on 64-bit OS. An alternative dll is available for 64bit OS from Sitecore.

Solution: If you’re are not using SQLite simply the delete the System.Data.SqlLite.dll from the bin folder and recycle your app to get back up and running. If you are using SQLite, download the alternative dll for 64bit OS.

While the similar method Context.Database.GetItem retrieves an item in the same language as the context item, it appears that SelectItems(selector, predicate) does not. So if you are in a language other than ‘en’ English, you will probably retrieve the ‘en’ version of your non-’en’ language items, resulting in large areas of missing data.

I’m not sure if this only effects SelectItems()  when run outside of the context of a web request, or if it happens all the time, but the work-around is to explicitly change your context.language before calling SelectItems().

ContextItem.Database.SelectItems(selector, predicate);     …..will return ‘en’ language items regardless.

Instead you must use:

Sitecore.Globalization.Language.Current = ContextItem.Language;
ContextItem.Database.SelectItems(selector, predicate);    

….which then retrieves items in the same language of contextItem.