Sitecore xDB Personalisation at the client

Nick Hills’ headed over to SUGCON last week to share his infinite wisdom with the rest of the Sitecore community, if you missed it or want a recap you can listen to his talk below:

This talk shows off techniques that allow you to combine two great tools: the personalisation capabilities and power of xDB along with the benefits of CDN edge-caching. Editors can configure and design personalisation rules as normal yet still harness all the power and gains that CDN’s offer.

He’s also handily put together a whitepaper weighing up the pros and cons of the various approaches – http://blog.boro2g.co.uk/personalization-at-scale-whitepaper/

Get in touch if you want to find out more on +44 (0)117 932 7700 or info@trueclarity.co.uk

Going digital to save Taylor & Francis 150 hours a week

“Our e-copyright system was a relatively simple idea that required some fairly complicated problem-solving to make a reality. We worked side-by-side with True Clarity throughout the process to make sure the product we ended up with didn’t limit our flexibility. We appreciated the willingness of True Clarity to have people doing the development work available to us throughout the process. ”

Edward A. Cilurso Vice President – Production

Taylor & Francis Group are one of the world’s leading publishers, releasing over 2,100 journals and 4,000 books annually, with a backlist in excess of 60,000 specialist titles.

What was needed

Taylor and Francis (T&F) have to handle around 250,000 copyright agreements annually in order to publish articles on their site. Getting these agreed involved a lengthy process of emailing out forms, toing and froing through lots of amends and relying on scanning, faxing and emails to log author’s approvals electronically. The business case was crying out for moving this process online, in order to eliminate the pain for authors and staff, save time and keep up with the competition’s online offering.

How we tackled it

Prototypes and early delivery

We kicked off with a discovery phase, involving a few workshops and sessions to completely map out the process, then put together some flow charts to illustrate how we would automate the copyright process. At this stage we carried out some proto-types to validate the approaches, flesh out the unknowns and start delivering to T&F as early as possible in the project. Beyond the author approval on-site, we also needed to map out the integration points with internal legacy sites, such as the production and content management system, and map out some complex workflows to keep the site as self-sufficient as possible.

Designed for Flexibility

The site was designed to present the author with a set of questions, which dynamically change based on their response. The outcome is then used to generate a personalised version of the copyright agreement for the author to agree and submit. Key to this project was to ensure migrating the process online didn’t take away the flexibility of the manual process, so we worked with T&F to create a sophisticated back end allowing T&F to define the routing of questions and answers used to generate the copyright agreements. We also created a number of customisable email templates sent out for different statuses, including reminders for authors who may not have completed an agreement after X amount of time.

The Value

Copyright agreements turned around much faster

T&F sent out 193 author publishing agreements upon launch. 118 were approved within a week, with the vast majority clicking standard options, requiring no further intervention from T&F and minimal enquiries. The approval of 118 licenses would have averaged a longer turnaround, with inspection of each form, using the old process.

Saving staff time on menial tasks

T&F are able to compile detailed weekly reports from the author approval system based on the current status of manuscripts, licenses assigned per country and APAs completed by day. T&F have calculated from the initial launch stats that the new system will save them up to 150 man hours per week – the equivalent of four full-time staff members.

Building Personalised Experiences with Sitecore

The Conservative Party is the current party of government in the United Kingdom, having won a majority of seats at the General Election in May. As part of their election efforts, we used Sitecore to build a personalised map that helped undecided voters understand how the party’s policies had helped secure a brighter future for people across the country. By entering their postcode, users could see exactly what that meant in their area.

We integrated various electorate information systems APIs to render the map information based on the content in Sitecore. The Conservative Party managed all of this content in Sitecore and placeholders were used to pull in the relevant numeric stats and personalised data. True Clarity knew that Conservatives.com would be one of the first places undecided voters would go to find out about the Party’s policies in the days leading up to the election – and on polling day itself. Therefore, it was imperative that the site focused on helping people understand not just what those policies were but what they meant for them, their family and their area. This is a great example of how Sitecore can be used to enable personalised experiences for customers.

https://www.conservatives.com/yourarea
Map of Data for True Clarity’s London Office

View your own personalised area map here.

Winning the Digital Election with Conservatives.com

“True Clarity’s impressive team helped us – again – to deliver a website of the highest possible standard. Thanks to their input, Conservatives.com played a key part in helping undecided voters understand exactly how our plan would secure a brighter future for Britain while also empowering supporters to help the campaign in a number of ways. True Clarity’s commitment to responsive design meant that the site worked seamlessly across all devices, and their hard work in the months leading up to polling day meant the site was both stable and secure in those crucial final days.”

Craig Elder, Digital Director, The Conservative Party

What was needed

5 years is a long time in politics. In 2010, we helped the Conservatives build what was widely recognised as the best of the UK political parties’ websites but for 2015 we needed to adapt to a changing digital landscape.

In particular, the huge shift towards consumption via mobile devices meant we’d have to approach the election with a “mobile first” mentality, and work to deliver simple user journeys that worked as well on a phone as they did a desktop.

We also had to focus on delivering a tightly-focused website that gave two key audiences what they needed: helping undecided voters understand exactly what the Conservatives’ long-term economic plan meant for them, while also giving Conservative supporters the tools they needed to get more involved in the campaign.

And of course, we had to achieve all of this while ensuring the site could cope under the huge amount of traffic we could expect during the busy election period (and on polling day itself).

How we tackled it

A responsive site for a mobile-first audience

Working with the Conservatives’ in-house design team, we created site templates that worked responsively across all devices – be it phone, tablet, laptop or desktop. This had to be achieved without compromising site editors’ ability to quickly add, edit and remove copy, content and widgets as required.

Giving undecided voters the information they need

Just as it was in 2010, we knew that Conservatives.com would be one of the first places undecided voters would go to find out about the Party’s policies in the days leading up to the election.

Therefore it was imperative that the site focused on helping people understand not just what those policies were – but what they meant for them, their family and their area. We worked with the Conservatives to produce a series of interactive pages throughout the site that allowed users – upon answering questions on location, salary and so on – to find out exactly how the Conservatives’ plan would help them.

We took this approach as far as making the 404 page on the site an interactive “find out what our plan means for you” page to ensure even users who couldn’t find what they were looking for could learn what the Conservatives’ long-term economic plan meant for them and their family.

Making it easy for supporters to support

Of course, one of the other key audiences for a political party’s website is its supporters, and it was important that Conservatives.com gave them the tools they’d need to make the campaign a success. We worked closely with the Conservatives’ team to optimise three key user journeys – membership, donations and volunteering – to ensure supporters could complete these actions as quickly and easily as possible. This was a huge success, with the donation page in particular seeing a huge leap in conversion rate, leading to the Party raising more money in small online donations than at any previous election. In addition, the new Volunteer page played a huge part in helping to assemble the ‘Team2015’ volunteer army which played a key part in winning the election.

Finally, we worked with Dynamic Signal to put gamification at the heart of a new ‘Share the Facts’ section of the website, which rewarded Conservative supporters every time they shared campaign images and videos on their own social networks – significantly increasing participation and reach for each piece of content.

The Value

Conservatives.com played an important part in the Party’s overall election efforts, which saw them win 331 seats and an overall majority – confounding the predictions of pollsters and commentators alike.

The Party raised more money in small online donations than ever before, and also assembled a 100,000+ strong ‘Team2015’ volunteer army thanks to people signing up via the website.

‘Share the Facts’ was hugely successful, helping Conservatives supporters reach an additional 3 million people every week – over and above the direct reach of the Conservatives’ existing digital channels – by empowering people to quickly and easily share campaign content.

And perhaps most importantly, the site performed seamlessly on polling day (just as it had in 2010), meaning voters in key constituencies all around the country were able to get the information they needed.

http://www.conservatives.com

Work, Rework and Maintenance in Software

I thought it would be worth clarifying the terminology used to express the different types of work, rework and maintenance in software development.

Refactoring

Refactoring is the art of making your code clean and refining the design in small incremental steps towards the SOLID principles of software development. Refactoring should be performed many times a day to ensure code is clean.

Prototyping

In order to learn enough about a problem, a prototype is often built that will be thrown away. If the prototype is not thrown away, it is not a prototype. Prototyping should be used to gain knowledge and understanding of a problem. A prototype will generally have a substandard design because the problem is not well understood until the prototype is finished. That’s why you throw a prototype away and redesign what you have just done with increased knowledge. Don’t continue working on a prototype when there is no more learning taking place. Do not mix user interface prototyping with software design prototyping. Always keep the two separate.

Tracer Bullet

A tracer bullet is an implementation of part of a piece of software. A tracer bullet is used to learn enough about the larger piece of software, usually in order to inform estimates and tackle technical risks early. Tracer bullets are risk management for developers. A tracer bullet is only finished if the code is production ready, tested and has been kept clean by constantly refactoring the code during development otherwise it is a prototype and should be thrown away.

Redesign

Redesign is usually undertaken when existing code needs to be drastically remodeled. Redesign is an indication of one or more of the following problems:

  • the code is not clean and does not following the SOLID principles
  • there was inadequate knowledge during development
  • the project was rushed
  • technical risks were not properly flushed out

Redesign is caused by a lack of refactoring, prototyping and tracer bullets.

Conclusion

Every project should have a healthy mix of prototyping to inform design, tracer bullets to inform estimates and reduce risks and refactoring that should be done regularly throughout the day. Redesign implies that you’ve got an unhealthy mix or are ignoring important steps in the software development cycle.

Also, changes to requirements would potentially precipitate one or more of refactoring, prototyping, tracer bullets, or even redesign. The later suggesting quite a major change.

Catch 22: Building Great Software

Building great software is hard. It starts with the choices you make and making great choices is really hard. After all, given the choice, we’d all choose to make great software right? So what’s the catch?

Picture a young guy out to buy a car. He really wants a convertible, it really fits with what he wants out of life right now. So he buys a convertible. He meets a girl, settles down, and before you know it, a baby is on the way. Now the convertible isn’t so great. What helped him when he was dating isn’t much help to him now. Now he wants a saloon!

Our current choices often conflict with future circumstances, however, if we focus too heavily on the future, we may actually make decisions which conflict with the present. A decision which is great for the future might be terrible news right now and vice versa. So how do we deal with this conflict? As it turns out, we tend to deal with it rather badly.

Decisions go stale. As business needs change, what’s relevant today is no longer useful in the future. In software development this means each feature will at some point fail to be relevant. Developers can see this as a failure to make the right choice rather than accept that ultimately our decisions will at some point be irrelevant. It’s likely that given such failure there will be the belief that enough learning took place to make a better decision in the future. We completely ignore the fact that at some point those choices will also become stale. Rather than turn our attention to being good at handling change, we repeat the cycle of trying to improve our decision making process on the basis that we’ve learnt enough that it won’t happen again. We naively think that next time our decisions will stand the test of time!

In extreme cases decisions can go stale before we’ve even acted on them. Businesses don’t wait for you to finish, they continue to change and learn. Being mechanically minded, let’s say we decide to build a saloon. This will take some time. While we’re doing this our child could grow up, leave home and we’re left wishing we’d built a convertible.

The conflict between present and future choices creates a problem in defining a successful software project. Success or failure at this level is outside our control: we can’t predict the future so we don’t know how relevant our choices are going to be over time.

In order to handle this conflict, we have to make it a non-issue. We have to be successful at adapting to change. What really matters to our success or failure is our ability or inability to respond quickly to change. When we can’t adapt quickly to changes, such as new or revised requirements, we have ultimately failed to manage the conflict. The good news is, if we understand that we have failed to respond quickly, we can invest time in understanding why that is and learn from this failure in order to improve our software development process to better handle change.

For a software team to be successful at adapting to change, you need to understand that you’re biggest asset or liability is your code. No decision you make can ensure your code outlives its usefulness, all you can do is ensure you can change it quickly, easily and without risk. To do this you will need clean, loosely coupled, well structured, unit tested code.

Similarly, business stakeholders need to help break the mind-set that features are perpetual. Software is evolutionary and features carry an amount of speculation about current and future value. Rather than telling developers what the business needs (or developers interpreting needs as perpetual), a “let’s try this next” approach to implementing features helps to reinforce the understanding that change comes with the territory.

Focus on handling change well. Your process should be about improving the ability to handle change by understanding and identifying your failure to respond quickly to change. Loose the ego and stop being frightened to admit failure and maximise the opportunity to learn as much as you can for the next change – it’s definitely coming!

Context Switching

The dreaded context switch. I hear about it all the time. I have to drop what I’m doing several times a day to listen to people discuss it (chuckle). Jokes aside, I’m genuinely interested in the reasons why people want to avoid switching context.

Up until very recently I was convinced context switching was a bad idea. I would also have assumed any books that deal with context switching would justify my belief Q.E.D. style, case closed. I’ve been wondering lately if my thinking is broken and that I’ve been fighting for the wrong cause.

So what is context switching? In my world we are talking about changing from one deep train of thought to another. As a developer it takes me a while to get my head round the code I’m looking at. If I have to break my concentration, clear my registers and push what I was thinking about onto my mental stack for later, it takes times. A familiar example would be working on the code for a project and an urgent problem comes in that requires my attention to be diverted to the codebase for another project. Everything I was holding in my head about the code I’m currently working on gets lost forever as I bring my thoughts to bear on a different problem.

Now I’m sure you might be thinking there are also sorts of time management practices you could apply to handle this. However, I would urge you to think twice about how you tackle context switching. Is it really a good idea to rush into this problem with your productivity hat on and try to optimize the process to avoid context switching? If you do, much simpler and more important questions may go unanswered: are people finding context switching hard? If they are: why is that?

Recently I read two very different books. Although they turned out not to be so different after I’d read them. One was on psychology, the other development.

The book on psychology “Think Fast, Think Slow” discusses how we handle the process of decision making and problem solving. It will come as no surprise that we need to engage a large percentage of our brain to solve problems. The harder the problem the more time and thought you will need to understand the problem and resolve it. My take away is that there are types of work that we struggle with. We’ve all put a book down because it was just too damn complicated. We’ve all read a book where each page was jammed so full of information it took ages to read. I’ve also read similar sized books in a few hours. Those books I’ve read quickly were just… well, less complicated!

The book on software development was called “Clean Coder”. It challenged some of my long held beliefs about good software development. It especially challenged how I think about being in the zone. I love being in the zone. I love being so engrossed in difficult problems that hours seem like minutes. I love holding hundreds of lines of code in my head trying to figure out where it’s going wrong. I’ve woken in the morning knowing how to fix a bug in some thousands lines of assembler I’ve memorized. It’s addictive and gratifying. There can be a certain amount of bravados about being better at handle complicated problems than others. Is boasting about fixing really hard code like totally unreadable assembler a good thing? Is being so engrossed in a problem that you lose sight of the big picture the best approach?

Both these books were telling me something about complicated work. It can be difficult to get your head round. We don’t want to be disturbed when we’re engrossed in complicated work because it’s difficult to get engrossed. If you like a challenge, it can be a lot of fun. We don’t want to be disturbed when we’re engrossed in complicated work because we’re enjoying ourselves. Whether context switching is time consuming or a killjoy, the fact is: difficult challenging work is time consuming.

If context switching is a problem, you need to understand if the work is the underlying cause.

Maybe you’re thinking: “Duh? Of course software development is hard. We need to focus more and have less distractions to deal with it!”

Maybe you’re thinking: “I kind of get it, but we can’t make the work less complicated, we have no control over the work coming in!”

Maybe you’re thinking: “How can I make the code I have to read all day simpler and clearer?” If you’re thinking about the possibility of making things simpler, you understand.

“My code is awesome” – Why we should strive to embrace third-party code

Sitecore and .NET are staples of the True Clarity diet, so to speak – we are both a Microsoft partner and a Sitecore partner, and we’re using these technologies to do some very cool things! Outside of these, most of the developers here are comfortable using a few other frameworks and libraries such as NUnit and jQuery, but are we really being adventurous enough with the technologies we’re using? I’d argue that we aren’t (yet), and I’d imagine the same goes for plenty of other developers elsewhere.

Why should we be more adventurous?

To help illustrate, here’s an example of a jQuery accordion (minus the styling) which I’ve thrown together for the purpose of this post:

<ul class="accordion">
  <li class="accordion-panel">
    <h3 class="accordion-header">
      <a href="#">SitAmet Security 2.0 (Awesome Edition)</a>
    </h3>
    <div class="accordion-content">
      Lorem ipsum dolor sit amet
    </div>
  </li>
  <li class="accordion-panel">
    ...
  </li>
</ul>
...
<script type="text/javascript">
  (function ($) {
    $(function () {
      $('.accordion-content').each(function (index) {
        if (index > 0) {
          $(this).hide();
        }
      });
      $('.accordion-header').click(function () {
        var content = $(this).parent().find('.accordion-content')[0];
        $('.accordion-content').each(function () {
          var $this = $(this);
          if ($this[0] === content) {
            $this.slideDown('fast');
          }
          else {
            $this.slideUp('fast');
          }
        });
      });
    });
  })(jQuery);
</script>

So, is it any good? Well it works, the markup is reasonably semantic, the Javascript is nicely wrapped and I’ve seen very similar accordions get deployed to QA (e.g. by yours truly). But there are still a lot of unanswered questions about this piece of code, for instance:

  • How flexible can we make the markup without breaking the Javascript?
  • How easy is it to change the slide speed or the default content panel?
  • Will it work with multiple such accordions on the same page? (Nope)
  • Does it do anything unexpected? (Yes, it jumps to top of page when you click – href=”javascript:void(0)” is much better for Javascript anchors)
  • Does it conform to accessibility guidelines such as AA and ARIA? (Note – maybe it doesn’t have to, but it’s certainly not a bad thing if it does!)
  • Does it work in IE6 and on an iPad? (See the above note)
  • Does the markup lend itself to looking consistent with your fancy custom dropdowns sitting next to it?
  • How gracefully does it fall back without Javascript?

All these questions will likely be asked at some point, whether by a fellow developer, a tester or the client. Sometimes the answer will be “it’s fine”, but often it won’t be, and this contributes very significantly to the overall development time and cost of the piece of work. Additionally, bugs will occasionally slip past QA and UAT and only be spotted once it’s on a production environment, resulting in less client satisfaction and more time spent fixing bugs, which is neither fun nor profitable for anyone!

By contrast, a plugin such as the jQuery UI accordion has had countless eyes on it and many man hours spent turning it into a robust, reusable and flexible piece of code that’s already proven its worth on many commercial websites. Remember when IE6 was playing peekaboo with you? Or when you accidentally used an ID instead of a class in your jQuery selector? Well, these little annoyances have all been ironed out for you – why open yourself up to them again by trying to reinvent the wheel? It’s a classic case of “work smart, not hard”.

While I’ve used a UI example here, this applies to any area of coding, whether it’s an accordion, a search index or a Sitecore module. Which frameworks and libraries to use may be a matter of business requirements or personal preference but there’s certainly no shortage of options to explore.

So what’s holding us back?

One reason is what I refer to as “my code is awesome” syndrome. There’s a natural tendency to attach extra value to your own code just because it’s “yours”. Granted, you may feel more comfortable working with your own code, but to think that no-one else can write good code is pretty silly. How awesome did my accordion end up being, compared to the plugin? Not very!

Of course, it’d be rather cynical to think that every developer has an ego problem (those are in HR, ahem…). There are plenty of other reasons, the main one being unfamiliarity and its immediate consequences. When you’re pressed for time it can be difficult to justify setting aside some time for investigation (e.g. a spike), but when you consider the amount of time it can save it really becomes a no-brainer. Though this is typically viewed as more of a long-run reward, the investigation and learning time will often pay for itself within the scope of the current piece of work. A much-overlooked fact is that the back-and-forth of a piece of work between Dev, QA and UAT often dwarfs the initial development time, and that’s the main reason why this approach can provide such great benefits.

It’s worth noting that even if a library or framework doesn’t quite do what you need (or doesn’t do it as elegantly as you would like), it may still be your advantage to beat it into submission and adapt it to suit your needs. At True Clarity we have done this very successfully on several occasions! Often however, it’s a case of simply doing some investigation to find out what does and doesn’t work well out-of-the-box.

Sometimes you may not even realise a library exists for the task at hand or that it’s an appropriate time to look for one. But if you find yourself, say, copying and pasting hundreds of lines of SQL and data-mapping code that does basic CRUD operations, that should set off alarm bells! A quick Google search will bring up plenty of widely used object-relational mappers: NHibernate, ActiveRecord, Linq to SQL and Entity Framework to name just a few. You may not have used an ORM before, or even have known what an ORM was, but whenever you find yourself doing something boring, fiddly or repetitive, there’s a good chance that plenty of others did too and found a way to tame it. Those people may even be sitting right next to you and may have a robust in-house solution that you just haven’t used before. But to find the right answers, we have ask the right question: “Is there an easier way of doing this?”. If I were writing that accordion on a real project, it could easily have slipped my mind to ask that question, and that’s a habit that needs to be broken.

How can we make this easier?

As always, Google / Bing / <insert search engine here> is your friend and ally. But there are other ways to explore all the options out there: asking a fellow developer (whether sitting next to you or on a site like Stack Overflow) is often a great start. For .NET developers specifically, NuGet is an excellent way to try new things: if you’re looking for a search library for instance, simply type in “search” and all the most popular options will be a single click away. The fact that NuGet takes care of adding (or removing) the necessary DLLs, dependencies, references, and bits of configuration is absolutely invaluable and takes away a lot of the friction involved in trying out new libraries. Not a fan of the library you’ve just tried out? Just click “uninstall” and try the next one on the list!

But what about the learning experience?

None of this is to say that you shouldn’t bother trying to understand how these libraries and frameworks work behind the scenes – that’s still very important from a learning perspective! But as developers in a professional environment I feel we need to focus more of our efforts on learning to use third-party code and integrate it effectively – we certainly shouldn’t be spending a significant percentage of our total development efforts on re-writing, testing and fixing common pieces of code that are already freely available (or available at a fraction of the cost of developing it from scratch). So I think it’s time we really embrace this approach and make a habit of saying “I’m going to do something awesome with your code.”