Week Ten (19th-23rd April): "Add some more jokes"

The first weeknotes in a while, not because I've not been doing anything (far from it!) but more because the last few weeks were all gearing up to yesterday - the final presentations.

We left our prototypes as long as we could (three weeks) before analysing them so that we could use the maximum amount of data.  This all formed part of the final slidedeck along with all of the other things we'd done over the last ten weeks.

Well, actually... not "all".  There has been so much done that fitting it into five to eight minutes was impossible.

Tuesday's session with We Are Open was a chance to put some finishing touches to the presentation before a chance to give it a practice run through with the rest of the cohort.  Before that was the important business of setting the running order through the use of a "wheel of fortune" style name picker.

We were lucky with a good draw of second place, not first up but able to get over with quickly!

While (hopefully) far from disaster, this first run through clocked in at over 10 minutes and produced some good constructive feedback.  "Less text", "more images", and most importantly "add some more jokes", while also lopping about a third off the running time.

No pressure then?

Wednesday was a day for editing.  Taking things out, putting them back in, combining two slides into one and trying to pull it all together.  The running time got a bit shorter and a few things seemed to click into place.

Thursday afternoon and we are joined by a few members of Catalyst for the final session and the presentations.  The introduction by Laura and Doug from We Are Open gave a good overview of the whole ten weeks along with some of the results from our bi-weekly check-ins (confirming once and for all that Back to the Future is the greatest film ever made).

Taking all of the feedback on board form Tuesday, I managed to squeeze everything into nine minutes, not too bad under the circumstances and the whole thing felt better for the changes.  A few nice comments in the chat and direct messages hopefully meant it went down well with everyone else.

As for the jokes?  I'll leave you with this (anonymised) message I got shortly after finishing

Awesome presentation, I seriously almost spit smoothie out of my nose

Mission accomplished.

It was great to then sit back and listen to the rest of the charities give their presentation.  Although there were threads of similarity in them, it was amazing to see how we all sat in the same sessions over ten weeks and yet produced so many different ideas and things to test.  They are all doing great things.

• Beats Bus Records
• Caxton Youth
• Collage Arts
• Action Hampshire
• Kent Coast Volunteering
• Parochial Church Council of the Ecclesiastical Parish of Totnes with Bridgetown
• Pennysmart CIC
• WECIL Ltd
• Worcester Cathedral Education Department

Analysis of the Prototypes

The four different splash pages we created were designed to test two design choices.

These were done across four splash pages, so that each design choice could be tested independently.

• Splash 0: No login, Section buttons on side
• Splash 1: Login, Section buttons on side
• Splash 2: No login, Section buttons on top
• Splash 3: Login, Section buttons on top

We used Google Analytics to test these pages over a three week period, and compared them to the original home page with figures based over the preceding three weeks for a fair comparison.

Login form at the top of the page

One hypothesis with the old website, was that the login form could be confusing to users.  There is only one section of the website that requires a log-in (the Debt Management Plan) so to have it on every page may make a user think they need to log-in to access other parts of the site.  This may also be the reason why a large number of users sign-up for an account, but then never progress through the Debt Management Plan.

Two of the prototype pages (1 and 3) had the login form, the other two (0 and 2) did not.

Results

With the results we got, the login form didn't actually make much difference. The bounce rate (percentage of people leaving the page without going any further into the website) was almost identical to the old home page regardless of whether the login form was there or not.

This would be better to run over a longer period, to ensure that the results aren't being skewed by only a few users.

Also, over a longer period, we would be able to see if more people actually progressed through the Debt Management Plan after signing up (as at present a large percentage do not).  To help analyse this, we can save the referring page to our database when someone signs up.  This will help us analyse whether people are signing up from the home page (presumably without knowing what the signup is for) or if they are signing up only after reading about the Debt Management Plan.

Placement of section buttons (Mobile only)

As horizontal space is more limited on mobile displays, two different layouts were tested for the main section buttons.  

The full-width buttons above the image and text, allowed for a larger button and image but looked more like a header than a button, they also meant that the user had to scroll further down the page to access all of the content.

The buttons to the side of the image meant less scrolling, and looked more like a traditional button that could be pressed, but made the image and button smaller.

Two of the prototype pages (2 and 3) had the buttons above the text, the other two (0 and 1) had them to the left.  On desktop views, the button placement was not affected, so analysis was only carried out on mobile users.

Results

These results were much more conclusive.  

For the button on the side, the bounce rate was 45.45% and for the button on the top the bounce rate was 75%.

This is a big enough difference to indicate a much better performance for the left-aligned buttons.  This is possibly due to the buttons on top appearing more as a "heading" than something that can be clicked. 

Future prototypes will include different buttons to make them more obviously "clickable" as even on the left-aligned buttons, they can look a bit flat and some people may not realise they can be clicked.