Friday, April 23, 2021

Week Ten (19th-23rd April): "Add some more jokes"

The first weeknotes in a while, not because I've not been doing anything (far from it!) but more because the last few weeks were all gearing up to yesterday - the final presentations.

We left our prototypes as long as we could (three weeks) before analysing them so that we could use the maximum amount of data.  This all formed part of the final slidedeck along with all of the other things we'd done over the last ten weeks.

Well, actually... not "all".  There has been so much done that fitting it into five to eight minutes was impossible.

Tuesday's session with We Are Open was a chance to put some finishing touches to the presentation before a chance to give it a practice run through with the rest of the cohort.  Before that was the important business of setting the running order through the use of a "wheel of fortune" style name picker.


We were lucky with a good draw of second place, not first up but able to get over with quickly!

While (hopefully) far from disaster, this first run through clocked in at over 10 minutes and produced some good constructive feedback.  "Less text", "more images", and most importantly "add some more jokes", while also lopping about a third off the running time.

No pressure then?

Wednesday was a day for editing.  Taking things out, putting them back in, combining two slides into one and trying to pull it all together.  The running time got a bit shorter and a few things seemed to click into place.

Thursday afternoon and we are joined by a few members of Catalyst for the final session and the presentations.  The introduction by Laura and Doug from We Are Open gave a good overview of the whole ten weeks along with some of the results from our bi-weekly check-ins (confirming once and for all that Back to the Future is the greatest film ever made).

Taking all of the feedback on board form Tuesday, I managed to squeeze everything into nine minutes, not too bad under the circumstances and the whole thing felt better for the changes.  A few nice comments in the chat and direct messages hopefully meant it went down well with everyone else.

As for the jokes?  I'll leave you with this (anonymised) message I got shortly after finishing

Awesome presentation, I seriously almost spit smoothie out of my nose

Mission accomplished.

It was great to then sit back and listen to the rest of the charities give their presentation.  Although there were threads of similarity in them, it was amazing to see how we all sat in the same sessions over ten weeks and yet produced so many different ideas and things to test.  They are all doing great things.

Friday, April 16, 2021

Analysis of the Prototypes

The four different splash pages we created were designed to test two design choices.

These were done across four splash pages, so that each design choice could be tested independently.

  • Splash 0: No login, Section buttons on side
  • Splash 1: Login, Section buttons on side
  • Splash 2: No login, Section buttons on top
  • Splash 3: Login, Section buttons on top
We used Google Analytics to test these pages over a three week period, and compared them to the original home page with figures based over the preceding three weeks for a fair comparison.

Login form at the top of the page

One hypothesis with the old website, was that the login form could be confusing to users.  There is only one section of the website that requires a log-in (the Debt Management Plan) so to have it on every page may make a user think they need to log-in to access other parts of the site.  This may also be the reason why a large number of users sign-up for an account, but then never progress through the Debt Management Plan.

Two of the prototype pages (1 and 3) had the login form, the other two (0 and 2) did not.

Results

With the results we got, the login form didn't actually make much difference. The bounce rate (percentage of people leaving the page without going any further into the website) was almost identical to the old home page regardless of whether the login form was there or not.

This would be better to run over a longer period, to ensure that the results aren't being skewed by only a few users.

Also, over a longer period, we would be able to see if more people actually progressed through the Debt Management Plan after signing up (as at present a large percentage do not).  To help analyse this, we can save the referring page to our database when someone signs up.  This will help us analyse whether people are signing up from the home page (presumably without knowing what the signup is for) or if they are signing up only after reading about the Debt Management Plan.

Placement of section buttons (Mobile only)

As horizontal space is more limited on mobile displays, two different layouts were tested for the main section buttons.  

The full-width buttons above the image and text, allowed for a larger button and image but looked more like a header than a button, they also meant that the user had to scroll further down the page to access all of the content.

The buttons to the side of the image meant less scrolling, and looked more like a traditional button that could be pressed, but made the image and button smaller.

Two of the prototype pages (2 and 3) had the buttons above the text, the other two (0 and 1) had them to the left.  On desktop views, the button placement was not affected, so analysis was only carried out on mobile users.


Results

These results were much more conclusive.  

For the button on the side, the bounce rate was 45.45% and for the button on the top the bounce rate was 75%.

This is a big enough difference to indicate a much better performance for the left-aligned buttons.  This is possibly due to the buttons on top appearing more as a "heading" than something that can be clicked. 

Future prototypes will include different buttons to make them more obviously "clickable" as even on the left-aligned buttons, they can look a bit flat and some people may not realise they can be clicked.



Tuesday, March 30, 2021

Recipe: Easy A/B testing

This is how we set up a really quick and easy A/B test for our new splash pages.

Firstly, we made our fours different splash pages and named them the same but with a number after them.  Start at zero and add one for each page you want to test:

  • splash-0.asp
  • splash-1.asp
  • splash-2.asp
  • splash-3.asp

Then, we set up a new page that nobody will ever see, this only contained one line of code (three if you count the "script" tags before and after).

<script>
      window.location.href = "splash-"+ Math.floor(Math.random() * 4) +".asp";
</script>

Let's break down what this is doing

Math.floor(Math.random() * 4)

This generates a random number between 0 and 3.  If you want to test more or less pages, just change the 4 to the number of pages you want to test.

So when the page loads, it will generate a random number (0-3) and add that to the text strings "splash-" and ".asp" to make up one of the webpages above.  You can change these text strings to match your own pages before and after where the number needs to go.

window.location.href = 

All this bit does is tell the webpage to instantly redirect to the pagename we've just created.  As soon as the page loads, it will do all of this and send your user to one of your splash pages, chosen at random.

If you want to use this yourself, all you have to change are the parts in red:

<script>
      window.location.href = "splash-"+ Math.floor(Math.random() * 4) +".asp";
</script>

  • splash- (the first part of your splash page names, before the number)
  • 4 (how many different splash pages you are testing)
  • .asp (the last part of your splash page names, after the number)

Making it work

You then need to make sure that this page is set as your default homepage on your webserver.  For us, using IIS7 it's just a matter of adding the new page in as the "default document", but this may be different on other servers.


So now, instead of automatically going to our old index,asp homepage users will be sent to this new randomising script first if they enter our website at the root (ie not on a specific page, just using the address nedcab.cabmoney.org.uk).

This script then seamlessly redirects them to one of the four test pages.

Analytics

The last part is to ensure that all of the splash pages have your Google Analytics code in them, so you can see how they are being used.  Because they all have different names (because of the different numbers in the page names) they will show up separately on your Analytics pages where you can check their performance.



Thursday, March 25, 2021

Week Six (22nd-26th March): Making a Splash

All of this week centred around one thing - making a prototype that we can get in front of users to test next week.

So rather than the usual daily breakdown of what happened, it all fits together better just to go through the process.

Step 1: Paper!

Good, old fashioned, paper and a pen.  Following on from last week's decision to focus on a new "splash page" that sits in front of our current home page, I first sketched something out on paper as a rough idea of what I was thinking of.  One main remit of this was to make sure we had a layout for both monitor and mobile displays.

Monitor display at the top, with three different mobile designs underneath,

Step 2: Digital

We then moved onto looking at a few different websites that can be used to design a wireframe.  I settled on Whimsical as it looked easy to use for what I needed and also had some presets for monitor and mobile layouts.

Effectively the same design as above, but now with added legibility!

I used this layout to get my first bit of user feedback from my colleagues who had participated in the consultation a few weeks ago.

Due to the time constraints, rather than call another meeting I set up a shared document and emailed a call to arms. The shared document was a clearer way for collaboration, rather than everyone "replying to all" in an email, which can get out of hand quickly (and we all get too many emails anyway!).  From this I asked for thoughts on the layouts, and also ideas for the wording and imagery we could use.

In a short period of time, I had got the first stage of the prototype in front of a small user group and already had feedback.  I took it on board and moved onto the next stage.

Step 3: Actual

With already having a background in website design (albeit a bit rusty), it was easier for me to go straight into coding at this point.  I put together a splash page based on the designs and feedback that we could get out in front of other users as soon as possible.

The splash page (v1)

I took the wording that had come back from the focus group, and ran it through a readability checker.  With a few tweaks, I was able to get the scores better which hopefully means a page that will be easier for people to read.  This helps us towards one of our main goals in redesigning CABmoney.

Another goal could be met (for this splash page at least) by using Bootstrap to ensure that the layout could be created for both monitor and mobile devices.

I also ran the finished page through the WebAim Web Accessibility Evaluation Tool to make sure that it was fully accessible.

With a finished framework and content now ready to go online, I needed some variations so I could do some A/B testing.  One of the areas for concern on the current CABmoney website is the amount of people signing up for an account that then go no further.  One hypothesis is that because the "sign up" link is quite prominent on every page, users might think that they have to sign up (when in reality, it's only needed to save their progress on a Debt Management Plan).

For A/B testing, I am going to have one version of the splash page with the login/sign up form and one without.  This might show if people are distracted by this, when they could just go into the main sections directly.

At this point in the week, the page still needs a little tidying up, and I may look at a different A/B test on the wording to see if the simpler text makes a difference in how likely the user is to click a particular link.  For now though this project has moved into something much more tangible which we can put up on the website very soon.

Monday, March 22, 2021

What's "What's Up?"

This article is not part of our Catalyst Programme, but we are looking to use this blog to talk about other projects and innovations we are working on here at Citizens Advice North East Derbyshire.

Friday, March 19, 2021

Week Five (15th-19th March): "You've Got Red on You"

Tuesday

Another shortened week due to being off work on Monday, attempted to clear the decks of "normal" work in the morning before heading to the afternoon Catalyst session.

Ch-ch-ch-ch-changes.
- David Bowie

During that session, our focus (particularly our Minimum Loveable Product (MLP)) changed somewhat from what it was just a week before.  Originally, the thought was to introduce video tutorials for the website, which would show users how to complete certain sections.  After working through our MLPs again today, I "borrowed" an idea from one of the other organisations in our cohort to instead focus on the home page of CABmoney.

Although the video tutorials are something we still want to add at a later date, the new homepage (or after a bit more refining later in the week, a "splash page") would be a more achievable target (ie easier to do!) and would give us instant feedback through Google Analytics.

Through A/B testing, we could create two new splash pages and test which was the most effective live on the actual website.  This data could also be analysed against our current home page (already with years worth of analytics available) to see how a simplified approach would assist users get to the information they need quicker.

Also today, while talking about readability, the Flesch test was mentioned which would also be a good indicator on how effective changing our wording would be.

These two ideas combined would tie in well with some of the highest ranking ideas from our staff consultation, namely:

  • Clearer, Simple Menus
  • Move the Budget Calculator to Front and Centre
  • Easy Read / More Visual Explanations
These were all high scoring ideas that could at least be partially addressed by a splash page before the main current homepage.

This revelation (and major change of focus) led to our Design Constraint to be as follows:

And that was only the first session on Tuesday!

The second session went into Service Blueprints, showing the flow through the service and how things connect.  A bit tricky as some people will use different sections of our website, but the general flow is shown in our blueprint below.

Thursday

Today's session was how to communicate our Minimum Lovable Product (MLP) as a "zine". Having only a red marker pen, which bled through the paper onto my other notes and fingers the end product looked a little gory.  If you want to get in a mess like this yourself, get an old leaky red marker pen and make your own zine.

Here's one I made earlier!

Had this been for the full CABmoney website, we could have filled a full book with problems and solutions, but this seems much more manageable for the next few weeks (which is the point!)

The contents worked through the issues with the current home page that we've identified, and show the solutions that should be achievable for our prototypes.  I cheated a little, by going over two pages for one of the images but with quite a small, focussed MLP there wasn't too much to say without it turning back into a Big Hairy Audacious Goal (BHAG).

Friday

A bit of a catch up day for Catalyst Project work today.  I had a good catch up with my line manager about the project now it's reached the mid-way point which was very positive.

The last couple of blog posts went up (including this one!) and managed to do a bit more work on some of the other aspects but still a bit of catching up to do on a few others.

Consultation Ideas

The best way to have a good idea is to have a lot of ideas

- Linus Pauling

At our staff consultation last week, we gathered together a number of ideas.  Some had been submitted through our CABmoney feedback form, and the rest came from the group during the meeting.  As we discussed them, we were able to produce a longlist (definitely not a shortlist!) of improvements and ideas that could be explored through the Catalyst Project and beyond.

Below is a summary of the ideas, roughly grouped by category (although a few ideas could easily span into multiple categories).  The scores for each idea are an average of the scores given by the group (a small group of seven people, so any further user research would be more valuable if scaled up).  These scores were for level of impact that the idea would have on users, out of ten where 1 was "least impact" and ten was "most impact".

Navigation

General navigation of the website was a key issue for a lot of people, with poor support for mobile devices and overly complicated menus being specifically highlighted.

CABmoney App

Score: 9.86

Having CABmoney as an app would make it much more convenient for people to access.  Similar, to just making the website more mobile friendly, an increasingly large number of people would benefit from being able to access CABmoney via their mobile phone or tablet.  The advantages of an app also include getting push notifications when key dates are approaching or for other reminders.

Mobile Friendly Layout

Score: 9.71

The current website was not designed for mobile devices, so can be difficult to use.  Our Google Analytics data shows that the percentage of mobile/tablet device users is increasing year on year.

Clearer, Simple Menus

Score: 9.14

Reducing the number of menu options would make navigating the website much easier.

Move the Budget Calculator to Front and Centre

Score: 9.14

It was widely agreed that the Budget Calculator (Financial Statement) should take “centre stage” on the website’s homepage as it was the section most people referred clients to and the most useful for a large portion of users.  Currently, it is behind a few screens and takes a few clicks to access.

Traffic Light System

Score: 8.43

A traffic light system was suggested that would clearly show what sections had been completed or still needed working on.   It could also be implemented to show how healthy somebody’s finances were in the Budget Calculator by highlighting any areas of concern.  A more visual approach was thought to be better for people to follow and understand.

Search Bar

Score: 6.30

A search bar would make finding some of the information on the website easier and potentially improve navigation, but it was generally thought that the site is better used when people work through the sections in order.  It may have more use as a tool for advisers looking up information, than for other people using the site for themselves.

Access and Support

This group of ideas mainly look at making the wording and processes easier for people to understand (alongside the navigation).  By using plainer English, and offering more levels of support and explanation, it was thought that the user experience would be much smoother.  

This was also coupled with ideas that sought to reassure a user before they started using the site, as it was thought that with a taboo and sometimes complicated subject, many users would be wary of using the site if it seemed too complicated.

Easy Read / More Visual Explanations

Score: 9.14

The text and information on CABmoney should be as easy to read as possible.  The current wording can be too technical in places, and may put people off using the website fully.

Tutorial Videos

Score: 8.86

Explainer videos throughout the site showing how to complete each section.  These would be screen captures of somebody filling in an example.  For added human interaction, the person doing the example could be shown in a thumbnail within the video talking the user through the process.

Links Out to Citizens Advice Contact Details

Score: 8.29

As some people using CABmoney may be outside our usual catchment area, it would be better to link out to the national Citizens Advice website where they can find the contact details of their local office if they run into difficulties, or need the help of an adviser.

Separate Adviser Section with Additional Information

Score: 8.00

As some of the information on CABmoney is aimed more at advisers, than general users it was suggested to move this to an “adviser’s only” section (with login).  This information could still be displayed in the current flow of the website, but only displayed if logged in as an adviser.

Case Studies of Other Users

Score: 7.86

In an attempt to reduce some of the stigma associated with managing debts, it was suggested that “case studies” (or success stories) from real users who had used CABmoney and were now “out the other end” would encourage people to use the system and see a light at the end of the tunnel before they began the process.

Introduction/Reassurement at the Beginning

Score: 7.43

Similar to the idea of including Case Studies (see above), a more general introduction that reassures people that there is a way out of debt was suggested to be included either on the home page, or early in the Debt Management Plan process.

Language/Translation Function

Score: 6.86

Not everybody using CABmoney will have English as a first language, so a translation option would help them read and navigate the site.  These days however, third party tools such as Google Translate can translate an entire website independently of having a specific function on the website.  Ensuring that the text was clearly written in English (see above) would aid any translation tool that was used though.

Debt Management Plan Improvements

These improvements were specifically for users of the Debt Management Plan (DMP) and/or the Budgeting Sheet sections of CABmoney. These are more involved processes but incredibly useful for the users (and form the main backbone of the site).  Although some users will just access CABmoney for the information available, the interactive sections that allow them to manage their own money are key.

Ability to Email Creditors Directly

Score: 9.14

As not everybody has access to a printer, alongside the fact that postage costs are always rising, it was suggested that being able to email letters to creditors would be a quicker, cheaper and more convenient method.  This would rely on creditor details being up-to-date and knowing if they were open to receiving correspondence by email.

Links to Access Credit Reports by Online Forms or Email

Score: 9.00

The forms required to request a credit report are now available online, this would also be a quicker and cheaper process than completing paper copies and sending/receiving in the post.

Expanding Budget Sheet Sections

Score: 8.86

Currently, some sections of the budget sheet actually consist of a number of values that need to be added together before entering (ie “Food/Housekeeping”).  These entries could have an option to expand out, so that individual items can be entered separately.  This would act as a prompt for the sort of things that could be included, and also mean that no manual addition was needed to enter the total value.

Self-Referral to Adviser

Score: 8.29

An option for people using the site who may run into difficulties to be able to self-refer to a Citizens Advice adviser.  Although useful, this would need to ensure that people are referred to the correct local office (as a website can be used outside of our geographical catchment area).

Reminders/Warning to Keep Reviewing Income/Expenditure

Score: 8.00

Generally, people enter their income and expenditure into their budget sheet or Debt Management Plan once and never return to it.  By reminding people (either by automated email, or notification if we were to develop an app) to keep these up-to-date, they will have a better understanding of their financial situation.  This is especially important at the moment as there are a number of outside factors that can affect somebody’s income and expenditure in these uncertain times.

Up-to-date Creditor Lists

Score: 7.57

As part of our Debt Management Plan section, we provide a number of contact details for various creditors.  Over the years, these have gotten out of date and need updating, though it would be a continuous process to keep up to date going forwards.

Disbursement of Payments

Score: 6.86

There was a suggestion that a system that allowed users to pay their creditors directly through CABmoney could be added.  Although useful, this would require a large amount of technology and security to be implemented along with other checks that the money was going to the right place and being received and accounted for properly.

Everything Else

Not everything fits into neat categories.  The ideas below looked at expanding CABmoney to incorporate other money management information (most of which we already have housed on a separate site), and also using it to help with our internal processes.

Not Just Debt

Score: 8.29

We currently have a separate site focussing on budgeting and income maximisation (http://moneysmart.nedcab.org.uk/).  It was felt that this should be combined with CABmoney so that the information was all in one place, and also so that there was information to help people avoid getting into debt, alongside information for those trying to get out of debt.

Easier Internal Referrals to Our Debt Team

Score: N/A (Idea surfaced after the scoring, but the feedback was very positive)

We currently use a PDF form to record client information which then goes to our Debt Team if the client is to be referred for specialist advice.  By replicating this form on CABmoney, the referrals could be made more seamlessly, and as a lot of the information is the same as on a budget sheet or Debt Management Plan, it could form the basis of completing one of those for the client.