Week Ten (19th-23rd April): "Add some more jokes"

The first weeknotes in a while, not because I've not been doing anything (far from it!) but more because the last few weeks were all gearing up to yesterday - the final presentations.

We left our prototypes as long as we could (three weeks) before analysing them so that we could use the maximum amount of data.  This all formed part of the final slidedeck along with all of the other things we'd done over the last ten weeks.

Well, actually... not "all".  There has been so much done that fitting it into five to eight minutes was impossible.

Tuesday's session with We Are Open was a chance to put some finishing touches to the presentation before a chance to give it a practice run through with the rest of the cohort.  Before that was the important business of setting the running order through the use of a "wheel of fortune" style name picker.

We were lucky with a good draw of second place, not first up but able to get over with quickly!

While (hopefully) far from disaster, this first run through clocked in at over 10 minutes and produced some good constructive feedback.  "Less text", "more images", and most importantly "add some more jokes", while also lopping about a third off the running time.

No pressure then?

Wednesday was a day for editing.  Taking things out, putting them back in, combining two slides into one and trying to pull it all together.  The running time got a bit shorter and a few things seemed to click into place.

Thursday afternoon and we are joined by a few members of Catalyst for the final session and the presentations.  The introduction by Laura and Doug from We Are Open gave a good overview of the whole ten weeks along with some of the results from our bi-weekly check-ins (confirming once and for all that Back to the Future is the greatest film ever made).

Taking all of the feedback on board form Tuesday, I managed to squeeze everything into nine minutes, not too bad under the circumstances and the whole thing felt better for the changes.  A few nice comments in the chat and direct messages hopefully meant it went down well with everyone else.

As for the jokes?  I'll leave you with this (anonymised) message I got shortly after finishing

Awesome presentation, I seriously almost spit smoothie out of my nose

Mission accomplished.

It was great to then sit back and listen to the rest of the charities give their presentation.  Although there were threads of similarity in them, it was amazing to see how we all sat in the same sessions over ten weeks and yet produced so many different ideas and things to test.  They are all doing great things.

• Beats Bus Records
• Caxton Youth
• Collage Arts
• Action Hampshire
• Kent Coast Volunteering
• Parochial Church Council of the Ecclesiastical Parish of Totnes with Bridgetown
• Pennysmart CIC
• WECIL Ltd
• Worcester Cathedral Education Department

Analysis of the Prototypes

The four different splash pages we created were designed to test two design choices.

These were done across four splash pages, so that each design choice could be tested independently.

• Splash 0: No login, Section buttons on side
• Splash 1: Login, Section buttons on side
• Splash 2: No login, Section buttons on top
• Splash 3: Login, Section buttons on top

We used Google Analytics to test these pages over a three week period, and compared them to the original home page with figures based over the preceding three weeks for a fair comparison.

Login form at the top of the page

One hypothesis with the old website, was that the login form could be confusing to users.  There is only one section of the website that requires a log-in (the Debt Management Plan) so to have it on every page may make a user think they need to log-in to access other parts of the site.  This may also be the reason why a large number of users sign-up for an account, but then never progress through the Debt Management Plan.

Two of the prototype pages (1 and 3) had the login form, the other two (0 and 2) did not.

Results

With the results we got, the login form didn't actually make much difference. The bounce rate (percentage of people leaving the page without going any further into the website) was almost identical to the old home page regardless of whether the login form was there or not.

This would be better to run over a longer period, to ensure that the results aren't being skewed by only a few users.

Also, over a longer period, we would be able to see if more people actually progressed through the Debt Management Plan after signing up (as at present a large percentage do not).  To help analyse this, we can save the referring page to our database when someone signs up.  This will help us analyse whether people are signing up from the home page (presumably without knowing what the signup is for) or if they are signing up only after reading about the Debt Management Plan.

Placement of section buttons (Mobile only)

As horizontal space is more limited on mobile displays, two different layouts were tested for the main section buttons.  

The full-width buttons above the image and text, allowed for a larger button and image but looked more like a header than a button, they also meant that the user had to scroll further down the page to access all of the content.

The buttons to the side of the image meant less scrolling, and looked more like a traditional button that could be pressed, but made the image and button smaller.

Two of the prototype pages (2 and 3) had the buttons above the text, the other two (0 and 1) had them to the left.  On desktop views, the button placement was not affected, so analysis was only carried out on mobile users.

Results

These results were much more conclusive.  

For the button on the side, the bounce rate was 45.45% and for the button on the top the bounce rate was 75%.

This is a big enough difference to indicate a much better performance for the left-aligned buttons.  This is possibly due to the buttons on top appearing more as a "heading" than something that can be clicked. 

Future prototypes will include different buttons to make them more obviously "clickable" as even on the left-aligned buttons, they can look a bit flat and some people may not realise they can be clicked.

Recipe: Easy A/B testing

This is how we set up a really quick and easy A/B test for our new splash pages.

Firstly, we made our fours different splash pages and named them the same but with a number after them.  Start at zero and add one for each page you want to test:

• splash-0.asp
• splash-1.asp
• splash-2.asp
• splash-3.asp

Then, we set up a new page that nobody will ever see, this only contained one line of code (three if you count the "script" tags before and after).

<script>
      window.location.href = "splash-"+ Math.floor(Math.random() * 4) +".asp";
</script>

Let's break down what this is doing

Math.floor(Math.random() * 4)

This generates a random number between 0 and 3.  If you want to test more or less pages, just change the 4 to the number of pages you want to test.

So when the page loads, it will generate a random number (0-3) and add that to the text strings "splash-" and ".asp" to make up one of the webpages above.  You can change these text strings to match your own pages before and after where the number needs to go.

window.location.href = 

All this bit does is tell the webpage to instantly redirect to the pagename we've just created.  As soon as the page loads, it will do all of this and send your user to one of your splash pages, chosen at random.

If you want to use this yourself, all you have to change are the parts in red:

<script>
      window.location.href = "splash-"+ Math.floor(Math.random() * 4) +".asp";
</script>

• splash- (the first part of your splash page names, before the number)
• 4 (how many different splash pages you are testing)
• .asp (the last part of your splash page names, after the number)

Making it work

You then need to make sure that this page is set as your default homepage on your webserver.  For us, using IIS7 it's just a matter of adding the new page in as the "default document", but this may be different on other servers.

So now, instead of automatically going to our old index,asp homepage users will be sent to this new randomising script first if they enter our website at the root (ie not on a specific page, just using the address nedcab.cabmoney.org.uk).

This script then seamlessly redirects them to one of the four test pages.

Analytics

The last part is to ensure that all of the splash pages have your Google Analytics code in them, so you can see how they are being used.  Because they all have different names (because of the different numbers in the page names) they will show up separately on your Analytics pages where you can check their performance.

Week Six (22nd-26th March): Making a Splash

All of this week centred around one thing - making a prototype that we can get in front of users to test next week.

So rather than the usual daily breakdown of what happened, it all fits together better just to go through the process.

Step 1: Paper!

Good, old fashioned, paper and a pen.  Following on from last week's decision to focus on a new "splash page" that sits in front of our current home page, I first sketched something out on paper as a rough idea of what I was thinking of.  One main remit of this was to make sure we had a layout for both monitor and mobile displays.

Monitor display at the top, with three different mobile designs underneath,

Step 2: Digital

We then moved onto looking at a few different websites that can be used to design a wireframe.  I settled on Whimsical as it looked easy to use for what I needed and also had some presets for monitor and mobile layouts.

Effectively the same design as above, but now with added legibility!

I used this layout to get my first bit of user feedback from my colleagues who had participated in the consultation a few weeks ago.

Due to the time constraints, rather than call another meeting I set up a shared document and emailed a call to arms. The shared document was a clearer way for collaboration, rather than everyone "replying to all" in an email, which can get out of hand quickly (and we all get too many emails anyway!).  From this I asked for thoughts on the layouts, and also ideas for the wording and imagery we could use.

In a short period of time, I had got the first stage of the prototype in front of a small user group and already had feedback.  I took it on board and moved onto the next stage.

Step 3: Actual

With already having a background in website design (albeit a bit rusty), it was easier for me to go straight into coding at this point.  I put together a splash page based on the designs and feedback that we could get out in front of other users as soon as possible.

The splash page (v1)

I took the wording that had come back from the focus group, and ran it through a readability checker.  With a few tweaks, I was able to get the scores better which hopefully means a page that will be easier for people to read.  This helps us towards one of our main goals in redesigning CABmoney.

Another goal could be met (for this splash page at least) by using Bootstrap to ensure that the layout could be created for both monitor and mobile devices.

I also ran the finished page through the WebAim Web Accessibility Evaluation Tool to make sure that it was fully accessible.

With a finished framework and content now ready to go online, I needed some variations so I could do some A/B testing.  One of the areas for concern on the current CABmoney website is the amount of people signing up for an account that then go no further.  One hypothesis is that because the "sign up" link is quite prominent on every page, users might think that they have to sign up (when in reality, it's only needed to save their progress on a Debt Management Plan).

For A/B testing, I am going to have one version of the splash page with the login/sign up form and one without.  This might show if people are distracted by this, when they could just go into the main sections directly.

At this point in the week, the page still needs a little tidying up, and I may look at a different A/B test on the wording to see if the simpler text makes a difference in how likely the user is to click a particular link.  For now though this project has moved into something much more tangible which we can put up on the website very soon.

What's "What's Up?"

This article is not part of our Catalyst Programme, but we are looking to use this blog to talk about other projects and innovations we are working on here at Citizens Advice North East Derbyshire.

Week Five (15th-19th March): "You've Got Red on You"

Tuesday

Another shortened week due to being off work on Monday, attempted to clear the decks of "normal" work in the morning before heading to the afternoon Catalyst session.

Ch-ch-ch-ch-changes.

- David Bowie

During that session, our focus (particularly our Minimum Loveable Product (MLP)) changed somewhat from what it was just a week before.  Originally, the thought was to introduce video tutorials for the website, which would show users how to complete certain sections.  After working through our MLPs again today, I "borrowed" an idea from one of the other organisations in our cohort to instead focus on the home page of CABmoney.

Although the video tutorials are something we still want to add at a later date, the new homepage (or after a bit more refining later in the week, a "splash page") would be a more achievable target (ie easier to do!) and would give us instant feedback through Google Analytics.

Through A/B testing, we could create two new splash pages and test which was the most effective live on the actual website.  This data could also be analysed against our current home page (already with years worth of analytics available) to see how a simplified approach would assist users get to the information they need quicker.

Also today, while talking about readability, the Flesch test was mentioned which would also be a good indicator on how effective changing our wording would be.

These two ideas combined would tie in well with some of the highest ranking ideas from our staff consultation, namely:

• Clearer, Simple Menus
• Move the Budget Calculator to Front and Centre
• Easy Read / More Visual Explanations

These were all high scoring ideas that could at least be partially addressed by a splash page before the main current homepage.

This revelation (and major change of focus) led to our Design Constraint to be as follows:

And that was only the first session on Tuesday!

The second session went into Service Blueprints, showing the flow through the service and how things connect.  A bit tricky as some people will use different sections of our website, but the general flow is shown in our blueprint below.

Thursday

Today's session was how to communicate our Minimum Lovable Product (MLP) as a "zine". Having only a red marker pen, which bled through the paper onto my other notes and fingers the end product looked a little gory.  If you want to get in a mess like this yourself, get an old leaky red marker pen and make your own zine.

Here's one I made earlier!

Had this been for the full CABmoney website, we could have filled a full book with problems and solutions, but this seems much more manageable for the next few weeks (which is the point!)

The contents worked through the issues with the current home page that we've identified, and show the solutions that should be achievable for our prototypes.  I cheated a little, by going over two pages for one of the images but with quite a small, focussed MLP there wasn't too much to say without it turning back into a Big Hairy Audacious Goal (BHAG).

Friday

A bit of a catch up day for Catalyst Project work today.  I had a good catch up with my line manager about the project now it's reached the mid-way point which was very positive.

The last couple of blog posts went up (including this one!) and managed to do a bit more work on some of the other aspects but still a bit of catching up to do on a few others.

Consultation Ideas

The best way to have a good idea is to have a lot of ideas

- Linus Pauling

At our staff consultation last week, we gathered together a number of ideas.  Some had been submitted through our CABmoney feedback form, and the rest came from the group during the meeting.  As we discussed them, we were able to produce a longlist (definitely not a shortlist!) of improvements and ideas that could be explored through the Catalyst Project and beyond.

Below is a summary of the ideas, roughly grouped by category (although a few ideas could easily span into multiple categories).  The scores for each idea are an average of the scores given by the group (a small group of seven people, so any further user research would be more valuable if scaled up).  These scores were for level of impact that the idea would have on users, out of ten where 1 was "least impact" and ten was "most impact".

Navigation

General navigation of the website was a key issue for a lot of people, with poor support for mobile devices and overly complicated menus being specifically highlighted.

CABmoney App

Score: 9.86

Having CABmoney as an app would make it much more convenient for people to access.  Similar, to just making the website more mobile friendly, an increasingly large number of people would benefit from being able to access CABmoney via their mobile phone or tablet.  The advantages of an app also include getting push notifications when key dates are approaching or for other reminders.

Mobile Friendly Layout

Score: 9.71

The current website was not designed for mobile devices, so can be difficult to use.  Our Google Analytics data shows that the percentage of mobile/tablet device users is increasing year on year.

Clearer, Simple Menus

Score: 9.14

Reducing the number of menu options would make navigating the website much easier.

Move the Budget Calculator to Front and Centre

Score: 9.14

It was widely agreed that the Budget Calculator (Financial Statement) should take “centre stage” on the website’s homepage as it was the section most people referred clients to and the most useful for a large portion of users.  Currently, it is behind a few screens and takes a few clicks to access.

Traffic Light System

Score: 8.43

A traffic light system was suggested that would clearly show what sections had been completed or still needed working on.   It could also be implemented to show how healthy somebody’s finances were in the Budget Calculator by highlighting any areas of concern.  A more visual approach was thought to be better for people to follow and understand.

Search Bar

Score: 6.30

A search bar would make finding some of the information on the website easier and potentially improve navigation, but it was generally thought that the site is better used when people work through the sections in order.  It may have more use as a tool for advisers looking up information, than for other people using the site for themselves.

Access and Support

This group of ideas mainly look at making the wording and processes easier for people to understand (alongside the navigation).  By using plainer English, and offering more levels of support and explanation, it was thought that the user experience would be much smoother.  

This was also coupled with ideas that sought to reassure a user before they started using the site, as it was thought that with a taboo and sometimes complicated subject, many users would be wary of using the site if it seemed too complicated.

Easy Read / More Visual Explanations

Score: 9.14

The text and information on CABmoney should be as easy to read as possible.  The current wording can be too technical in places, and may put people off using the website fully.

Tutorial Videos

Score: 8.86

Explainer videos throughout the site showing how to complete each section.  These would be screen captures of somebody filling in an example.  For added human interaction, the person doing the example could be shown in a thumbnail within the video talking the user through the process.

Links Out to Citizens Advice Contact Details

Score: 8.29

As some people using CABmoney may be outside our usual catchment area, it would be better to link out to the national Citizens Advice website where they can find the contact details of their local office if they run into difficulties, or need the help of an adviser.

Separate Adviser Section with Additional Information

Score: 8.00

As some of the information on CABmoney is aimed more at advisers, than general users it was suggested to move this to an “adviser’s only” section (with login).  This information could still be displayed in the current flow of the website, but only displayed if logged in as an adviser.

Case Studies of Other Users

Score: 7.86

In an attempt to reduce some of the stigma associated with managing debts, it was suggested that “case studies” (or success stories) from real users who had used CABmoney and were now “out the other end” would encourage people to use the system and see a light at the end of the tunnel before they began the process.

Introduction/Reassurement at the Beginning

Score: 7.43

Similar to the idea of including Case Studies (see above), a more general introduction that reassures people that there is a way out of debt was suggested to be included either on the home page, or early in the Debt Management Plan process.

Language/Translation Function

Score: 6.86

Not everybody using CABmoney will have English as a first language, so a translation option would help them read and navigate the site.  These days however, third party tools such as Google Translate can translate an entire website independently of having a specific function on the website.  Ensuring that the text was clearly written in English (see above) would aid any translation tool that was used though.

Debt Management Plan Improvements

These improvements were specifically for users of the Debt Management Plan (DMP) and/or the Budgeting Sheet sections of CABmoney. These are more involved processes but incredibly useful for the users (and form the main backbone of the site).  Although some users will just access CABmoney for the information available, the interactive sections that allow them to manage their own money are key.

Ability to Email Creditors Directly

Score: 9.14

As not everybody has access to a printer, alongside the fact that postage costs are always rising, it was suggested that being able to email letters to creditors would be a quicker, cheaper and more convenient method.  This would rely on creditor details being up-to-date and knowing if they were open to receiving correspondence by email.

Links to Access Credit Reports by Online Forms or Email

Score: 9.00

The forms required to request a credit report are now available online, this would also be a quicker and cheaper process than completing paper copies and sending/receiving in the post.

Expanding Budget Sheet Sections

Score: 8.86

Currently, some sections of the budget sheet actually consist of a number of values that need to be added together before entering (ie “Food/Housekeeping”).  These entries could have an option to expand out, so that individual items can be entered separately.  This would act as a prompt for the sort of things that could be included, and also mean that no manual addition was needed to enter the total value.

Self-Referral to Adviser

Score: 8.29

An option for people using the site who may run into difficulties to be able to self-refer to a Citizens Advice adviser.  Although useful, this would need to ensure that people are referred to the correct local office (as a website can be used outside of our geographical catchment area).

Reminders/Warning to Keep Reviewing Income/Expenditure

Score: 8.00

Generally, people enter their income and expenditure into their budget sheet or Debt Management Plan once and never return to it.  By reminding people (either by automated email, or notification if we were to develop an app) to keep these up-to-date, they will have a better understanding of their financial situation.  This is especially important at the moment as there are a number of outside factors that can affect somebody’s income and expenditure in these uncertain times.

Up-to-date Creditor Lists

Score: 7.57

As part of our Debt Management Plan section, we provide a number of contact details for various creditors.  Over the years, these have gotten out of date and need updating, though it would be a continuous process to keep up to date going forwards.

Disbursement of Payments

Score: 6.86

There was a suggestion that a system that allowed users to pay their creditors directly through CABmoney could be added.  Although useful, this would require a large amount of technology and security to be implemented along with other checks that the money was going to the right place and being received and accounted for properly.

Everything Else

Not everything fits into neat categories.  The ideas below looked at expanding CABmoney to incorporate other money management information (most of which we already have housed on a separate site), and also using it to help with our internal processes.

Not Just Debt

Score: 8.29

We currently have a separate site focussing on budgeting and income maximisation (http://moneysmart.nedcab.org.uk/).  It was felt that this should be combined with CABmoney so that the information was all in one place, and also so that there was information to help people avoid getting into debt, alongside information for those trying to get out of debt.

Easier Internal Referrals to Our Debt Team

Score: N/A (Idea surfaced after the scoring, but the feedback was very positive)

We currently use a PDF form to record client information which then goes to our Debt Team if the client is to be referred for specialist advice.  By replicating this form on CABmoney, the referrals could be made more seamlessly, and as a lot of the information is the same as on a budget sheet or Debt Management Plan, it could form the basis of completing one of those for the client.

Week Four (8th-12th March): Big and Hairy

Slightly shorter weeknotes this week, due to me having a shorter week at work, and also having already gone into some things in more detail earlier in the blog.

Monday

We had our first staff and volunteer consultation today, managing to gather six people with varying degrees of experience with CABmoney but lots of enthusiasm and ideas.  Thanks to everyone involved! Getting insights from the people who actually use CABmoney (either for themselves as an adviser, or assisting clients to sign in and enter their details) is invaluable.

First we went through the personas we already had for our clients, a range of six people who may need CABmoney (whether they knew it or not).  The one type of user persona missing was the "adviser" group.  If only I had a group of advisers at hand to help put some together.

Splitting into two groups, the team were able to define two new personas to cover different types of advisers who would use CABmoney under slightly different circumstances but still fit into our persona spectrums,

As it's quite a large part of these first few weeks, our work on personas and persona spectrums has been posted separately on the blog earlier today.

We then went through what ideas and thoughts people had about CABmoney.  As people were already bringing ideas, I left it a bit more informal which was good as it let people chat about and evolve the ideas more organically.  We still managed to get them all down and scored them for how much impact they would have on users (using a method that "kind of" worked, detailed here) and I'm going to do a more detailed breakdown of the ideas in a future blog post.

All in all, a great session, and thanks again to all who attended - I will no doubt be calling upon you again soon!

Tuesday

The first session was to start looking at a content strategy, with a number of personas to cater for and a lot of content on CABmoney, this will be a task for the zombie garden for now, and will definitely spill out onto a spreadsheet rather than a single slide!

Content by Visual Thinkery is licenced under CC-BY-ND

The second session focussed on finding our Minimal Lovable Product (MLP) by breaking down our "Big, Hairy, Audacious Goal" (BHAG) into more manageable chunks.  Breaking it down into smaller problems and plotting on a graph was very similar to work done in the discovery phase and I think that's where a few in our cohort (self included) veered more to identifying the solutions than the underlying problems.  It also meant the "big reveal" of showing the different quadrants of prioritisation was less of a revelation than it was intended!

BHAG by Visual Thinkery is licenced under CC-BY-ND

Wednesday

A bit of a catch-up day, with the personas and persona spectrums finalised (and now fully illustrated) along with the user journeys.  Felt better to get a few things ticked off the list and into the final playback slidedeck, but still a lot to do.

Thursday

Last day of the working week (for me), and with full intentions to work more on our Content Strategy, the phone didn't stop ringing (apart from when the email notifications were going off), so for now it stops in the zombie garden until at least next week.

The afternoon session was a recap of the BHAG into MLP exercise to make sure everybody had their problems defined, followed by "fast fives".  Again, this was a repeat of something we did in the discovery phase (only there were eight of them, not five!) and for my MLP (How might we improve the tutorials to make it easier for people to refer to and understand) seemed a little bit too narrow to find five solutions for (in five minutes), so some may have strayed into other aspects of the website.

They also didn't lend themselves very well to illustration outside of boxes and lines!

With a couple of days off to come, I'm just hoping the zombie garden isn't too overgrown when I get back!

Personas Assemble!

Personas are used in a project to identify different users, usually digging deep into who they are and what they want.  Once identified, they can be used to walk through different situations to ensure that their needs are met by the finished product or service.

Although started earlier in the project, we've only just managed to get all of our personas together to cover a range of users of CABmoney.  This was achieved by pooling the resources of a number of our advisers who know our client's needs as well as their own.

The full set of personas is available here with expanded profiles, but for the sake of clarity, I've shown a summary of our main eight below.

Although there are a number of differences between the personas, the main thing that connects them, is the level of involvement of both the end user (or client) and the adviser.  As we use CABmoney internally to help our advisers actually advise clients (and also for training purposes), the advisers are as much a "user" as our clients are.

We have split our personas into groups based on the amount of involvement of both the clients and the advisers, both of these increase and decrease for different circumstances (with a middle ground of 50:50 involvement from both).

Explorers

Some users will find CABmoney "out in the wild" either through search engines, links, or recommendations.  At the far end of our spectrum, these users will be able to get the information they need or be able to complete a Debt Management Plan (DMP) totally independently of us, and we would have no direct contact with them.  They will generally be more comfortable with technology and could be based anywhere in the country.

Assisted Users

Some "explorers" may get part way through the DMP and then require assistance, so they would contact us for help, therefore moving along the spectrum into the "assisted user" category.

The more common type of "assisted user" is somebody who access the Citizens Advice service through other means (telephone or pre-COVID face-to-face) and is assisted by an adviser to enter their details onto CABmoney and shown how to use it.  Some of these users may then be able to complete the process on their own, or may require further assistance as their circumstances change or if they struggle to work independently.

These users will always be local to our office and benefit from the support of an adviser as and when they need it.

Advisers

The third grouping on the spectrum are Advisers.  Some of our advisers use CABmoney as a tool for themselves to help them assist people.  As well as helping "assisted users", there will be some clients for whom it wouldn't be suitable to use CABmoney themselves.  This would be assessed by an adviser and is usually dependent on how comfortable they are with either technology or managing their own money.

In these cases, the adviser would still use CABmoney, entering the client's details and using it for calculations, producing letters, and keeping track of dates and finances, but the client themselves would have no involvement and likely not even know that CABmoney exists.

Summary

These three different groups highlight different needs and levels of support that would be needed when using CABmoney.  Depending on the level of involvement form both client and adviser, different features can be developed to help both whether working together or independently.

The user journeys below highlight how the experience of using CABmoney can be improved for all three groups on the persona spectrum.

Idea Prioritisation (Recipe - missing a few ingredients)

As part of the Catalyst project, we are encouraged to share "recipes", things we've built that can be shared with others who may be looking to solve the same problem.  I fully intended to do that with something I trialled for creating an automated "Idea Prioritisation Matrix", and while it wasn't a 100% success I think the process is still worth sharing and it may be something that can be refined in the future.

The Problem

The usual method for an Idea Prioritisation Matrix, is to let people put each idea on a sticky note (physically or digitally) and move them about the matrix depending on two criteria: the user value, and the feasibility (or effort).  Like this:

The problem I had, was that in the session I was running, I didn't know how many ideas were going to be generated (though I assumed it would be quite a few) and thought that a number of people moving that many sticky notes around would get a bit messy and cluttered.

Also, for the project in hand (the redevelopment of our CABmoney website) I thought that the "effort by organisation" part might be difficult for people to decide upon.  Without knowing what is easy or hard to develop for a website, this could have been skewed.

The Solution

Rather than attempting online sticky notes, I dropped back to a simpler method of scoring the ideas - Google Forms.  Most people are familiar with Google Forms, so I set up a fairly basic one with a question for each idea and a ranking of 1-10.

Because most of the ideas were being formulated during the session, I was able to easily duplicate the questions and add the new ideas in as we went, meaning that the form was populated as soon as we needed it.

I wanted to be able to visualise this in a way that Google Forms wasn't able to (the matrix), so I then linked up Google Data Studio.  After a bit of trial and error, it seemed that the data format directly from the form wasn't suitable for Data Studio, so I went back to the form responses spreadsheet to make some adjustments.

To save Google Data Studio having to do some of the "heavy lifting", I set up two new tabs in the form responses table to make it more manageable for Data Studio.

The first tab transposed the raw data, so that it was "long" rather than "wide".

In cell A1, I used the code

=transpose(Responses1!A1:AAN208)

which changed the data layout from this

to this

Then, in the other new tab, I set up cell B2 to again transpose the ideas with

=transpose(Responses1!B1:1)

Cell C2 as an average of the scores from the new "Transposed" tab. This formula would need to be manually copied down depending on how many ideas there were.

=average(Transposed!A2:2)

and an empty column for the "Ease of implementation" score that I would add manually once the results were in from the form.

The finished form (after the results were in) looked like this:

Over in Data Studio, I added this worksheet as a data source. I knew that I wanted the results to be displayed with the axes going through the center of the matrix, so as these were scores out of ten I simply added two new calculated fields that subtracted five from each score, so that the score range was now -5 to +5.

Plotting these new fields on a scatter graph would automatically create the matrix as soon as the Google Forms were filled in, this seemed much easier than getting people to move multiple sticky notes around.

For the visual effect, I added the image below (created by me, free to use) behind the graph so that the plotted points would fall into the various coloured bands.  The image below is rectangular as that was the shape of the matrix, but it can be stretched to fit any other dimensions if needed.

Did it work?

Technically, yes.

Though not quite as well as I'd have liked.

One issue was that in asking everyone to score the ideas out of ten, most people thought most of the ideas were good, so there was a bottleneck of ideas up in the top half of the matrix.  Nothing had scored lower than 6.4 for impact (or 1.4 on the adjusted matrix with minus 5 applied).

I was able to remedy this by changing the Y-axis so that it started at 0 instead of -5, which effectively stretched the results out from top to bottom.

Also, for the sake of clarity, I removed the data labels from the matrix as there was so much overlap of text.  Within Data Studio this works well, as the points can be hovered over to view the text but if exported elsewhere would need a key (the labels could be numbered ideas with a key to show what they are).

What could be done differently?

Ranking the ideas rather than scoring would give a better spread of results.  I had thought of that before asking people for scores, but Google Forms doesn't have a ranking option.  A few other online surveys did have the option in their paid for tiers.  Ideally an option to drag the ideas into place would make it easier for people.

With 21 ideas it's still preferable to moving all of the sticky notes in the session, which is a problem when taking the ideas from a group of people and then taking the votes.  If this was a finite number of ideas before the session, there would be more control over the voting but I wanted something more dynamic for the session.

Week Three (1st-5th March): Ignore the Jargon

Monday

The start of a new month and the slew of jobs that come with it (usually revolving around monthly reporting) meant Monday was a day for getting on with "the day job" so there was more time clear later in the week for Catalyst work.

Tuesday

Thankfully, the Tuesday WAO session was partially set aside for catching up a little and checking where everyone was at.  We were able to get a better handle on personas and feel a bit more "caught up" now after missing this in our Discovery Phase.

Following that we looked at Systems Ecosystems, using a very busy Miro board with many post-it notes, arrows and cursors flying about from all of the charities in our cohort.  Being able to show what we use and what we use them for highlights a few duplicates we already knew about (due to different pieces of software sometimes being better suited to different tasks or users) but always worth keeping tabs on this sort of thing so as not to reinvent the wheel (or not using three different programs to reinvent it!)

Wednesday

I think our Systems Ecosystem is finished, even if at times it feels a little like this:

But really, it's more like this:

Mapping Out Our System Ecosystem by Visual Thinkery is licenced under CC-BY-ND

Ours is certainly making full use of coloured arrows and post it notes.

On a side note, our trial of Trello has gone down well with our team of Social Media advisers, so we already have one "quick win" from the Catalyst Project that we've been able to put into action within the first three weeks.  This doesn't tie directly into the redevelopment of CABmoney, but just goes to show where the tools and methodologies can be used elsewhere across our organisation.

Thursday

Thursday's tech session was all about Agile Development, a phrase I'd heard bandied around before when Citizens Advice built their latest database system.  I hadn't quite got a handle on it (only a rough idea), but now I have a list of new buzzwords to use the next time I do a standup before breaking an epic into timeboxed sprints.

Then we talked about not alienating people with jargon, so maybe just ignore that last sentence!

Into Office Hours shortly afterwards, to get a bit more confirmation over those troublesome personas.  I think we're nearly there and with a couple more to add next week, they should be ready to share on the blog next week.

Friday

The end of another busy week!  Finalised the slides for our internal consultation on CABmoney (more on that next week) and sorted out a running order for it (hopefully I'm overestimating the time at two hours).  Fingers crossed the tech (including my internet connection) holds up as I've set up a Google Form that links to Google Data Studio to "hopefully" create a live Ideas Prioritisation map.

If it all works, I'll share how I put it together on the blog next week.

If it doesn't, we'll be back to pen and paper!

Week Two (22nd-26th February): Hot Mugs and Zombie Gardens

Monday

Started the week by finalising a date for our internal consultation about the CABmoney website.  We've managed to get together a selection of advisers with a good range of experience using the site and their input will be valuable.  Amazingly, there was a time slot that everybody could make so we're just two weeks away from putting some of the methods learnt in our discovery phase in practice to "rediscover" against our new focus.

Tuesday

Zombie Garden by Visual Thinkery is licenced under CC-BY-ND

Other than sounding like a Netflix TV series that I would definitely binge watch, Zombie Gadens are effectively a way to put ideas to one side until a better time.  Something we'll definitely be utilising!

The rest of Tuesday's session focussed on personas and persona spectrums.  Unfortunately, we didn't cover personas during our Discovery phase, so felt a little left behind.  Fortunately, the team at We Are Open (WAO) covered this by splitting the cohort into two groups to catch us up.  Being honest, it was the first time in the project that it felt like things were slipping as we were at a bit of a disadvantage due to different groups running the Discovery phase differently.  It's certainly salvageable though, with video guides offered by WAO and the open invite to Office Hours most likely being useful to us going forward.

We also started working on our Architecture of Participation, to lay out how people in our organisation can get involved and where they can find information about the project.  With the consultation group already set up, and this blog already available we have a good head start on this and we can continue promoting the work through our internal newsletters and meetings.

The work from Tuesday is all very much a work in progress, with the personas and architecture of participation working documents with a lot of refining to do.  They'll all be up on this blog when we're happier with them, but we are moving forward.  Back at the "office" (virtually) I asked some colleagues to put together some personas for our clients, and we will be looking at covering other user groups soon with people.  Not quite in the Zombie Garden, but at least on the borders!

Thursday

With a bit of a clash of sessions today, WAO swapped around the tech session and the office hours, so a few members of the cohort were able to attend "Designing Human Connection in Digital Services with Deepr".

A number of people didn't get the memo about bringing a hot drink, so didn't get involved with holding onto a hot mug and looking at everyone else in the meeting to create a sense of warmth!

There were a LOT of ideas in this 90 minutes, most of which I need time to digest and look through again.  While I can see the reasons behind a lot of them under the right circumstances, a number of charities were quick to dismiss "showing your room" or "lighting a candle" due to either a risk of confidentiality or burning the house down!

I did pick out a few ideas though that could be applied to the new CABmoney, particularly Lo-fi Hi and Mirroring which I think we can utilise to make a better human connection with people using the site.

That was soon followed by the WAO "Tech Session", much like last week's tech session there were a few things we already knew about as ways to acquire software and new technology but also a few others that sound like being well worth a look through.  

• AlternativeTo
• Open Design Kit
• Open Practice Library

Lots more to go through, so for now, they're in the Zombie Garden (just beside the tomato plants) but definitely useful resources.

Friday

Onto Friday, and it's blog writing (which if you're reading this was a success), another tweak of our personas (now with added wonky drawings) and an outline plan for the staff consultation session which is now just over a week away - how time flies!

Week One (15th-19th February): Definition

A whirlwind week of workshops, meetings, breakout rooms, icebreakers and notes.  Lots and lots of notes.

I was only briefly involved in our Discovery phase but now found myself fully immersed in the Definition phase after a change of tact (see our previous blog posts) and a better focus (hopefully!) on exactly what we were looking to achieve from this project.

Tuesday

The first port of call was a workshop on "Working in the Open" (which sounds like it would be much too cold to do in a frost-tinged February) but covered ways of sharing our work throughout this project.  One form of sharing our workings and process was to post to a blog, so if you're reading this now it's a proven method!

Tuesday afternoon moved to the first session with We Are Open for the Kickoff workshop and a chance to meet the other organisations in our cohort.  More icebreakers and introductions, a short break and then into getting something down on paper (or on screen), Stakeholder Mapping.

Stakeholder Maps by Visual Thinkery is licenced under CC-BY-ND

Plotting various stakeholders along two axes (one for "interested" they are in a project, and one for how "involved" they are) helps show how these various groups should be dealt with through the project's life.

Adding a few extra stakeholders that fit into our project I was able to fairly quickly create a map thanks to the techniques used in the session.  The first bit of "homework" done!

Wednesday

No sessions today, but I realised we needed to think more about our project, especially with us changing focus relatively late on.  Our Discovery phase was still important, but with moving away from looking at video advice and onto redeveloping our CABmoney website, I felt we needed to re-Discover.

Fortunately, we had a staff meeting planned for the afternoon, so I was able to secure a spot to talk to people about the project.  Still formulating what it is we need, I explained why we were doing this and that I'd like to get a group together to do a consultation on what people want CABmoney to actually be.  One positive response from a colleague before the end of the day, made it seem all worthwhile!

Thursday

The afternoon session covered "Audience Analysis".  Well presented, but having worked with our websites for some years I was already familiar with Google Analytics and I just discovered Google Data Studio a month ago.  We don't have a "Data Model" though... hmm, one more for the list!

Session over and if nothing else, I learnt how difficult it is to come up with a three word "how are you feeling" at short notice!

I spent the remainder of the afternoon starting this blog and going back through our client consultation data now that we had a few extra month's worth to pore over.  A few extra thoughts on how and where we can gather data for CABmoney, but that can wait until Friday.

Friday

Using our Google Analytics stats plus information from the CABmoney database, I was able to put together some more data and post up in the blog about what CABmoney is, and the first steps into looking at what we need to do with it.

To get further data, I added a Google Form survey to CABmoney and posted about it on our social media channels. Hopefully, we get enough engagement to discover some more about user experience with the website that will help us get a clearer course of action.

Week one is over, and in some ways it feels like starting over from scratch but at least we have a plan!

What is CABmoney?

In our first blog post, we talked about how our initial thoughts had changed from finding different ways to offer advice and had we were now looking at how we can best revitalise our CABmoney website.

But what is CABmoney?

CABmoney is a debt advice information guide and resource developed and maintained by Citizens Advice North East Derbyshire.

The content was developed by utilising the many years of experience and knowledge of our debt team, other bureau advisers and external advice specialists.  The coding of the website was all written in-house by the bureau I.T co-ordinator.

CABmoney incorporates a unique online assisted self-help Debt Management Plan (DMP) that automatically generates letters / financial statements, does calculations and is all saved securely online. You can access and update your personal Debt Management Plan whenever you want and you are in control.

Based on an established, award winning system

Citizens Advice North East Derbyshire launched the "mymoney" debt resource in January 2012 and were awarded the IMA Money Advice Performance Award for Best New Initiative shortly afterwards.  The site has evolved since (including input from Citizens Advice Specialist Support), and "mymoney" has become "CABmoney" as we have made it available to all local offices in England and Wales.

As more local offices use CABmoney, it will continue improving through their feedback and we always look forward to hearing about innovative uses of the system. It has already been used to secure funding to launch new projects, giving more people free access across their community.

Why might it need changing?

CABmoney's current form was written in 2014, which in website terms is extremely old!  Technology has moved on so much in that time that there are better ways of doing things.  Back then, designing for mobile devices was less important and so CABmoney today can seem "fiddly" on a smaller screen.

We designed CABmoney with both our clients and our advisers in mind, what we don't know is whether this resulted in a website with "too much information", or was too "technical".  Do we need more information, or less?  Can it be streamlined and made easier for those who used it independently without ever contacting us directly for advice?

These are the questions that we will try to answer through this current stage.

CABmoney by the numbers

We are able to ascertain a lot of information about the current usage of CABmoney, both from Google Analytics and its own "back end" database.

Even though we are a regional Citizens Advice office (covering the Local Authorities of North East Derbyshire and Bolsover) CABmoney has been used all over the country. We can see this on Google Analytics (though this isn't an exact science due to how the location of a user is collected).

Also in Google Analytics, we can see that device usage has changed over the last seven years.  As a percentage, desktop computer usage is declining and mobile phone usage is increasing.  Surprisingly, tablet use is also in decline, meaning that the current trend is for more people to be using their mobile phones than more traditional larger screens.

In the CABmoney database, we can see more details on how people use the Debt Management Plan (DMP) area of the website.  This is the only part that somebody would need to set up a (free) account to use as it stores budgeting and debt details in order to generate budget plans and letters to help deal with their debts.

As of February 2021, just under 10,000 people have signed into CABmoney.  As these will only be the people signing up to use the DMP it is only a fraction of total users, but it is a good indication of engagement.

The rate of users signing up to use the DMP, has increased over time with a noticeable increase across 2020 (though the largest spike was in November 2019).  This shows that there is more demand than ever for people to manage their debts with an online system such as CABmoney.

Do new signed in users actually equate to more people completing the Debt Management Plan?

In order to allow people to pick up where they left off, the system tracks how far through the DMP they have progressed.  In analysing this, we can see that a high percentage of people (78%) sign up for an account but then go no further.

This is an important area to look at, as we cannot say why people are signing up for an account but then seemingly not using it.

Whether it is because they are signing up when they don't need to, or once they sign up they are having difficulties in using the system we will need to make the user journey as straightforward as possible. 

Our following research and consultations will focus on these areas as we aim to improve CABmoney, not just technically in the coding, but more importantly the user interfaces and clarity of information for users.

Update (24/02/2021): We have removed a number of obvious "bot" accounts from these graphs, though there may be many more.  This would at least partially explain the number of people signing up for an account but not using the resources.