Engage for ChangeIn September I joined a small group piloting an online course called Using Data in the Health and Social Care Environment. Here’s my initial impression of it.

The pilot is one of three being run by the Department of Health for NHS England to provide education and training for volunteers working to improve health and social care.

The courses

The three courses are:

“Understanding the Health and Social Care Environment” — exploring the stakeholders in the current health and social care environment. Course participants learn about influencing change and undertake a practical project.

“Facilitation and Engagement in the Health and Social Care Environment” — covering the engagement of seldom heard groups, and facilitation skills. Course participants plan and carry out a practical engagement exercise.

“Using Data in the Health and Social Care Environment” — focusing on key terms used in data, and where to get data to support a case for change. Course participants plan a report and find data to back it up.

The courses are provided by a group calling itself Engage For Change, which in turn is run by Artworks Creative Communities, a charity in Bradford that runs the Delius Arts & Cultural Centre, amongst other things. Engage For Change doesn’t appear to provide any courses other than the three that are being piloted.

The course I’m doing has three sections and includes six assignments. The other two courses have face-to-face assessments, but mine is completely online. There’ll be an evaluation day in December when participants from all three courses will come together and provide feedback.

Learning environment

The courses are provided online through the Engage For Change website, which is written in PHP and has a simple, clean design with just a few peculiarities. The course also uses PDF forms and Word documents. There doesn’t seem to be any reason to have used three different technologies when the whole thing could have been done on the website in PHP.

The assignments demand Adobe Acrobat Reader. But as Reader isn’t stable on this computer and doesn’t work at all on my Ubuntu laptop I’ll not be using it. Instead, on this computer I’ll use my usual PDF reader, PDF X-Change, and on my laptop I’ll use the built-in document viewer application. Both of them seem to handle PDF forms perfectly well.

There are some signs that the site’s design draws on the Moodle learning platform, but it doesn’t seem to acknowledge Moodle explicitly and the content doesn’t look much like other Moodle courses. There have been a few technical errors and peculiarities in the web pages. Some of them have been fixed quickly, indicating good technical support (despite poor design and testing).

The most obvious peculiarity of the site is a big button labelled “Switch to high contrast” that changes the fonts slightly. It has the unfortunate side effect of hiding keyboard focus, so that if you are using your keyboard to navigate around the site it’s much harder in high contrast mode. And changing the fonts slightly isn’t really the same as high contrast. People with visual impairments will already have set their whole computer to meet their needs as best it can, and it’s not obvious that this button is really much help to anyone.

High contrast mode uses one of several cookies, and there is (correctly) no warning. It’s impossible to tell whether this is correct by intention because someone is unusually knowledgeable, or by mistake because no one even thought about it.

The high contrast code uses jQuery downloaded from Google Inc., which is technically a privacy violation because it means Google gets data on everyone’s usage of the site, that it can then sell to advertisers and pass on to agencies of the US government. There’s no obvious privacy policy on the site.

Each course has an online forum. These seem to have been cobbled together by hand from some Moodle components, so they don’t have the rich features of normal online forums and there are some minor bugs.

When I left my computer to do other things my session timed out and I had to log in again, which seems pointless.

At this early stage the course content seems to be ordinary linear text that has been pointlessly webified to make it less easy to read. There’s no obvious way to print a module nicely or download it to my Kindle to read offline.

Part of the webification is that some paragraphs are detached from the main body of the text in pop-up windows. You have to click on something to pop up the paragraph. This is annoying because it makes it harder to understand the context of the paragraph in terms of the paragraphs around it. Also, a bug in the web design means that focus disappears when you close the popup, so you can no longer tell what you clicked to make the thing pop up.

The course is divided into sections and tasks. Really, each “section” is just a chapter and each “task” is just a section. Many of the “tasks” don’t actually give you anything to do, only a web page to read.

Using data?

In the first section of my course I’m told I will:

“look at how to begin a data collection exercise and begin to explore some of the terms associated with data, and look at the credibility of different types of data.”

This is a little surprising, because data collection is usually distinct from data use, and I signed up for a course about data use. Maybe the course uses the word “collection” in an unusual sense.

Later I’m told I’ll decide what the case I would like to make is:

“In order to support your case for change or improvement, it is not enough just to provide an argument. Your argument also needs to be supported by facts which have been processed, interpreted, structured and presented appropriately for the given context. When we do this, we create information.”

This is also surprising because making a case for change or improvement isn’t the kind of role I have, and isn’t the kind of role I’m ever likely to take on.

So, already, the course seems to be heading in unexpected directions based on unsupported assumptions such as that people who use data must also collect data, make cases for change and create information. The course title may simply be badly chosen. This seems to be a course about report writing in which using data plays a small part.

Task 1

Task 1 doesn’t have a title. It’s just some introductory stuff, a brief overview of the section.

Task 2: Key Steps in Using Data

Although this task’s title makes it look as if it’s about using data, all it does is briefly define the term data in a slightly unusual way. After that it barely mentions data at all. It’s mostly about structuring a case for change.

Case study

A case study supposedly describes how a body known as Voluntary Action A wished to encourage social prescribing by GPs. It’s not clear what the A stands for or when this happened. There is a body known as Voluntary Action Angus that is sometimes known as VAA, but it’s in Scotland. We’re told “they identified as their audience the people who commission the GPs – the local NHS Commissioners.” But in England GPs were commissioned nationally until April this year. If the case study is fiction it would be better to know for sure that it’s fiction.

A real case study would have been better, because we could have found real life background information about it. Some bodies with Voluntary Action in their titles have been involved in promoting social prescribing in various parts of the country, but at this stage I wasn’t able to tell which one (if any) was the basis for the case study.

Task 3: Analysing your Stakeholders

This says we:

“will need to identify the stakeholders to whom you wish to present your case. A stakeholder is defined as “a person who has something to gain or lose through the outcomes of a planning process, programme or project.”(Dialogue by Design 2008). Put simply, your stakeholders are all those who will be affected by what you are proposing to do.”

It would have been nice for the reference to be a link. Dialogue by Design is a London public engagement consultancy, it turns out.

Their definition isn’t terribly useful, as the text goes on to explain, because the stakeholders who matter are the ones who have influence. Indeed, the people who have influence matter even if they don’t have anything to gain or lose, making the definition of stakeholder a little suspect.

Task 4

Task 4 has no title. It starts by defining quantitative and qualitative data rather confusingly. The trouble is that, for example, if you count the number of people who say they like something, the fact that you’re using numbers to do your counting doesn’t make the data quantitative. The numbers aren’t measuring any well-defined quantity.

Then the task defines primary and secondary data rather confusingly, too. The problem here is that hardly any data is ever primary in practice. When decisions are made, the data they are based on is always secondary (unless the decision-maker personally made the measurements, which is highly unlikely in health and social care).

The task makes the point twice, in bold text both times, that you must always specify the sources of secondary data. For example, as part of its case study the task cites the source of some data as a report published in 2013 by Sheffield Hallam University. Curiously, this can’t be the same case study as before, because in this report the Voluntary Action group is VAR (Voluntary Action Rotherham), not VAA as before, and in this study the work was led by GPs:

“as part of a wider GP-led Integrated Case Management Pilot”

Whereas in the last case study the VAA was trying to persuade the GPs.

The text of the task implies that the figures show social prescribing caused reductions in hospital episodes, even though the report itself refutes this, warning:

“it is not possible to attribute these changes directly to Social Prescribing”

Most weirdly of all, although this task strongly emphasizes the need to specify sources of secondary data, the data it quotes doesn’t come from the report it cites. For example, the task says there were 173 A&E episodes before receiving social prescribing, but the number 173 doesn’t appear anywhere in the report. It must have come from somewhere else.

The next part of the case study, on qualitative data, contains what it says is a fictional case study without citing any source. But searching the Internet reveals that it has been taken from one of several other documents related to the same work in Rotherham as before. In those documents it’s presented as the story of a real patient, not fiction. For some mysterious reason the name of Crossroads Care in Rotherham has been redacted in the task.

The story tells of a carer, Mrs A, who gets help from Crossroads together with respite care, freeing her to go out when she needs to. This is part of a care plan following an assessment of the household’s needs. It’s presented as an example of social prescribing in action, although my understanding is that carers are entitled to assessments by law. So it’s not clear to me what the story is really an example of.

A big button labelled “Reveal Key Points” highlights almost all the text of the case study as key. I thought this was unhelpful. The button doesn’t change into a “Hide Key Points” button afterwards, leaving some of the text a bit difficult to read.

A tedious exercise on classifying data as Primary/Secondary and Quantitative/Qualitative follows. The conclusion is:

“This hopefully shows you that data from varying sources and of varying types should be used wherever possible, in order to give a full picture and a sound evidence base.”

It doesn’t show anything of the sort. I agree that having a full picture is nice, but this task doesn’t explain why it’s nice. I don’t agree that simply having a mix of different types of data makes the evidence base sound.

Task 5: How good is the data?

A paragraph on the options for data collection doesn’t say anything very useful about the options. It does clarify that “data collection” in this course doesn’t quite mean what it means anywhere else, which is unfortunate. In this course it seems to include re-using data that someone else actually collected.

Then there are a few paragraphs about the validity of data, again using the coloured boxes to make it more difficult to read.

The final paragraph, on bias, seems to suggest that “emotional or financial connection with the subject matter” could be a source of bias. But this seems to contradict a previous task that suggested “primary” data (data collected by the person making the case for change) is a good thing.

This task finally changes the subject from how good the data is to how it’s interpreted, introducing a new source of confusion between interpretation and presentation.

The example given concerns a self-help group for depression that takes part in a community art project. It asks how to interpret changes in participants’ medication, with a view to obtaining further funding for the project.

If you’re strictly honest about the outcome you can’t use it to make any case for funding. The only way you can make a case is to interpret the results in a biased way. It’s an interesting scenario whose moral implications aren’t explored further.

Task 6: Assignment

The final task is a marked assignment. The assignment is pointlessly presented as a PDF form, not a page on the website. On the form you have to write stuff in a dozen text boxes, and fill in some other text fields before saving it and uploading it back to the website. There’s also a cover sheet to complete and upload, but, again pointlessly, it’s a Microsoft Word document.

The text boxes are limited to 800 characters each. For the whole assignment I wrote about 900 words in total, but I didn’t try to fill all the boxes to their limit.

The assignment claims to support one of the course’s learning outcomes:

“Be able to plan a data collection exercise to support a case for change in health and social care”

However, only one of the dozen text boxes is anything to do with collecting data, and that just asks what data you might need to collect.

The assignment also says:

“You need to show through this assignment that you can:
• Analyse information sources to assess their usefulness
• Clarify information gaps and develop responses to them”

But the text boxes don’t ask anything about analysing information sources. The only relevant text box is the one I mentioned above that asks what data you might need to collect.

The rest of the assignment is to do with planning to write a report that makes a case for change. It reinforces my impression that this course was originally written as a “Writing a report” guide, and then tweaked a bit to make it look like a “Using data” course. It wasn’t tweaked enough.

Possibly the worst feature of the assignment is that most of the things it asks you to do are things the other five tasks in this section give you no help with. For example, the first question in the assignment is, “What are you trying to change?” But the course gives you no practical assistance in deciding how best to make a difference in health and social care. You’re on your own.

If the rest of the course is like this, the entire thing will be completely pointless. A course has to provide learners with some knowledge and skills, and then provide some opportunity to practice using that knowledge and those skills.

Section 1 of this course has provided only very limited and problematical knowledge, and the assignment then provided an opportunity to practice completely different things that weren’t in the course content. It’s a mess.

Maybe Section 2 will be better.


About Rod

Chairman of the Gloucestershire charity Suicide Crisis, Vice Chair of Relate Gloucestershire & Swindon, and an enthusiast for public involvement in the NHS.
This entry was posted in Engage for Change. Bookmark the permalink.

One Response to Data

  1. Francesco Palma says:

    Rod you do not disappoint.
    Just started using data course last week late start due to drop out while I was on waiting list, if not before see in London or Bradford in December for evaluation day which again I am sure that you will not disappoint.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s