Evaluation and assessment of the usefulness of a mail delivered personalised diabetes information booklet and the association of non-response with clinical risk: the WICKED Project

Syed MR Gillani,1 A Nevill,2 Baldev M Singh1

1 The Royal Wolverhampton NHS Trust

2 University of Wolverhampton

Address for correspondence: Dr Syed Gillani
Diabetes Centre, New Cross Hospital, Wednesfield Road,
Wolverhampton, WV10 0QP, UK


Introduction: Patient activation can promote partnership working between people with diabetes and their healthcare professionals. We sent to people with diabetes a personalised, structured information booklet containing the results of their latest nine key care processes in order to inform and activate them. We present the findings of a survey to assess the utility of this report, with an analysis of the association of non-response to the survey, a surrogate for poorer patient activation, with adverse diabetes and clinical outcomes.

Methods: All 14,559 people with diabetes in the Wolverhampton health economy received a mailed report of the results of their latest nine diabetes care processes. Of these, 6,282 patients aged <75 years were mailed this report twice; 1000 of these 6,282 patients were selected randomly to receive a structured questionnaire to assess the report’s effectiveness.

Results: Of 1,000 patients, 419 (42%) responded (mean age 62±10 years, 246 males, 249 Caucasians, 389 had type 2 diabetes). Patients found this report useful (89%), a source of knowledge (78%), a source of increased confidence (74%) and it helped them understand their diabetes (78%). Non-response was associated with significantly higher surrogate markers of micro- and macrovascular risk.

Conclusion: A structured and personalised diabetes report, without direct professional or health service intervention, may improve the understanding and confidence of people with diabetes in their self-care and it may help to activate them to take a stronger partnership role in their health care. Non-response as a marker of patient activation is associated with increased clinical risk.

Br J Diabetes 2016;16:179-184

Key words: care delivery, patient activation, patient engagement, information provision, diabetes


The increasing burden of long-term conditions such as diabetes1 has increased emphasis on improved self-management.2,3 Care planning is a high priority in modern healthcare, but achieving an equal partnership between healthcare professionals and people with diabetes needs a clear understanding of the nature of the transaction and certainly not the imposition of a plan by the healthcare professional.4-7 An effective partnership requires enablement, empowerment and engagement of the person with diabetes within a transactional model based on concordance, rather than compliance.8

The concept of patient activation encompasses engagement, empowerment and enablement,9 but seeks to extend this to the measure of specific outcomes, indicating that the person with diabetes has become demonstrably active in their own healthcare. Patient activation may be considered as an outcome of diabetes care provision in its own right,10 as people who are engaged and actively involved in the management of a long-term condition have better outcomes.11 Such empowerment is conventionally sought via diabetes education aimed at improving knowledge, awareness and expertise in self-care, facilitating engagement in the proposed dialogue. Unfortunately, the provision of structured education for people with diabetes in the UK is suboptimal with uncertain outcomes,12,13 and outcomes from current care planning processes are also little understood.14 Other client-focused interventions promoting health literacy, self-care, access, care experience, service development and, not least, decision making have been found to be effective,15 including pre-consultation information,16-18 goal setting19 and organisational measures.20,21 The provision of written, individualised information is known to have a positive effect in diabetes consultations.22 A person with diabetes provided with good information can enhance their own knowledge,23 perhaps enhancing further their need to know and act.

Written information is valued,24 but it remains unclear if such information per se can activate people with diabetes (in the sense that we have defined). Accordingly, we have recently demonstrated the perceived benefit to patients and doctors alike of a structured information booklet in the setting of specialist clinic consultations and care planning in our local model of diabetes care, Wolverhampton Interface Care, Knowledge Empowered Diabetes (WICKED).25 We also demonstrated in a large randomised controlled trial (RCT) that this booklet was associated with a significant increase in patient activation.26

We have surveyed the impact of a booklet providing up-to-date information on the results of nine care processes among people with diabetes. Responding to the survey may be a potential marker of patient activation, and we have also analysed the associations between failing to respond and adverse diabetes and cardiovascular outcomes. We believe this to be the first report of the perspective of people with diabetes regarding such information delivered across a whole health economy, independently of the influence of healthcare professional input. It is also, we believe, the first report of the association of adverse diabetes outcomes in relationship to a potential marker of patient activation independent of selection bias relating to surveys in patients who attend care.


A structured diabetes report, ‘My Diabetes, My Information, My Plan’, containing individualised information about nine key care processes in diabetes was designed by a three-phase development process, as described previously.25 It contained person- specific information on the nine key diabetes processes that inform UK National Health Service (NHS) diabetes healthcare delivery: HbA1c, body mass index (BMI), blood pressure, urinary albumin:creatinine ratio (ACR), serum creatinine, serum cholesterol, recorded smoking status, digital retinal photography and foot examination. The booklet is available to view online at Each process was scored as 1 if missing and 0 if completed within a 15-month period. Each individual thus had a summary score calculated to assess the accrued process failure, the Failed Process Score (FPS), with range 0–9.

The FPS is used as a direct measure of patient activation, with outcome data from the process measures recorded for the 15-month time frame. Primary coronary heart disease (CHD) risk was calculated using the Framingham method where complete data were available. Secondary macrovascular risk status was defined as the presence of cardiovascular, cerebrovascular or peripheral vascular disease. Data were accrued into the local diabetes information system via data linkage with multiple other systems including the primary care data warehouse. The rolling processes of data quality management have already been published, showing a very high level of data accuracy (>95%).27

The study follows on from our previously reported RCT involving 14,559 people with diabetes.26 The structured report containing individualised information was initially sent to 8,725 people with diabetes in Wolverhampton in the active limb of a RCT which has reported positive 3-month outcomes.26 The mailing was subsequently repeated after 3 months and, at this stage, included all people with diabetes in Wolverhampton. We selected all patients aged <75 years who had received the booklet twice (n=6,282) followed by a further random selection of 1,000 patients (by Excel 2011 random number generation), and sent them a questionnaire to evaluate the booklet. Initial non-responders received a reminder after 2 weeks and the receipt of returns was closed at 4 weeks. It is the response to this questionnaire that is used as a potential second measure of patient activation.

Data were analysed (SPSS Version 22) using the Student's t-test (means) or the chi-squared test (proportions) with statistical significance taken at p<0.05. Primary 10-year CHD risk was reanalysed in univariate regression analysis after log conversion (non-normal distribution) in order to adjust for age differences between groups.



There were no significant differences for any demographic or other characteristic (Table 1) between those who were surveyed (n=1,000) and those who were not (n=5,282).


Of the 1,000 people surveyed, 419 responded. Non-responders were younger and less likely to be of non-white Caucasian background compared with responders, but there were no significant differences between these groups for gender, index of deprivation or type or duration of diabetes.

Survey evaluation of ‘My Diabetes, My Information, My Plan’

The results of the evaluation of ‘My Diabetes, My Information, My Plan’ from the 419 respondents are presented in Table 2. A positive response was expressed for every question: 72–85% of respondents answered ‘Yes definitely’ or ‘Yes to some extent’ to every question relating to whether the report would improve their own diabetes care, and 87–90% gave one of these answers to questions relating to the utility of the report for anyone with diabetes. A frequency of once or twice/year for the report was supported by 85%.


Patient activation and outcomes

The FPS was significantly worse in non-responders than in responders, as was process attainment in every single one of the nine subsidiary processes (Table 3). Among those where outcomes were ascertainable and who had process measures within the 15-month time frame (Table 4), non-responders had a higher HbA1c, BMI, serum cholesterol and were more likely to be smokers (there was no significant difference in mean systolic BP). Significantly more non-responders had HbA1c >9% (74 mmol/mol) and serum cholesterol >6 mmol/L, while there was a trend towards a higher proportion with systolic BP >160 mmHg. The 10-year primary CHD risk score was adjusted for differences in age between groups in univariate analysis and was significantly worse in non-responders (Figure 1), independently of age (F=22.2, p<0.001). However, the effect was not statistically different between groups within separate age bands, despite the apparent separation among the middle age bands.




Based on these data, an estimated 27 additional adverse cardiovascular events might be expected to occur over the following 10 years in the non-responder group, based on their CHD risk at the time of the evaluation. There were no significant differences in the crude prevalence of established microvascular disease (eye, foot or renal) or established secondary macrovascular disease.


The intervention we described is a simple but systematically constructed and personalised information booklet for people with diabetes. In this article we have sought to evaluate this patient-centric intervention ability to promote patient activation; to extend our understanding of the co-associations of poorer patient activation with adverse demographic and clinical outcome; and to reiterate the feasibility of navigating poorer patient activation and thus potentially influence the recognised associated adverse outcomes.

Our explanation relates to Kilpatrick’s four levels of evaluation27 (KP 1–4), bearing in mind that the meaning of evaluation and assessment are often confused. In this context, evaluation should be taken as what the recipients – our patients – thought of what was done (KP 1) and assessment as being that which the service measured as an outcome relating to an intended benefit (KP 2–4).

In this sense, KP level 1 was initially considered in a small-scale published pilot evaluation of the information booklet by our patients.25 Here, a much more rigorous and systematic survey, without any form of selection bias, again indicates clearly that people with diabetes want to have their individualised information in a structured, easily understandable format and that they found it beneficial in a number of domains relating to their diabetes care. We emphasise that KP-1 does not and cannot prove that the booklet was effective in relation to patient activation, and it simply shows that patient evaluation of the booklet was very high; they liked it, hopefully as a result of the considerable effort in design and pilot testing. We acknowledge the limitation of this evaluation, and the response rate to our survey is a potential source of bias. The response rates to health surveys average around 60%28 when using maximal techniques,29 with response rates of about 50% typical of surveys with two reminders.30 There are many confounding factors determining response rates to surveys that are epidemiologically based, but a crucial key factor is the resampling of populations that have participated in previous surveys,31 which in itself is a perverse bias, as is the sampling of those that are engaged in care (e.g. attenders). We thus emphasise that our survey population was a random sample that did not differ significantly in any regard from the wider cohort, with no selection of any description. This mitigates the potential for bias arising from surreptitious selection, but certainly will be associated with a more realistic non- response rate. Additionally, the findings were exactly in line with the outcomes from our much smaller scale pilot in which the response rate was greater than 60%.25

We did not measure KP level 2 by intention, which would have amounted to a knowledge test. Here, our a priori interest was not in knowledge acquisition but in action outcomes (i.e. performance). However, we have published the 3-month outcomes of a large RCT equating to KP levels 3 and 4. The intended benefit, a true measurable outcome – in other words, our assessment – was demonstrating that patient activation occurred as the measureable impact of the patient booklet.26 We will shortly submit the 1-year outcomes which not only show improvements in failed processes but also a significant benefit in HbA1c. To our knowledge, this is the first prospective demonstration of the use of an information tool (as an intervention) to improve patient activation in that its use resulted in a significant improvement in the completion of diabetes processes as shown by a reduced FPS.26,32-34

Patient activation is a broad concept that encompasses patient engagement, empowerment and partnership working.33 Four levels of patient activation have been described, where the individual (1) does not play an active role in their health; (2) does play a role but lacks confidence or knowledge; (3) is beginning to take action to improve their health; or (4) is able to self-manage and self-care.33 Cross-sectional studies suggested that poorer patient activation may be associated with poorer outcomes in the general population as well as in people with diabetes.10,35 We did not use a validated patient activation measure tool,36 which was beyond the remit of our project, but we took FPS and a failure to respond to the survey as indirect measures of patient activation. This is a limitation of our study, but highlights the conundrum that formal measurement of patient activation can only be undertaken in those who engage in care. A poor FPS score and low survey response may reflect poorer patient activation which may in turn be associated with measurable increased risk. In the present study, non-response to the survey (suggesting potentially poorer patient activation) was not only significantly associated with a worse FPS as another marker of activation, but also with adverse clinical risk as reflected by HbA1c, BMI, cholesterol, smoking and primary CHD risk score.

However, poorer patient activation, reflected in non- response, does not mean that the information booklet is necessarily ineffective in modifying that risk. Our preceding publication clearly demonstrated a greater impact on diabetes access and process measure outcomes in the poorer FPS categories.26 Increased access and process attainment, followed by subsequent intervention, may therefore modify the adverse clinical profile in these hard-to-reach patients.


In summary, people with diabetes are able to understand their most important diabetes-related information when it is presented to them in a simple but structured format. The booklet is easy to generate and its low-tech presentation belies the high standard and governance of data integration and quality assurance required to produce it. The booklet should be easily reproducible in other health economies and it can be disseminated independently of healthcare professionals and distributed systematically across a whole population. It provides for people with diabetes an opportunity to reflect on their own status and take action, facilitates their consultations with healthcare professionals25 and promotes patient activation, even amongst those with a poorer baseline FPS score.26

Whether its prospective use improves key clinical diabetes outcomes via the enhancement of patient activation remains a key question. However, the ultimate intention of promoting patient activation is that of improving measurable surrogate and hard endpoint clinical outcomes. We hope to address this in the final analysis of our current RCT at 1 year post-intervention, potentially influencing the relationship between poorer patient activation and adverse clinical risk.

Conflict of interest None.

Funding British Medical Association (BMA) Joan Dawkins Grant 2012 partly funded this project but has no role in the writing of manuscript or decision to submit it for publication..

Study registration The study was registered in the UK national research database (UK CRN ref: DRN 795, available at Search/StudyDetail.aspx?StudyID=14324) and at (NCT02200965). Ethical permission was obtained from NHS Health Research Authority (NRES committee North East-York, REC Ref: 13/NE/0052) and further clarification was obtained from National Information Governance Board.


1.       World Health Statistics. Geneva: World Health Organisation, 2012. Available at: eng.pdf?ua=1 (last accessed Jan 2015).

2.       Cayton H. Patient engagement and patient decision-making in England. The Commonwealth Fund/The Nuffield Trust, 2004. Available at: (last accessed Jan 2015).

3.       Department of Health. Self Care – A Real Choice; Self Care Support – A Practical Option. London, 2005. Available at: (last accessed Jan 2015).

4.       Department of Health. High Quality Care for All: NHS Next Stage Review Final Report. London, 2008. Available at: (last accessed Feb 2015).

5.       Eaton S, Walker R. Partners in Care: A Guide to Implementing a Care Planning Approach to Diabetes Care 2008. Available from: (last accessed Feb 2015).

6.       Department of Health. The Expert Patient: A New Approach to Chronic Disease Management for the 21st Century. London, 2001. Available at: (last accessed Feb 2015).

7.       Department of Health and Diabetes UK Care Planning Working Group. Care Planning in Diabetes. London, 2006. Available at: http://www.diabetes. (last accessed Feb 2015).

8.       Chatterjee JS. From compliance to concordance in diabetes. J Med Ethics 2006;32:507–10.

9.       Department of Health. Our Health, Our Care, Our Say: A new direction for community services; Health and social care working together in partnership (White Paper). London, 2006. Available at: (last accessed Feb 2015).

10.     Remmers C, Hibbard J, Mosen DM, et al. Is patient activation associated with future health outcomes and healthcare utilization among patients with diabetes? J Ambul Care Manage 2009;32:320–7. http://dx.doi/org/10.1097/JAC.0b013e3181ba6e77

11. Wong CK, Wong WC, Lam CL, et al. Effects of Patient Empowerment Programme (PEP) on clinical outcomes and health service utilization in type 2 diabetes mellitus in primary care: an observational matched cohort study. PLoS One 2014;9:e95328.

12.     Health and Social Care Information Centre/Diabetes UK. National Diabetes Audit 2011–12 Report. Report 1: Care Processes and Treatment Targets. 2012. Available at: (last accessed Feb 2015).

13.     Loveman E, Frampton GK, Clegg AJ. The clinical effectiveness of diabetes education models for type 2 diabetes: a systematic review. Health Technol Assess 2008;12:1–116.

14.     Newbould J, Burt J, Bower P, et al. Experiences of care planning in England: interviews with patients with long term conditions. BMC Fam Pract 2012;13:71.

15.     Parchman ML, Zeber JE, Palmer RF. Participatory decision making, patient activation, medication adherence, and intermediate clinical outcomes in type 2 diabetes: a STARNet study. Ann Fam Med 2010;8:410–17. 10.1370/afm.1161

16.     Greenfield S, Kaplan SH, Ware JE Jr, et al. Patients' participation in medical care: effects on blood sugar control and quality of life in diabetes. J Gen Intern Med 1988;3:448–57.

17.     Doherty Y, Eaton S, Turnbull R, et al. Year of Care: the key drivers and theoretical basis for a new approach in diabetes care. Practical Diabetes 2012;29:183–6.

18.     Kidd J, Marteau TM, Robinson S, et al. Promoting patient participation in consultations: a randomised controlled trial to evaluate the effectiveness of three patient-focused interventions. Patient Educ Couns 2004; 52:107–12.

19.     Estabrooks PA, Nelson CC, Xu S, et al. The frequency and behavioral outcomes of goal choices in the self-management of diabetes. Diabetes Educ 2005;31:391–400.

20.     Renders CM, Valk CM, Griffin SJ, et al. Interventions to improve the management of diabetes in primary care, outpatient, and community settings, a systematic review. Diabetes Care 2001;24:1821–33.

21.     O'Connor PJ, Desai J, Solberg LI, et al. Randomized trial of quality improvement intervention to improve diabetes care in primary care settings. Diabetes Care 2005;28:1890–7.

22.     Hong YY, Lim YY, Audrey Lim SY, et al. Providing diabetes patients with personalized written clinical information in the diabetes outpatient clinic: a pilot study. Diabet Med 2010;27:685–90.

23.     Tusting K, Barton D. Models of adult learning: a literature review. Leicester: NIACE, 2006.

24.     Weinman J. Providing written information for patients: psychological considerations. J R Soc Med 1990;83:303–05.

25.     Gillani SM, Singh BM. A simple method for introducing care planning into specialist diabetes clinics. The WICKED project. Practical Diabetes 2014;31:337–41.

26.     Gillani SMR, Nevill A, Singh BM. Provision of structured diabetes information encourages activation amongst people with diabetes as measured by diabetes care process attainment: the WICKED Project. Diabet Med 2015;32:865–71. 10.1111/dme.12737

27.     Kirkpatrick, Donald L. "Techniques for evaluating training programs”. Classic writings on instructional technology 1.192 (1996):119.

28.     Asch DA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. J Clin Epidemiol 1997;50:1129–36.

29.     Edwards P, Roberts I, Clarke M, et al. Methods to increase response rates to postal questionnaires. Cochrane Database Syst Rev 2007;(2): MR000008.

30. Etter, J-F, Perneger TV. Analysis of non-response bias in a mailed health survey. J Clin Epidemiol 1997;50:1123–28.

31.     Melton LJ, Dyck PJ, Kames JL, O’Brien PC, Service FJ. Non-response bias in studies of diabetic complications: the Rochester Diabetic Neuropathy Study. J Clin Epidemiol 1993;46:341–8.

32.     Gillani SMR, Nayak AU, Thiruvenkatasamy K, et al. A method to validate the accuracy of a centralised district diabetes register. Practical Diabetes 2013;30:224–8.

33.     Hibbard J, Gilburt H. Supporting People to Manage their Health: An Introduction to Patient Activation. London: King’s Fund, 2014. Available from: file/supporting-people-manage-health-patient-activation-may14.pdf (last accessed October 2015).

34.     Gillani SMR, Holland MR, Sidhu M, Singh BM. A case control study of use of the Failed Access Score for determination of failed access to structured diabetes care: the WICKED Project. Practical Diabetes 2014; 31:107–10.

35.     Donald M, Ware RS, Ozolins IZ, et al. The role of patient activation in frequent attendance at primary care: a population-based study of people with chronic disease. Patient Educ Couns 2011;83:217–21.

36.     Brenk-Franz K, Hibbard JH, Herrmann WJ, et al. Validation of the German version of the Patient Activation Measure 13 (PAM13-D) in an international multicentre study of primary care patients. PLoS One 2013;8:e74786.


  • There are currently no refbacks.

The Journal of the Association of British Clinical Diabetologists