There was a strong Diabetes Prevention Research Group presence at last week’s IBNPA XChange 2021 Annual Meeting! Learn more about our presentations, below.
How do we know what to say? Refinement of a Bank of Messages Targeting Diet and Physical Activity Through End-User Evaluation
MacPherson, M.M., Johnston C., Cranston, K., Locke, S., & Jung, M.E.
Purpose: Given the increasing reliance on mobile technologies and widespread use of short messaging services (SMS) as a communication strategy, SMS provides an opportunity to improve the delivery of healthcare virtually. SMS has been shown to be an effective way to reach a broad audience engaging in physical activity and dietary interventions. However, limited evidence exists regarding the development or receptivity to SMS content. Understanding how interventions are developed is necessary for future research to implement effective SMS interventions.
Methods: The study design consisted of two phases. 1) Evaluation: Five SSBC trainers and 13 past SSBC participants were asked to review a bank of 124 SMS messages using a 5-point Likert scale on readability, usefulness, and relevance. Each message was followed by an open text box for suggestions on improvements. 2) Refinement: an average score was calculated for each message (ranging from 3-15). Any message with a score <14 was refined using participant feedback.
Results/findings: On average, messages received a score of 13.77/15±0.76, with SSBC trainers scoring messages 0.81 higher than did past participants. Questions received an average of 2.6 suggestions for improvement (range=0-5). 60 messages received a score <14 and were edited based on feedback from participants; 23 messages received a score >14 but had multiple suggestions so were refined based on feedback; 11 messages received consistent critique revolving around applicability to the program and were removed.
Conclusions: This study involved the evaluation and refinement of a bank of SMS messages which are suitable for use following the SSBC program. These messages are based on behaviour change theory and have been refined based on feedback from those with lived experiences as diabetes prevention program trainers and participants. The message bank, that was positively assessed by those representative of future message senders and recipients, will prompt participants to adhere with the diet and exercise changes they made during the SSBC program.
Implementation evaluation of a diabetes prevention program when delivered by a community organization
Dineen, T., Bean, C., & Jung, M. E.
Purpose: With type two diabetes on the rise, there is a need for more prevention programs to reach the large number of at-risk individuals. The purpose of this research was to examine the implementation process, strategies, and multilevel contextual factors as an evidence-based diabetes prevention program was implemented into two local community organization sites. In-depth reporting of implementation strategies and context are needed to support future studies.
Methods: Small Steps for Big Changes is a brief-counselling diet and exercise modification program for individuals at-risk of developing type 2 diabetes with demonstrated success. A one-year partnered planning process with a local not-for-profit community organization co-developed an implementation plan for the translation of this project. This research was guided by a pragmatic epistemology. Semi-structured interviews were conducted with community organization staff who delivered the program (n = 8), and a focus group was completed with implementation support staff (n = 5) from both community sites. Interviews were transcribed verbatim and thematically analyzed using a template approach. The consolidated framework for implementation research (CFIR) was used to guide the analysis of this study. The CFIR is a multilevel implementation determinant framework with strong theoretical heritage. Within the template approach, themes were first inductively identified to ensure all salient ideas were captured, then identified themes were deductively linked to CFIR constructs.
Results: Implementation strategies used were appropriate, well-received by staff and promoted successful implementation. Several CFIR constructs were identified from all five domains: (a) process, (b) intervention characteristics, (c) outer setting, (d) inner setting, and (e) individual characteristics. Specifically, results revealed the partnered one-year planning process, program components and structure, level of support, and synergy between program and context were important factors in the implementation.
Conclusion: This study describes the strategies and contextual factors used to implement a community-based diabetes prevention program into two community sites. Successful implementation was supported by a fully engaged, partnered approach to planning, and subsequently executing, an implementation effort. The CFIR provided a thorough framework to identify and evaluate multilevel contextual factors impacting implementation. Results demonstrate a successful approach to working with a community partner to support implementation.
Evidence-based diabetes prevention program implemented with fidelity by community organization
Dineen, T., Banser, T., Bean, C., & Jung, M. E.
Purpose: In-depth fidelity evaluations remain understudied. Fidelity evaluations examine the extent an intervention is delivered as intended and increases confidence that intervention results are due to the intervention itself. As a program translates from one context to another, knowing exactly what and how it was implemented is critical to understand program outcomes and provide feedback to subsequently scale and inform future research. The purpose of this study was to examine the delivery of, and engagement with, an evidence-based diet and exercise diabetes prevention program when delivered by fitness facility staff within a community organization.
Methods: This pragmatic study investigates the implementation of a community-based diabetes prevention program by a local community organization. Ten staff from this organization were trained to deliver an evidence-based program. Between August 2019 – March 2020, 26 clients enrolled in the program. Three fidelity assessments were completed. First, staff completed session-specific fidelity checklists (n = 156). Second, two audio-recorded counseling sessions from all clients underwent an independent coder fidelity check (n = 52). Third, staff recorded client goals on session-specific fidelity checklists and all goals were independently assessed for (a) staff goal-setting fidelity, (b) client intervention receipt, and (c) client goal enactment by two coders (n = 285). Fidelity scores were calculated for each checklist, converted into a percentage, and compared to independent coder fidelity checklists. Content analysis was used for staff goal-setting fidelity, client receipt of intervention, and client goal enactment.
Results: Average self-reported fidelity was 89.5% for all six sessions. Independent coder fidelity scores for the two assessed counseling sessions were 83% and 81%. Overall, staff helped clients create goals in line with program content clients indicated receipt of the program as intended and had a high goal achievement of 79%.
Conclusion: This study demonstrates relatively low-resource methods to assess program fidelity and client engagement. The program was implemented with high fidelity by fitness facility staff at a community organization and clients engaged with the program as intended. Findings increase confidence that program outcomes are due to the intervention itself and provide feedback to refine implementation strategies to support future scale-up efforts.
Using the Behaviour Change Wheel to Develop Text Messages to Promote Diet and Physical Activity Adherence Following a Diabetes Prevention Program
MacPherson, M. M., Cranston, K. D., Locke, S. R., Bourne, J. E., & Jung, M. E.
Purpose: Improvements to diet and physical activity (PA) can reduce the risk of developing type 2 diabetes (T2D); however, long-term adherence to diet and PA is poor. In order to influence T2D risk at a population level, scalable interventions are needed to facilitate behaviour change adherence. Text messaging interventions are a cost-effective way to improve long-term diet and PA changes; however, they often fail to report on theoretical underpinnings of message development. This study aims to describe the development of a bank of text messages to support diet and PA behaviour change adherence following a diabetes prevention program using the Behaviour Change Wheel (BCW). The BCW is a synthesis of 19 behaviour change frameworks which provides structure to intervention design and has been used extensively in health behaviour change interventions.
Methods: The following stages of the BCW were followed: (I) target behaviours and barriers/facilitators to engaging in them; (II) intervention options and policy categories; and (III) relevant behaviour change techniques (BCTs) associated with selected intervention options. A library of text messages was then written to map onto identified BCTs and was coded for BCT fidelity by an independent reviewer trained in BCT identification.
Results: Target behaviours included adherence to diet and PA recommendations. Sixteen barriers/facilitators and 28 associated BCTs were identified through previous qualitative work. One hundred and twenty-four messages were written based on selected BCTs and following the fidelity check, a total of 43 unique BCTs were included in the final bank of messages.
Conclusion: While other research has utilized the BCW to develop text messaging interventions, this is first, to our knowledge, to have an independent coder assess BCT classification in the final message development as a form of fidelity check. This additional step proved imperative in developing a comprehensive picture of the active components within the current bank of messages. By reporting on the theoretical underpinnings and mechanisms of action within text messages, future research can understand not only if these messages are effective, but also why certain messages may be more or less effective, and what combination and dose of messages can optimally influence behaviour change.
What makes up an effective diabetes prevention program: identifying behaviour change techniques and motivational interviewing techniques in a community-based program for adults at risk for type 2 diabetes
Cranston, K.D., MacPherson, M.M., Dineen, T.E., & Jung, M.E.
Purpose: Diet and physical activity programs are effective at reducing one’s risk of developing type 2 diabetes (T2D); however, it is not known what specific techniques lead to optimal outcomes. When developing and implementing diabetes prevention programs in the community, it is imperative that intervention components are thoroughly and consistently reported to understand program effectiveness and help with scaling up. Small Steps for Big Changes (SSBC) is an evidence-based diabetes prevention program that consists of six one-on-one sessions with a coach. Coaches deliver diet and physical activity content to clients using behaviour change techniques (BCTs) and motivational interviewing (MI). BCTs are the specific active intervention components, and MI techniques describe the relational aspects of how intervention components are delivered. By reporting the BCTs and MI techniques, future researchers can better replicate the components and the delivery style of an intervention. The purpose of this study was to comprehensively identify the BCTs and MI techniques within SSBC.
Methods: BCTs and MI techniques were coded within each of the six SSBC session protocols using the BCT Taxonomy v1 and the Table of MI Techniques. To ensure reliable coding, two coders completed the BCT coding and two coders completed the MI techniques coding. Interrater reliability was assessed.
Results/findings: Forty-three BCTs (interrater reliability [kappa]=0.77) and 20 MI techniques (kappa=0.63) were identified within the SSBC protocols (mean BCTs per session=30; mean MI techniques per session=16). SSBC sessions specifically focussed on the BCT categories of “Goals and Planning”, “Feedback and Monitoring”, and “Repetition and Substitution”, and the MI technique processes of “Engaging” and “Evoking”.
Conclusions: Detailed reporting of intervention components are necessary for future reviews to accurately test which intervention components are effective in eliciting change in desired outcomes. This study revealed the use of a greater number of BCTs in the SSBC program compared to the numbers of BCTs reported in a number of T2D programs. In general, more thorough reporting of BCTs used in interventions will progress the field of health behaviour change. Further, the detailed reporting of the specific techniques within SSBC can be used to inform development of future diabetes prevention programs.
