The Cochrane review [19] applied rigorous quality criteria well-known to influence the validity of randomised controlled trials [9]: allocation concealment, clear definition of inclusion and exclusion criteria, loss to follow-up, comparability of study groups, equal treatment of study groups, except of the intervention, and intention-to-treat analysis. Further criteria specially addressing hip protector trial methodology were applied: active or scheduled follow-up ascertainment and monitoring of adherence. However, appraising trials using formal checklists may overlook important clinical flaws in the original trials [7]. This can be illustrated exemplarily by critical appraisal of two included trials [18, 24]. The trials by van Schoor et al. [24] and O’Halloran et al. [18] were chosen, as they received the largest weight in the data pooling of institutional setting studies (Fig. 1).
Van Schoor et al. [24] investigated the effectiveness of a hip protector intervention programme in a heterogeneous population of elderly living in apartment houses, residential homes and nursing homes. The intervention did not result in a reduction of hip fractures, and the authors claimed the ineffectiveness of the hip protector.
However, the study has several important methodological shortcomings. The power calculation was based on a one-sided p value and a reduction of hip fractures of 75%. Such a pronounced effect has never been shown before. Sample-size calculation indicated the need of 700 participants. However, only 561 participants were enrolled. During the trial, follow-up was extended to 69.6 weeks to increase the number of events.
The education approach used in the intervention group was scarcely described. The underlying education theory was not mentioned. Precise details of the interventions intended for each group and how these were actually administered were not given. The implementation of the intervention may be less than optimal for the group of cognitively impaired persons because there was no structured involvement of caregivers.
Van Schoor et al. emphasized single randomisation as an important methodological strength of their study. However, the avoidance of cluster randomisation seems to be problematic in this setting. Contamination could have compromised the study, as the same caregivers looked after participants of both study groups.
Outcome assessment relied on participant-kept calendar. As baseline data indicated cognitive impairment in approximately 75% of participants, the reliability of the data is of major concern. None of these obvious shortcomings has been reflected in the Cochrane review, which classified the trial of high methodological quality with a score of 10 out of a possible 10.
The study by O’Halloran [18] is by far the largest trial investigating a hip protector intervention in institutionalised elderly. Thus, it received a weight of 46% in the meta-analysis of the Cochrane review. The sample consisted of 127 nursing and residential homes with 4,117 beds in Northern Ireland. The combined intervention of provision of hip protectors, education and information did not result in a reduction of hip fractures during 72 weeks of follow-up. However, this study also has important methodological shortcomings, which might have heavily affected the results. A mixed population was investigated, including nursing home residents and non-nursing home residents. Baseline data of participants are not reported. It remains unclear whether the two study groups were comparable at the beginning of the study. A flow of participants through the trial, which discloses the number of subjects with early study termination and the proportion of subjects that have been included after randomisation, is not given. The absolute number of participating residents remains unknown, as data are reported as events per 100 occupied beds or per 100 residents per year.
The multifaceted intervention consisted of an education session for home staff, distribution of manufacturer’s information material, provision of a videotape on hip protectors, information for residents and relatives on request, counselling by a nurse facilitator and provision of hip protectors for every resident agreeing to wear them. No details are reported on how the education and information programme was structured, whether it was based on an education theory and if so, whether it had been sufficiently explored before application. The reader cannot assess if an appropriate intervention was administered. Access to the education programme is not possible neither by further references nor authors’ offer to contact them.
It remains unclear whether newly admitted residents also received the information or whether residents and relatives requested information sessions. It is of special concern that the authors did not mention how and when the investigators determined agreement to wear the hip protector. Frequency of hip protector use in the control group was reported at baseline only. However, during the time course of the trial a successful hip protector trial has been published [10], and hip protector use became a more common prevention strategy. It is possible that an increasing number of control group residents also used hip protectors.
Outcome measurement was based on routine documentation of homes and chart review by the external nurse facilitator. Numbers of falls, fallers and total fractures were not reported and probably not ascertained. It remains unclear if participants of the intervention and control centres had a similar risk of suffering a hip fracture. The reported significant difference in pelvic fractures suggests that the use of the hip protector pads was not the only important difference among the study groups.
Furthermore, the trial was limited, as seven out of 40 homes were not compliant with the introduction of the intervention and outcome ascertainment compared to six out of 87 homes not cooperating in the control group. Management of missing data has not been discussed.
Although the clustering design effect was considered in the sample size calculation, the results were not adjusted for the cluster effect.
In conclusion, critical appraisal of the study by O’Halloran revealed several major methodological flaws. It seems that hip protectors were disseminated to a heterogeneous group of elderly using an unstructured, not theory-based education and information approach. These biases affecting the internal validity of the study have not been considered by the Cochrane review. On the contrary, quality assessment by reviewers’ checklist indicated a score of 9 out of 10 [19].
Want more information on Motorcycle Back Protector? Click the link below to contact us.