D4 Outcomes Measures and Uniform Data Set
Allen Lewis, PhD
The development of outcome measures and a Uniform Data Set (UDS) is an important step toward solving the methodology and policy barriers. This development task will be conducted in three phases: (1) meta-analysis, (2) development of a preliminary outcome set field tested in all projects in the proposed RERC, and (3) development of a UDS by collaborating with national and international experts in conjunction with professional tele-health organizations (American Telemedicine Association’s Telerehabilitation Special Interest Group, ATA-Telerehabilitation SIG).
The objectives are:
- To conduct meta-analysis of TR outcome measures
- To develop a preliminary outcome set, a model UDS database, and conduct a field test using projects in the proposed RERC
- To develop uniform TR outcome measures and a UDS through collaboration with national and international experts and with the TR community such as the ATA Telerehabilitation SIG.
The development of outcome measures and uniform data sets consists of the following activities:
- Conduct a meta-analysis of outcomes in TR
- Develop a preliminary TR outcomes set
- Coordinate the uniform data set development with the tele-health/telemedicine community.
Conduct a meta-analysis of outcomes in TR:
In the first year of the project, a meta-analysis will be done by conducting a search of articles containing outcome studies in journal databases and conference proceedings, and by contacting authors and experts in countries identified as having TR. The inclusion criteria for the outcome studies are that the studies should be randomized trials, controlled before and after studies, or interrupted time series studies of human subjects using TR for some modality. The database for the article searches will include MEDLINE, CINAHL, PsychLit, OpenSIGLE, INSPEC, Compendex, National Technical Information Service, Econlit, Public Affairs Information Service, Library Information Science & Technology Abstracts, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, CRD’s Health Technology Assessment Database, and DHSS-Data. Two investigators will independently review the results of the searches for study inclusion and extract data. Study quality (particularly method), randomization, and intervention, will be judged using accepted criteria and compared with the review protocol. Suitable studies will be identified, for inclusion in the review, and will be pooled for statistical analyses. To address the risk of publication bias, the review will include unpublished studies, and non-English language publications. A protocol will specify how the results of this
meta-analysis will be presented.
Develop a preliminary TR outcomes set:
Based on the data collected from meta-analysis, a preliminary outcomes set will be developed. We will use the RERC-TR R&D tasks and TIPs to conduct a field test of the preliminary outcomes set. The outcomes set will act as a specific measure that will be used to quantify (quantitative measure) or gauge (qualitative measure) the result or visible effect of TR intervention. The results of this field test will be a draft UDS that can be presented to a wider community.
Coordinate the uniform data set development with the tele-health/ telemedicine community:
In order for the TR uniform data set (TR UDS) to gain recognition and adoption, it needs to be developed in consultation with nationally and internationally recognized subject experts and further field tests. Toward this end, a TR Outcomes Task Force will be established in collaboration with the American Telemedicine Association (ATA) Special Interest Group (SIG) on Telerehabilitation. In addition to engaging national and international experts, this task force will function as a unit to coordinate the uniform data set development with TR standards/guidelines. We will also convene a TR UDS Consensus conference in conjunction with an ATA annual meeting to finalize data points and obtain commitment from health systems and community- based rehabilitation settings in the use of the uniform data set.
Dr. Ellen Cohn (the current Vice Chair of the ATA Telerehabilitation SIG, and the Chair-Elect for a subsequent two-year term) is an investigator on this RERC. She will establish a sub-committee to involve ATA Telerehabilitation SIG in convening experts and gaining consensus for a TR UDS.
Expected Findings and Deliverables:
Conceptual model and state-of-the-art:
The goal of the UDS is to provide researchers a standard set of assessment procedures, collected longitudinally. Data collection forms will be developed for initial and follow-up visits based on the consensus developed through the UDS Outcomes Task Force activities.
The deliverables of the project include 1) a validated TR outcome set; and 2) a TR Uniform Data Set that provides high-quality and comparative information on the impact of TR solutions. This data set will support decision and policy makers in guiding the application of TR solutions to improve and maintain the health of individuals, communities, and populations, and also improve the functioning of the health system for which they are responsible.
Protocol prepared. Systematic review of literature began in February 2010.
Before conducting the meta-analysis, an internal group familiar with the Telerehabilitation literature met to discuss key outcome areas to assist with guiding the literature search. The three main variables agreed upon included; cost-effectiveness, efficacy, and satisfaction. The highlights from the meta-analysis include, 1700 articles reviewed and 280 studies retained based on a set of exclusion and inclusion criteria. The lessons learned in conducting such a complex methodology was to include a librarian cognizant of complex database systems and past history of conducting evaluations. The review is in its second phase conducted with the assistance of the Uniform Data System-International Consensus Group (UDS-ICG) and other individuals who have heard about our research and have an affinity for outcomes within Telerehabilitation.
The UDS-ICG was a formulation of both national and international experts within the Telerehabilitation field assisting to systematically solicit, organize, and structure collective judgments and opinions on addressing a UDS for TR outcomes. Members of the RERC-TR found that in order not to bias the results and to gain a better understanding of what is happening outside of the US, we identified and enlisted 13 TR experts across academia, public policy, and management roles based on their acknowledged expertise, publication, and programs within the TR discipline. A two day meeting was coordinated in Pittsburgh to discuss the current meta-analysis and the three distinct areas in which we were able to discern and describe services along with what stakeholders should be looking for when developing a UDS for TR outcome tools. In order to continue discussions among such a diverse group geographically, the RERC has set up a collaboration portal and has given access to all group members to review, edit, and add materials. Within the portal there are specific sections for the three outcome categories. Currently, a Telerehabilitation Usability Questionnaire (TUQ) has been developed and will be tested within the center along with our UDS-ICG group.
UDS team members understand that an accepted, standardized set of definitions and outcome measures describing TR is critical for clinical decision making, optimal communication among caregivers, and research in different settings; however have identified challenges that include, changing of technology, lack of appropriate study designs, applications and variation in health outcomes and implementation resistance.
MetaAnalyze on TR outcomes commenced.
International TR community consensus sought for UDS at ATA 2010 Annual meeting.