Mobilizing Knowledge : The Evidence Gap for Assistive Devices Edward

Knowledge mobilization relies on evidence of sufficient quality to enable appropriate decision making. Although many established sectors produce well-controlled research with sufficient sample sizes to guide decision making confidently, the knowledge mobilization process is challenged in sectors where the pace of change outpaces the ability to complete high-quality research methodologies. For example, technology-sector research involving products and humans can take much longer to complete that the annual or faster product-revision cycles. Therefore, the outcomes from a large multi-site, randomized control study may not apply to a new product that replaces the tested technology. Although well-controlled research with appropriate sample sizes is needed, this approach must be balanced with other evidence sources to address the knowledge-immediacy requirements. The assistive device sector is an interesting group for exploring this human–technology knowledge mobilization issue.


Introduction
Knowledge mobilization relies on evidence of sufficient quality to enable appropriate decision making.Although many established sectors produce well-controlled research with sufficient sample sizes to guide decision making confidently, the knowledge mobilization process is challenged in sectors where the pace of change outpaces the ability to complete high-quality research methodologies.For example, technology-sector research involving products and humans can take much longer to complete that the annual or faster product-revision cycles.Therefore, the outcomes from a large multi-site, randomized control study may not apply to a new product that replaces the tested technology.Although well-controlled research with appropriate sample sizes is needed, this approach must be balanced with other evidence sources to address the knowledge-immediacy requirements.The assistive device sector is an interesting group for exploring this human-technology knowledge mobilization issue.
Assistive devices that improve mobility for people with disabilities cover a range of technologies, from wheelchairs to prosthetics and orthotics to robotic exoskeletons.Although the target populations and core technologies vary between these mobility assistive devices, a common element is the rapid pace of innovation in relatively small markets (Baljko & Hamidi, 2014).Modern assistive devices are on annual or biannual revision cycles, often with faster revision cycles for devices controlled by high-end microprocessors.While this pace of innovation should be encouraged to provide the best technology for people with mobility disabilities, constant change presents challenges for evidence-based practice and knowledge mobilization.
Knowledge mobilization can be hindered in healthcare technology settings where the pace of change outpaces the ability to perform high-quality research methodologies that provide timely knowledge to enable informed prescription and technology application to the end user.Although well-controlled research with appropriate sample sizes is needed, this approach must be balanced with other evidence sources to address the knowledge immediacy requirements.Using carbon-fibre ankle-foot orthoses (i.e., lower-limb braces that improve stability, alignment, and foot-to-ground placement) as a case study, various sources of assistive device evidence were explored for their contribution to the continuum of knowledge in this area.A basic level of knowledge exists, but the quality is insufficient to inform the physical rehabilitation community on selecting from the almost 70 different devices on the market and the expected clinical outcomes for a target population.A combination of enhanced single-participant reports should be considered as an important part of the knowledge continuum and essential for knowledge immediacy.This approach must also be expanded to national and multinational database initiatives that provide a better base from which to extract knowledge on assistive device performance and mobilize this knowledge to provide optimal care for people with physical disabilities.
The temptation to form premature theories upon insufficient data is the bane of our profession.

Sherlock Holmes Fictional private detective
In The Valley of Fear by Arthur Conan Doyle  " " Mobilizing Knowledge: The Evidence Gap for Assistive Devices

Edward D. Lemaire
In the field of physical rehabilitation, several challenges have been identified for knowledge development and mobilization (Johnston et al., 2009).These challenges include: • the complexity of interactions between people with a disability and their environment (WHO, 2016) • the need to evaluate people as they participate in their daily lives • small sample sizes • the difficulty or impossibility of implementing blinding or placebos • ethical issues for control groups who do not receive treatment • integrating assistive technology evaluation with other research methods and design factors • the lack of funding for large effectiveness studies • the difficulty of addressing issues between large social systems and physical, economic, and environmental factors These challenges are directly related to knowledge mobilization for assistive devices.In fact, poor consideration of assistive technology use in rehabilitation research can be a confounding factor when interpreting research results (Rust & Smith, 2005).
To positively affect practice, the appropriate evidence with the appropriate quality must exist.The study by Gebrosky and colleagues (2013) demonstrated the usefulness of these standard tests by performing the ANSI/RESNA standard tests on a series of lightweight wheelchairs.Most of the wheelchairs tested did not meet durability standards and a recommendation was made for stronger regulations and testing by independent and certified facilities.Ensuring that standard tests are appropriately administered is essential, but access to these test results is also lacking given that companies, test facilities, and regulators are not required to share test outcomes.Even if the intent to share this information existed, a reference and indexing source that can easily be used by clinicians is not available.For knowledge mobilization, systems such as clinical trial registries (such as ClinicalTrials.gov)are needed to provide an access gateway to the standard evaluation outcomes for assistive devices.This would enable direct access by the healthcare community and provide a base for knowledge synthesis by researchers and other stakeholders.
Assistive devices are typically categorized under Class 1 or Class 2 for regulatory approval.In Canada, Class 1 devices have the lowest medical risk and do not require a Medical Device License.Therefore, much of the background evidence is not provided through the regulatory process.Even with regulatory information, independent research evidence to guide clinical decision making is usually lacking when new or modified devices appear in the market.
From a biomechanical perspective, a modest amount of literature exists for the effect on movement and body structures from assistive device use.A search using the Scopus database with keywords "wheelchair biomechanics" returned 953 results, and 151 results since 2012.
A search using "(prosthetics or orthotics) and biomechanics : excluding implants (surgical), Animal, Arthroplasty, Bone Cements, Biomedical Engineering -Surgical Implants" returned 2,689 publications, with 406 results since 2012.In comparison, a search using "physical rehabilitation and biomechanics" returned 55,042 publications.Most of the biomechanics literature involves motion laboratory studies, with the majority of locomotor research on level ground.However, the emergence of wearable sensor systems and room-scale virtual reality systems (Sinitski et al., 2015) (Jutai et al., 2005;Lenker & Paquet, 2003).
Ideally, a continuum of evidence can be amassed that covers the need for immediate information on new technologies, broader research that quantifies how a device compares with similar assistive devices in the market or literature, and high-quality evidence for assistive device concepts that exist throughout a device class.To explore this continuum concept, a case study involving carbon-fibre ankle-foot orthoses is presented here.

Evaluating the Current State of Carbon-Fibre Ankle-Foot Orthoses
A carbon-fibre ankle-foot orthosis (CF-AFO) is a lower limb brace that controls movement of the foot relative to the shank for people with lower limb weakness, paralysis, or excessive tone/spasm.Carbon-fibre materials produce a device that is thin and light, stiff, or capable of storing and returning energy when walking.A market survey conducted by the investigator returned 68 distinct CF-AFO products from 13 manufacturers (i.e., devices with distinct designs or functions, not including different sizes).Most CF-AFOs consist of a carbon-fibre foot and shank section that inserts into the shoe and a strap at the proximal end to anchor the orthosis to the shank (Figure 1).The energy storage design consists of separate foot and shank sections that both connect to a centre energy storing part (rods or rectangular section) that stores energy after foot strike and then releases stored energy at toe off.An expiring patent on this technology should lead to many more devices entering the market place in the next few years.
The first level of evidence for device function and unique contributions is patents.The investigator identified 25 relevant patents with a Google Patent search using the keywords "carbon fibre ankle foot orthosis AFO".This information provides claims and design concepts for many of the commercial devices.Peer review is from patent office experts.Although this level of evidence is insufficient to guide clinical practice, it does provide information to help understand the objectives, concepts, and novel contributions that differ between devices.
From a regulatory perspective, CF-AFOs are Class 1 medical devices and therefore the manufacturer is not required to provide evidence on device safety or function.Although regional regulatory requirements may provide a level of evidence, this information cannot be expected for this class of assistive device.In general, when assembling knowledge for mobilization, regulatory testing results are typically not considered, but could be a useful source of information that would be available as a technology enters the market.
From the literature, searches of Scopus and Google Scholar databases with the keywords "ankle foot orthosis afo carbon fibre" produced 29 relevant articles published between 2006 and 2016.Of these, 9 were related to AFO mechanical testing, 15 using biomechanical and physiological analysis, and 4 using clinical tests.
For mechanical testing, a series of experimental protocols were used to apply loads and measure the resulting AFO movement.These methods evaluated AFO stiffness, engineering modelling and structural analysis, and repeated loading to determine how the device could fail.However, the lack of a standard measurement method prohibits valid comparisons between studies and meta-analysis.The orthosis-limb interac-

Mobilizing Knowledge: The Evidence Gap for Assistive Devices
Edward D. Lemaire tion is also a consistent deficiency.Orthoses are designed to intimately interact with the foot and lower limb; therefore, the limb is part of the structural system and therefore must be considered when testing device performance.Most studies did not consider the limb and, of the three studies that tested the AFO with a limb surrogate, a rigid plaster limb shape with a single axis joint was employed.For carbon-fibre AFO with multiple axes of possible movement, a solid single-axis foot-leg model is insufficient for replicating AFO movement under daily activity loads.Future research should develop a more appropriate foot-leg surrogate to enable mechanical testing so that the results can be more directly translated to the real world.For knowledge mobilization, this example applies broadly to knowledge obtained through simulation and modelling.Insufficient or inappropriate identification of the person within the system being evaluated could lead to poor evidence that would not be easily identified by the knowledge user following knowledge translation.
Biomechanical testing can be grouped by design: custom AFO with a posterior strut that stores and releases energy during gait, or lighter one-piece carbon-fibre AFO that is typically prefabricated.3D quantitative motion analysis demonstrated that devices with a posterior strut typically stored and returned more energy, thereby helping the person walk more naturally and with less effort, and fine control of strut stiffness may not be required for prescription but can affect comfort and range of movement.Although the biomechanical evidence is adequate, most papers originated from one study at one laboratory and many of the other studies did not provide appropriate comparative data.Biomechanical research on the one-piece carbon-fibre designs included different populations in each study, such as able-bodied, Charcot-Marie-Tooth, polio, multiple sclerosis, peroneal nerve injury, and stroke.Given that each population had different walking characteristics and requirements, comparisons across populations become difficult.Analyses also vary, from full 3D motion analysis to simple stride analysis to physiological energy expenditure.Although the available biomechanical information provides a basis for understanding how various CF-AFOs perform during walking, the level of evidence remains insufficient to support clinical decisions when choosing between the many designs available in the marketplace.
Biomechanical laboratory-based analyses have difficulty meeting the immediacy requirement for knowledge mobilization, where barriers for providing timely outcomes include costs and time for completing the study (fitting devices for each person), laboratory access, and recruitment fatigue from a regional participant pool that are approached for multiple evaluation studies.Economically, funding these studies may be difficult for small manufacturers where profits from individual products are much less than other health sectors, such as pharmaceuticals.Motion-laboratory analyses may be better positioned for assessing broader classes of devices and first occurrence of disruptive technologies.
Methodologically, many biomechanical studies compared AFO gait to walking without a device.Given that the literature has previously established that any appropriately prescribed AFO improves gait over walking without the orthosis (Tyson et al., 2013), this comparison does not inform the clinical community as to the device's relevance in the marketplace.This is in contradiction to the typical placebo methodology where the placebo is a harmless intervention prescribed for the psychological benefit of the participant.For knowledge mobilization, comparisons between the assistive technology and the current standard of practice for the target population are required to ensure that a clinical decision maker can use the translated knowledge to recommend the most appropriate device for the patient.
Clinical testing also varied between the four studies.Across all studies, different combinations of 14 tests were employed to evaluate balance, muscle strength, functional status, walking and movement capacity, mobility capacity, quality of life, and disability.As with the biomechanical studies, different populations participated in each study, and each population had different movement characteristics.These clinical tests provided functional performance measures that helped to complete the knowledge base for CF-AFO; however, the quantity of information is insufficient for making evidence-based decisions for CF-AFO prescription and fitting.The advantage of these clinical tests are relatively quick implementation (minutes), portability, quick reporting, and standardized protocols.However, these tests only provide a high-level assessment of the parameter (i.e., we could learn that a person walks faster with a new assistive device, but not why they walk faster).
As with much of the assistive technology literature, participants were not well described in reference to how their disability could affect CF-AFO performance.Detailed research participant descriptions are required to enable the reader to assess the study and make decisions about the relevance of outcomes to a specific

Conclusion
As demonstrated by the CF-AFO case study, it is apparent that typical medical research methods are insufficient to generate timely evidence to guide assistive device prescription and consumer/clinician decision making and enable effective knowledge mobilization (Johnston et al., 2009).Immediacy is a core requirement that is contrary to the time required to complete controlled research with a sufficient sample size.Overall, a series of recommendations can be considered.
First, as outlined by Ottenbacher and Hinderer (2001), single-participant studies should be considered for evidence-based practice when quick reports on new devices or practices are needed.The lack of generalizability from single-participant studies can be partially addressed by adding additional replications of the measured treatment effect, systematic replication of the program or treatment over a wider range of situations, and meta-analysis.However, consistent data collection and reporting in the field are needed to provide the appropriate information.
Second, beyond single-participant studies, the assistive device field requires a multisite, multinational clinical database approach to support data assembly, knowledge extraction, and knowledge mobilization (Fuhrer, 2001).Secure international data networks are now available for collecting de-identified information on the user, assistive device characteristics, and outcome measures (e.g., SwedeAmp quality registry [Emilsson et al., 2015]).By engaging multinational clinical, non-government, and government organizations, the infrastructure to participate efficiently in a global registry can be created, thereby providing timely data on device outcomes that can be synthesized computationally to reveal patterns, trends, and associations related to assistive-technology utilization (i.e., Big Data approaches).The obstacles to achieve such an objective include data-security requirements across jurisdictions, engagement at the clinician level to collect high-quality standardized outcome measures, engagement at the local administration level to enable de-identified data sharing, and partnerships to enable timely and relevant analysis and reporting from the system to the appropriate recipients (i.e., knowledge mobilization).Differences in clinical practices between sites is also a potential problem because the local therapy methods may influence outcomes beyond factors attributed to the assistive device.
Third, in addition to clinician-generated information, assistive-device consumers may also participate by sharing their wearable mobility device information (smart watch, smart phone, etc.) or stored data from their intelligent assistive devices (microprocessor controlled prosthesis, powered wheelchair, etc.).Industry must be engaged by multiple stakeholders (including end users, healthcare providers, professional associations, and privacy regulators) to make the transition from only using device sensor data for device control and product development to making this device-user-centric data an integral part of the evidence base.
The research process cannot keep pace with innovation, so evidence to guide prescription is lacking; however, the continuous knowledge generation era is upon us and the assistive technology field is well positioned to take advantage and bring immediacy to knowledge mobilization.For assistive devices, partnerships and action are required to "mobilize" knowledge mobilization.

The Evidence Gap for Assistive Devices
are providing research tools to generate device-function evidence that better reflects movement environments encountered in daily living.Clinical research on assistive technology includes questionnaire-based tools for evaluating user perceptions of www.timreview.caMobilizing Knowledge:

Mobilizing Knowledge: The Evidence Gap for Assistive Devices
Edward Lemaire, PhD, is actively involved with research on technologies that improve mobility for people with physical disabilities.He is a Clinical Researcher at The Ottawa Hospital Research Institute's Centre for Rehabilitation Research and Development; Professor at the University of Ottawa Faculty of Medicine; and Adjunct Professor in Human Kinetics, Mechanical Engineering, and Systems Design Engineering.He is also active with the International Society of Prosthetics and Orthotics, as a board member and international congress scientific chairman.Dr. Lemaire's research has resulted in over 350 published papers and presentations that include intelligent prosthetics and orthotics, biomechanical walking analysis in 3D virtual environments, smartphone approaches to improve decision making, and eHealth technology to enhance access to education and rehabilitation services.