Measuring Innovation Skills Acquired by College and Polytechnic Students through Applied Research

The past decade has seen increasing investment in applied research in Canadian colleges. A primary output of applied research conducted in the college system is the impact on highly qualified and skilled personnel (HQSP): the students and graduates of programs that engage in applied research with industry partners. Although HQSP can be considered an input to business innovation, it is important to understand how HQSP are also important outputs of the college system. HQSP engaged in applied research at a college gain useful skills as an outcome of this experience. The ultimate outcome of this experience is increased business innovation. To understand how skills are acquired through student engagement in applied research and the potential downstream impact on firms, we must articulate the kinds of skills and activities that result in applied research activities in colleges.


Introduction
The past decade has seen increasing investment in applied research in Canadian colleges.A primary output of applied research conducted in the college system is the impact on highly qualified and skilled personnel (HQSP): the students and graduates of programs that engage in applied research with industry partners.Although HQSP can be considered an input to business innovation, it is important to understand how HQSP are also important outputs of the college system.HQSP engaged in applied research at a college gain useful skills as an outcome of this experience.The ultimate outcome of this experience is increased business innovation.To understand how skills are acquired through student engagement in applied research and the potential downstream impact on firms, we must articulate the kinds of skills and activities that result in applied research activities in colleges.
A key program supporting business innovation in Canada is the College and Community Innovation Program (CCIP; tinyurl.com/3uwknht),which was formally instituted with the 2007 science and technology strategy (Industry Canada, 2007;tinyurl.com/lkz6lqk).The CCIP has two objectives: i) increased R&D and innovation capacity by local firms in a college's catchment area and ii) increased capacity of colleges to engage local firms in applied research.Although there are several significant government funding initiatives that focus on business innovation and applied research activities in colleges (e.g., FedDev Ontario's Applied Research and Commercialization Initiative: tinyurl.com/7qetygt;Alberta Innovation Vouchers from Alberta Innovates Technology Futures: tinyurl.com/3f6kj8a; and Quebec's support for College Centres for the Transfer of Technologies: tinyurl.com/myw55v4), this article will focus on the CCIP as a program that is generally representative of programs that aim to enhance college capabilities in support of business innovation.
There is a two-fold benefit arising from colleges conducting applied research with industry partners: i) industry gains access to talent, facilities, markets, networks, and capital, along with support to launch new products and services into the marketplace, and ii) engaging students in applied research fosters innovation skills (i.e., "innovation literacy") in graduates, This article provides an overview of how colleges and polytechnic institutes are fostering innovation literacy via support for business innovation, and it outlines models for measuring innovation literacy for improved downstream innovation and productivity in industry.The article demonstrates how we can innovate innovation by taking a specific, proactive, and instrumental approach to fostering business innovation and skills acquisition gained through applied research work experience by students as part of their college education.This approach is being used by George Brown College in developing a framework for measuring this innovation potential with a long-term, outcomes-based analysis.
The goal of education is to make people privately happy and publicly useful.
John Godfrey Educator, journalist, and politician

Measuring Innovation Skills Acquired by Students through Applied Research
Robert Luke thereby increasing the innovation potential of the workforce.In order to understand how these two benefits can be measured, this article provides an overview of the applied research activities undertaken by colleges, how these activities relate to innovation skills, how these skills might be measured in students and graduates, and what subsequent impacts graduates can have on firm-level innovation performance.A logic model is presented that articulates the relationships between activities, skills, and measurement.Such measurement is necessary for the college system to show impact on improved downstream innovation and productivity in industry as part of accountability for innovation programs as linked to education.

Applied Research in Colleges and Polytechnic Institutes
The college system is funded explicitly to

Measuring Innovation Skills Acquired by Students through Applied Research
Robert Luke opt a user-centred firm perspective.The NAICS was used to delineate the type of industry accessing the site.This enabled us to design the site to support a firm working in a particular industry sector (as defined by NAICS) who needed a certain service, as shown in Box 2.
The activities listed in Box 2 reflect the kinds of applied R&D services offered by colleges in which students are engaged as part of their experiential learning.The list is directly related to the types of skill outcomes associated with college programs.In other words, each applied research service offered to support firms relates to the kinds of skills students will acquire by performing these services.Students work on applied research as part of their program curricula.The demonstration or performance of these skills can be part of their core curriculum or ancillary to it.By specifically linking skills acquired and demonstrated through the performance of applied research conducted for firms, students are encouraged to embrace and understand innovation as it is practiced at the firm level.
The learning outcomes associated with college education are augmented by experiential learning, leading to greater innovation capacity; the hypothesis is that this will lead to greater innovation capacity in the economy in which our graduates eventually will work.These activities should therefore have outcomes and impact in firms, both those that partner with colleges on applied R&D, and those that employ graduates who have experience with providing applied R&D as part of their vocational training.Accordingly, these activities represent the practice of skills relevant to innovation in firms. When

From Innovation Skills to Firm-Level Innovation
In colleges, applied research is focused on business innovation as an outcome.This outcome presents a measurement challenge insofar as the demonstration of impact of college-supplied R&D services will be affected by many variables outside of the college system's span of control, making direct causation or attribution difficult.Despite this challenge, we can still undertake efforts to link the provision of applied R&D services to the ultimate success of firms in terms of innovation capacity.A useful way to do this will be through college graduates, who can be proxies for our measurement of outcomes, as measured by the acquisition of innovation skills.
Skills acquisition, demonstration, and deployment are well defined areas of research and practice.The Ontario college system articulates essential employability skills (see tinyurl.com/3nvsxk5) as attributes of a college education.The OECD Innovation Strategy (Organisation for Economic Co-operation and Development, 2010; tinyurl.com/otrrs9k)articulates the connection between affective domain skills and the hard and soft skills as key to enabling innovation.These essential employability skills and those skills articulated by the OECD are foundational to innovation literacy.However, there is work to be done in terms of ensuring that both graduates and employers see the links between innovation skills and their effect on productivity in the economy (Dwyer and Luke, 2012; tinyurl.com/lm2qe3c).Of key importance is the application of skills into workplace settings, with a focus on the capacity to engage in innovation and entrepreneurship activities, thereby ensuring that students see the link between applied research activities with firms and skill acquisition as being directly related to supporting firm-level innovation.
Given that the explicit mandate for colleges in Ontario is to ensure that graduates are prepared for the workforce (Ontario Ministry of Attorney General, 2003; tinyurl.com/lqvns6t),our discussion of skills and innovation literacy must be grounded in how these skills and attributes may affect downstream economic performance.This performance will need to be measured both in the firms that partner with colleges on applied research as well as in the graduates themselves.Both of these settings present their own measurement challenges.
There is a compounding variable in the readiness of a college to engage in applied R&D with partners.There are therefore two intertwined variables with each needing discrete treatment: i) the college system's readiness for and effectiveness at applied R&D, and ii) the acquisition of innovation skills in students and graduates.Together, these variables have a downstream effect on firms.The readiness of the college system to engage in applied research is beyond the scope of this Each of these audiences requires specific measures and instruments.All need a consistent focus on outcomes.
There is a well-developed framework for linking objectives, activities, and outcomes: the logic model.Logic models are well-established frameworks for evaluation, and they are used widely within the Government of Canada.For example, the Treasury Board of Canada Secretariat publishes detailed descriptions of their approach to performance management, including the use of logic models (tinyurl.com/k5rx2yd).A program logic model can aid us in defining and articulating the links between the activities, outputs, and outcomes associ-

Measuring Innovation Skills Acquired by Students through Applied Research
Robert Luke ated with applied research and the acquisition of skills by students.Such links are important to all audiences listed above.Tracking outcomes against stated objectives lets us link inputs to outputs and ultimately outcomes and impacts.
The following components of a logic model are used to measure progress towards a desired objective over time: 1. Purpose: the strategic aim/objective of change/effect to be measured These elements of a logic model provide a tool for linking the applied R&D activities supported by colleges for firms and the skills students gain as a result of engaging in this form of experiential learning.A key aspect of this approach is that it allows us to test -over time -the acquisition and demonstration of skills by students, through to downstream innovation support in firms after the students have graduated.Thus, this approach allows us to develop a logic model for training highly qualified and skilled personnel (HQSP) through applied research, as shown in Table 1.

Measurement components
There are several components to our logic model for measuring the acquisition of innovation literacy skills and their deployment in firms post-graduation.The components presented below are from work conducted to date on measuring innovation literacy in students and graduates: 1.An innovation skills measurement tool: The Conference Board of Canada's General Innovation Skills Aptitude Test (GISAT; tinyurl.com/metjy87)offers a tool to measure the acquisition of skills and their application in firms.The Conference Board of Canada has recently updated their Innovation Skills Profile (tinyurl.com/q7yhafk),which provides a basis for understanding the kinds of skills relevant to fostering innovation in firms and how these are related to applied research activities as noted above.

General and college-specific key performance indicators (KPIs):
The provincially mandated KPI survey offers a rich dataset that lets us examine how students and graduates feel about the acquisition of the skills that comprise innovation literacy.We can use the KPI survey questions that are relevant to skills acquisition to link to innovation literacy skills acquisition as measured by the GISAT.Whereas the GISAT can be deployed specifically to students engaged in applied research, the KPIs provide a context or baseline against which to measure the general population.In addition to the KPIs as provincially mandated, colleges are allowed to put in five college-specific questions asked only of their students and graduates.George Brown College has included questions that ask students and graduates if they participated in an "Applied research project/course project with industry" and if so, what is their level of satisfaction with the experience.We also ask this of employers.

Toronto Next survey:
In October 2012, George Brown College released the results of a survey of Greater Toronto Area firms and their understanding of innovation and productivity, and the inputs required for these.(CCA, 2012;tinyurl.com/k4v47x2)The challenge faced by the college system, then, is to come up with measures that can be used for future assessments.These measures will need: i) to be easily and consistently applied and collected across the country; ii) to be segregated by industry; iii) to focus on impact and outcomes (i.e., not just activities); and iv) to stand externally as viable measures of success.It will thus be increasingly important to link applied research activities to student learning outcomes given the growth trajectory of applied research in colleges.In so doing, we can innovate innovation by taking a specific, proactive, and instrumental approach to fostering business innovation and skills acquisition gained through work experience in applied research.

Conclusion
There are two key outputs for colleges engaging in applied research: i) the support of firm-level innovation and ii) the training of highly qualified and skilled personnel, who gain innovation skills ancillary to their program outcomes.As noted above, these skills are directly related to the provision of applied R&D services to firms.The applied research services offered by colleges offer a strong platform on which to base innovation skills, and measuring these skills over time is strongly related to the success of the college system and its ability to provide for the innovation capacity of Canada.The learning outcomes associated with college education, as augmented by experiential learning such as applied research, will lead to greater innovation capacity in partner firms, as well as those firms that employ graduates equipped with innovation literacy.

Innovation Skills Acquired by Students through Applied Research
Innovation literacy is an amalgam of skills that encompasses the cognitive, psychomotor, and affective domains of learning; it is the ability to engage in the types of business innovation activities outlined in Box 2. www.timreview.caMeasuring

Table 1 .
A logic model for training highly qualified and skilled personnel (HQSP) through applied research www.timreview.ca

Important to Measure the Impact of Applied Research in Colleges
As well, in many cases, data are available only for specific institutions, sectors, or regions, and are not available consistently across the country.As a result, while general statistics of this kind may illuminate certain facts about Canada's applied R&D strengths in specific institutional settings, their piecemeal nature precludes a systematic identification of Canada's research and technology strengths.The Panel thus concludes that there remains a need for more systematic and detailed data collection of metrics related to applied research and technology development activity in Canada."