Enhancing Early Innovation in an Urban Living Lab: Lessons from Espoo, Finland Soile Juujärvi and Virpi Lund Urban Living Labs for Sustainability in Suburbs in Need of Modernization and Social Uplift

A number of urban living labs have been set up in recent years, with the aim of developing innovation processes within a multi-stakeholder partnership in an urban context. Several urban living labs focus on sustainable development, which is a visible and urgent issue in less valued suburbs in need of modernization and social uplift. We argue that, when applying the living labs approach in the context of sustainable development in suburbs, the primary focus should be society's collective goals, as expressed through municipalities and users. The aim of this article is to show examples of how urban living labs can be applied in less valued suburbs in order to contribute to sustainability based on societal goals. We build on analyses from the research project SubUrbanLab, where urban living labs were set up in Alby and Peltosaari, two suburban areas in Sweden and Finland, respectively. We draw lessons regarding how to use urban living labs for sustainable development in order to create favourable conditions for ongoing engagement with the municipality and users towards long-term sustainability. People worry about costs addressing sustainability in less valued suburbs, but what are the costs of not responding to the residents' concerns about their living environment?


Publisher
The Technology Innovation Management Review is a monthly publication of the Talent First Network.

Overview
The Technology Innovation Management Review (TIM Review) provides insights about the issues and emerging trends relevant to launching and growing technology businesses. The TIM Review focuses on the theories, strategies, and tools that help small and large technology companies succeed.
Our readers are looking for practical ideas they can apply within their own organizations. The TIM Review brings together diverse viewpoints -from academics, entrepreneurs, companies of all sizes, the public sector, the community sector, and others -to bridge the gap between theory and practice. In particular, we focus on the topics of technology and global entrepreneurship in small and large companies. We welcome input from readers into upcoming themes. Please visit timreview.ca to suggest themes and nominate authors and guest editors.
The books are available in ebook format on the Amazon store. Please see the Best of TIM Review website (timbooks.ca) for details and ordering information. Note that all of the net proceeds from the sales of our Best of TIM Review ebooks will be used to offset the operational costs of publishing future issues of the TIM Review.
We hope you enjoy this issue of the TIM Review and will share your comments online. We welcome your submissions of articles on technology entrepreneurship, innovation management, and other topics relevant to launching and growing technology companies and solving practical problems in emerging domains. Please contact us (timreview.ca/contact) with potential article topics and submissions.

From the Guest Editors
It is our pleasure to be able to kick off a fresh year of the Technology Innovation Management Review with the second issue on the theme of Living Labs and User Innovation. This issue includes a second batch of articles that were carefully selected and reworked from the living lab track at the ISPIM 2015 Innovation Conference in Budapest, Hungary. Accordingly, we would already like to invite you to the ISPIM 2016 Innovation Conference (conference.ispim.org) to be held in Porto on June 19-22, 2016. The conference will feature another designated living lab track including an invited speaker session hosted by the European Network of Living Labs (ENoLL), the catalyst and forerunner of living labs. Since its inception in the 2006 Helsinki Manifesto, exactly 10 years ago, ENoLL has grown in "waves" up to this day. To this date, nine waves have been launched, resulting in 395 historically accepted living labs all over the world and a strong core of 170 living labs of active members present in 20 of the 28 EU Member States and on all continents. This rapid growth in the number of living labs accompanied a rather high attrition rate over the years indicates the growing pains suffered by this largely practice-based concept. However, together with EN-oLL's growth, the number of academic papers on living labs has also started to blossom (Figure 1), which fosters a better understanding of the different aspects related to living labs.

Editorial: Living Labs and User Innovation
Chris McPhee, Seppo Leminen, Dimitri Schuurman, Mika Westerlund, and Eelko Huizingh This transition from practice-based to more theoretically grounded concept that links up with broader innovation theories enables the optimization of living lab practices, but it also puts forward living labs as interesting study objects for a variety of innovation scholars. Recent work has focused on describing the roles of stakeholders and users, network structures, and innovation outcomes resulting in 17 propositions for living lab research (Leminen, 2015;. Other work has shifted attention towards the multi-facetted nature of the living labs phenomena, distinguishing an organizational layer of collaborating stakeholders at the macro level, an intermediary innovation project layer at the meso level, and a methodological layer consisting of the different research and project steps at the micro level . The five articles in this issue further build on these arguments, including two studies (articles 1 and 4) on methodological aspects and the outcomes in living lab projects, and one article (5) looking at the intermediary process of individual actors in a living lab project and beyond. The two remaining studies (2 and 3) focus on the emerging concept of urban living labs -a specific type of living lab organization in a city context -to examine sustainable innovation supported by all city stakeholders.
The first article, by Dimitri Schuurman, Lieven De Marez, and Pieter Ballon, from iMinds-MICT-Ghent University and from iMinds-SMIT-VUB, investigates the impact of methodological characteristics of living lab projects on the outcomes of the innovation process. Based on a sample of 27 projects, they discovered that a multi-method approach and a real-life intervention increase the chances of obtaining valuable user contributions. The article adds to the theoretical and empirical foundations of the added value of living labs.
Next, Soile Juujärvi and Virpi Lund from Laurea University of Applied Sciences and from the University of Helsinki in Finland, examine urban living labs based on a case study of Espoo, Finland. Their study advocates combined bottom-up and top-down processes, which implies collaboration and co-creation between the different city stakeholders. They highlight the important role that citizens play in the early stages of the innovation process (i.e., the "preject"), while emphasizing that policy makers and city developers play an increasing role in later stages, to effectively realize the bottom-up ideas and projects. An urban living lab thus functions as an intermediary (actor) to facilitate and manage co-creation and collaboration between these different stakeholder groups and during the different stages of the innovation development process.
In the third article, Katarina Buhr, Maija Federley, and Anja Karlsson, from IVL Swedish Environment Research Institute and VTT Technical Research Centre of Finland, also elaborate on the topic of urban living labs. They base their article on two urban living labs in two different countries and focus on their contribution to sustainability based on societal goals that emerged primarily from the citizens and municipalities. They conclude that urban living labs need to be embedded within the local city context and require a long-term continuation to enhance user engagement. Therefore, the involvement of the municipality seems necessary, although it does not need to be the driving actor.
The fourth article is by Annabel Georges, Dimitri Schuurman, and Koen Vervoort from iMinds-MICT-Ghent University and from iMinds Living Labs. Based on an analysis of real-life tests within multiple living lab projects, they propose a model to analyze attrition during living lab field trials. Two types of attrition occur: non-usage attrition (which can provide valuable feedback) and drop-out attrition (which refers to the problem of users who stop providing feedback). The article concludes with a set of practical guidelines for practitioners.

Finally, Louna Hakkarainen and Sampsa Hyysalo from
Aalto University, Finland, examine the roles of innovation intermediaries in living labs. By means of an indepth longitudinal case study of a living lab project in the eHealth domain, the authors demonstrate that the intermediary process is versatile and cannot be reduced to facilitation. Their analysis shows that intermediation work in a living lab project consists of a range of tasks, including configuring, brokering, as well as facilitating. They contribute to the research on living labs by focusing on stakeholder and single-actor interactions, and also extend their findings to the broader domain of innovation intermediaries in open innovation.
Together, the articles in this second issue on the theme of Living Labs and User Innovation illustrate the theoretical progression made in the field of living lab research, as more and more authors succeed in demonstrating the added value created through living labs and in connecting living labs phenomena to broader innovation theories and discussions (articles 1, 4, and 5). Moreover, living labs also seem capable of providing answers to current challenges posed to society, as can be witnessed in urban living labs (articles 2 and 3). Through their focus on co-creation and multi-stakeholder engagement, living labs give society a voice in innovation, which facilitates sustainable innovation that is broadly supported.

Introduction
In academic theory, open innovation has been regarded as the norm for studying innovation management ever since Chesbrough's seminal and widely cited Open Innovation: The New Imperative for Creating and Profiting from Technology (2003). However, in practice, a balance should be found between open and closed innovation, which calls for innovation management approaches that deal with finding this balance (Lakhani & Panetta, 2007). Although many principles and phenomena from the open innovation literature, such as economic spill-overs (Arrow, 1962) and dynamic capabilities (Teece & Pisano, 1994), were already described a long time ago, and open innovation as a domain has already fostered a large body of research (West & Bogers, 2013), many companies and innovation practitioners are still struggling with the concrete implementation of strategies to cope with these distributed innovation processes (Chiaroni et al., 2011).
Three main issues and gaps will be examined in this article. First, there still is a lack of adequate innovation management models for implementing open innovation (Lichtenthaler, 2011). Second, Enkel, Gassmann, and Chesbrough (2009) found that few studies try to put forward measurement systems and key performance indicators to evaluate open versus closed approaches. Third, Huizingh (2011) argues that open innovation became an umbrella that connected a range of already existing activities. However, most of the principles and research relating to open innovation are tailored to large companies with abundant resources (van de Vrande et al., 2006), despite the fact that SMEs are usually more flexible, less formalized, and quicker to make decisions -meaning that they present many opportunities for the implementation of open innovation (Lee et al., 2010). Therefore, with this article, we will assist in filling these gaps by investigating living lab projects as an organized (as opposed to an ad hoc) approach to open innovation consisting of real-life experimentation and active user involvement by means of different methods involving multiple stakeholders . More specifically, we will explore the value of a living lab approach for open innovation in SMEs.
Open innovation scholars as well as practitioners are still struggling with the practical implementation of open innovation principles in different contexts. In this article, we explore the value of a living lab approach for open innovation in small and medium-sized enterprises (SMEs). Using a case study approach, we compared 27 SME projects conducted by iMinds Living Labs from 2011 to 2015. The results suggest that a real-life intervention and a multi-method approach -both of which are methodological characteristics of living lab projects -increase the chance of generating actionable user contributions for the innovation under development. Moreover, the results also suggest that a living lab project yields maximal value when evolving from concept towards prototype. Besides these exploratory findings, this article also demonstrates that living lab projects are a perfect "playground" to test and validate assumptions from the open innovation literature.
If you look at history, innovation doesn't come just from giving people incentives; it comes from creating environments where their ideas can connect. Steve Johnson Science author and media theorist " " www.timreview.ca

Dimitri Schuurman, Lieven De Marez, and Pieter Ballon
We conducted a comparative case study analysis of 27 SME projects in the domain of new media and ICT conducted by iMinds Living Labs from 2011 to 2015. Our aim was to examine the impact of the methodological setup of living lab innovation projects on the innovation contribution of end-users and on the eventual outcome of the new product development process. In this way, we assist in addressing the need for impact assessment and measurement systems of open innovation approaches, and we demonstrate the viability of a living lab project as a "playground" to test and validate assumptions from the open innovation literature.

Living Labs as a Structured approach to Open Innovation
The first premise of open innovation is that, from the perspective of a single firm (the usual level of analysis in open innovation research), opening the internal innovation process of a firm yields extra value (Chesbrough et al., 2006). According to Chesbrough and Bogers (2014), the critical conceptual distinction between the previous literature on spillovers in innovation is that open innovation transforms these spillovers into inflows and outflows of knowledge that can and should be purposively managed. Many open innovation studies deal with the economic (pecuniary) implications and opportunities provided by external sources of innovation and commercialization, and mainly focus on the revenue-generating practices from a firm perspective (Brunswicker & Vanhaverbeke, 2014;van de Vrande et al., 2009). Enkel and colleagues (2009) conclude that the future of innovation processes lies in an appropriate balance between open and closed innovation approaches, because too much openness can lead to a negative impact on companies' long-term innovation success, loss of control, and loss of core competences, whereas a too closed innovation approach does not serve the demands of increasingly shorter innovation cycles and reduced time-to-market. Pisano and Verganti (2008) also add to this discussion by identifying four types of open innovation collaboration models, based on the governance model (hierarchical versus flat) and participation (open versus closed), which indicates that there are many options and trade-offs between "open" approaches. However, there seems to be a gap between theory and practice given that multiple studies have indicated that many companies struggle with implementing open innovation practices (Lichtenthaler, 2008;van de Vrande et al., 2009), and that there are major differences between different firms and organizations (Laursen & Salter, 2006).
From the introduction, we gathered that there is a lack of structural approaches and guidelines for implementing open innovation, a lack of measurement systems that allow impact assessment, and a lack of research into open innovation in SMEs. One specific approach that offers a structured approach to open innovation and that has been used specifically by startups and SMEs are "living labs" . Living labs are put forward as an institution to overcome the socalled "European Paradox" or the gap between research leadership and (commercial success of) innovation . Living labs are physical regions or virtual realities, interaction spaces, in which stakeholders form public-private-people partnerships (4Ps) of companies, public agencies, universities, institutes, users, and others that follow the philosophies of open and user innovation to collaborate for improving, developing, creating, prototyping, validating, and testing of current or new technologies, services, products, and systems in real-life contexts , and are driven by two main factors: involving users in the early stages of the innovation process and experimentation in real-world settings that aim to provide structure to user participation (Almirall & Wareham, 2008). Therefore, living lab projects are a specific case of open innovation where companies open up their innovation processes to users or customers , which can be linked to the user innovation paradigm (von Hippel, 1976(von Hippel, , 2009).
In terms of methodological deconstruction of the living labs approach, the work of Pierson and Lievens (2005) remains unique in describing the different phases of a living lab project: i) contextualization, ii) selection, iii) concretization, iv) implementation, and v) feedback. However, the methodological basis of these five phases is left unexplored, as are the actual outcomes and added value when engaging in living lab projects.  suggested that this methodology is very similar to a quasi-experimental design, with a pre-test, a real-life intervention, and a post-test ( Figure 1).
By adopting this methodological approach, living lab projects would be able to overcome the barriers to user contribution, because it implies triangulation of different methods and a real-life contextualization (Frissen, 2000). However, only very few studies try to assess the impact of the methodological design of living lab projects . For living labs, there generally is a gap in measurement systems; as Katzy and Turgut (2010) state, a valid research methodology still www.timreview.ca The Impact of Living Lab Methodology on Open Innovation Contributions and Outcomes Dimitri Schuurman, Lieven De Marez, and Pieter Ballon needs to be developed for the innovation performance of individual living labs. According to them, the measurement of the efficiency of living lab processes and structures would serve two purposes: i) legitimating the (EU) research budget that has been used to stimulate the establishment of living labs and ii) enabling modification of the concept, or at least certain aspects of it. Therefore, by studying living lab projects that are an emanation of open innovation, a more systematic study of open innovation processes and principles is enabled, which may help overcome one of the key problems with open innovation: conceptual ambiguity (Dahlander & Gann, 2010). Moreover, in terms of the typology of Pisano and Verganti (2008), the studied living lab projects can be considered as collaboration projects with hierarchical governance, given that the project instigator (the SME) decides how the user contributions are implemented within the innovation in development.

Methodology
Based on a comparative case study analysis, we wanted to assess the impact of the methodological set-up on innovation contribution of end users and on the outcome of the innovation project. To gather and analyze empirical data, we used the case study technique, which is a common method in social sciences to describe and explore poorly understood processes and events. Case studies are especially suited to such work because of their emphasis on detailed contextual analysis of a limited number of events or conditions and their relationships (Eisenhardt, 1989).
Within a case study design, careful consideration should be dedicated to the selection of the cases to be included in the analysis (Dion, 2003). We analyzed all SME living lab projects that have been carried out by iMinds Living Labs in the period from 2011 to 2015. This approach yields a slightly larger sample of cases than usual, but enables a more quantitative, yet still exploratory, analysis coupled with more in-depth qualitative investigation. Therefore, the case studies are neither prospective (i.e., criteria are established and cases fitting the criteria are included as they become available) nor retrospective (i.e., in which criteria are established for selecting cases from historical records for inclusion in the study), but can be labelled as comprehensive for the analyzed time frame (2011)(2012)(2013)(2014)(2015), which is in line with the "sustained period of time" criterion for data collection of Shepard (2001). The Impact of Living Lab Methodology on Open Innovation Contributions and Outcomes Dimitri Schuurman, Lieven De Marez, and Pieter Ballon As researchers from iMinds, we had access to the following data sources from the iMinds Living Lab projects: i) transcripts of semi-structured interviews with representatives of the SMEs and ii) all project deliverables. Within the living labs community, iMinds Living Labs has played an important role in developing, applying, and studying the living labs approach, and is regarded internationally as a "best practice" example (Almirall et al., 2012;Dell'Era & Landoni, 2014). This view is reinforced by the fact that iMinds Living Labs also acts as secretary of the European Network of Living Labs (ENoLL; openlivinglabs.eu). Therefore, the availability of rich data, first-hand experiences, and the leading role of iMinds Living Labs in the living labs landscape justifies the choice of these 27 projects (Yin, 1984). Later in the article, we briefly summarize each case; more detailed descriptions are provided by  and the website of iMinds Living Labs (tinyurl.com/zzowv4m).
For this article, we gathered the following data for the 27 projects: i) the presence of the living lab methodology, ii) evolution in terms of stages of new product development (NPD), iii) user contribution generated by the living lab project, iv) and outcome of the innovation.

Presence of the living lab methodology
If the project included a quasi-experimental design (i.e., pre-test -real-life intervention -post-test) and a multi-method user involvement approach, this criterion was coded as "yes". If only one or none of these characteristics was present, it was coded as "no".

Evolution in terms of NPD stages
For all projects, the evolution of the innovation in terms of NPD stages was logged during the interviews with the project instigators. We discerned between the following stages, based on Jespersen (2008): ideaconcept -prototype -pre-launch -launch -postlaunch. We recoded the project into three categories (iMinds, 2015), which are also used to describe the type of living lab projects, which are summarized in Figure 2: 1. Exploration: a project where the innovation starts at the idea or concept stage and ends in the idea or concept stage. These projects focus on exploring new knowledge for innovation development.
2. Experimentation: a project that includes the prototype stage. These projects focus on experimenting with the innovation.
3. Evaluation: projects that start at the pre-launch stage or later. These projects focus on evaluating the innovation.

User contribution generated by the living lab project
This aspect indicates the instigators' perceptions of what has been done with the user contributions generated during the living lab project. We discern three categories of contribution -during, after, or none -to indicate whether the results were used to modify the innovation during or after the project or to indicate that the results did not modify the innovation.

Outcome
The final variable identifies the current status of the innovation (as of May, 2015). On the market indicates that the innovation is launched and available for end-users, pipeline indicates that the innovation is still planned to be launched, but is not available yet, and reoriented is used when the instigator has decided not to launch the innovation. The Impact of Living Lab Methodology on Open Innovation Contributions and Outcomes Dimitri Schuurman, Lieven De Marez, and Pieter Ballon Results and Discussion Table 1 below gives an overview of the variables for all the 27 projects. In terms of the methodological approach, we notice that the majority of the projects (14 of 27; 52%) did not include all the methodological elements of an "ideal" living lab project. Nineteen out of 27 projects contain a real-life intervention (70%), and only 13 out of 27 (48%) also include a post-assessment.
These methodological elements can be regarded as forms of user contextualization, which is proposed as a means to overcome barriers related to user involvement, or the so-called "real-life experience" of living labs (Frissen, 2000). Another method to overcome these barriers was to include triangulation of different methods, which reflects the "multi-method" characteristic of living labs. The majority of the projects adhere to this criterion, with 23 out of 27 projects (85%) containing triangulation of user involvement.
The absence of some of these characteristics can be ascribed to various reasons. First, startups and SMEs are constrained by time and budget, which did not allow them to have all elements in a project. Second, the NPD stage also impacted the possibility of a real-life intervention. Projects that remained in the exploration stage have greater difficulty in organizing a field trial given that there is no working prototype yet. As a solution, a proxy technology assessment (Pierson et al., 2006), which means a simulation of the innovation by means of existing technologies, can be used (e.g., the Veltion and Unicorn projects), but this requires extra effort and expertise.
In terms of the types of living lab projects, 7 projects (26%) can be labelled as exploration, which means that, at the end of the project, there was no working prototype, 15 (56%) are experimental in nature, including the prototype stage, and 5 (18%) were coded as evaluation because these projects consisted of innovations that were already in a pre-launch stage at the start of the project.
The first variable that refers to an outcome of the living lab project is user contribution: it indicates what happened with the user contributions generated during the project. For 13 cases (48%), modifications were made during the project; for 7 cases (26%), they were made after the project; and in the remaining 7 cases, instigators stated that they did not use the living lab results to modify the innovation.
The second outcome variable refers to the market introduction of the innovation after the living lab project. In total, 10 innovations (37%) were launched on the market, 8 innovations (30%) were still in development, and 9 instigators (33%) reoriented themselves and abandoned the innovation development.
These results indicate that, in nearly three-quarters of the projects, the user contribution had an impact on the innovation development, but that iteration of the innovation development during the living lab project, or the so-called "pivots" out of the lean startup literature, is less common and occurred in only about half of the projects. However, when comparing the projects in which the "full" living lab methodology was used, there are some pronounced differences, as shown in Table 2 and described below.
For the cases that did contain all methodological living lab elements, only 2 did not generate user contributions that led to modifications in the innovation. Stated differently, 85% of these projects generated actionable user contributions, and more than half of the cases included iterations during the project. In comparison, for projects that did not contain all methodological elements, two-thirds generated actionable user contributions. In terms of outcome, the "real" living lab projects The Impact of Living Lab Methodology on Open Innovation Contributions and Outcomes Dimitri Schuurman, Lieven De Marez, and Pieter Ballon resulted in 4 market introductions and 6 innovations still in development. Only 3 cases resulted in the innovation being abandoned. For the other 14 projects, the number of reoriented cases is twice as high (6 or 43%), but the number of successful market introductions is also slightly higher (6). A potential explanation could be that the living lab projects generated more input for the innovation, which requires more time to implement these changes and induces a longer time to market.
Finally, when looking at the stages in the NPD process, the majority of the "full" living lab projects were labelled as "experimentation", whereas the other projects were more evenly distributed among the three categories. Given that the experimentation projects lend themselves more to incorporating all methodological living lab elements, our results seem to suggest that the best time for a living lab project is when advancing from concept to prototype, or at least to include this stage in the project.
Because of the time and budget constraints of startups and SMEs, this was not always realistic within the cases we studied. One of the strategies that was used to overcome this issue was to carry out multiple projects in sequence. As an example, the Coxo case was an exploratory project that did not include all methodological elements, but was aimed at studying all stakeholders in the complex ecosystem of the innovation. The Planidoo project was the follow-up project of Coxo, with the innovation carrying a changed name (something which followed out of the results of this first project). This project did carry all living lab characteristics, because it started in the pre-launch stage and evolved towards market introduction during the project. This illustrates that a lean and agile approach and attitude are necessary when carrying out these type of projects with SMEs, both from the researchers and from the project instigators.

Conclusion
Within this article, we looked at 27 innovation projects from Flemish startups and SMEs carried out within the iMinds Living Labs constellation. To conclude, we summarize and translate our findings in three propositions. First, the discussed living lab projects are aimed at opening up the company boundaries towards user contributions, thus facilitating outside-in open innovation. Moreover, in terms of the collaboration typology of Pisano and Verganti (2008), the projects can be labelled as hierarchical and shifting between open and closed participation. The user contributions were successful for al-most two-thirds of the projects, leading to modifications of the innovation during or after the project based on user contributions. Moreover, for two-thirds of projects, this innovation resulted in a market introduction or in further development. These findings show that living lab projects are a means to successfully facilitate open innovation in startups and SMEs.

Proposition 1: Living Lab projects foster successful open innovation in SMEs
Although the open innovation literature was inconclusive regarding the relationship between open innovation and SMEs, it seems that the agility and flexibility of the living lab projects from our study link up with the needs of SMEs.
When taking into account the methodological set-up of these projects, it seems that a real-life intervention, a quasi-experimental design, and a multi-method approach increase the chances of user contributions that lead to modifications in the innovation, as this was the case for 85% of the projects against 64% for the projects that lacked one or more of these elements.
Proposition 2: The living lab characteristics "real-life intervention" and "a multi-method approach" foster valuable user contributions Although the literature on living labs stresses the importance of real-life and multi-method research approaches, few actual evidence-based arguments can be found that support this claim. However, our research supports this proposition: within our studied sample, not all projects displayed all the methodological characteristics, but those that did scored higher in terms of user contribution. Moreover, the projects with the most positive outcomes could be characterized as "experimentation", which indicates a transition from concept to prototype during the living lab project.

Proposition 3: The ideal maturity level of an innovation for a living lab project is the transition towards a testable prototype
This proposition is also in line with the emphasis that is put on the "real-life" testing within the living labs literature. Moreover, our findings also support the thesis that triangulation and real-life experience lower the barriers for user contribution, as was suggested by Frissen (2000).  Dimitri Schuurman, Lieven De Marez, and Pieter Ballon account, such as the attitude of the instigator, the characteristics of participating end users, or the nature of the innovation in development. This broader approach would enable assessment of their impact on the outcomes of living lab projects. In any case, the characteristics of a living lab, where a given constellation carries out multiple innovation projects following a given methodology, allows researchers to test hypotheses on a supra-case level. This facilitates grasping the mechanics and nature of open innovation processes and phenomena beyond a single project. Moreover, this also allows investigation of the iterative learning processes that take place on the constellation level when conducting multiple open innovation projects.

The Impact of Living Lab Methodology on Open Innovation Contributions and Outcomes
We regard this as the way to go for living lab researchers and practitioners, in order to more clearly understand the mechanisms by which living labs operate and to assess the added value they are able to generate. This type of knowledge is necessary to further develop the potential of living labs and to outgrow their status as a promising, but fuzzy innovation concept. www.timreview.ca

Introduction
Living laboratories have increasingly been used as platforms for innovation and experimentation in urban areas, involving key features of open innovation, a multistakeholder approach, real-life environments, and residents as users (Friedlich et al., 2013;Veeckman & Graaf, 2015). The goals of urban living labs can vary according to their environments, from small-scale experiments of new technology and services to large-scale social and economic improvement (Franz et al., 2015). In addition to complex problems in physical environments, there are social and economic problems that are difficult to understand and handle due to their multidimensional nature, such as stigmatization, unemployment, and segregation of ethnic minorities. There are also problems due to organized complexity: a multiplicity of organizations steering the region can result in competitive and overlapping systems of administration (Baynes, 2013;Wallin, 2013). Due to multi-layered problems, urban living labs call for practice-based innovation with diffuse and heterogeneous knowledge production, instead of homogenous accumulation of knowledge and clearlydefined problem solving (Melkas & Harmaakorpi, 2008). Thus, an urban living lab usually starts as a bottom-up process setting additional challenges for problem definition and the composition of actors.
The purpose of this article is to describe an urban living lab initiative in a suburban area by examining the early phase of its innovation process, which is also called the front-end phase in research literature. The front-end phase refers to the starting point of the project where opportunities are identified and concepts are created through adaptive interactions between participants. In our case, participants represented a range of living lab roles: enablers, providers, utilizers, and residents as users (Juujärvi & Pesso, 2013a;, whose further analysis is beyond the scope of this study. The present article is focused on advancing the urban living lab approach as an innovation method for urban development. We start by discussing the concept of the urban living lab and its implications for the early innovation process.
Urban areas are often characterized by complex problems, such as social and economic deprivation, segregation, or bureaucratic administration. Urban living laboratories provide a promising approach to redefining and tackling such problems in novel ways by enabling bottom-up innovation with various actors. The present study examined an urban living lab initiative in a suburban area of Espoo, Finland, where guided workshops based on the Change Laboratory method were arranged. The findings show that, before development projects are launched, it is important to dedicate sufficient time to the early innovation process, which includes building relationships, sharing knowledge, exploring ignorance, and innovating new concepts. The study emphasizes the importance of distinguishing early innovation processes from later ones, which means separating the "preject" from the "project". We conclude that successful management of an urban living lab combines bottom-up and topdown approaches.
It is in the field of ignorance that the spark of something new is most often ignited.

Urban Living Labs
The emerging interest in urban living labs calls for more precise definitions of the concept. An urban living lab has been defined as a forum for innovation that integrates residents and other stakeholders to develop and test new ideas, systems, and solutions in complex and real contexts (see Friedlich et al., 2013). Referring to Almirall and Wareham (2008), it can be seen as a specific type of open innovation network that acts as an intermediary between residents, public organizations, and private organizations to capture and codify user insights in their living environments. Franz, Tausz, and Thiel (2015) further distinguish technologically and socially oriented urban living labs, with the former ones focusing on co-developing new products and services and the latter ones dealing with the wider scope of urban and city development to improve living environments involving technological, social, and political questions. In socially oriented urban living labs, users have versatile roles as residents and citizens, and consequently, citizen participatory and co-creation processes are intertwined (Franz et al., 2015;Juujärvi & Pesso, 2013a). Consistent with this view, an urban living lab has been defined as a regional forum for innovation and dialogue focusing on solving challenges in the urban area (Friedlich et al., 2013).
Previous studies suggest that urban living labs may have various goals ranging in size and contents and calling for different forms of collaboration (see . In this article, we focus on socially oriented urban living labs, which are characterized by citizen participation, strong collaboration with local stakeholders, and the aim to create concepts and methodology that can be transferred into other contexts (see Franz et al., 2015). Proactive networking, experimentation as a bottom-up process, as well as commitment and longevity in development work has been previously suggested to be success factors for urban living labs (Juujärvi & Pesso, 2013a). Juujärvi and Pesso (2013a) have further elaborated actor roles for successful urban living labs. City representatives as enablers and public authorities bear an important role in creating a vision and allocating public resources. They also provide strategic leadership, promote networking across administrative units, and create public-private-people partnerships. Utilizers, such as firms and non-governmental associations, produce place-based knowledge and set small-scale objectives, and they pursue the creation of products and services suitable to the area and its residents. Research institutions engage researchers and students in development work, provide innovative methods, and take responsibility for systematic knowledge augmentation. Residents as users produce placebased user experiences, participate in experiments, and empower other citizens through co-creation.
When starting an urban living lab, it is first important to bring multiple actors together and engage them in creating a shared vision. However, collaboration in living laboratories is challenged by power struggles and inadequate cooperation skills, and therefore, actors need to learn to interact with others in the first place . In particular, the role of residents is vulnerable, because their local knowledge and use of natural language is not compatible with the jargon of experts (Staffans, 2014).
Previous studies raise the question of how urban living labs should be coordinated in order to utilize their full innovation potential, which lies on the boundaries between different groups and actors (Melkas & Harmaakorpi, 2008). Socially oriented urban living labs are based on so-called Mode 2 innovation activity that is organized around a particular application, and innovators need to combine different types of information from scattered sources over lengthy periods (Gibbons et al., 1994;Melkas & Harmaakorpi, 2008.) This raises the question of how Mode 2 innovation activity could be best coordinated. Leminen (2013) distinguishes bottom-up and top-down approaches for coordinating innovation activities, with the former operating at the grassroots levels and focusing on local needs, and latter pursuing centralized and official targets. A bottom-up approach is facilitated rather than managed, whereas a top-down approach is managed rather than facilitated .  further points out that enabler-driven living labs (e.g., driven by city representatives) and user-driven labs (e.g., driven by residents) are characterized by bottom-up coordination, whereas provider-driven labs (e.g., driven by R&D institutions) and utilizer-driven labs (e.g., driven by companies) tend to be top-down coordinated.
Consistent with some previous studies (Lievens et al., 2011;Sauer, 2012) we argue that urban living labs should combine bottom-up and top-down developments. Whereas a bottom-up approach helps to identify needs and unanticipated ideas, a top-down approach is needed to validate ideas and concepts and to provide a formal structure. Urban living labs have usually been led by enablers collecting needs identified within the region through networking, involving a risk that innovation activities remain as information-sharing networks (see . Although the involvement of www.timreview.ca

Soile Juujärvi and Virpi Lund
enablers gives urban living labs authority and legitimacy, they rarely possess R&D methodologies to transform harvested ideas towards realizing large-scale societal goals. As a result of ineffective collaboration, residents and other actors may become frustrated and give up their participatory efforts (Friedlich et al., 2013).
The aim of the present study is to advance understanding about the early innovation process and its facilitation or management.
In-depth studies of microprocesses in living lab development are rare , and we wish to examine how early innovation process develops within emerging urban living labs activities. For this purpose, we first introduce a model developed by Darsø (2003) based on team processes in an international company. While doing so, we presume that successful innovation processes are basically team processes regardless of the context (Juujärvi & Pesso, 2013b).
The remainders of the article is organized as follows. After discussing the early innovation process, we describe our intervention method for urban living lab development using community workshops. Next, we present our research design and findings. Finally, we provide conclusions.

The Early Innovation Process
Several models of innovation emphasize the importance of the early innovation process for nurturing creativity at the level of interaction. Among the most well-known ones is the model of knowledge creation by Nonaka and Takeuchi (1995) and further developed by Nonaka and Konno (1998). In their model, the innovation process is seen a collective learning spiral that increases knowledge through four arenas (bas). In brief, the phases are described by Nonaka and Konno (1998) as follows: 1. Socialization to originating ba involves sharing of tacit knowledge, that is, each individual's mental model, through physical proximity and face-to-face contact, which creates common understanding and mutual trust among group members.
2. Externalization to interacting ba means the expression of tacit knowledge and its translation into concepts, and making it understandable to others through dialogue.
3. Combination to cyber ba combines new knowledge with existing knowledge into explicit knowledge, transcending the group through different media.

Internalization to exercising ba converts explicit knowledge into tacit knowledge in practice through experiments or simulations
The model of knowledge creation points out the importance of sharing tacit knowledge and explicates much of what happens in urban living lab activities. Although the model creates general understanding about the innovation process, it seems too theoretical in practical matters of urban living lab management. Therefore, we turn to a model developed by Darsø (2003), who investigated innovation teams of a large international company and identified two distinctive phases for a successful innovation process: a project and a preject.
The project refers to the usual project management with goal definition and limited time; it seeks results, prefers linear progress towards goals, and employs convergent thinking and fast decision making. However, in successful innovation processes, the project is preceded by prolonged goal seeking and the emergence of divergent thinking in an open decision space, where a group of people searches for novel knowledge and probes new possibilities. From the perspective of management, this period -the preject -may seem chaotic, but it is crucial for generating radical innovations. Darsø (2003) emphasizes that the preject needs a different type of management that utilizes diverse leadership roles and functions, as identified previously in the group theory literature (e.g., Johnson & Johnson, 2002). Most importantly, to enhance preject development, one needs to know the critical parameters of the preject, which Darsø has crystallized in the diamond of innovation model: building relationships, developing knowledge, exploring ignorance, innovating new concepts) ( Figure 1).

Soile Juujärvi and Virpi Lund
Knowledge in innovation processes is under constant development and has different modes, such as scientific knowledge and personal knowledge that is developed through experience, reflection, and practice. In the innovation literature, personal knowledge is seen mostly as a positive contribution, but Darsø warns that it may include personal beliefs and attitudes that may hinder group development, rather than open up new possibilities. In addition to these modes of knowledge, it is important to acknowledge the role of tacit knowledge that is transformed into explicit concepts in innovation processes (Nonaka & Takeuchi, 1995).
Relationships have a great influence on the quality of the results; mutual trust and honest communication is needed to venture into areas of new possibilities. Relationships can be determined by discussing each participant's expectations and wishes, and their level of ambition in relation to the project. Possibilities to link evolving common goals with personal ones are important, because personal goals motivate participants and enhance commitment (see Bandura, 2001). In the living lab context, it is especially important to identify user motivations that are usually based on personal, rather than professional, interests. Their participation is voluntary, and consequently, strong motivation is needed for long-term engagement.
Ignorance is the most important parameter in the diamond of innovation, because it provokes questions that boost the innovation process. However, participants who reveal their ignorance are susceptible to criticism, which again underscores the need for mutual trust and a supportive atmosphere. Finally, developing new concepts signals the emergence of innovation ideas. Words are often insufficient, and therefore, conceptualization can be advanced through drawings, figures, or 3D models. Whether incremental or radical, concepts are not yet innovations, but can become them through further development (Darsø, 2003). Darsø (2003) further emphasizes that the poles of the axes in the model are not contradictory but complementary and reinforcing, and they can be worked on at the same time. Knowledge and ignorance can be present simultaneously, and continuous movement between them is important. Similarly, conversation about personal interests may stimulate and expand understanding about concepts, and vice versa.
Thus, the rationale of the present study is based on the assumption that, due to the multidimensional nature of urban problems, innovation processes in urban living labs are at high risk of inadequately defining problems. Therefore, a successful urban living lab initiative may require project management to nurture the early innovation process (i.e., the preject), which can be complemented by project planning, as shown in the case study that follows.

Case Study: Espoo Centre, Finland
This study is part of a three-year participatory action research project (Kemmis &McTaggart, 2000) to examine and enhance residents' participation in urban development and to develop efficient means for residents and stakeholders to collaborate in urban development. The research project included wide context mapping with interview with 32 residents and 64 stakeholders, participation in local development networks, and two main interventions: i) special workshops for residents (Juujärvi & Lund, unpublished) and ii) residents and stakeholders (i.e., community workshops). The focus area is Espoo Centre, a part of the municipal district in the City of Espoo in southern Finland, which consists of the administrative centre of the city and two surrounding neighbourhoods, with a total population of 17,000. The area is characterized by different historical layers in terms of construction of social housing, and waves of migration, mainly refugees, from the 1970s onwards.
Cultural diversity in daily life is reflected in a high proportion of immigrants and more than 70 spoken languages. According to social and economic indicators, the area represents the least advantaged suburb in the City of Espoo. The area's strengths include good transportation and services, and access to surrounding natural areas that enable outdoor activities (Hirvonen, 2011;Residents' Welfare in Espoo, 2013). Several academic research projects have pinpointed challenges of the area, and consequently, the city of Espoo has undertaken several projects to improve the environment and launch a regeneration process. In recent decades, non-profit agencies in particular have been eager to start different kinds of development initiatives to improve social cohesion and the wellbeing of citizens.

Community Workshops
The present study was motivated by two main observations within the overall project: i) resident have so far shown low engagement in development endeavours and ii) there has been a lack of systematic collaboration among various stakeholders and developers. The living lab approach was assumed to provide an appropriate innovation platform for a systematic collaboration initiative that would bring together actors who do not www.timreview.ca

Soile Juujärvi and Virpi Lund
necessarily know each other, but who would be expected to exploit each other's resources and expertise. For this purpose, a special method called "community workshops" was designed and implemented. The community workshops method represents an application of the Change Laboratory, which is based on the theory of expansive learning (Engeström & Sannino, 2010) and has been widely applied for promoting innovation and learning within organizations. The Change Laboratory is a formative intervention method where new ideas are developed and put into action in a social process of innovation. Researchers act as interventionists in the process by providing tools for envisioning, designing, and experimenting with novel forms of activities. The rationale behind interventions is to expand participants' understanding about the objects of development work, thereby enabling shared goals and enhancing collaboration. Each workshop has a specific purpose to deepen the innovation process (Virkkunen & Newnham, 2013). Based on our previous pilot (Juujärvi & Pesso, 2012), we expected that the community workshops would combine bottom-up and top-down approaches, because they engage stakeholders and residents in transforming grassroots ideas into new activities and allow enablers to play an active role in shaping the shared vision and boosting activities.
The community workshops included five successive workshops in early 2015 as follows: i) charting the situation; ii) analyzing contradictions and issues with the situation; iii) creating new models; iv) concretizing new models and then experimenting during a period of two months; and v) evaluating the experiments and making decisions about their consolidation. The workshop process yielded four experiments, which each represented new forms of practices in urban development. They represented social innovations, such as co-planning of a local community house, a multi-actor steering group for regional development, a multicultural food festival, and a multi-event square for citizens. Common characteristics in each new practice were that they required developing partnerships and coordinating multiple resources.
The approximately two-hour workshop programme included presentations of pieces of research data, speeches about future lines of development, and various innovation methods. Between 30 and 40 people attended each of the five workshops. The participants were residents, members of resident associations, managers of regeneration projects, city planners, civil servants, representatives of non-profit organizations and local parishes, and managers of shopping malls. The workshops were managed by a consultant qualified to practice the Change Laboratory method in collaboration with four researchers, including the authors, who acted as group facilitators.

Research Design
We employed ethnographic methods of participatory action research (Kemmis & McTaggart, 2000), meaning that we recorded video of all workshop activities, recorded and partly transcribed small-group discussions, and documented the materials. We acted as group facilitators and made observations on interactions and each member's role therein, and we later checked those observations by reviewing the recordings. We acquainted ourselves with the early innovation process and reviewed material from all workshops several times in order to obtain an overall view of the process and re-plan activities for the forthcoming workshop.
It became evident that the third and fourth workshops were the most critical ones. In the third workshop, newly-formed teams started to innovate and plan experiments based on the shared interest (the third workshop); In the fourth workshop, teams finalized and cross-evaluated their plans for experimenting with new ways of collaborating. Therefore, we decided to limit the analysis to these two workshops, which yielded approximately 19 hours of recordings that were partially transcribed.
Qualitative directive content analysis was employed, meaning that data are initially coded with categories derived from existing theories then are complemented with themes emerging from the data. The ultimate purpose is to validate or extend a conceptual framework or theory (Boyatzis, 1998;Zhang & Wildemuth, 2000). This analysis was guided by the following research questions: 1. How are the features of the preject manifested in workshop activities?

How can the process of the preject be enhanced by workshop interventions?
The analysis proceeded as follows. The first author listened to recordings and wrote down observations that were cross-checked by the second author and compared with earlier observations made by the group facilitators. Then, the written observations were coded into categories derived from the components of the preject (i.e., building relationships, developing knowledge, exploring ignorance, innovating new concepts). The content of each category was compared with the theory, and some www.timreview.ca

Soile Juujärvi and Virpi Lund
sub-categories providing evidence of new knowledge were added. Finally, the categories were written as descriptions presented in the findings. With regard to the second research question (How can the process of the preject be enhanced by workshop interventions?), we decided to describe the interventions at the third workshop because they reveal the dynamics between them and the innovation of new concepts.

Workshop Interventions
The objective of the third workshop was to shape goals for near-future development and start to plan experiments based on the previous analysis of contradictions and conflicts in current practices that have hampered development actions in the area (Virkkunen & Newnham, 2013). The participants were guided to choose a group with a pre-determined theme that emerged from the previous workshops and that would match their interests (e.g., multicultural integration, common premises, coordination of urban development, "wild card"). Following the guidelines of the Change Laboratory method, three sets of stimuli were given at different points of the workshop to provoke innovative thinking: i) a synopsis of contradictions in current development presented by a principal researcher, ii) a speech about future urban development given by a director of the city planning department, and iii) a shared group reflection on future possibilities by five volunteering participants. The purpose of these interventions was to enhance common understanding on goal setting and to help participants shape their roles in planned experiments, but the retrospective analysis also revealed unintended positive effects. The most powerful one was the director's speech, which triggered creating new concepts among the participants.
The director told about future lines of local urban planning up to 2030, based on an envisioned zoning scheme of the area. City planning would be focused on laying foundations for a physical environment of high quality, which in turn shall enhance residents' wellbeing and sense of community, for example, by creating meeting places. However, he also emphasized that urban planning procedures do not involve means that would directly address to the problems identified by the participants in the current workshops. Social and cultural aspects are not sufficiently taken into account in the zoning process, and there is a lack of multi-professional cooperation due to the rigid boundaries of administrative units. He admitted his ignorance about how to proceed with these deficiencies and invited the audience to give him some advice and ideas.
The speech was followed by reflections in pairs and a lively plenary discussion, in which participants pointed out critical aspects lacking in city planning: the plan did not cover aspects of social and cultural development, and more specifically, it did not provide any means to prevent further segregation of immigrant groups. The recordings in small-group reflections revealed that the speech had triggered innovative new concepts to overcome limitations of the current city planning. The concept was later explicated in the plenary discussion as social zoning, as illustrated by the following participant comments: After the interventions, the groups were instructed to start planning an experiment for a new way of stakeholder collaboration within the following two months. The participants were encouraged to change their group choice if it no longer matched their interests. The planning continued in the successive workshop, and the evolving plans were cross-evaluated at several points.

The Preject's Group Process
All established teams succeeded in generating a solid plan for their experiments, which represented new cooperational initiatives, including a regional development committee, the co-design of a local community house, a citizen square, and a multicultural food festival. Each planning process involved the elements of the early innovation process: building relationships and sharing knowledge preceding exploring ignorance and innovating new concepts complemented with refinements, as shown in Figure 2. The order of the elements is logical rather than chronological; the teams changed www.timreview.ca

Soile Juujärvi and Virpi Lund
their focus back and forth during the process. In addition, even though all elements could be discerned from the flow of discussion, they were not equally balanced across the teams.
Building relationships took place through the whole process. In the beginning, group members introduced themselves and gave some background information, but later on, they started to ask about and reveal their personal motivations and interests, leading to in-depth discussions. Participants also built friendships and working alliances across the teams. Multiple discussions as a part of the Change Laboratory procedure encouraged them to informally reveal to each other their personal interests, motivations, and feelings.
The important step in building relationships was the emergence of actor roles, which took place after the teams started to plan their innovation projects on the later rounds of the cycle (see Figure 2). In particular, city representatives as enablers had a distinguished role in creating a vision and allocating resources. The representatives of the firms and non-governmental organizations (NGOs) were eager to pursue new services and operational models, and residents showed nuanced knowledge of local conditions and empowered other citizens to participate in forthcoming experiments, which is consistent with previous findings (Juujärvi & Pesso, 2013a). These roles, however, emerged as a result of the planning process and were situation-contingent rather than based on participants' acknowledged positions or expertise . One team member could play multiple roles while being a key driver in the process. Some participants adopted the role of provider/developer, which is strongly encouraged by the Change Laboratory methods. The teams also missed some roles in beginning, typically utilizer or enabler, to accomplish intended actions, and started to look for them outside.
Sharing knowledge was critically tied to building relationships, because participants got to know each other by exchanging knowledge. The exchange of tacit knowledge included sharing personal ways of thinking and emotional outbursts revealing values and attitudes, and its exchange was present through the process. Delivering tacit knowledge was supported by physical proximity in small groups and intensive working periods, and it enabled building a highly positive, lighthearted with plenty of joking and laughing (Nonaka & Takeuchi, 1995). Knowledge included participants' specific local knowledge that was especially useful when defining problems. The expert knowledge of team members was exploited when the plans were realized; during the ideation process, it was largely ignored.
Scrutinizing limitations of current practices is a starting point for development actions in the Change Laboratory method, and therefore, exploring one's own ignorance grows a collective effort to explore the limits of shared knowledge. Ignorance was explored to varying degrees among teams and it was largely induced by the interventions, especially the director's speech, which revealed his own ignorance, as described above. When reflecting on ignorance, participants also hinted about their own skills and expertise, and therefore, this element was renamed as exploring limits of expertise (see Figure 2). This in turn prompted innovative ideas, which were further elaborated into new concepts. The teams struggled to find an appropriate name for their future experiments, because the familiar ones did not align with the core intent of their innovation, as illustrated by the following comments from participants: "Even though the regional welfare group we used to have in past has been ceased, it should be something like this. But it must have a different name. But unlike it, this group must have responsibility, duties, and resources; it cannot be any sort of discussion or coffee drinking club." "Based on its tasks, it ought to be a regional development group. It is an awfully dull name, but it is what this all is about." "We are not satisfied with this name. It must be much cooler, more attractive. Let's put it in quotation marks." Figure 2. Group process of a preject www.timreview.ca

Soile Juujärvi and Virpi Lund
The complexity of the concept was related to the length of the joint group process. In two teams, most members had joined together in the previous workshops, and had already shared ideas to some extent. These teams had lively, enriching discussions. In contrast, two other teams had several member changes between the workshops. They spent a lot time exchanging knowledge and getting to know each other and were forced into premature decisions due to time limits in the previous workshop, which caused them to spend time reshaping ideas in the successive workshop.

Conclusion
This article elaborated the early innovation process in an urban living lab initiative and resulted in several new practices in urban development. They represented social innovations created through collaboration between participants representing diverse living lab roles. It became evident that all living lab roles were needed to realize innovation intents. The strength of socially oriented urban living labs situated in a certain geographical region lies in the dissemination of local knowledge along emerging social networks (Melkas & Harmaakorpi, 2008). The bottom-up approach is a suitable starting point for problem definition and brainstorming, but it must be adjusted by a top-down approach that provides information about official visions, goals, plans, and procedures. The top-down approach can be empowered by civil servants and politicians who can be equal participants or otherwise engaged in living lab activities. Our findings suggest that urban living labs provide a promising approach for neighbourhood renewal, which has been dominated by centralized top-down planning and urgently needs tools for citizen involvement (Pennen & Bortel, 2015). Urban living labs could work as an intermediary bringing self-organizing groups and city developers together to co-create urban space (Horelli et al., 2015). However, this potential can be lost if urban living labs are poorly managed. In conclusion, three key lessons can be taken from the study: 1. Successful urban living lab activities require sufficient time dedicated to early innovation process. Urban living labs are usually established to solve complex problems for which several unsuccessful attempts have already been made. Urban living labs provide an opportunity to bring together stakeholders with diverse knowledge and experience, and to collaborate in tackling those problems. In order to actualize innovation potential, sufficient time should be dedicated to building relationships, exchanging know-ledge, and establishing shared goals. Even in lowthreshold settings, the participation of residents seems to require some citizen skills and relevant basic knowledge, and consequently, additional support and encouragement are needed (Veeckman & Graaf, 2015). Even though not all participants would engage in development projects as a result of the bottom-up process, building alliances across different boundaries is a valuable result from the perspective of networking.

Innovations result from successful team processes.
Early innovation process took place in teams that enable relationships to be built and relevant knowledge to be constructed, as well as the exploration of ignorance and concepts (Darsø, 2003;Darsø & Høyrup, 2011). The present findings suggest that building relationships and sharing knowledge precede the exploration of ignorance and the innovation of new ideas and concepts. Iterating cycles form a progressive spiral, leading to the exploration of actor roles and the limits of expertise, and to the development of more nuanced concepts. There must be sufficient trust and confidence before members dare to reveal their ignorance and to express unconventional ideas, as well as to confront the opinions of others. Members also need to understand and share an evolving innovation concept in order to commit themselves to development projects.

Prejects can be managed.
The present study provides some insights into preject management. First, the alternation of plenary discussions and working in groups created tension between general goals and the interests of participants, and it cross-fertilized innovative thinking among participants. The plenary discussions helped spread ideas across the teams and encouraged the sharing of feedback and additional resources. Second, special attention should be given to protecting and nurturing group processes, for example, using group facilitators. Under time pressure, teams tend to make premature decisions leading to rather conventional projects with limited commitment; therefore, the team should process their idea until something truly new to them emerges and they become emotionally engaged. However, group processes are difficult to maintain across the workshops due to fluctuations in participation. It could be advisable to build teams around key people who are strongly motivated. Third, of crucial importance is a constructive atmosphere that encourages discussions of ignorance and welcoming questions and criticism. Innovations can further be enhanced www.timreview.ca

Soile Juujärvi and Virpi Lund
through specific interventions aimed at exploring ignorance and the limits of expertise. This can be simply done by asking questions, but more sophisticated tools are available (e.g., Virkkunen & Newnham, 2013). With these conditions, an urban living lab can provide a forum for creative collaboration and problem-solving in community and urban development.  Voytenko et al., in press). In the words of Allen, McKeever, and Mitchum (1996), a sustainable community is created through "the deliberate effort to ensure that community development not only enhances the local economy, but also the local environment and quality of life". This definition emphasizes the local dimension of the three conventional sustainability pillars. In practice, many cities and municipalities have operationalized what sustainability means to them and what aspects are considered particularly important to address (e.g., Botkyrka Municipality, 2009).

About the Authors
Many sustainability issues are highly visible in numerous suburbs across Europe that were built in the 1960s and 1970s and are characterized by outdated urban systems in urgent need of modernization and social uplift. These suburbs share many challenges related to local aspects of sustainability and quality of life, which makes it relevant to speak of suburban sustainability. In general, the population of these less valued suburbs is relatively demographically homogenous in terms of, for example, income level, education level, and social background, although they may represent a range of ethni-A number of urban living labs have been set up in recent years, with the aim of developing innovation processes within a multi-stakeholder partnership in an urban context. Several urban living labs focus on sustainable development, which is a visible and urgent issue in less valued suburbs in need of modernization and social uplift. We argue that, when applying the living labs approach in the context of sustainable development in suburbs, the primary focus should be society's collective goals, as expressed through municipalities and users. cities. Segregation often becomes obvious, as marginalized groups live largely separated from other societal groups -this situation is sometimes reinforced by poor transport solutions that cut off an area from other parts of the city. It is also common that poor urban planning and lack of modernization has contributed to perceptions of unsafe urban environments. Moreover, the buildings themselves often have high energy consumption and lack modern technology that could have limited their negative environmental impacts. Massive renovation of these urban areas is required, but investment opportunities are often quite limited. Modernizing these suburbs in a sustainable way will not only require comprehensive investments in advanced technology but also must take into account social, ecological, and economic objectives. Urban living labs offer opportunities to bring existing groups of urban actors together in new ways to allow for local sustainable development (Voytenko et al., in press), thereby responding to calls for methods and arenas that promote stakeholder collaboration and learning in urban development (Elbakidze et al., 2015). Urban living labs can develop suburbs through an advanced form of stakeholder and resident involvement, which may increase the chances that modernization actions contribute to sustainability in a way that resonates with local people's ideas and needs.
Many living labs have been used to test information and communication technology (ICT) and services (e.g.,  and have featured companies with commercial goals or educational institutions with research goals as a main driver. One research branch uses living labs to study "smart cities" (Hirvikoski, 2014;McPhee et al., 2015), often with a focus on ICT such as digital development services (Eskelinen et al., 2015). However, many urban living labs do not serve as company-driven technological research environments, but rather as platforms for citizens to participate in city planning (Juujärvi & Pesso, 2013). Voytenko and colleagues (in press) found that, among five major European urban living lab projects that address sustainability, private sector involvement was not particularly salient. Although business has an important role to play in sustainable development, we focus in this article on how contributions to sustainability in less valued suburbs can be made without or with only minimal commercial goals.
We argue that, when applying a living labs approach with sustainability objectives in less valued suburbs, we should first and foremost start with the collective goals of the society, expressed through municipalities and the users themselves, while acknowledging the goals of all participants. Urban living labs offer an excellent platform to develop less valued suburbs in a sustainable direction through multi-stakeholder collaboration. Against this backdrop, the aim of this article is to show inspiring examples of how urban living labs can be applied in less valued suburbs, starting from the society's collective goals, in order to respond to local sustainability challenges. In line with previous research on the early development of living lab methodologies (Ståhlbröst, 2008), we focus on the design phase of the urban living labs. Our article also contributes to the growing collection of empirical studies of urban living labs (e.g., Juujärvi & Pesso, 2013;Voytenko et al., in press). We share lessons learned and hope to inspire others to use living labs to contribute to suburban sustainability.

Living Labs in a Suburban Context
There is no generally accepted definition of living labs (Leminen, 2015;, but they are frequently described as consisting of elements of cocreation, exploration, experimentation, and evaluation (e.g., ENoLL, 2015). Leminen (2015) emphasizes that living labs are used by communities and for innovation. An urban living lab has been defined as "a forum for innovation, applied to the development of new products, systems, services, and processes in an urban area; employing working methods to integrate people into the entire development process as users and co-creators to explore, examine, experiment, test and evaluate new ideas, scenarios, processes, systems, concepts and creative solutions in complex and everyday contexts" (JPI Urban Europe, 2015b). Rather than repeating previous reviews of various definitions, we explain below how two key ingredients of urban living labs -citizens and innovation -were operationalized in the context of this research.
In an urban perspective, it is common to refer to citizens as important co-creators (e.g., Eskelinen et al., 2015;Hirvikoski, 2014;Juujärvi & Pesso, 2013). Citizens include residents as well as other people who spend time in the area, through work, school, leisure activities, etc., and stakeholders who are concerned with or may be affected by an activity there. With regards to innovation, we apply a broad perspective in the sense that it is not necessarily a brand new product or service, but rather a new valuable solution in a particular context. To illustrate this perspective, the use of the living labs approach in an urban context is relatively new, but it is innovative to use living labs in the context of adwww.timreview.ca

Urban Living Labs for Sustainability in Suburbs in Need of Modernization and Social Uplift
Katarina Buhr, Maija Federley, and Anja Karlsson dressing sustainability issues in less valued suburbs. Although participatory methods have been used in these suburbs before, urban living labs go well beyond common dialogue practices.
Urban living labs can be initiated by various actors (Voytenko et al., in press), and it has been suggested that the type of actor that drives the activities within a living lab affects its characteristics.  distinguished between four types of living labs depending on the driving actor: i) utilizer-driven living labs, which are driven by companies to develop their business; ii) enabler-driven living labs, which typically are public sector projects built around regional development objectives; iii) providerdriven living labs launched mainly by developer organizations such as educational institutes, universities or consultants, to promote research and knowledge creation; and iv) user-driven living labs established by the user community itself, focusing on solving specific problems for the users and benefitting other stakeholders only indirectly. We find this distinction useful in that it highlights how living lab objectives are intimately linked with the driving actors and their essential goals.
When applying living labs focusing on sustainability in suburbs that are in need of modernization and social uplift, it makes sense to start with a focus on the collective goals of the society (i.e., municipalities and users). It can be a way to address important sustainability issues that often fall outside the responsibility and interest of single actors, such as creating meaningful and inexpensive activities for residents and improving the safety and appreciation of public spaces. Although these issues are typically on the agenda of public authorities, there is an added valued when involving additional actors through a living lab approach. Also, urban living labs are often closely linked to city development processes that normally span several years and therefore need long-term commitment to achieve their full potential (Juujärvi & Pesso, 2013).
We argue that there are at least three reasons why living labs are a useful approach to address sustainability challenges in suburbs in need of modernization and social uplift. First, many of these suburbs face major modernization measures both in the indoor and the outdoor environments (Häkkinen, 2012), and living lab methods increase the chances of gaining broad support for such large changes. Second, taking users' ideas into account in urban development increases the chances of users valuing, taking pride in, and appreciating the at-tractiveness of their local area, which is beneficial for these suburbs. And, third, living lab methods can support interaction between municipalities and residents, giving residents a feeling that they are being listened to, which may be particularly important in suburbs where a relatively large share of the population can be described as marginalized. Engaging residents in urban development can in itself be a way to contribute to social sustainability (Weingaertner & Moberg, 2011).

Case Studies from Alby and Peltosaari
This article builds on analyses from the JPI Urban Europe research project SubUrbanLab (http://suburbanlab.eu), in which researchers from VTT Technical Research Centre of Finland and IVL Swedish Environmental Research Institute cooperated with the municipalities of Botkyrka and Riihimäki to set up urban living labs in two suburban case areas located in the outskirts of the capitals of Sweden and Finland, respectively (cf. Thörn et al., 2015).
The Peltosaari neighbourhood is located next to Riihimäki city centre, north of Helsinki, Finland, and has approximately 3,000 residents. The buildings in Peltosaari represent typical concrete apartment buildings constructed during the 1970s and 1980s. A large share of the municipality's social housing is located in Peltosaari and the population structure is biased, with a considerably larger share of unemployed, low-income households and residents with lower educational levels than average in Riihimäki. The challenges in the area include the physical condition of buildings, low interest from private investors, social problems and general untidiness. The market prices of the apartments in the area are remarkably lower than in other areas with similar locations close to railway stations and services. On the positive side, Peltosaari is known for its many activities organized by volunteer residents and its laid-back atmosphere.
The Alby neighbourhood of the Botkyrka municipality is a suburban area in the south of Stockholm, Sweden, with around 13,300 inhabitants (Botkyrka Statistikportal, 2015). The housing stock in Alby was built in the early 1970s, during a time when approximately one million new dwellings were built in Sweden due to new living standards and an increased demand for apartments. The area is characterized by large-scale uniform buildings and sterile public spaces. Like many other neighbourhoods built during this time, Alby is in urgent need of comprehensive renovation and renewal of both the housing stock and its surroundings. One imwww.timreview.ca

Katarina Buhr, Maija Federley, and Anja Karlsson
portant challenge is to bring the housing up to today's environmental standards and needs. The area also faces considerable social challenges, such as high unemployment rates and segregation, and is at the same time constrained by a lack of economic resources. Approximately 60% of the inhabitants originate from other countries than Sweden (Botkyrka Municipality, 2015). Alby is located in a part of Botkyrka where citizens have expressed that they do not feel secure in public spaces (SCB, 2015). However, Botkyrka is also famous for its rich cultural life with a salient artistic vibe.
Both Peltosaari and Alby have previous experiences of involving the citizens in their decision-making, for example through dialogue forums in Alby and "Peltosaari Parliament" in Peltosaari, but it has been challenging to receive commitment from broad and representative groups of residents (Thörn et al., 2015). Urban living labs offer a possibility to test new working methods, but it is important to carefully consider how they can be designed in order to contribute to the suburbs' sustainability challenges and how to make participation rewarding for the stakeholders. In the following section, we show examples of these opportunities and challenges by describing two of the six urban living labs carried out in the research project SubUrbanLab.

Alby (Sweden): New Light on Alby Hill
The "New Light on Alby Hill" living lab was set up to contribute to local sustainability by transforming a walkway lined by vegetation that residents had previously identified in surveys as insecure. The walkway is one of the few stretches through which residents can ac-cess public transportation and downtown from the residential area of Alby Hill. We set up new LED technology along the walkway and four so-called "Gobos", which allowed for artistic decoration through light installations to be projected on two rock walls and two spots in the grass near the walkway (Figure 1). LED technology is energy efficient and inexpensive and was in this project used as a way to allow light to be distributed over a larger area to increase the residents' sense of security, as compared to the existing lighting conditions. The images used as artistic decorations on the rock walls and spots in the grass were drawings submitted by local residents and elementary school students on the theme "Our Alby". In this way, the artistic decorations gave a voice to the users of this walkway, while highlighting the surroundings. Out of 20 images submitted by residents, two images were selected by a jury of local stakeholders and two images were selected by residents via an Internet-based poll. Three of the selected images gave expression to anti-racism messages.
"New Light on Alby Hill" was designed to address several sustainability challenges that are typical for many less valued suburbs, such as the perceived lack of security in public spaces, old-fashioned lighting with relatively large energy consumption, lack of aesthetic public spaces, unattractiveness of public transport, and need for meaningful activities for youth. These are important issues that fall mainly under the responsibility of the municipality, which is why Botkyrka municipality participated in dialogue with residents from the start. Several of these sustainability challenges had previously been identified by Botkyrka Municipality and were formalized as goals in their program for sustainable de-

Urban Living Labs for Sustainability in Suburbs in Need of Modernization and Social Uplift
Katarina Buhr, Maija Federley, and Anja Karlsson velopment (Botkyrka Municipality, 2009). This work already built on dialogue with local residents, and within "New Light on Alby Hill", the needs and ideas of local residents were further explored and taken into consideration. Residents -the main users of this walkwayparticipated in the urban living lab through, for example, the Alby Hill Residents' Council, a private housing company (Mitt Alby), and a local school. The residents were particularly involved in the planning of the project and the design of ambient light, although everyone had the opportunity to submit drawings and vote on what drawings they would like to see projected along the walkway. Lighting designers, local entrepreneurs, and landscape architects supported the technical aspects of the urban living lab, and researchers at the IVL Swedish Environmental Research Institute both supported and studied the process, although product development or knowledge creation was never their main focus. Methods used included an interactive website and different channels for information, open Internet-based polls, and regular meetings between the municipality, the housing company, and the researchers. In summary, "New Light on Alby Hill" allowed for an innovative co-creation activity to renew a walkway in Alby that will be better appreciated by its users.

Peltosaari (Finland): Together More
The aim of "Together More" was to improve people's appreciation of Peltosaari and strengthen social cohesion. These were important priorities for the City of Riihimäki, which was a driving actor behind the urban living lab from the start. The municipality built on the identified needs of the community, striving to enable activities run by third parties and let users take responsibility over them to support continuity. Providing methodological support for the urban living lab, the research institute VTT performed a survey at the beginning of the lab to explore the needs and ideas of the citizens of Peltosaari who were the users of public services in the area. The citizens expressed that meaningful and lowcost activities were needed for youth and families with children, for example, but so far, these groups had been difficult to engage. Activities across groups, such as senior citizens, children, immigrants, and the unemployed, were also needed to increase the communal feeling, as well as improved dialogue between the municipality and residents. Another challenge in Peltosaari was that several residents had experience of being involved in previous development projects where they felt frustrated with the uncertainty of implementation and few visible improvements.
"Together More" became an umbrella to implement two types of activities, both of which had been identified on the basis of local development priorities and user interests. First, a former grill kiosk was renovated in a central location to create a meeting place that would act as a "living room" for local residents ( Figure  2). The venue was made available free of charge for potential leaders of leisure activities, and it offered selected municipal services. Second, a range of events and activities were organized, such as senior gymnastics, urban gardening, a multicultural café, school break activities for children, the building and opening of a fishing place, and a mid-summer party (Figure 2). Several of these events and activities were set up so that people could spontaneously join in, which reached all kinds of people, including youth and families with children. The urban living lab coordinated facilities, events, and people and supported the implementation of these activities. What these activities had in common was the aim of improving appreciation of the area and strength-

Urban Living Labs for Sustainability in Suburbs in Need of Modernization and Social Uplift
Katarina Buhr, Maija Federley, and Anja Karlsson ening social cohesion, hence addressing sustainability challenges that are common in less valued suburbs, such as segregation, loneliness, inequality, and frustration among citizens -phenomena that also lead to increased social costs and decreased welfare among citizens. There are important issues for the society and it is in the interest of the public sector, including municipalities, to address them.
"Together More" was planned, designed, and implemented on the basis of the priorities of the City of Riihimäki and residents, who expressed themselves through third sector organizations, the "Peltosaari Parliament" and the "Peltosaari Association", as well as directly through a range of dialogue fora organized by researchers at VTT. Some of the activities were launched in co-operation with other concurrent projects of the municipality, such as "Youth First", which offered personalized support for youth employment, and "Liikuta" and "Kulttuuriviritys", which organized cultural activities for local residents, such as urban gardening. This deliberate strategy by the municipality was designed to reach and engage more people. VTT organized an early survey sent out to residents in which prioritized areas for development were identified, discussion events in which users were encouraged to influence the plans, face-to-face discussions with residents that participated in the activities, and meetings with the Peltosaari Parliament and the Peltosaari Association. Facebook, Twitter, and Instagram accounts were established to further improve communication about the events and observations in Peltosaari. In summary, "Together More" launched processes for co-creating a more attractive neighbourhood that would appeal to residents, visitors, and other stakeholders.

Conclusion
We have shown examples of how urban living labs in less valued suburbs can contribute to sustainability based on societal goals (i.e., the goals of municipalities and users), building on analyses from two urban living labs in Alby and Peltosaari. Both urban living labs should be regarded as a combination of enabler-driver and user-driven living labs (cf. , given that societal goals were primarily derived from the goals of municipalities and users. The networks formed around municipalities and users, rather than around the research institutes or other stakeholders. Key purposes revolved around local development objectives and problem-solving for the community, not around research and knowledge creation or commer-cial goals. To conclude, we draw lessons about how urban living labs can be applied in a suburban sustainability context in order to create favorable conditions for: i) municipality and user engagement and ii) continuation towards long-term sustainability.
Applying urban living labs in two different countries allows for insights that may otherwise have passed unnoticed. Besides cross-national learning throughout the process, at least two observations regarding the co-creation part in the design phase deserve attention. As many living labs have noticed, engagement among users should not be taken for granted even though the activities focus on improving their everyday lives (e.g., . The experiences from Alby and Peltosaari suggest that user engagement can be spurred by addressing sustainability challenges that are particularly salient in the suburb and allow people to express themselves on issues that already engage and interest them. "New Light on Alby Hill" was designed not only to improve the outdoor illumination and the sense of security, but also to draw on the esthetic "vibe" that exists there. With no steering, a significant share of the contributions, expressed through art, came to focus on anti-racism messages: a salient issue in Alby, which has a large proportion of immigrants. The Peltosaari-based urban living lab "Together More" was designed to enhance appreciation of the area and strengthen social cohesion. The urban living lab took advantage of the laidback Peltosaari spirit by developing activities that people could spontaneously join into, such as urban gardening, the multicultural café, fishing, and midsummer celebrations. These activities also fulfilled the wish among citizens to improve tidiness of the area and enhance dialogue with the municipality. Both urban living labs experienced enhanced user engagement by launching activities with visible results in public spaces that allow for long-term continuation.
To spur municipality engagement, one needs to take into consideration the institutional and cultural preconditions when introducing the living lab approach. Even though the municipalities involved had previous experience of dialogue with citizens, they had to challenge and expand their idea of co-creation into something more extensive. In public organizations, challenges of communication, collaboration, and coordination between departments may be more evident compared to the private sector. Different departments may face different demands and have different goals. Also, the cross-national comparison revealed different traditions with regards to ideas about the extent to which residents should be involved in decision making. www.timreview.ca

Katarina Buhr, Maija Federley, and Anja Karlsson
The issue of long-term continuation of the living lab initiatives may be particularly challenging when the urban living lab is based on societal goals and with few commercial objectives yet is important in light of the pressing sustainability issues in many suburbs. The urban living labs presented here paved the way for long-term continuation by setting up facilities and improving social relations, both of which need maintenance. In addition, both urban living labs need involvement of the municipality in order to continue, although it does not necessarily have to be the driving actor. In the case of "New Light on Alby Hill", the municipality owns the walkway and the lightning, but the drawing contest can be delegated to the users of the walkway. In a similar vein, "Together More" offered a venue, or "living room", free of charge thanks to the municipality, but many activities could be run by residents. If the urban living labs are transformed into user-driven urban living labs primarily run by the residents, it may further strengthen residents' feelings of inclusion and participation in the local society. But, residents may not have the time, interest, knowledge, and skills to drive a living lab. On the other hand, an enabler-driven urban living lab, run by the municipality, may increase the chances of sufficient resources; however, the working methods of living labs need to be fully embraced and integrated into existing organizational routines. Experiences from "Together More" also show that residents valued the municipality's initiatives to co-create activities that would improve social cohesion and the general appreciation of Peltosaari. The research project SubUrbanLab offers additional lessons from six urban living labs that can provide further inspiration to continue exploring the opportunities for such labs to address sustainability in less valued suburbs.

Recommended Reading
This article builds on the research project SubUrban-Lab, funded by VINNOVA and Tekes through Joint Programming Initiative -Urban Europe, 2013-2016. The project includes six urban living labs based in Alby and Peltosaari. Information and reports from the project, and each urban living lab, is available at suburbanlab.eu

About the Authors
Katarina Buhr is a Social Scientist at IVL Swedish Environment Research Institute. She holds a PhD in Business Administration (Organization and Management) from Uppsala University in Sweden and has been a post-doctoral researcher at Linköping University in Sweden. She has worked in several research projects related to urban sustainable development and has published widely on policy processes and public engagement in the environmental and sustainability field. In SubUrbanLab, she was particularly involved in the evaluation and scientific writing activities for the urban living labs in Alby, Sweden.

Maija Federley is a Senior Scientist at VTT Technical
Research Centre of Finland. She holds a Master of Science degree in technology from Helsinki University of Technology (currently Aalto University) in Finland. She has worked in several research projects related to co-development of digital services and environmental sustainability communication in stakeholder networks. In SubUrbanLab, she was particularly involved in designing and observing all urban living labs in Peltosaari, Finland, with a special interest in participatory methods and development of urban living lab practices and evaluation. Anja Karlsson has worked at IVL Swedish Environment Research Institute since 2011. She holds a BSc in Political Science and Environmental Science from Gothenburg University, Sweden and an MSc from Uppsala University, Sweden. Her studies have focused on stakeholder and public participation in local and national decision making. She has worked in research projects related to sustainable development in urban areas, focusing on social sustainability and the involvement of residents and other stakeholders in urban development. In SubUrbanLab, she was particularly involved in the urban living labs in Alby, Sweden, and the evaluation of the urban living labs.

Introduction
Within living lab research, end users are involved actively to develop an innovation that is adapted to their needs and wants. A living lab environment is defined as "a user-driven open innovation ecosystem based on a business-citizens-government partnership which enables users to take an active part in the research, development and innovation process" (European Commission, 2009). In addition to this active user involvement, a multi-method approach and real-life interventions make up the three central characteristics of the living lab approach . Although questions have been raised about the extent to which living labs are capable of achieving the necessary levels of user engagement and keeping in mind that their interests are sometimes overlooked (Dutilleul et al., 2010), users are generally seen as very important actors.
A living lab study by Ebbesson and Eriksson (2013), in the context of an online platform to gather input from end users, showed good support for the end users during the startup phase of the projects, but also showed an increasing number of users dropping out or lowering their activity level. When studying the motivations of end users participating in open innovation processes, Ståhlbröst andBergvall-Kåreborn (2011, 2013) found a close relationship between motivational factors and the values achieved, and thus that most voluntary contributors are satisfied when learning new things. Intrinsic motivations such as learning, being entertained, and stimulating curiosity are seen as the most important motivators to participate in an innovation intermediary context. Baccarne, Logghe,  also found that the main motivator to participate in living lab research is intrinsic in nature, but for repeated participation, material incentives become more important as motivators. They also argue that the motivations to participate tend to differ according to the research step.
With this study, we wanted to dig deeper into the reasons why people participate or drop out during living lab research. Because there seem to be differences between research techniques (e.g., surveys, field trials, co-creation workshops) , we decided to focus on one research step in particular: field trials. Field trials can be defined as "tests of technical and other aspects of a new technology, product or service in a limited, but real-life environment" (Ballon et al., 2005).
Next to active user involvement and a multi-method approach, a third major principle within living lab research consists of capturing the real-life context in which an innovation is used by end users. Field trials are a method to study the interaction of test users with an innovation in the context of use. However, when conducting field trials, there are several reasons why users stop participating in research activities, a phenomenon labelled as attrition.
In this article, we elaborate on drop-outs during field trials by analyzing three post-trial surveys of living lab field trials. Our results show that several factors related to the innovation, as well as related to the field trial setup, play a role in attrition, including the lack of added value of the innovation and the extent to which the innovation satisfies the needs and time restrictions of test users. Based on our findings, we provide practical guidelines for managers to reduce attrition during field trials.
Motivation is the art of getting people to do what you want them to do because they want to do it. Dwight D. Eisenhower (1890-1969 34th President of the United States

Annabel Georges, Dimitri Schuurman, and Koen Vervoort
They also link up with the "real-life intervention" characteristic of living lab projects . Field trials enable researchers to study the use of the innovation by test users in a natural use context and allow them to discover and understand how technologies are being used and adopted in a real-life setting, which is one of the key principles within living lab research (Ballon et al., 2005;Kjeldskov & Skov, 2014;. In contrast to other research methods, participation in field trials requires a prolonged engagement of test users because they are expected to test an innovation during a specific period. Moreover, in most field trials, users are asked to actively provide feedback regarding their usage. However, following participants over a prolonged period also increases the risk of drop-out before the end of a test period (Schuurman & De Marez, 2009).
In previous research on attrition during field trials, some studies have been conducted in the field of eHealth. Within this domain, Eysenbach (2005) introduced the law of attrition, which is "the phenomenon of participants stopping usage and/or being lost to follow-up, as one of the fundamental characteristics and methodological challenges in the evaluation of eHealth applications". Simons, Hampe, and Guldemond (2013) mention time and timing issues as reasons why people stop participating. For eHealth trials on the internet or with self-help applications, high dropout rates "may be a natural and typical feature"; however, it is important to further analyze the attrition data, because it may give an indication of real-life adoption problems (Eysenbach, 2005). Eysenbach (2005) also identified two sorts of attrition, namely dropout attrition, which is "the phenomenon of losing participants to follow-up (e.g., participants do not return to fill in follow-up questionnaires)" and non-usage attrition in which participants "have lost interest in the application and stopped using it". In a field trial, an example of dropout attrition would be test users continuing to use the innovation but no longer providing feedback, whereas non-usage attrition occurs when test users stop using the innovation but can still give feedback regarding their non-usage. The second type of attrition provides important information for the innovation development process, whereas the first type of attrition generates less information. Therefore, it is especially relevant to minimize the rate of dropout attrition.
Multiple studies have illustrated the occurrence of attrition in the context of eHealth applications, without digging into the causes of the attrition (Grudin, 2002;Korn & Bødker, 2012). Kanstrup, Bjerge, and Kristensen (2010) argue that the stability of the ICT infrastructure and some kind of user support are factors that decrease the rate of attrition, but do not make a distinction between dropout and non-usage attrition.
More in-depth research regarding the attrition within ICT field trials or living lab projects is lacking, despite the specific testing opportunities in multiple real-life contexts of new media innovations and because of their ubiquitous nature (Grudin, 2002;Korn & Bødker, 2012). Therefore, within this paper we want to tackle two main research questions:

Methodology
The main goal of this study is to find factors that are related, either positively or negatively, to different types of attrition during field trials. Therefore, we conducted a qualitative analysis within three Living Lab field trials. The field trials were carried out in living lab projects from iMinds Living Labs (tinyurl.com/zqm6qsn), a division of the iMinds ICT research institute of Flanders, Belgium. The attrition rates per field trial (based on project documents) are described in the results section.
In order to find as many factors as possible, we selected three cases that differ in multiple ways, such as sample size, type of innovation, field trial setup, and communication with test users. First, we conducted a quantitative analysis on the attrition rates. The qualitative analysis was done by coding the answers test users gave to open questions from the post-trial survey. Thus, during the analysis and interpretation of the results, we must consider that the survey data will only include information about non-usage attrition, because test users subject to dropout attrition will already have dropped out. The answers to these open questions were analyzed using QSR International's NVivo 10 qualitative data analysis software. When analyzing the factors related to attrition, we coded the factors that the same codes could be used for the three field trials.
Below, the field trials are further described and the answer rates of the post-trial surveys are given. One general finding is that, in all cases, the dropout attrition rate is high. This high dropout attrition rate must be kept in mind when interpreting the results. www.timreview.ca

Annabel Georges, Dimitri Schuurman, and Koen Vervoort
Field trial 1 The first field trial was part of a living lab project to develop a location-based service application. The application was tested for seven weeks and participants received weekly emails with updates, tasks related to the innovation, and a feedback form where technical problems could also be reported. At the end of the trial, a survey was sent to 558 test users to receive feedback regarding their experience during the field trial ( Figure 1).

Field trial 2
The second field trial was part of a living lab project in which an application to meet up with friends was co-created. The application was tested for five weeks and the participants received weekly emails to give feedback about the innovation, and they were given a weekly assignment. At the end of the field trial, a post-trial survey about the innovation was sent to the 55 participants ( Figure 2). The test users could also send the survey to their friends or family that also tested the application. In total 35 test users and eleven contacts of the test users filled in the survey completely.

Field trial 3
The third field trial was more data-driven. Participants had to read 30 news articles for the first try-out and 60 for the second and third try-outs. They could choose to participate in one or more try-outs. Within this field trial, participants did not co-create the innovation, but instead received different assignments that generated data that was needed to test an underlying technology. They did not know the exact intention of the field trial. Participants were rewarded with a cinema ticket for each finished assignment. At the end of the field trial, a survey was sent to 350 participants to get feedback about the trial (Figure 3).

Results
Attrition rates during field trials Within this section, we dig deeper into the first research question: To what extent can different types of attrition be distinguished within ICT living lab field trials? Within the first field trial (Figure 4), we see two dips in the attrition rate: i) when respondents have to fill out an intake survey to participate during the trial (dropout attrition) and ii) at the end of the field trial, when many test users stopped using the innovation (non-usage attrition). Thus, many test users did not participate for the entire duration of the field trial.
For the second field trial, we see that the pattern of attrition ( Figure 5) is similar to the first field trial. There is high dropout attrition when people have to complete the intake survey, and then there is further (non-usage) attrition during the field trial. However, at the end of the trial, 35 participants filled out the post-trial survey.
Concerning the third field trial, the highest attrition rate was observed when test users had to complete the assignments ( Figure 6). For the first assignment, for which the users were asked to read 30 news articles, the non-usage attrition rate was approximately 10% lower than for the two subsequent assignments, each of which required them to read 60 articles. Thus, the lower attrition rate in the first assignment may be explained by it being less cumbersome than the other two assignments. Because it was not expected from the test users to give feedback about an innovation via several research methods, the dropout attrition during this field trial was rather low.
In general, we can conclude that, within living lab field trials, dropout attrition occurs during different phases of the trial. A crucial moment for dropout attrition seems to be the intake survey. This increased attrition is most pronounced in the first and second field trial, which seems to be caused by the fact that these surveys had more than 20 questions, whereas in the third field trial, users only had to fill in five questions. Within the first and the third case, there was a delay of several days between the intake survey and the start of testing. However, compared to the second trial, in which the participants received a link to test the application immediately after filling in the survey, there were no substantial differences in attrition.
Non-usage attrition occurs especially after the first time the test-users are confronted with the innovation. Within the next section we will dig deeper into the reasons why participants dropped out during living lab field trials.

Annabel Georges, Dimitri Schuurman, and Koen Vervoort
Factors related to participation in field trials Next, we examine factors that can play a role in the attrition during field trials. The data used for this analysis is based on the post-trial surveys at the end of the field trials. The respondents were asked to explain why they stopped using the application or why their use decreased, increased, or stayed constant throughout the entire field trial. These answers were coded according to the different factors related to attrition.
First, we analyzed the factors that are positively related to participation in field trials (Table 1). When analyzing the first field trial, the assignments that were given to test the innovation were seen as particularly positive, likely because of the users' curiosity: they wanted to know what the innovation was about. For the second field trial, only a few people kept on testing the innovation during the field trial. Therefore, only three factors were mentioned: i) the "fun factor" of testing the innovation, ii) the added value of the app, and iii) the fact that friends also started to use and test the innovation. For the third field trial, extrinsic motivation (incentives) and intrinsic motivation to participate in scientific research played a role as a factor that motivated test users to finish the assignments.
When comparing the factors that are related positively to participation in field trials with the motivational factors mentioned by Ståhlbröst andBergvall-Kåreborn (2011, 2013), we see that learning new things (e.g., increasing one's own skills), being entertained (e.g., fun), and stimulating curiosity were also mentioned by the participants in the field trials. During the three field trials, the fun factor played a motivating role.
Next, we analyzed the factors that are negatively related to participation in field trials ( Table 2) and found that different factors are of importance for each field trial. Within the first field trial, users stopped using the innovation because they did not see the benefit of using it. There were only a limited amount of features available, which made the innovation less interesting to test and made it less likely that test users would test it for a longer period, keeping in mind their time restrictions. For the second field trial, participants mentioned that the innovation did not satisfy their needs. Furthermore, technical issues and a small user base generated dropout among the participants. This finding is in line with Kanstrup, Bjerge, and Kristensen (2010), who argue that users dropout when the technology is unstable. Finally, for the third field trial, time restrictions caused non-participation in the assignments, as was also found by Simons, Hampe, and Guldemond (2013).
When analyzing the factors across the three field trials (Table 3) and when digging deeper into the difference between dropout and non-usage attrition, our studied cases suggest that dropout attrition is mainly linked to the research setup, whereas non-usage attrition is mainly linked to factors related to the innovation itself.
When comparing the non-usage attrition over the three field trials, we see that it is high for the first and second field trial because these projects focused more on user co-creation of the innovation, which corresponds with the active user involvement characteristic of living lab research. In the third field trial, the non-usage attrition was lower. The focus in this project was more on the users generating data that allowed testing of the underlying technology, which made the co-creation aspect less important. Next to this, the participants also received cinema tickets after completing their assignments. In the first and second field trial, the participants were not certain they would receive a material incentive, however they did not mentioned this as a factor in their decision to participate in the field trial. Thus, incentives helped when participants had to finish a certain assignment, but when test users had to co-create, intrinsic motivations became more important.
The higher non-usage attrition for the first and second field trial is interesting for the instigator of the project: it points to factors related to the innovation (e.g., usability problems or users not seeing the benefit of the application), which should lead to iteration of the innovation or of the use cases. This finding is in line with Eysenbach (2005), who argues that attrition data can give clues about real-life adoption problems.
Also, network externalities, or the nature of the innovation itself, can cause non-usage attrition. For example, during the second field trial, the testing involved an application for meeting up with friends, which implied that the friends of the test users also had to use the application. These network externalities related to the innovation had a negative influence on the sustained usage of the innovation as the factor "not enough users" scored very high for this field trial.
Also, differences in the dropout attrition are noticeable between the trials. These factors are mostly related to the design of the field trial. For the first field trial, we see, for example, high interest among participants to start the field trial, but a very high attrition rate subsequently. This high level of interest in participating can be explained by the communication strategy that was used. A narrative was generated for the field trial, www.timreview.ca

Factors Affecting the Attrition of Test Users During Living Lab Field Trials
Annabel Georges, Dimitri Schuurman, and Koen Vervoort

Factors Affecting the Attrition of Test Users During Living Lab Field Trials
Annabel Georges, Dimitri Schuurman, and Koen Vervoort

Factors Affecting the Attrition of Test Users During Living Lab Field Trials
Annabel Georges, Dimitri Schuurman, and Koen Vervoort which asked the test users to help as "undercover agents" and to go on missions to test a new secret application. The mysterious nature of the narrative seemed to have a positive influence on the willingness to participate by triggering the curiosity of the test users. The long intake survey, which was cumbersome, and the lack of perceived added value of the application caused the highest attrition rate.
For the second field trial, the participants were clearly briefed regarding the innovation and were stimulated to provide feedback that would be taken into account by the project instigator. This trial attracted a lower number of test users, but we noticed a lower rate of dropout attrition: some test users kept on giving feedback although they stopped testing the innovation itself. This relatively low rate of dropout attrition seems to be caused by the intrinsic motivations of the participants, who were involved in active co-creation, coupled with reminders that were sent for filling in the feedback surveys.
Concerning the non-usage attrition, the duration of the trial also can play a role in attrition. For example, the first field trial lasted for six weeks, the second trial lasted for four weeks, and the third trial lasted one week per assignment. When comparing the trials, we see a bigger attrition rate for longer field trials.

Guidelines for Project Instigators and Managers
Although the results presented here are exploratory in nature, and further research is needed, we have summarized the main lessons learned in the form of practical guidelines related to: i) the innovation and non-usage attrition and ii) the field trial setup and dropout attrition.
Guidelines related to the innovation and non-usage attrition 1. Introduce the innovation clearly and underline its benefits.
2. Stress the co-creation aspect: test users can be motivated by knowing that their contributions can impact the innovation.
3. Conduct usability testing before the start of the field trial so that any technical issues can be solved beforehand. If there are still technical issues during the field trial, then provide a clear help channel and manage the expectations of test users by, for example, re-minding them that the innovation is still in its development phase.
4. Try to anticipate network externalities, because the number of test users can impact the relevance of certain functionalities of an innovation.
5. Communicate clearly at the beginning of the trial what is expected from the test users. Define tasks for the test users to stimulate usage.
6. Remind test users to perform the requested tasks. Some may not otherwise set aside time for testing or they may not remember that a task is to be completed.
Guidelines related to the field trial set-up and dropout attrition 1. Create an accessible helpdesk and make it clear who is responsible for operating it. By including a helpdesk, test users can always give useful feedback when they have the time.
2. Ensure that the testing initiation process is clear and straight forward (e.g., by providing a clear test link at the start).
3. Provide incentives to encourage test users to complete tasks. However, note that incentives do not trigger test users to give valuable feedback.
4. Include some fun (or even funny) tasks or assignments that challenge the users or trigger their curiosity. Appeal to the motivating factor that encourages participation just for fun.

Conclusion
Within living labs, field trials help researchers study the extent to which innovations are being used by test users in a real-life environment. However, several authors have highlighted the difficulty in finding motivated and engaged (long-term) users (Ebbesson & Eriksson, 2013;Kaasinen et al., 2013;Schuurman & De Marez, 2009). This challenge can be problematic, because the setup of a field trial is very time consuming and expensive. Currently, the literature on user participation in field trials during living labs is scarce. In the research domain of eHealth, Eysenbach (2005) explained the law of attrition within field trials and the difference between non-usage attrition and dropout attrition. Although it is difficult to extrapolate these results to field trials in a living lab context, we used this Koen Vervoort manages and coordinates panels within living lab settings within one of the first living lab organizations in Europe: iMinds Living Labs in Belgium. He also represents iMinds within large Flemish and European living lab projects, hosts workshops, benchmarks internal processes within the entire living lab community (and beyond), oversees an internal quality survey, and organizes fieldwork for iMinds' flagship, Digimeter (digimeter.be), a report that tracks the ownership and use of media (technology) among the Flemish population.
framework to analyze attrition within living lab field trials. With this study, we conducted a qualitative analysis of open questions in post-trial surveys of three living lab field trials and an analysis of attrition data from project documents.
This research has some limitations, including for example, that dropout attrition occurred when the posttrial surveys were sent to the test users. Future research could elaborate on this aspect by exploring how to minimize the dropout attrition so that there is information about why test users dropped out. Future studies should also ask why people stop testing an innovation and how many people stop testing an innovation. Although data logging can be used to measure attrition during field trials, it does not help researchers understand why the users stopped. There are thus many opportunities within this domain for quantitative as well as qualitative research. Although the results of this research are exploratory and difficult to generalize to other field trials, we believe the results are valuable for other researchers, practitioners, and idea owners of new products and services to organize and follow-up field trials. Researchers can pro-actively take into account the factors that play a role in the attrition of test users during the preparations for these trials. The idea owners can also practically gain from these findings because some attrition factors relate directly to the innovation itself.
Within this exploratory research, we can conclude that non-usage attrition as well as dropout attrition occurs. Whereas dropout attrition is mainly linked to the research setup, non-usage attrition is mainly linked to the innovation itself. The factors that affect attrition differ for each field trial because of the differences in the innovation and design of the trial. In this study, the main factors why participants stopped testing is because of time restrictions, because they did not see the benefits of using the application, or the application did not address the user's need as well as intended. We also provided practical guidelines to help instigators and managers reduce attrition in their living lab field trials.
Here, the main outcome is that communication with test users plays an important role in minimizing dropout attrition, which in turn yields valuable information regarding non-usage attrition. Project instigators and managers should take care to recognize the factors that affect attrition and consider how they can predict future adoption behaviour.

Factors Affecting the Attrition of Test Users During Living Lab Field Trials
Annabel Georges, Dimitri Schuurman, and Koen Vervoort The Evolution of Intermediary Activities:

Introduction
Living labs are real-life experimentation environments in which new products and services are given shape through collaborative efforts of users and developers. They aim to extend co-design and open innovation activities from mere concept design and ideation to design-in-use, which is often requisite for co-realizing the true value points of new technologies and services (Botero & Hyysalo, 2013;Hartswood et al., 2002;Hillgren et al., 2011;Hyysalo, 2010;Voss et al., 2009).
The success of such real-life collaboration, which aims to promote learning between different stakeholders, hinges on how the co-design process has been orchestrated, facilitated, and managed. In discussions about living labs notions such as "quadruple helix" and "public-private-people partnerships" flag the issue promin-ently. However, research on collaboration dynamics in living labs remains nascent, and it seems that often the complex knowledge exchange tends to be taken for granted, overlooked, or simplified beyond what, for instance, the kind of guidance practitioners would benefit from the most.
This article on intermediation work in a living lab project is based on a longitudinal qualitative study of a four-year (2005-2009) living lab project that took place in four units of a large public nursing home in Finland.
The data allows us to describe and analyze how the user-side innovation intermediaries facilitated learning between developers and users during a long-term codesign project. We focus on the intermediation work done by three living lab project workers, whose educational background was in nursing and elderly care. After the four-year living lab project, the developer company hired the key project worker as a customer Innovation intermediaries play an important role in open innovation endeavours. In living lab projects, where different professional identities and organizational cultures are at play, intermediary actors facilitate learning between stakeholders and manage tensions and conflicts of interest. The current living lab literature recognizes the importance and multifacetedness of these actors, but does not shed light on the work they do at a more practical level. Our study seeks to capture the variety and evolution of work tasks of user-side innovation intermediaries during and after a four-year technology project in a living lab. The study explores how these mediating actors tackle the everyday challenges of a living lab project. This article is grounded on a longitudinal qualitative case study of a innovation process for a floor monitoring system for elderly care -the "smart floor".
It is hardly possible to overrate the value… of placing human beings in contact with persons dissimilar to themselves, and with modes of thought and action unlike those with which they are familiar. … Such communication has always been… one of the primary sources of progress.

Louna Hakkarainen and Sampsa Hyysalo
care specialist. This made it possible to extend the scope of our research to a total of eight years and to include the after-market launch period, when the locally tailored product was "generified" to serve a widening clientele (Hyysalo, 2010;Pollock & Williams, 2008).
In order to address the variety of intermediation work in the case, we have turned to research on innovation intermediaries. Innovation intermediaries have been central in social learning processes in technological innovation (Stewart & Hyysalo, 2008;Williams et al., 2005). In innovation studies, these mediating actors have been studied for some time. Howells (2006) describes an innovation intermediary as "[a]n organization or body [or an individual] that acts an agent or broker in any aspect of the innovation process between two or more parties".
For a long time, research around the topic focused on supply-side actors, such as industry associations and knowledge-intensive business services, but lately, work has been done to highlight the significance of innovation intermediaries in the user-side activities and processes of social learning: "The highly visible supply-side intermediaries […] and the easily identifiable middleground agencies […] tend to overshadow the often more informal yet just as crucial intermediaries at the user-end of the supply-use relation. Intermediate users, local experts and 'tailors' facilitate, configure and broker systems, usages and knowledge about systems and their deployments, helping users to domesticate them and suppliers to respond to actual, realised uses." (Stewart & Hyysalo, 2008). Our present study focuses on the role of public sector user-side innovation intermediaries in a collaborative innovation process.

Theoretical Framework
Our understanding of living labs relies on findings from science and technology studies -especially around social learning (Hyysalo, 2009;Williams et al., 2005) and domestication of technology (Berger et al., 2006;Silverstone et al., 1992;Sørensen, 1996).
The social learning in technological innovation approach (Williams et al., 2005) grew out of research on the social shaping of innovation (MacKenzie & Wajcman, 1999;Williams & Edge, 1996). The concept of social learning places particular emphasis on the activity of the users during the appropriation of new technology and highlights the importance of simultaneously studying processes of design, implementation, and use.
Social learning refers especially to two simultaneous, complementary, and intertwined processes: innofusion (Fleck, 1988) and domestication of technology (Sørensen, 1996). Innofusion (innovation that takes place during diffusion) refers to "processes of technological design, trial and exploration, in which user needs and requirements are discovered and incorporated in the course of the struggle to get the technology to work in useful ways, at the point of application" (Fleck, 1988). The concept of domestication has its origins in cultural consumption studies, and it refers to the work users go through in "fitting [technologies] into the preexisting heterogeneous network of machines, systems, routines and culture" (Sørensen, 1996).
From these perspectives, we see living labs as a codesign infrastructures in which users' creativity around technology use and their efforts to fit technology to cultural, organizational, and material contexts become resources for product development. However, the potential of this kind of collaboration does not realize automatically, which is why we focus on the crucial work done by innovation intermediaries in living lab networks. Stewart and Hyysalo (2008) define user-side innovation intermediaries as organizations or individuals that "attempt to configure the users, the context, the technology and the 'content', but they do not, and cannot define and control use or the technology". They are thus actors who seek to influence users and developers, but do not have final say over how the technology is eventually used (this is what users and managers at user organizations do) nor do they hold decision-making power, or necessary skills, to alter the form of the technology at the developer end.

Innovation intermediaries
In their seminal studies, Howells (2006) and Bessant and Rush (1995) have listed functions and bridging activities of innovation intermediaries (Box 1). Shortcomings of these kinds of listings are that they leave aside the common types of engagements that these actors are involved in during their "bridging activities". Stewart and Hyysalo (2008) have attempted to move from a mere ordered list of functions to an analytically ordered set of concepts that describe how intermediaries act and what are the different facets of their work in innovation. They have recognized three user-side innovation intermediary roles with respect to social learning: facilitating, configuring, and brokering. www.timreview.ca

Louna Hakkarainen and Sampsa Hyysalo
Facilitating means providing opportunities to other people, by educating, gathering and distributing resources, influencing regulations, developing the local rules, and creating "spaces" for others to act. Configuring means material and symbolic alteration of technology, adjusting its form and content (often in minor ways), as well as how it is interpreted and used. Brokering refers to the establishing, nurturing, adjusting, and altering of connections between different actors. This work on connections is not just neutral bridging, but is often selective and occasionally self-serving to the position of the intermediary actor itself.

Intermediation work in living labs
In recent years, living labs also have been analyzed as innovation intermediaries (e.g., Baltes & Gard, 2010;Katzy et al., 2013).  define living labs as " […] open innovation intermediaries that seek to mediate between users, research, public and private organisations, advance our concept of technology transfer by incorporating not only the user based experimentation, but also by engaging firms and public organisations in a process of learning and the creation of pre-commercial demand." Some attempts have been made to shed light on the interaction dynamics inside living labs on a more detailed level. Such research has focused on communities of practice and boundary objects (Johansson & Snis, 2011), living lab actors' roles and role patterns  Box 2), living lab networks' modes of coordination and participation , functions and roles of public open innovation intermediaries (Bakici et al., 2013), strategic capabilities of living labs (Katzy et al., 2013), paradoxical tensions in living labs , complexity in the stakeholder interactions (Pade-Khene et al., 2013), and possibilities of social and cognitive translation between stakeholders (Svensson & Ebbesson, 2010). Part of this work has been attempts to also identify the roles of intermediary actors in living labs ; see Box 2).
Although helpful in gaining a sense of what functions actors perform in collaborative innovation, empirically derived listings and classifications bear close similarity to previous empirically derived listings of innovation intermediaries such as those of Howells (2006) or Bessant and Rush (1995) (see Box 1). Gregor (2002) has characterized such listings as "naming theory", the most rudimentary form of theory within a research domain, a stepping stone on which more analytically ordered typologies and gradually more explanatory theory building can take place. One of the steps needed to move beyond naming and answering simple "what" questions is to conduct empirical studies that expose the situatedness and context-specific aspects of the innovation process and can shed light on "how" questions. This is important also for gaining practical sense of what works (Gregor, 2002;Woolrych et al., 2011) Thus, with regard to actor roles in living labs, further work is called for, particularly in two respects. First, there is a need to empirically gain better specificity in what kinds of engagements the roles relate to. The current lists of actor roles by Nyström and colleagues (2014) have been derived from multiple projects and multiple different actors and beg for further clarification, as do the contents of the different roles. Furthermore, only some of the roles are present in different projects and, at that, different phases of projects. Existing analysis of processes of intermediation in or by living labs address the systemic or organizational level, but fail to describe in detail how individuals tackle the challenges posed by everyday life in living labs.

Louna Hakkarainen and Sampsa Hyysalo
Second, although the more detailed empirical examination of roles and their prevalence in actual living lab projects is in order, the research on actor roles in living labs would also benefit from seeking to move beyond mere naming towards better understanding of the interrelations of different roles, as was done with innovation intermediaries previously (Stewart & Hyysalo, 2008). Our focus on living lab facilitators happens to reside within the broader notion of innovation intermediary, and hence we shall examine whether our previously developed typology of configuring, brokering, and facilitating would be fit for further organizing the findings in the present article.

Research Approach
Our work enriches the previous research by focusing on the innovation intermediaries' work on the level of tasks and activities. We map the evolution of the intermediation work during and after the living lab project, covering almost eight years' time on the biography of the maturing artefact.
The study continues an analysis started in licentiate study by Hakkarainen (2013) and continued during the follow-up phase of study Hyysalo & Hakkarainen, 2014). The living lab project workers documented nearly all the collaboration meetings held with different assemblies over the course of the four-year project. In addition to memos, the data included project reports, plans, and marketing material -altogether 151 different documents related to the development and use of the "smart floor", which we describe later in the article. The overall number of qualitative in-depth interviews is 21: 16 during the living lab project and five after it. Four of the latter interviews were conducted with the developer company's sales manager and customer care specialist (who was previously a living lab project worker), and one was conducted with the customer care specialist alone. The last interview was conducted after the both interviewees had quit working for the company.
The units of analysis are intermediary activities and tasks of the living lab project personnel. By task, we Box 2. Identified actor roles www.timreview.ca

Louna Hakkarainen and Sampsa Hyysalo
mean an organized set of actions that can be either a one-time effort or a repeated pattern in the practices of the mediating personnel -in any case, a set of actions that formed a mutually recognized whole by both the mediating personnel and their colleagues (Strauss, 1993).
The coded tasks were ordered chronologically and reorganized under higher-level activities. The result of the analysis were 31 different tasks, which were categorized under 13 activities. The results were organized in a matrix (see Table 4) that shows how the activities and tasks evolved over time in different phases of the innovation process.
The smart floor innovation process has been divided in four phases (Figure 1). The division is based on empirical work done by Van de Ven and colleagues (1999) on innovation journeys and by Pollock and Williams (2008) on biographies of artefacts as well as process dynamics observed in the study by Hakkarainen (2013). Each transition represents significant changes in the innovation network as well as in the smart floor artefact.
In the final step of the analysis, we structured the tasks according to facilitating, configuring, and brokering (Stewart & Hyysalo, 2008) to see if there are changes in the broader-level orientation of the intermediaries in the course of the innovation project.

Case Study: A Smart Floor System
The origins of smart floor system are in the Helsinki University of Technology (now Aalto University), where the motion-tracking technique behind it was discovered in the early 1990s. Years later, a group of re-searchers and students created the first version of the smart floor -a simple floor monitoring system -and a company was founded around it in 2005. The idea for creating a gerontechnological device originally came from the user side: a well networked, innovation-oriented nursing home manager became aware of the discovery and encouraged the engineers to advance the technique into a system for elderly care.
The technology was next developed in an enabler-driven living lab , which was established in 2006 as part of Helsinki Living Lab, an early member of the European Network of Living Labs. The lab focused on a large public nursing home. The publicsector actors were the initiators of the collaboration and were also responsible for applying funding and hiring of the project personnel that acted as innovation intermediaries. The nursing home manager later became the head of the innovation undertaking, wherein the smart floor was one of the four sub-projects. The main stakeholders of the project are presented in the Figure 2. The number of project workers varied between two and three fulltime workers in different stages of the project.
The smart floor system -the outcome of the collaboration -consists of a sensor foil, which is installed under the flooring material; a user interface, which is accessed on a computer situated in the office; and cell phones, which the nurses carry with them during their work shifts. The movements of the residents generate alerts, which the nurses receive through the cell phones. The system can inform the nurses about, for example, a situation where a frail elderly person is getting out of bed, entering or leaving the room, entering the toilet, or occupying the toilet for an unusually long time. The alarms are tailored individually to each person. The Evolution of Intermediary Activities: Broadening the Concept of Facilitation

Louna Hakkarainen and Sampsa Hyysalo
Setting the stage for co-design Technology development was not the purpose of the collaboration project from the beginning. The initial plan was to explore ways to efficiently utilize the smart floor technology in the everyday life of the nursing home. However, due to the immaturity of the product, the focus of the collaboration changed to technology development.
The project workers had background in care work and, during the first months of the project, they participated in regular care duties in the units. This meant that the project workers had a profound understanding of the users, their work practices, and the context of use. However, they were not familiar with formal co-design or participatory design methods.
The collaboration started officially with a workshop in which the intermediaries, developers, and care workers defined the first user requirements for the system. After this, the information exchanges took place mostly in regular meetings. The project workers could organize the collaboration as they saw best, and the goals and methods were reassessed regularly and adjusted to the needs of the project.
The project was formally divided in two sub-projects: the main purpose of the first part was to test the smart floor in two rooms and to develop it further, especially by fixing technical bugs and getting rid of false alarms, so that the second part, a larger-scale implementation, was possible. The project workers had significant responsibility in diagnosing and weeding out technical problems.
From the beginning, the engineers and the nursing home staff and management -project workers included -had strongly differing understandings about the maturity of the product and each other's roles in the collaboration. The company was in a hurry to launch their product, but from the users' perspective, the smart floor was not even ready for the test implementation. The client -as represented by nursing home staff and project workers -was frustrated with the functioning of the system and severity of its bugs; they saw the engineers as arrogant and indifferent to the welfare of the residents and nursing home staff. The developers, for their part, saw the users' requests as unreasonable and unrealistically scheduled. The goal of the company was to create a generic product instead of a tailored system, and they were sceptical about the representativeness of the client's demands.
Finally, the nursing home management and project workers refused to proceed with the implementation unless their demands were met. At the end of 2007, two out of three members of the living lab project staff -including the project manager and project co-ordinatorresigned, as did technology company's CEO, bringing the whole undertaking to the verge of collapse.
A summary of intermediary activities and tasks in the first phase is presented in Table 1.

Implementation and design-in-use
Changes in staff eased the tensions, and the collaboration continued, after the developers, two project workers (one newly hired), and management of the nursing home found common ground prior to the implementa- The hiring of a new project worker was pivotal for the new consensus. At this point, the project management had better understanding of the requirements of the intermediary position. This time, they were looking for an independent and innovative negotiator, someone who would be technology-oriented and able to change perspectives when needed. In a delicate situation, the project workers needed to convince different stakeholders of each other's good intentions, recognize shared interests, and react quickly to changing circumstances. Nevertheless, they had to be practical enough to push through the demanding implementation phase and support the care workers by taking part in the regular care duties.
The implementation phase invoked a new kind of division between the living lab project stakeholders: many of the end-users -the nursing home staff -reacted negatively to the smart floor. The nursing staff was unwilling to study new things alongside their normal workload or to change their work routines. Their job was demanding enough on its own. In addition, the nurses saw themselves as caregivers, not machinists, and were generally reserved about complex gerontechnological devices. Many care workers boycotted the project and the system, for example, by not carrying cell phones with them during their shift and continuing to work as they used to. Pushing forward with the rollout of the system required developers, project workers, and nursing home management to ally themselves against the care personnel, among who many were reluctant to put the system to use let alone participate in its improvement and to make the use of the system. Attendance at the feedback meetings was made obligatory for the nurses.
During the implementation, the strict discipline was counterbalanced by the devotion of the project workers, who were also care professionals by education. They spent time in the living lab units on a daily basis and helped the nurses in the implementation of the system, even occasionally assisting them with normal care duties. The weekly (later monthly) feedback meetings provided the care personnel an opportunity to speak out, comment on the system, and express new development ideas. The project workers and the nurses discussed how the system had been utilized, what its benefits were, and how it affected the care practices and the elderly people. This feedback was complemented by observing the smart floor's daily use, which the project workers valued as the most important way to collect information for the improvement of the system. Their background as care workers helped them to make sense of the daily work in the units, which was needed because the burden of developing the system further was placed on their shoulders. The project workers ob- The Evolution of Intermediary Activities: Broadening the Concept of Facilitation

Louna Hakkarainen and Sampsa Hyysalo
served use, identified problems and solutions with the engineers, and thought of ways to utilize different functionalities and properties of the system with the care personnel. Another important area was how the system should be used in order to produce optimal results: for example, how to determine the right mix of alarms for each resident, how the system affects elderly people in the long term, and what should be done when a nurse receives overlapping alarms. They also had to think about the challenges that the living lab project created, for example, what practical actions to take when the system does not work the way it is supposed to.
In addition, the project workers were active in planning, organizing, and executing effectiveness research of the impact of the smart floor on, for example, resident safety and nursing work. The work was done primarily for the client (the City of Helsinki), but the results were highly valuable for the company as well. Later in the project, the project workers were also active in showcasing the system and the project to numerous potential customers from all over the world.
A summary of intermediary activities and tasks in the design-in-use phase is presented in Table 2.
After the living lab project: Generification and broadening the clientele In the course of the living lab project, the startup company had merged with an established electronics company. When the living lab project was coming to an end, the company hired, as a customer care specialist, the key project worker -the one that had started in the middle of the project and who managed to turn the confrontation into fruitful cooperation.
After the market launch of the product, the clientele of the company grew, and new contextual problems arose, for example, in new buildings where the concrete was more humid and disrupted the normal functioning of the system. There were also minor differences in work practices at different institutions, which required some changes to the system.
From the onset, the company adopted a tailoring strategy, which meant that the system was customized to each customer organization's needs. After a while, this strategy was found to be unviable, and a more generic product was needed. Hence, the company sought to repackage its offering as a more standard product and servicing, where the customer care services, that previously were offered freely, were billed separately.
The customer care specialist organized user training and took care of the customer concerns, but she also continued to participate in the R&D activities by collecting user feedback, ideating improvements in the system, and networking with potential partners. She acted as a link between the customers and the company, and for this reason she had a very realistic understanding of the customers' reactions, concerns, and preferences. Her technical know-how, which had accumulated during the living lab project, allowed her to participate actively in the technical installation, testing, and problem solving in new client organizations. She also had credibility and the ability to consult management of the client organizations in renewing their care practices in order to get the biggest benefit out of the system.
Committing the client organizations to the use of the system remained as one of the biggest challenges for the company. The use of a complex system such as the smart floor can easily degenerate in new client organizations, because the end users and mid-level managers are usually not the ones making the purchasing decision.
The customer care specialist also participated in the marketing and sales negotiations. Because of a shared professional identity, she was able to ally herself with the client organization and even make some critical comments if the sales manager's pitch was too direct.
In 2013, the company was sold once more and the sales manager was laid off. At this point, the customer care specialist also decided to resign, because she was expected to assume the sales manager's responsibilities in addition to her existing responsibilities. By the start of 2016, the smart floor had become a stable product in the market and it has been installed in over 2000 apartments, mostly in northern Europe.
A summary of intermediary activities and tasks in the design-in-use phase is presented in Table 3.

Evolution of Intermediary Activities
The mapping of the responsibilities of the project personnel shows how intermediary activities and tasks are spread out through the course of the innovation process, and how they continue and change along with the project (see Table 4). Above all, it reveals the diversity of responsibilities undertaken by the intermediary actors.
The most intensive engagement took place at the implementation and design-in-use phase, during which the www.timreview.ca

Louna Hakkarainen and Sampsa Hyysalo
largest number of tasks were performed. The case history underscores, however, that despite fewer tasks in other phases, they are equally crucial for success: effective collaboration in the design-in-use phase requires great effort, and achieving the goal of a profitable, widely applicable technology after the living lab phase was equally crucial for the innovation projects' success.
With respect to our analysis considering facilitation, configuring, and brokering, we can see three patterns emerging: i) all three engagements are quite evenly distributed in the first part of the living lab project; ii) the design-in-use phase is dominated by facilitation and brokering; and iii) brokering played the most important role after the project.
The three types of engagement do indeed appear to characterize the tasks of living lab intermediariesnone of these more abstracted roles appear redundant or absent. They underscore how the common way to denote such people as living lab "facilitators" seems to be a misleading way to characterize what such people do as innovation intermediaries: this role comprises only one third of their engagements and is strongest only in the design-in-use phase of collaborative innovation in living lab. Without a longitudinal perspective that reaches beyond the design-in-use phase, the illusion of the centrality of facilitation would prevail in our data as well.

Conclusions
Our study shows that the nature of intermediation in living lab projects cannot be reduced to facilitation. Intermediation work in a living lab project consists of a range of tasks, including configuring of technology and use practices, brokering contacts and interactions between different actors, as well as facilitating their work, learning, and interactions. Furthermore, the content and form of intermediary work evolves in the course of successful living lab project. Altogether, we recognized the intermediaries participating in 13 different intermediary activities and 31 tasks. Engagements that are typically thought of as "facilitating" comprise only a third of what these mediating personnel need to handle and comprise the most common form of engagement only in the phase after implementation, when design-in-use efforts are most active.
Previous research has approached the topic of intermediation in living labs mostly through cross-case comparisons of multiple organizations participating in multiple projects and networks (e.g., Heikkinen et al., 2007;. Because of this approach, the granularity of the findings has remained coarse and has resulted in "naming theory" of identifying lists of "actor roles". Following Gregor's (2002) framework for theory development, this is the most rudimentary form of theory in a given area that merely answers "what" questions. In the present article, we have shown how moving to longitudinal in-depth case studies of particular projects conducted in living labs helps to reveal process descriptions and answer "how" questions: both how living lab projects are shaped over time and how actor roles play out. This approach offers a richer understanding of the tasks and actions of particular actors as well as how they evolve over the course of an innovation project, allowing us to further connect living lab actor roles to wider theoretical development within innovation studies on innovation intermediaries (Bessant & Rush, 1995;Howells, 2006;Stewart & Hyysalo, 2008), as well as in-depth process studies on innovation (e.g., Hyysalo, 2010;Van de Ven et al., 1999;Williams & Edge, 1996;Williams et al., 2005).
Considering the pivotal role that the intermediary actors play in open innovation processes, such as those using living labs, we are surprised how under-researched the topic is to date. Recent living lab research has actively focused on the network composition and different methods that are used in living labs, but we want to highlight the importance of focusing, in detail, on the active engagements between different stakeholder groups and between people and technology.
The complexity of the intermediary work also reveals important practical insights for living labs: in a real-life context with multiple stakeholders, the direction of the innovation and challenges the project has to face are very difficult to predict. Thus, the capability of intermediaries to adjust their role and actions to changing circumstances is essential. This view holds implications for the recruitment of employees to living lab projects and for the management of living lab activities. Intermediaries hired in a living lab project need to engage in technical configuration and substance issues of the user domain, and not only in the brokering and facilitating tasks. Our study also lends support to the findings by Nyström and colleagues (2014) regarding the need for role ambidexterity, temporality, and multiplicityan actor's capability to flexibly change, create, adjust, and adapt to roles with respect to the evolving network structure as well the ability to hold multiple roles at the same time.
www.timreview.ca TIM is a unique Master's program for innovative engineers that focuses on creating wealth at the early stages of company or opportunity life cycles. It is offered by Carleton University's Institute for Technology Entrepreneurship and Commercialization. The program provides benefits to aspiring entrepreneurs, employees seeking more senior leadership roles in their companies, and engineers building credentials and expertise for their next career move.