The Defining Characteristics of Urban Living Labs

The organization of supported and sustainable urban interventions is challenging, with multiple actors involved, fragmented decision-making powers, and multiple values at stake. Globally, urban living labs have become a fashionable phenomenon to tackle this challenge, fostering the development and implementation of innovation, experimentation, and knowledge in urban, real-life settings while emphasizing the important role of participation and co-creation. However, although urban living labs could in this way help cities to speed up the sustainable transition, urban living lab experts agree that, in order to truly succeed in these ambitious tasks, the way urban living labs are being shaped and steered needs further research. Yet, they also confirm the existing variation and opaqueness in the definition of the concept. This article contributes to conceptual clarity by developing an operationalized definition of urban living labs, which has been used to assess 90 sustainable urban innovation projects in the city of Amsterdam. The assessment shows that the majority of the projects that are labelled as living labs do not include one or more of the defining elements of a living lab. In particular, the defining co-creation and development activities were found to be absent in many of the projects. This article makes it possible to categorize alleged living lab projects and distill the “true” living labs from the many improperly labelled or unlabelled living labs, allowing more specific analyses and, ultimately, better targeted methodological recommendations for urban living labs.


Introduction
By 1969, before the term "sustainable development" was commonly known, the United States Congress had already emphasized the need to "create and maintain conditions that permit fulfilling the social, economic and other requirements of present and future generations" (U.S. Congress, 1969). It is apparent that the current systems of urban life and development do not succeed in achieving this sense of sustainability in cities, thereby demonstrating the need for urban innovation to contribute to the sustainability transition of cities (Joss, 2015;Wheeler & Beatley, 2015).
However, many sustainable urban innovations at present do not systemically find their way to the market, inhibiting their wider implementation and limiting their positive impact potential in the context of this urban sustainability transition (Ashuri & Durmus-Pedini, 2010;van Bueren & Broekhans, 2013). The living lab approach is believed to close this gap between production and actual market uptake by directly involving all end users and other urban stakeholders into the development of new products. This would ensure a match with the actual needs and aspirations of the users while taking into account the local and institutional contexts, cultures, and creativity potentials (ENoLL, 2006;Kresin, 2009;Lemke, 2009;Lesnikowski et al., 2016;van Bueren & De Jong, 2007). Furthermore, the new partnerships and the inclusion of new, multidisciplinary stakeholders in the development process is believed to provide new insights and offer new, more integrated solutions to the investigated problems, thereby advancing the generation of innovations (ENoLL, 2006;Vincent, 2016).
The organization of supported and sustainable urban interventions is challenging, with multiple actors involved, fragmented decision-making powers, and multiple values at stake. Globally, urban living labs have become a fashionable phenomenon to tackle this challenge, fostering the development and implementation of innovation, experimentation, and knowledge in urban, real-life settings while emphasizing the important role of participation and co-creation. However, although urban living labs could in this way help cities to speed up the sustainable transition, urban living lab experts agree that, in order to truly succeed in these ambitious tasks, the way urban living labs are being shaped and steered needs further research. Yet, they also confirm the existing variation and opaqueness in the definition of the concept. This article contributes to conceptual clarity by developing an operationalized definition of urban living labs, which has been used to assess 90 sustainable urban innovation projects in the city of Amsterdam. The assessment shows that the majority of the projects that are labelled as living labs do not include one or more of the defining elements of a living lab. In particular, the defining co-creation and development activities were found to be absent in many of the projects. This article makes it possible to categorize alleged living lab projects and distill the "true" living labs from the many improperly labelled or unlabelled living labs, allowing more specific analyses and, ultimately, better targeted methodological recommendations for urban living labs.
www.timreview.ca Given these alleged benefits, it is no surprise that, in recent years, the "urban living lab" has become a popular development approach on which hope is fixed to accelerate the generation and adoption of sustainable innovations in the urban system in the light of the urban sustainability transition. However, evaluative accounts of urban living labs in practice indicate that many of these experience difficulty in achieving the full benefits of this approach (Bulkeley & Castán Broto, 2013;Franz et al., 2015;Karré et al., 2015;. These authors point to a lack of methodological deepening of the living lab approach as a reason for such difficulties. Yet, in order to provide targeted methodological recommendations, it is necessary to take a closer and more precise look at the aim of urban living labs. Analysis of the (urban) living lab literature and the variety of existing urban living labs, for example those presented at the ENoLL website (openlivinglabs.eu), shows that there is no consensus on this issue. The way an urban living lab is defined and used is diffuse, with many different and often abstract definitions of the concept, leaving participants involved in urban living labs in the dark as to how the "living lab dimension" makes this project different from other urban innovation projects.
This article focuses on establishing a definition of urban living labs, which will show what can be expected from urban living labs and which will provide guidance to actors engaged in urban living labs or contemplating starting one. The definition presented in this article has been based on the existing living lab literature. Additionally, a scan of 90 Amsterdam-based sustainable urban innovation projects has provided insights on the representation of "true" urban living labs, fostering cocreative innovation among the more traditional sustainable urban innovation initiatives. We conclude by identifying some key challenges to the implementation of urban living labs.

Defining "Urban Living Lab"
The term "urban living lab" is to refer to a variety of local experimental projects of a participatory nature. It is often used interchangeably with the terms "testing ground", "hatchery", "incubator", "making space", "testbed", "hub", "city laboratory", "urban lab", or "field lab". Although there are many publications on living labs and urban living labs, even in these texts, this notion of the urban living lab has not been defined clearly. In literature discussing the "theory" of (urban) living labs, they have been explain ed as a methodology (Eriksson et al., 2005), as an environment (Ballon et al., 2005;Schaffers et al., 2007), as a system (CoreLabs, 2007), and as a governance approach (Bulkeley et al., 2016).
To explore the variety of definitions and understandings of the concept, we have analyzed articles on (urban) living labs published in this journal, the Technology Innovation Management Review (TIM Review) and other journals. The articles in the TIM Review are particularly relevant because this journal has played a considerable role in the transformative debate on the essence, role, and shape of urban living labs, with, in addition to separate articles on this topic, special issues attributed to this topic in 2012, 2013, 2015, 2016, and 2017. Appendix A presents an overview of the definitions used in these issues.
Almost all the articles we found on (urban) living labs, including those published in other journals, referred to the variation and opaqueness in the definition of the concept. Often, these articles, which are presented in Appendix B, adopt existing definitions, such as the one used by the European Network of Living Labs (ENoLL): "Living labs are defined as user-centered, open innovation ecosystems based on systematic user co-creation approach, integrating research and innovation processes in real life communities and settings" (ENoLL, 2016). These definitions aim to grasp the essence of living labs in one sentence, mentioning the main aspects and the relations between these aspects, often using complex concepts. Besides these single-sentence definitions, there are authors who provide a rich description of the phenomenon, but do not define it. Other authors distinguish different "types" of living labs (e.g., Leminen, 2013) or imply that living labs are environments offering a "multitude of different projects" (Tuija Hirvikoski, President of ENoLL, in an open discussion on February 13, 2017). Finally, there are authors that simply refer to an empirical example (e.g., Femenías & Hagbert, 2013) or skip the usual explanatory paragraph altogether and discuss "living labs" as an already accepted notion (e.g., Coorevits & Jacobs, 2017). We see only a few examples where authors operationalize this definition by formulating key principles, defining elements, or building blocks of living labs Ståhlbröst, 2012;Veeckman et al., 2013). Yet, gradually, in the current living lab literature, a stabilized conception of living labs with roughly similar characteristics has emerged.

Characteristics
As described in the subsections below, we have drawn on the living labs articles published in the TIM Review and in other journals and documents to capture the de-

Kris Steen and Ellen van Bueren
www.timreview.ca fining characteristics of urban living labs. In total, we identified nine characteristics in four dimensions: i) aim ii) activities, iii) participants, and iv) context. For each dimension, the defining characteristics of urban living labs are described in the subsections that follow, and the complete list of defining characteristics of urban living labs, presenting the full urban living lab definition, is summarized in Table 1.

Aim
The overall aim of living labs is to learn and experiment, by integrating processes of research and innovation (ASC, 2016;Bijsterveldt, 2016;ENoLL, 2016). The innovation aspect refers to the development of new products (i.e., an object, service, technology, application, process, or system) and to the discovery of new solutions to existing problems. Learning and experimenting (McCormick, 2016;Pallot & Pawar, 2012;Schaub, 2016;Vincent, 2016) refers to the production and exchange of knowledge among participants (ASC, 2016;Bijsterveldt, 2016;Friedrich et al., 2013;Heijden, 2016;McCormick, 2016;Pallot & Pawar, 2012). Thus, the aim is not only to learn from the experiences from the particular lab environment, but also to replicate the innovation elsewhere, in real life, or to further future innovation (ASC, 2016;Bijsterveldt, 2016;Franz et al., 2015;Juujärvi & Lund, 2016). It is this emphasis on formalized knowledge production -lessons that are formulated and that can be disseminated -that sets living labs apart from other policy experiments and niches of innovation (Evans & Karvonen, 2014).
"Urban" living labs distinguish themselves from living labs by unanimously displaying an explicit territorial focus on finding local sustainable solutions addressing wicked problems that tend to be global, such as climate change and energy transition. The use of cities or parts of cities as laboratories is well in line with the current emphasis on the city as the impactful governance level for economic development (e.g., Glaeser, 2011;Barber, 2013;Katz & Wagner, 2014) and for sustainable development (e.g., van Bueren et al., 2012;Bulkeley & Betsill, 2013). It also responds to calls for citizen empowerment (e.g., Saurugger, 2010;Fung, 2015).

Kris Steen and Ellen van Bueren Activities
There are a number of activities that are supposed to take place in living labs. Living labs should comprise the development of a product, which can be all sorts of artifacts, including process innovations, and not only, for example, testing or implementation (Budweg et al., 2011;Coenen et al., 2014;Feurstein et al., 2008;Gray et al., 2014;Leminen, 2015;Schaffers et al., 2007;Tang & Hämäläinen, 2012). A key element in this development process is co-creation (Bijsterveldt, 2016;ENoLL, 2016;Feurstein et al., 2008;Franz et al., 2015;Gómez-Barroso et al., 2009;Heijden, 2016;Schaub, 2016). The essence of a living lab is that the solution is sought together with the user, rather than just applying a fixed solution and involving the user only for testing. To qualify as cocreation, the targeted users need to be involved in the various development phases of the living lab process: not only should they be asked for their opinions, they should have decision-making power throughout the phases (Prahalad & Ramaswamy, 2004). Furthermore, the development process of living labs is iterative, which implies that, after being designed, the (prototype) product is used and evaluated by the stakeholders. The feedback and evaluation gathered from these steps are used to further develop and improve the product (Feurstein et al., 2008;Pallot & Pawar, 2012;Pierson & Lievens, 2005).

Participants
The literature further shows that the living lab is a systematic innovation approach in which all stakeholders -all actors who have a stake in the developed product and the process leading to it -participate directly in the development process. Stakeholder participation, including users, is an integral part of the development process in living labs (Feurstein et al., 2008).
The actors who need to participate in the living lab activities are, at a minimum: users (the end users of the final product that will be developed; in many cases, citizens), private actors (businesses, firms, companies), public actors (governments and public institutions), and knowledge institutes. The participation of knowledge institutes emphasizes the aim of formalized knowledge production (Krueger & Buckingham, 2009;Perry, 2006). As mentioned, these actors not only need to participate in these activities but also need to have power to influence the process (Prahalad & Ramaswamy, 2004). This power allows these actors, including the end users of the product, to be active partners in the innovation and development during the whole process, rather than just passive consumers and receivers and subjects of R&D activities (Almirall et al., 2012;Feurstein et al., 2008;Friedrich et al., 2013;Higgins & Klein, 2011;Leminen, 2015;Schaub, 2016).

Context
Authors on living labs unanimously agree that a defining characteristic of living labs is that they take place in real-life environments. The complexity and multi-contextuality that is connected to real-life environments is part of the challenge of living labs (Feurstein et al., 2008;Friedrich et al., 2013). Whether these real-life environments are limited to physical environments (Feurstein et al., 2008;Friedrich et al., 2013;Kieboom, 2013;Leminen, 2015;McCormick, 2016;Westerlund & Leminen, 2011) or also comprise virtual realities (Feurstein et al., 2008;Leminen, 2015) is contested. In this research, the criterion used is that the product central to the living lab is being developed and evaluated in its real-life use context. Given that this product can also be a virtual product, such as a digital data-gathering system, virtual realities can also form the context of living labs. More often, however, the living labs will take place in a physical place, such as a neighbourhood, city, or other area. The wide attention for urban living labs emphasizes the need or desire of involved stakeholders to capture the real-life context in all its complexity while assuming that such experiments, despite the highly uncontrolled conditions, nevertheless produce useful, transformative knowledge (Evans & Karvonen, 2013).
An overview of the defining living lab characteristics mentioned in the living lab definitions presented in the literature is provided in Figure 1. This figure shows a high level of agreement among the authors of the texts studied on the characteristics of living labs, leading to the synthesised definition presented in Table 1. In addition, Appendix B lists how authors have worded these living lab characteristics, which shows the original variety of concepts to which authors referred to convey these characteristics and demonstrating the need for operationalization of these concepts to be of guidance to stakeholders involved in urban living labs.

Living labs vs. urban living labs
The characteristics of living labs and urban living labs are very similar, and we assume that the general characteristics of living labs and recommendations for their design and operation are generally applicable to urban living labs as well. The difference between living labs and urban living labs is the explicit focus on finding solutions meant to increase urban sustainability. The inclusion of this explicit aim of increasing urban sustainability in the problem and goal statement of a living www.timreview.ca

Kris Steen and Ellen van Bueren
lab is a concrete feature of urban living labs -a feature that is often vaguely referred to by urban living lab authors as the "urban" dimension. This view is also consistent with the intuitive conception of scholars and practitioners that there is a difference between "normal" living labs and "urban" living labs in terms of increased complexity. The intricate number of variables and relationships influencing the process and outcome of urban living labs compared to "normal" living labs is exactly the learning environment that urban stakeholders tend to look for and appreciate. It helps them to develop real-world solutions to real-world problems while emphasizing the need for these solutions to work.
The use of living labs in cities may also raise questions about the inclusiveness and democratic accountability of the selection of areas for living labs, and about the focus of the innovation. Indeed, many innovations tend to be technological and are driven by ICT and decentralized technologies with ambitions to become "smart" and "low carbon". These are highly relevant questions, but they are beyond the focus of this article, which focuses on the living lab phenomenon itself.

Living Labs in Practice
Despite the popularity of urban living labs, stakeholders being engaged in living labs or wanting to start a living lab are struggling with what it actually is that they are supposed to do in an urban living lab. To answer this question, we have identified and quickly scanned the urban living labs in the Amsterdam metropolitan region. Given that urban living labs are not necessarily labelled as such, the identification of urban living labs in Amsterdam was not based on the projects calling themselves a living lab or an urban living lab, but on projects potentially being classified as an urban living lab based on their characteristics. This approach led to a sample of 90 sustainable urban innovation projects (for details, see  with attention for the citizens or end users in their project descriptions. The projects were identified through snowball sampling using a mixture of sources: policy documents; research papers; the Internet; and suggestions from experts working at the municipality of Amsterdam, Amsterdam SmartCity (a public organization aimed to collect and disseminate knowledge on smart city initiatives in the Amsterdam region), AMS (the Amsterdam Institute for Advanced Metropolitan Solutions), and other Amsterdam-oriented researchers. Based on the project descriptions found in online and offline sources, one researcher veri-fied which of the defining living lab characteristics (Table 1) were present in each of the 90 projects.
During this search, some notable observations were made. Of the nine characteristics of urban living labs, the characteristics of development taking place in the lab and of co-creation applying to the living lab activities were found to be the two urban living lab characteristics most frequently absent in sustainable urban innovation projects in Amsterdam. Of the 90 scanned projects, only 12 displayed these two characteristics. Of these 12 projects, all of them contained the remaining seven urban living lab characteristics (Table 1), thus qualifying as urban living labs. Surprisingly, most of these projects are not the ones calling themselves "living labs" or "labs". In fact, only three of the 14 projects with "living lab" in their name and two of the 14 projects with "lab" in their name met the identified defining characteristics of an urban living lab. The other seven projects from the 90 urban innovation projects in Amsterdam that were identified as urban living labs did not use any reference to "lab" of "living lab" in their name.
In this article, we hone in on the challenges experienced with these two urban living lab characteristics of "developing an innovation" and "co-creation", which have been identified as the two urban living lab characteristics most frequently absent in sustainable urban innovation projects.

Innovation challenges
The analysis of 90 sustainable urban innovation projects in Amsterdam shows that, besides development of an innovation, which is a criterion for urban living labs, a number of other innovation-related activities can be central in the projects.
From the literature on the innovation process -whether it concerns new product development (Cooper, 1988), agile development (Beck et al., 2001), or userdriven (open) innovation processes (Von Hippel, 2005; Mikkela, 2008) -we have identified five overarching phases in the innovation process and used them to categorize the projects: 1. Research: "An investigative process of revising current knowledge employed to reach understanding of a subject for the purpose of making decisions" (Cambridge Dictionary: dictionary.cambridge.org). www.timreview.ca

Kris Steen and Ellen van Bueren
2. Development: "The process of the product being created, growing, and changing in order to become more advanced" (Business Dictionary: businessdictionary.com).
3. Testing: "Application of a product for the purpose of observing and assessing its functionality" (Merriam-Webster: merriam-webster.com).
4. Implementation: Implementation of a finished product in a real-life environment.

5.
Commercialization of the developed product: Demonstrating a previously developed and implemented product.
The first four phases largely coincide with the natural stages of product maturation in the product development process (Cooper, 1998). The fifth phase is emphasized in the literature on user-driven, open innovation, and new product development, where the convincing of users who have not been involved in the development process needs more attention. It should be noted that this phasing is presented as a linear process. In reality, the sequence of these phases is not fixed, and processes take place in iteration.
Ideally, an innovation project would cover all these phases. In practice, we observed that each of the projects focussed on a particular phase of the innovation process (according to the pattern visualised in Figure   2). The projects aimed specifically at researching, developing, testing, implementing, or commercializing a solution. According to the living lab characteristics, only the projects (also) conducting development activities qualify as urban living labs.

User involvement challenges
Even though innovation in urban living labs theoretically should take place in co-creative processes in which participants can participate and influence decisions in all phases of the innovation process, this was rarely seen in the 90 investigated sustainable urban innovation projects in Amsterdam. When looking at the literature, we consider Sherry Arnstein's "ladder of citizen participation" (1969) the most usable elementary scale for measuring the degree of user involvement in the projects studied, with the three main levels of involvement being decision-making power, tokenism, and non-participation. As in the literature on co-creation (Prahalad & Ramaswamy, 2004), Arnstein associates true participation with the possession of (decision-making) power. The second level of participation identified by Arnstein is tokenism: the action of symbolically allowing people to "participate" without actually providing them with power in the decisionmaking process. The third main level distinguished is non-participation.
When projecting Arnstein's model onto the 90 potential living lab projects, we simply considered the first level as "participation" and the other two levels (tokenism

Kris Steen and Ellen van Bueren
and non-participation) as "no participation", because decision-making power by stakeholders, including users, has been shown to be conditional for living labs. Following this criterion, user participation only occurred in 51 of the 90 urban innovation projects, and, although this may sound as if co-creation takes place, user participation in the project activities is in fact quite common, if not inherent, when the project activities concerned are testing, implementation, and demonstration. However, development with the user (i.e., with the user participating and having decision-making power in the development process of the developed product) is the only form of user involvement that actually enables co-creation, which is one of the characteristics of living labs. The scan of the 90 sustainable urban innovation projects in Amsterdam found development with the user in only 12 of the projects.
When looking more closely at the sustainable urban innovation projects in Amsterdam associated with a living lab approach not including user participation, these projects do display a significant focus on user-related activities ( Figure 3). Two-thirds of these projects either clearly adopt a user perspective and specifically aim to deliver solutions for the end user rather than for public or industry partners, or they perform their project activities with the help of digital user data that the user actively or passively provides, for example through iBeacons, smartphone applications, or smart meters. The first form of this user focus -in which the user does not directly participate (i.e., there is no co-creation) but the user is included in other ways -can be referred to as "user oriented", following the definition of "user-oriented design" by Veryzer and de Mozota (2005). The second form can be referred to as "user sourced", which indicates that project activities are performed using virtual user data, which is actively or passively provided by the user.

Conclusions
Although urban living labs are widely implemented in urban contexts and are popular projects to allegedly catalyze sustainable urban innovation and the sustainable transition, their definition remains unclear, both in real life as well as in the literature. This article has set up a demarcating definition of the concept of urban living labs in order to allow researchers to further investigate this topic. After a literature review on the common denominators of living lab projects, a framework has been set up outlining the defining features of living labs in terms of goals, activities, participants, and context, making it possible to categorize and diversify alleged living lab projects and distill the real living labs from the many improperly labelled or unlabelled living labs, allowing more specific analyses and recommendations.  Her research and teaching focuses on the development of tangible concepts, tools, and principles for an integrated, area-centred approach to contemporary urban challenges. She is the author of scientific publications and a member of the editorial board of scientific journals in the field of policy management and urban development, and she teaches in various (inter)disciplinary educational settings for students and professionals.

Kris Steen and Ellen van Bueren
Based on a scan of 90 sustainable urban innovation projects and alleged living lab projects in Amsterdam, the majority of the projects that are labelled as living labs in fact do not include one or more of the defining elements of living labs, and sometimes are not even so much different from traditional top-down product development processes. However, given that many of the benefits attributed to living labs are linked to precisely to these defining characteristics, with co-creation and innovation as the most distinguishing ones, the absence of some of these basic components of living labs will also lead to disappointing performance in the field of these alleged benefits.
In order to develop an urban living lab design that does achieve the sustainable outcomes that the texts and definitions on living labs outline, the methodology of urban living labs should be more closely studied. For example, in this context, it would be valuable to extract the details of the methods recommended in the living lab literature and how they compare to the methods being used in practice, what the drivers and barriers are for the emergence and implementation of sustainable urban innovations, and which other methodological lessons can be learned from previous experiences with urban living labs. Reformulating these findings in the form of recommendations regarding the methodology, design, and governance of urban living labs would enhance the ability of urban living labs to achieve successful development, adoption, and replication of sustainable urban innovations. These aspects were beyond the scope of the current study, but could be addressed in future research.
In closing, we emphasize that today's popularity of living labs, also among subsidizing programmes preferring or demanding living labs as a research method, sometimes leads to a choice for a living lab approach without relating it to the project aims. With their specific co-creation requirement, living labs are certainly not applicable to every urban innovation project. Especially with regards to projects developing highly technological innovations, often pursued in the search for smart and sustainable cities, co-creative development, as is expected in living labs, can pose large challenges while not necessarily contributing to the quality of the solution. www.timreview.ca

Kris Steen and Ellen van Bueren
www.timreview.ca