If you look at history, innovation doesn't come just from giving people incentives; it comes from creating environments where their ideas can connect.
Science author & media theorist
Living Labs (LLs) are complex multi-stakeholder environments that enable real-life testing and experimentation of products, services, and systems. Despite increasing attention by practitioners as well as policymakers, and growing scholarly interest in the field, the literature exploring congruency between organisational objectives and outcomes when utilising LLs is still scarce. To fill this gap, a qualitative case study is employed to gain an in-depth understanding of objectives and project outcomes of organisations utilising LLs. The LL JOSEPHS® was chosen as this study's empirical context, in which 14 different projects were analysed. In-depth interviews revealed eight categories of measurable project outcomes: market acceptance, price acceptability, exposure, product testing, market intelligence, legitimisation, method testing, and networking. This study not only highlights what companies have achieved in comparison to their original project objectives, but also identifies additional unplanned outcomes that they accomplished. The findings offer important project-level insights into the potential and limitations of LLs. The results form a basis upon which to develop a better understanding of how innovation performance can be nurtured in LLs. Insights from the study may also help firms and facilitators by providing a deeper understanding of LLs at an individual project-level, and by articulating potential objectives and outcomes associated with organisations' involvement in LLs.
Living Labs (LLs) are complex multi-stakeholder environments that enable real-life testing and experimentation of products, services, and systems. Commonly viewed as a practical tool for pursuing innovation through co-creation, LLs have enjoyed increasing attention from scholars, policymakers, and practitioners. Despite a growing literature (Greve et al., 2020), the actual performance of LLs remains under-researched (Paskaleva & Cooper, 2021). Rudmark, Arnestrand, and Avital (2012)suggested that “understanding the key to co-creation success must draw on the motivations of the relevant stakeholders to engage in the process”. While practitioners and academics have discussed the benefits gained from co-creation, little is known about what motivates different stakeholders to participate in co-creating innovations (Pedrosa, 2009). Research tends to focus on understanding the motivation of users to engage in co-creative activities (Zwass, 2010; Ståhlbröst & Bergvall-Kåreborn, 2011; Roser et al., 2013; Roberts et al., 2014; Georges et al., 2015), however, the literature on objectives that organisations wish to address and achieve in LLs is scarce.
To guide firms and facilitators on how to utilise LLs, more knowledge is needed regarding company drivers for participating in innovative co-creation processes. Furthermore, an organisation’s objectives need to be compared against the results of engaging in a LL, in order to understand how far the co-creation project has achieved its purpose. Veeckman et al. (2013)recommended, simply, that “the innovation outcome must be considered”. Yet meanwhile, others have commented that, “the emerging LLs research fails to highlight innovation outcomes” (Leminen & Westerlund, 2015). As “value is always uniquely and phenomenologically determined by the beneficiary” (Vargo & Lusch, 2008), companies utilising the facilitation service of LLs, therefore, determine the value derived from it. While, the success of co-creation projects in LLs can be based on the congruence or discrepancy between planned objectives and outcomes (Gardner, 1977), the literature does not offer such insights. Following Paskaleva and Cooper (2021)who argue that “LLs could be evaluated by whether they deliver the benefits they set out to achieve”, this study addresses the following research questions:
- What are the project objectives of organisations that utilise a LL?
- What are the realised project outcomes of organisations that utilise a LL?
Following this introduction, we discuss the role of LLs, organisational objectives, and project outcomes, as well as the effectiveness of LLs. The third section outlines the case study approach, introduces the case JOSEPHS®, and defines this study’s data collection and analysis. The findings, in section four, highlight what companies have achieved in comparison to their original project objectives. The fifth section discusses the findings in light of extant literature and presents the study’s contributions to our understanding of LL projects. Finally, we identify the theoretical and practical contributions from this study, as well as limitations of the research.
The Role of Living Labs
LLs find application in many sectors. Their fields of application, as well as attention from policymakers and academics, have grown, particularly during the last decade (Paskaleva et al., 2015; Schuurman et al., 2015; Hossain et al., 2019; Greve et al., 2020, 2021; Paskaleva & Cooper, 2021). LLs are often described as bridging the gap between “open innovation” (Chesbrough, 2003)and “user innovation” (von Hippel, 2005).
LLs are discussed in the literature as performing multiple roles, while also being described as intermediaries, platforms, and networks. Almirall and Wareham (2011)claimed that LLs function as an intermediary between various stakeholders. LLs can perform a variety of activities in the innovation process in their intermediary capacity (Howells, 2006), and thus can also be labelled as agents, brokers, or marketplaces. Katzy et al. (2013)suggested a strategic position for these innovation intermediaries as facilitators with strategic innovation capabilities. Their study recognised matchmaking and innovation process design, management of collaborative projects, project valuation, and portfolio management as such strategic capabilities. For such an intermediary role to be performed effectively, Lapointe and Guimont (2015)remarked on the need for an organisational culture of openness and permeability, in regard to the external environment of companies. They also confirmed that stakeholders utilising LLs identify the need to be sensitised and supported in the development of open innovation know-how through intermediaries. Agogué, Yström, and Le Masson (2013)suggested that innovation intermediaries, such as LLs, can play a valuable role, even when the technologies, markets, and stakeholders are unidentified, and where there is a need for communal action beyond the sole company to discover new opportunities.
Users play a vital role in LLs as they contribute to the co-creation of new products, services, and systems. Extant literature commonly discusses the drivers of customers and users to participate in such activities (Antikainen et al., 2010; Roberts et al., 2014; Georges et al., 2015). However, LL literature focusing on the specific objectives that drive companies to utilise such environments is scarce.
Organisations utilising Living Labs
Organisations engage in co-creation projects as a way to understand their customers better, and, as a consequence, they can turn the insights they derive from this into innovation and a competitive advantage. For organisations utilising a LL, the process begins with setting project objectives. Bhalla (2014)identifies three categories that classify such objectives. Firstly, Generation refers to cases where the company’s objective is to obtain ideas, suggestions, or designs from customers and other stakeholders. Secondly, Refinement includes cases where collaborators work with the firm’s representatives to refine the features of a product or service. Thirdly, Creation refers to situations where both collaborators and a company’s professionals work together to develop a completely new product or service (Bhalla, 2014). Leminen and Westerlund (2012)point out that a LL serves as a platform for addressing both the shared goals of LLs and the goals of individual stakeholders. Existing literature documents the benefits associated with engaging LLs, but also some of the potential challenges and risks with co-creative activities.
Hoyer et al. (2010)recognise several positive co-creation outcomes, such as increased productivity and efficiency gains through cost-minimisation. Furthermore, a faster speed to market (Alam, 2002)and a closer fit with customer needs (Fang, 2008)can be achieved through co-creation. However, Hoyer et al. (2010)also acknowledged the costs and risks associated with co-creation. For example, companies may experience diminished control with regards to strategic management and business planning. In addition to decreasing control, the empowerment of consumers may lead to greater complexity in managing the company’s objectives, given the interests of diverse stakeholders involved in the co-creation process (Hoyer et al., 2010). Edwards-Schachter, Matti and Alcántara (2012)suggested that LLs help to recognise peoples’ needs, their preferences, as well as expectations for innovation opportunities using a specific methodology. Aside from identifying community needs, the findings also show that LLs are a beneficial instrument for improving local development and support, as well as integrating technological and social innovations in policies and local governance processes. In LLs, the knowledge emerging in experimentation phases often delivers unexpected insights, whereas more predictable knowledge is often produced in the co-creation and exploration phases (Lehmann et al., 2015). The same study also suggests that emerging knowledge might increase in complexity along the phase progression of a LL project, as stakeholders and users become more informed and experienced about the services they are co-developing (Lehmann et al., 2015). Magadley and Birdi (2009)offered insights into micro issues, such as creative outcomes, human–technology interaction, group dynamics, and facilitators. The findings suggested that innovation labs may positively influence creativity. This positive impact can be credited to the main conceptual ingredients of innovation-oriented facilities, such as a time and place to participate in creative thinking and the technology needed to facilitate such a process. Yet, the study stressed another important characteristic, which is human facilitation, or the impact of people. In spite of the potential positive outcomes associated with LLs, Grotenhuis (2017)highlighted that some LLs remain underutilised. Building on the experience of several LLs, the scholar emphasise
s the importance of better coordination between LLs, the companies, and ecosystems they serve to fully exploit LL potential. LLs can indeed offer many benefits to organisations, facilitating the provision of a wide variety of services, ranging from new R&D projects to a joint business development.
Effectiveness of Living Labs
Ballon, Van Hoed and Schuurman (2018)suggested that LL aims are manifold, as they “bring digital innovation processes and outcomes more in line with user preferences and practices, discover unexpected uses, identify potentially sound business and revenue models, stimulate cooperation between stakeholders, enable specific stakeholder groups to influence design features, increase acceptance, understand and tackle inhibiting factors, minimise failures, or study effects of introduction”. Supporting earlier findings (Schuurman et al., 2016), Ballon et al., however, stated that impact assessment of LLs remain anecdotal.
Lewis and Moultrie (2005)proposed a framework as the foundation for analysing the structure, infrastructure, benefits, and dis-benefits of innovation labs. Similarly, Magadley and Birdi (2009)assessed the effectiveness of an innovation lab as a new approach for endorsing creativity in companies. The study expanded on the research of Lewis and Moultrie (2005), not only by assessing an innovation lab by means of various research approaches, but also by viewing the phenomenon entirely from the user perspective. Veeckman et al. (2013)put forward five recommendations to achieve a successful implementation of projects. They suggested that a LL should establish: (i) a clear strategic intention, (ii) a minimum of shared value creation and sharing among all stakeholders, (iii) a minimum level of openness, (iv) a minimum set of users that establish strong communication, and (v) a mixed set of LL tools to discover new opportunities. Paskaleva and Cooper (2021) examine the effectiveness of LLs through a systematic review of extant literature. The study criticises that the benefits of using LLs are often only presented as leading to “innovation” and “development”. The scholars further critique the high-level, non-specific, nature of authors’ discussions about benefits that they claim occur from the use of LLs. Paskaleva and Cooper (2021) conclude that “outcomes from LLs are still poorly understood”.
Our study contributes to filling this research gap. While substantial efforts have been made to understand the motivations of users to engage in co-creative activities in LLs, a more fine-grained understanding of companies’ objectives to engage with LLs is required, to help more properly understand the effectiveness of LLs. Although, Bhalla (2014)identified three broad categories of objectives for companies to engage with LLs, which refer to the generation, refinement, and joint creation of ideas, the study does not provide a list of specific and measurable objectives that companies would like to address. To guide firms and facilitators on how to fully utilise LLs, more knowledge is needed regarding the companies’ specific objectives for participating in a co-creation process. Paskaleva and Cooper (2021) argue that “LLs could be evaluated by whether they deliver the benefits they set out to achieve”. While the success of co-creation projects in LLs can be based on the congruence or discrepancy between planned objectives and outcomes (Gardner, 1977), extant literature does not cover such insights.
Given the exploratory approach in this study and the research question, a qualitative case study was employed to gain an in-depth understanding of the outcomes of organisations using LLs (Yin, 2015). Only a limited number of studies discuss specific LL project objectives and outcomes; furthermore, LL projects are commonly studied across various empirical contexts. To eliminate potential biases due to the heterogeneity of LLs (Ballon et al., 2018), one LL was chosen for the empirical context of this study, facilitating a comparison across 14 different projects taking place in such an environment, each represented by a different company.
The case of JOSEPHS®
JOSEPHS® is a LL based in Nuremberg (Germany). It incorporates key LL features as defined by Westerlund and Leminen (2011). In line with their definition, JOSEPHS® offers a real-life context in which authentic use situations are created and studied. In this physical space, various stakeholders can contribute to the innovation process. JOSEPHS® has also received multiple awards for its innovation, as well as its research activities.
The 400 m² open setting of JOSEPHS® attracts co-creators through four different areas: the living lab, a think tank, Café, and the Gadget Shop. The LL area is where companies have their products or services tested by users. This open space is divided into five business islands, each occupied by a company for three months under one common theme. JOSEPHS® also has a “Think Tank”, which is often used to run university seminars, events with an external speaker, or lead user workshops for companies to further deepen their co-creation activities. JOSEPHS® in addition hosts an Italian Café. Positioned right at the entrance, the café attracts visitors without them necessarily knowing that JOSEPHS® has more to offer, which helps in lowering barriers to interaction. Finally, the smallest space within JOSEPHS® is occupied by Ultra Comix’s “Gadget Shop”. The shop offers gift ideas, such as board games and books.
JOSEPHS® projects can be described by reference to three key phases. First, a briefing takes place to clarify the JOSEPHS® concept and set realistic expectations for collaboration. One of the key objectives in this first phase, is to establish the project’s research design. A research question that the company would like to find answers to is articulated. Second, the three months test phase starts. The prototype is presented at JOSEPHS®, and facilitators encourage users to test it and provide feedback. User feedback is then presented back to the companies in order for them to review their prototypes and make possible adjustments throughout the testing phase. Third, qualitative and quantitative analyses are performed on the feedback collected throughout the three months. In accordance with the individual agreement, a report, a presentation, or both is presented back to the company. Feedback to companies comprises results as well as recommended actions.
Data Collection and Analysis
Semi-structured interviews with 14 individuals from various organisations were carried out between April and June 2017. The interviews were recorded and transcribed. An overview of the organisations interviewed is provided in Table 1.
Table 1. Study Participants
During the interviews, study participants were asked questions about their objectives to engage in a LL project, as well as the anticipated project outcomes, including planned and unexpected results. Based on grounded theory, the data was analysed with no preconceived hypothesis (Glaser & Strauss, 1967). In line with Glaser and Strauss (1967), this study employed the constant comparison method by following a non-linear process of coding, comparing, and memoing of data, along with identifying project objectives and outcomes. Through this iterative process, concepts that explained patterns in the data were developed. Our data analysis continued until theoretical saturation was reached (Glaser & Strauss, 1967) and no further categories of project objectives and outcomes were identified. Taking into consideration the challenges of measuring project success in LLs, we adopted a goal-based approach in examining the congruence or discrepancy between planned objectives and outcomes (Gardner, 1977) as an indicator for project success.
Project Objectives and Outcomes
For LLs to be effective and to be able to facilitate co-creation, first it is helpful to understand what motivates organisations to utilise such spaces. This study finds that companies engage in co-creation for different reasons, which mainly belong to two broad categories. On the one hand, companies wish to gain access to co-creators, and, on the other hand, they would like to gain access to the LL itself. Within these two areas, the interviews reveal seven different types of objectives involved in why companies engage in co-creation. Table 2provides an overview of project objectives against realised outcomes, distinguishing between planned and unplanned outcomes.
Table 2. Project objectives and outcomes
Symbols: X objective achieved; -- objective not achieved; O objective not achieved for company internal reasons; unplanned project outcomes
In total, we identified eight realised co-creation outcomes. Seven out of these eight correspond to the objectives driving companies to engage in co-creation at JOSEPHS®.
Companies stressed the importance to them of receiving feedback from a diverse range of co-creators. The feedback they seek relates to market readiness, price acceptability, exposure, product testing, and market intelligence. The only category not mentioned as a project objective by any company was “legitimisation”. For four companies, however, involving co-creators in the project enhanced their internal decision making. Therefore, legitimisation is an unplanned project outcome that companies did not anticipate or consider. Four companies that were interviewed pointed out the importance of accessing JOSEPHS® as a resource in itself. Companies would like to access JOSEPHS® as a way to test the suitability of the LL as an innovation method, or to access its wider ecosystem and to network with many different stakeholders.
Only HomeGrown has not achieved all its project objectives. Most companies, instead, achieved more project outcomes than what they had intended. All unplanned, and therefore, additional project outcomes were considered beneficial to the company, apart from CitizenChampions that received insights on market acceptance of their product but did not consider this feedback useful to them.
The project objective that was mentioned the most was market acceptance. Out of 14 companies, 13 stated that they are interested in understanding if their product or service is satisfying customer needs. HomeConcepts stated that, “we want to present our ideas and concepts, before they are fully finalised and are on the market” (HomeConcepts). The company wanted to examine customer perception of the product and identify “what is important for the user”.
The interviewee from SleekSoftwareSolutions stated that their “aim was to get as much user feedback as possible” because “we don’t have in our sector many research projects for the end customer [as] we offer mostly b2b solutions”. As the development of the app was already well on the way, the company implemented a continuous development cycle of one to two weeks, when the developers were posing questions that required further investigation through the co-creation project at JOSEPHS®. Simultaneously, they were incorporating feedback from the users at JOSEPHS® to update the app according to the suggestions received.
LearningLounge presented a new company website and posed questions related to navigating their main page as a way to test new features. In contrast to clickstream analytics, JOSEPHS® offered a more comprehensive platform to gather information about users experiences of navigating the LearningLounge website and their cognitive associations with different ideas and concepts.
Establishing contact with end-customers was stated to be of particular importance to SmartComfort, which operates in a B2B context: “We had in the past […] not much direct contact to the end-customer and can’t really accurately say how the end-customer […] perceives our products, how they assess it and what suggestions of improvement the customer may have” (SmartComfort interviewee). In the context of their specific products, SmartComfort stated, “We have realised that we need to get much closer to the end-customer”. Through JOSEPHS®, SmartComfort was able to test market acceptance of two technologies in comparison to one another, directly with end-customers.
All 13 companies achieved their original project objective with regard to understanding market acceptance of their product or service. Overall, four different kinds of results could be observed.
Firstly, five companies received completely new insights through their co-creation project at JOSEPHS®. For example, Technology Institute was confronted with “some uncomfortable questions” indicating where the ideas from the company differ from “what the customer actually wants”. While some of the feedback given was already known to the company, they also realised “that there are sometimes expectations or ideas - sometimes quite funny ideas, that we didn’t think of before and that motivated people on our side to think again about what direction we want to develop the product”.
In other cases, co-creators confirmed pre-existent assumptions from companies, or provided feedback that led companies to readjust their approach during the testing phase. While IT4Tomorrow Institute met their objectives, they also experienced challenges in receiving content-related feedback, as they were less interested in the product’s design aspects:
“We are interested in opinions regarding the content and not design. Many, many visitors said the box on the [product] is too big, it is way too big and way too heavy. […] Many still focused on the design and just after we told them, that we are already aware of it then they told us other content related feedback. It was really difficult to make people not think about the design, but about the content. That was tricky”.
For this reason, it was important for the company to reflect on interim feedback and thereby adjust their approach. Integral to the success of the project were the information guides who conveyed guidance to the co-creator, by steering them to aspects about which the company wished to receive feedback.
Finally, one company completely changed their business model as a result of their co-creation project at JOSEPHS®. HomeConcepts reflected on their experience at JOSEPHS® and stated that it “opened our eyes”. The interviewee explained that the project had a far-reaching impact on the overall offering: “We completely left our original thought [about] how to offer [a] technology supported [service] to our [clients]”.
Another objective for companies to engage with JOSEPHS® related to price acceptability. Five of the companies wanted to find out what customers are willing to pay for their products and services. The comment from HomeGrown was particularly direct in its meaning: “We wanted to find out especially what price range people are expecting. What are people willing to pay for the product?” HomeConcepts initially developed a concept that was very pricey and potential customers pointed out that they could not afford such a high price point. The company asked JOSEPHS® co-creators what they would be willing to pay for a specific service. To avoid overengineering a concept that would not be affordable for a mass market, HomeConcepts posed some questions to JOSEPHS® co-creators: “What is important for them? What can they imagine?”
YoungStar even “increased the price [of their product] by 200 Euro […]. In this case, [it had] just positive, and no negative effects”. While four companies met their objective, one of them also had to test aspects that could influence the price. Technology Institute wanted to understand if customers would prefer an entire device or an app. JOSEPHS® co-creators favoured an app: “We already had that idea, but we didn’t know how much people would be interested in that, but it became apparent that it […] is worth pursuing” (Technology Institute interviewee).
HomeGrown, on the other hand, did not meet its initial objectives and was unable to define a price range for their product. HomeGrown explained that the quality of the data was not satisfactory: “It was difficult because the quality of the data was not sufficient. For example, we have 20 questionnaires that state that the customers are willing to pay two euros for the device, which is of course not very useful for me”. Considering the complexity of the device, the suggested price is far below any reasonable assessment: “As a consequence, we intend to further test the price, maybe in a different setting”.
Although JOSEPHS® openness was beneficial to companies, staff also had to consider the specifics of a prototype and the suitability of co-creators to provide input. This could be considered in the data processing phase to allow for more differentiated feedback.
JOSEPHS® offers exposure to companies and their prototypes. While some companies may see this as a natural consequence of their engagement with JOSEPHS®, three companies we studied specifically identified exposure as one of their project objectives. This is different to the companies trying to examine market acceptance through co-creator feedback, as these companies were identifying exposure as their objective, and did not actively seek feedback from co-creators at JOSEPHS®. CitizenChampions stated that, “we didn’t really have an objective. We just wanted to introduce [the product]”. Similarly, RadicalRethink explained that, “my expectation was mainly the exhibition”. Furthermore, the latter interviewee explained that the objective was to exhibit the product to “the walk-ins, but also the people that have been invited by JOSEPHS®”. YoungStar, a start-up from the region, also articulated product exposure as an objective of their project: It was “somewhat also about showing the [product]”.
Different to the three companies, HomeConcepts and AmazingAccessories did not define “exposure” as their original project objective; however, both explained that this was nevertheless an unplanned project outcome. For example, AmazingAccessories explained that they “received good media coverage”. The local newspaper “Nürnberger Nachrichten published an article”, which the interviewee described as “a good side effect”. AmazingAccessories did not plan to achieve such exposure, but acknowledged the positive impact it had: “We were able to communicate it well locally that we are currently having an exhibition at JOSEPHS® and that was positive”. Similarly, HomeConcepts recognised that the project was “also beneficial for the [company] image. You are at JOSEPHS®; that raises awareness, [and] who knows what people go in and out there”.
Two companies explicitly used JOSEPHS® to test their products from a technical point of view. For example, an interviewee of SmartComfort stated, “My objective was to see how the installation of the two systems work out in general. […] Just the fact that something like that was installed on-site; to find out how smoothly it works”. The results of this product test were not only “used for the development of their own products but also to assess the products of [an Acquisition Company]”. Product testing for the Acquisition Company was an important consideration for the possible acquisition:
“At that point the ‘Acquisition Company’ was not part of SmartComfort. It was in the preliminary stage in [the] context of ongoing discussions, so that one could also test the format of JOSEPHS® to find out where does this company stand, what can they do, what can’t they do, to strengthen our assessment, which was very valuable”.
HomeGrown also stated that testing the device and its functionality at JOSEPHS® was one of their project objectives. However, it did not achieve their objective due to internal reasons. HomeGrown intended to conduct product testing with their device: “Originally, we wanted to observe how the device copes for one and a half months without supervision”.
One company, YoungStar’s objective is to collect information that can be used in defining market opportunities, market penetration, or market development. Their objective is to gather market intelligence: “To know where customers are from helps us with the decision where we want to open a shop. Where can we expect good returns?” To understand where the customers are from, YoungStar offered “some coupon codes on a blanket, worth 10 euros”. As a result of this initiative, the company had co-creators redeem their coupon: “We could see who used them and know that we benefited financially from the project” (YoungStar). Furthermore, the company could also draw a conclusion from the initiative in terms of the location of their customers.
“Legitimisation” was the only category of project outcomes that was not mentioned as an initial objective to engage in co-creation at JOSEPHS®. Yet, four companies explained that the involvement of co-creators in the project legitimised their actions and endorsed decision making internally. Additionally, it supported communication external to the company.
For example, MyMoney identified that the feedback from JOSEPHS® co-creators provided more legitimacy internally to the company: “We have clear user-feedback and this user-feedback is taken more seriously than the feedback of our family and friends. And our board completely agreed”. SmartComfort similarly used co-creator insights from JOSEPHS®: “I also used it internally, not only to raise awareness for JOSEPHS®, but also used the results to bring on certain decisions. […] We discussed it with the board, because it is quite rare that we do these sorts of activities”. External to the company, IT4Tomorrow Institute acknowledged that, it “helps us when we talk to our clients and producers. We can tell them that we did end-user polling and we know that you can produce this in price range”. Similarly, based on the feedback from co-creators, AmazingAccessories stated, “Through JOSEPHS®, you get rid of your gut feeling and get a rational profound sample size, that you can rely on and that you are able to work with. You no longer have to act blindly, because you know, okay, I now have the numbers to confirm this”. Involving co-creators at JOSEPHS® also legitimised AmazingAccessories’ actions: “When I tell the distributors that we went to Fraunhofer and tested it over three months in a LL and that we have a solid base of results, that is of course completely different than if we say, we tested it on one colleague and he said it is this way and that’s now how it is. That has a completely different weight when you have actual data behind it”.
Two of the interviewed companies stressed their interest in testing JOSEPHS® as a method for co-creation. SmartComfort explained that they wanted to understand the following,
“How does such a probe work with JOSEPHS®? How many people come? How many people participate? How does the supervision work on-site? How much do you have to directly engage in the supervision and evaluation as a company and how much does JOSEPHS® do? I would say also [it is] a test of the service of JOSEPHS®, because for us it is obvious, that we want to use those kinds of format more often in the future and for that you have to start somewhere.”.
Innovation4Society explained, “We gained some interesting methodological insights that we will make use of in the future”. Also, SmartComfort stated, “We met our internal objectives, and we also were able to meet our hidden objectives”. The latter refers to the method testing of JOSEPHS® as a LL, which SmartComfort did not openly communicate to JOSEPHS® staff. The interviewee described their experience as “a very smooth cooperation, that was implemented well. One never had the feeling of being left alone, because we received information proactively, which we could use. We [will] consider repeating it [this experience] for different products”. SmartComfort was satisfied with using JOSEPHS® as a method and would use the LL again for future projects.
Although, only two companies explicitly articulated method testing as an objective, all companies that utilised JOSEPHS® are likely to have reflected on their experience and whether JOSEPHS® met their expectations or not. On a scale from 1 (not successful) to 7 (extremely successful), all companies except CitizenChampions (1), and P (4) rated their project success as 6 or 7. Therefore, we conclude it is likely that they were also satisfied with JOSEPHS® as a method. Considering the variety of companies that utilised JOSEPHS®, their satisfaction provides evidence of the LL's adaptiveness.
Establishing new contacts through JOSEPHS’® wider network of stakeholders was a stated objective for two companies. AmazingAccessories and Imagine Institute explicitly aimed to expand their network. Furthermore, Technology Institute, LearningLounge, and RadicalRethink also benefitted from networking opportunities, yet they did not define this as one of their original project objectives.
AmazingAccessories was hoping “that maybe one or two distributors might come by, see it and buy a few for their shops”. Imagine Institute met their objective to expand its network: “Through the feedback new contacts were made”. The interviewee stated, “There was one project on smart school gear and then there was one evening at JOSEPHS®, where some people from schools and the education industry were there. And there was also a school backpack manufacturer there and we were able to connect. So customer engagement at JOSEPHS® held true”.
Regarding unplanned project outcomes, three additional companies reported that they benefitted from the networking opportunities that arose as a result of their co-creation project at JOSEPHS®. LearningLounge was able to expand its network by establishing contacts with JOSEPHS® staff “and, of course, behind that [was] also the Fraunhofer Institute”. An employee from LearningLounge “facilitated workshops here, and he attended as a guest the re-launch [of JOSEPHS®] as well”. As a result of the co-creation project, Technology Institute received “two additional but really interesting enquiries that came through the JOSEPHS® exhibition”. Aside from the originally defined objectives, RadicalRethink also benefitted from the project through events and networking at JOSEPHS®: “I got to know JOSEPHS® and I was able to listen to other presentations that were really interesting, and also visit a[nother] project”. Those examples also stressed the value of connectedness that JOSEPHS® has to offer to the stakeholders that engage in their facilities.
This study aimed to investigate the objectives and the congruence with realised project outcomes of 14 companies that utilised a LL. Interviews with companies revealed that they have difficulties in quantifying the success of a project. MyMoney, for instance, said that “one can’t evaluate it on one figure alone, because there are too many factors that one has to consider, and that can’t be expressed in a number”. Similarly, LearningLounge stated that, “coming up with a number is very, very, very difficult”. Indeed, companies emphasised that their success is expressed through the attainment of their often-qualitative goals. Taking into consideration the challenges of measuring project success in LLs, our research examined the congruence or discrepancy between planned objectives and outcomes (Gardner, 1977)as an indicator for project success. From a practical perspective, comparing project objectives to realised outcomes also allowed companies to learn from their experiences, and thereby adjust their actions and expectations for future co-creation projects.
Supplementary to previous studies, this research provides a list of specific objectives, which can be measured and are associated to particular organisational activities and functions. LL literature in this area is, indeed, particularly scarce. Our paper expands the framework from Bhalla (2014) by adding more specific co-creation objectives to the three high-level ones identified.
Some of the objectives discussed in the paper have already been identified by existing studies (see Table 3). Market acceptance (Ponce De Leon et al., 2006; Hsiao & Yang, 2010; Buhl et al., 2017)and networking (Niitamo et al., 2006; Juujärvi & Pesso, 2013), for example, are mentioned in the literature, but only as assumed co-creation objectives, and without them being really explored. Product testing is also identified as an objective in the work of Schumacher and Feurstein (2007). The authors state that LLs carry out product tests with users prior to the final launch of new products and services. In the context of this research, however, the value of product testing was recognised also when done earlier in the development phase.
Results from this research, therefore, expand current knowledge about the objectives firms have to carry out co-creation in LLs, which is done in two ways. Firstly, we provided a list of measurable objectives, associated with access to the LL itself or its co-creators. Secondly, while some of the objectives identified in this paper are partially acknowledged in the literature, the empirical evidence gathered allows for deeper discussion and understanding, thereby adding to current knowledge about co-creation in LLs.
This research makes important contributions to knowledge about co-creation outcomes in LLs. In addition to identifying specific project outcomes and highlighting what companies achieved in comparison to their original project objectives, the research discusses additional unplanned outcomes that companies accomplished. In this study, seven out of the eight categories of co-creation outcomes were found to be consistent with categories in the project objectives; this section referred to them as “planned outcomes”. In addition to planned outcomes, we also identified one new category of unplanned project outcomes: Legitimisation.
Moreover, this research recognised that companies not only seek input from co-creators, but also from JOSEPHS® itself. Although the current literature acknowledges the involvement of multiple stakeholders in the co-creation process (Schaffers & Kulkki, 2007; Feuerstein et al., 2008; Almirall & Wareham, 2011; Westerlund & Leminen, 2011), the focus predominantly lies on co-creators, their ideas, suggestions, and feedback, which ultimately produces value to the companies. Thus, these findings expand on the current literature, which predominantly stresses the value user feedback generates for companies (Dutilleul et al., 2010; Nyström et al., 2014), by accentuating the value that the LL, itself, can deliver to firms as beneficiaries of LL services. The project objectives and outcomes identified in this study are summarised in Table 3and compared against existing studies.
Table 3. Summary of Project Objectives and Contributions from this Research
This study highlighted the innovation potential of LLs through three particularly interesting observations.
Firstly, a variety of objectives drive companies to engage in LL projects (see Table 2) and companies tend to pursue a multiplicity of them simultaneously. Further, more than half of the companies examined in this study achieved more outcomes than what they had anticipated. Indeed, some companies obtained user insights and answers to questions that they had not posed or even considered, which underpins the innovation potential of LLs. One possible explanation for achieving unplanned and unexpected project outcomes is the open enquiry process and role played by facilitators in eliciting feedback from users.
Secondly, this study also highlighted that while most companies do not identify “legitimisation” as an initial project objective, almost one third of the companies studied recognise it as an unplanned outcome. The findings revealed that companies value testing with users as it reinforces and supports their communication with internal and external stakeholders.
Thirdly, while extant literature has discussed several benefits derived through the involvement of customers and other stakeholders such as universities and suppliers in the innovation process, this study also recognised the value that LL facilitators add to the process. For example, companies wanted to engage with JOSEPHS® also because of their experience with LL projects. Since the opening of JOSEPHS®, LL staff have been able to acquire considerable tacit knowledge there over time that is not easily accessible to others. Yet, through the interaction and guidance of JOSEPHS® staff, companies were able to tap into the tacit knowledge gained in the LL and make use of it in the context of their own project.
All of these reasons might lead organisations using LLs to achieve unexpected results that go beyond their original objectives. Despite the mismatch between some companies’ project objectives and outcomes, the innovation potential of LLs seems evident as the projects we studied generated new insights, verified existing assumptions, provided contextual information, or delivered completely unplanned insights triggering a virtuous learning process.
Overall, we identified seven categories of co-creation objectives in this study, four of which have not been addressed in the LL literature: price acceptability, exposure, market intelligence, and method testing. Furthermore, this study also recognised that companies not only derive value from the involvement of co-creators in the innovation process, but also through interaction with LL facilitators. Furthermore, this study also reported eight categories of co-creation outcomes, both planned and unplanned, that companies achieved through their LL-based project. Extant literature has not provided insights about the measurable outcomes of co-creation projects in LLs, thus making this a novel contribution.
In addition to the theoretical contributions, this study has also made several contributions to practice. This research will be of particular interest to managers, LL facilitators, and policymakers. An understanding of various objectives that can drive involvement with LLs is fundamental for managers to fully understand the potential associated with co-creation activities. Co-creation offers companies and their network of stakeholders important opportunities for innovation, as each stakeholder provides access to new resources. The interaction process between stakeholders, therefore, can provide them with opportunities to facilitate value creation for and with each other (Grönroos, 2008).
The research findings show that this is relevant for companies across industries, irrespective of their size, who want to utilise a LL as effectively as possible. This study provides project-level insights that can support companies’ innovation endeavours and highlights the potential, as well as the limitations, of LLs. The examples provided throughout the paper, even when anecdotal, can trigger reflection from managers about the applicability of LLs to various contexts, as well as the suitability of LL methods to achieve different purposes.
This study is also of importance to LL facilitators. As the core service of a LL is to facilitate co-creation by acting as an interface between multiple stakeholders (Mulder & Stappers, 2009), it is important for a LL's staff to understand the objectives of various stakeholders involved in the process. To encourage companies to engage in the co-creation process, it is important to understand what they expect from co-creation (Füller, 2010). Therefore, it is an essential prerequisite to first investigate what drives organisations before a facilitator can help develop the capacity to address their aims. This study identified seven reasons why companies engage in co-creation at JOSEPHS®, which helps outline how the processes can be facilitated. Furthermore, the planned and unplanned project outcomes, identified through this study, can also inform companies about LLs’ communication strategies, so that they can articulate their value proposition more clearly, to help set and manage expectations consistently.
Finally, this study is highly relevant for public body stakeholders. As policymakers and local governments support LL activities by providing financial and legislative resources, as well as geographical space (Katzy, 2012; König & Evans, 2013; Karvonen et al., 2014), the findings of this study offer important insights in explaining how innovation performance can be nurtured in LLs. This study can help policymakers to better understand what works in practice, and what kind of policy environment is needed in order to support regional and national innovation efforts more effectively. For example, based on this study’s insights, public authorities can establish conditions to enable more companies to engage in LLs. Public funding can be made available for companies that utilise a LL in accordance with specific requirements.
Despite the study’s contribution, some areas call for additional consideration. Firstly, given the exploratory character of the research, the identified objectives and outcomes can be tested in a larger and different sample of organisations and with respect to involving various other LLs. Furthermore, from a methodological perspective, employing a longitudinal perspective would allow researchers to map changes in the way organisations engage with LLs, including the ability to define and measure their objectives.
Agogué, M., Yström, A., & Le Masson, P. 2013. Rethinking the Role of Intermediaries as an Architect of Collective Exploration and Creation of Knowledge in Open Innovation. International Journal of Innovation Management, 17(2): 1-24. DOI: http://dx.doi.org/10.1142/S1363919613500072
Alam, I. 2002. An Exploratory Investigation of User Involvement in New Service Development. Journal of the Academy of Marketing Science, 30(3): 250-261. DOI: http://dx.doi.org/10.1177/0092070302303006
Almirall, E., & Wareham, J. 2011. Living Labs: arbiters of mid- and ground-level innovation. Technology Analysis and Strategic Management, 23(1): 87-102. DOI: https://doi.org/10.1080/09537325.2011.537110
Antikainen, M., Mäkipää, M., & Ahonen, M. 2010. Motivating and Supporting Collaboration in Open Innovation. European Journal of Innovation Management, 13(1): 100-119. DOI: http://dx.doi.org/10.1108/14601061011013258
Ballon, P., Van Hoed, M., & Schuurman, D. 2018. The Effectiveness of Involving Users in Digital Innovation: Measuring the impact of living labs. Telematics and Informatics, 35(5): 1201-1214. DOI: 10.1016/j.tele.2018.02.003. DOI: http://dx.doi.org/10.1016/j.tele.2018.02.003
Bhalla, G. 2014. How to Plan and Manage a Project to Co-create Value with Stakeholders. Strategy & Leadership, 42(2): 19-25. DOI: http://dx.doi.org/10.1108/SL-01-2014-0006
Buhl, J. et al. 2017. Rebound Effects in Living Labs: Opportunities for monitoring and mitigating re-spending and time use effects in user integrated innovation design. Journal of Cleaner Production, 151: 592-602. DOI: http://dx.doi.org/10.1016/j.jclepro.2017.03.001
Chesbrough, H. 2003. Open Innovation: The new imperative for creating and profiting from technology. Boston, Massachusetts: Harvard Business School Press.
Dutilleul, B., Birrer, F.A.J., & Mensink, W. 2010. Unpacking European Living Labs: Analysing innovation’s social dimensions. Central European Journal of Public Policy, 4(1): 60-85. DOI: https://doaj.org/article/bb52e65bd2a84a57b531ac5a3ebdb949
Edwards-Schachter, M.E., Matti, C.E., & Alcántara, E. 2012. Fostering Quality of Life through Social Innovation: A living lab methodology study case. Review of Policy Research, 29(6): 672-692. DOI: http://dx.doi.org/10.1111/j.1541-1338.2012.00588.x
Fang, E. 2008. Trade-off between New Product Customer Participation and the Innovativeness and Speed to Market. Journal of Marketing, 72(4): 90-104. DOI: http://dx.doi.org/10.1509/jmkg.72.4.90
Feuerstein, K. et al. 2008. Living Labs: A new development strategy. In Schumacher, J. (ed.) European living labs: A new approach for human centric regional innovation. Berlin, Germany: Wissenschaftlicher Verlag, 1-14.
Gardner, D.E. 1977. Five Evaluation Frameworks: Implications for decision making in higher education. The Journal of Higher Education, 48(5): 571-593. DOI: https://doi.org/10.2307/1981598
Georges, A. et al. 2015. User Engagement in Living Lab Field Trials. Info, 17(4): 26-39. DOI: http://dx.doi.org/10.1108/info-01-2015-0011
Greve, K. et al. 2020. Unveiling the Diversity of Scholarly Debate on Living Labs: A bibliometric approach. International Journal of Innovation Management. DOI: http://dx.doi.org/10.1142/S1363919620400034
Greve, K. et al. 2021. Living Labs: From niche to mainstream innovation management. Sustainability (Switzerland), 13(2): 1-24. DOI: http://dx.doi.org/10.3390/su13020791
Grotenhuis, F.D.J. 2017. Living Labs as Service Providers: From proliferation to coordination. Global Business & Organizational Excellence, 36(4): 52-57. DOI: http://dx.doi.org/10.1002/joe.21790
Von Hippel, E. 2005. Democratizing Innovation. Cambridge, MA: MIT Press.
Hossain, M., Leminen, S., & Westerlund, M. 2019. A Systematic Review of Living Lab Literature. Journal of Cleaner Production, 213: 976-988. DOI: 10.1016/j.jclepro.2018.12.257.
Howells, J. 2006. Intermediation and the Role of Intermediaries in Innovation. Research policy, 35(5): 715-728. DOI: https://doi.org/10.1016/j.respol.2006.03.005
Hoyer, W.D. et al. 2010. Consumer Cocreation in New Product Development. Journal of Service Research, 13(3): 283-296. DOI: http://dx.doi.org/10.1177/1094670510375604
Hsiao, S.L., & Yang, H.L. 2010. A Service Experience Engineering (SEE) Method for Developing New Services. International Journal of Management, 27(3): 437-447.
Juujärvi, S., & Pesso, K. 2013. Actor Roles in an Urban Living Lab: What can we learn from Suurpelto, Finland? Technology Innovation Management Review, 3(11): 22-27. DOI: http://doi.org/10.22215/timreview/742
Karvonen, A., Evans, J., & Van Heur, B. 2014. The Politics of Urban Experiments: Radical change or business as usual? In Marvin, S. and Hodson, M. (eds) After Sustainable Cities? London, UK: Routledge.
Katzy, B. et al. 2013. Innovation Intermediaries: A process view on open innovation coordination. Technology Analysis & Strategic Management, 25(3): 295-309. DOI: http://dx.doi.org/10.1080/09537325.2013.764982
Katzy, B.R. 2012. Designing Viable Business Models for Living Labs. Technology Innovation Management Review, 2(9): 19-24. DOI: http://doi.org/10.22215/timreview/604
König, A., & Evans, J. 2013. Experimenting for Sustainable Development? Living laboratories, social learning, and the role of the university, in König, A. (ed.) Regenerative sustainable development of universities and cities. Cheltenham, UK: Edward Elgar Publishing: 1-13.
Lapointe, D., & Guimont, D. 2015. Open Innovation Practices Adopted by Private Stakeholders: Perspectives for living labs. Info, 17(4): 67-80. DOI: http://dx.doi.org/10.1108/info-01-2015-0003
Lehmann, V. et al. 2015. Living Lab as knowledge system: An actual approach for managing urban service projects? Journal of Knowledge Management, 19(5): 1087-1107.
Leminen, S., & Westerlund, M. 2012. Towards Innovation in Living Labs Networks. International Journal of Product Development, 17(1/2): 43. DOI: http://dx.doi.org/10.1504/IJPD.2012.051161
Leminen, S., & Westerlund, M. 2015 Incremental and Radical Service Innovation in Living Labs. In Al., D.M. et (ed.), Economics: Concepts, methodologies, tools, & applications. Information Resources Management Association, 445-459. DOI: https://doi.org/10.4018/978-1-4666-8468-3.ch0025
Lewis, M., & Moultrie, J. 2005. The Organizational Innovation Laboratory. Creativity & Innovation Management, 14(1): 73-83. DOI: http://dx.doi.org/10.1111/j.1467-8691.2005.00327.x
Magadley, W., & Birdi, K. 2009. Innovation Labs: An Examination into the Use of Physical Spaces to Enhance Organizational Creativity. Creativity & Innovation Management, 18(4): 315-325. DOI: http://dx.doi.org/10.1111/j.1467-8691.2009.00540.x
Mulder, I., & Stappers, P.J. 2009. Co-creating in Practice: Results and challenges. In Proceedings of the 2009 IEEE International Technology Management Conference (ICE): 1-8.
Niitamo, V.P. et al. 2006. State-of-the-art & Good Practice in the Field of Living Labs. In Proceedings of the Technology Management Conference (ICE), 2006 IEEE International: 1-8. DOI: https://doi.org/10.1109/ICE.2006.7477081
Nyström, A.G. et al. 2014. Actor Roles and Role Patterns Influencing Innovation in Living Labs. Industrial Marketing Management, 43(3): 483-495.
Paskaleva, D.K., & Cooper, D.I. 2021. Are Living Labs Effective? Exploring the evidence. Technovation. Elsevier Ltd, 106(April): 102311. DOI: 10.1016/j.technovation.2021.102311.
Paskaleva, K. et al. 2015. Smart City Stakeholder Engagement: making Living Labs work. In Bolívar, M. (Ed.), Transforming City Governments for Successful Smart Cities. Springer: 115-145.
Pedrosa, A. 2009. Motivating Stakeholders for Co-Created Innovation. Technology Innovation Management Review, December. DOI: http://timreview.ca/article/311
Ponce De Leon, M. et al. 2006. Creating a Distributed Mobile Networking Testbed Environment - Through the living labs approach. In Proceedings of the 2nd International Conference on Testbeds and Research Infrastructures for the Development of Networks and Communities, TRIDENTCOM: 135-139.
Roberts, D., Hughes, M., & Kertbo, K. 2014. Exploring Consumers’ Motivations to Engage in Innovation through Co-Creation Activities. European Journal of Marketing, 48(1/2): 147-169. DOI: http://dx.doi.org/10.1108/EJM-12-2010-0637
Roser, T., DeFillippi, R., & Samson, A. 2013. Managing your Co-Creation Mix: Co-creation ventures in distinctive contexts. European Business Review, 25(1): 20-41. DOI: http://dx.doi.org/10.1108/09555341311287727
Rudmark, D., Arnestrand, E., & Avital, M. 2012. Crowdpushing: The flip side of crowdsourcing. in Proceedings of the Ecis 2012.
Schaffers, H., & Kulkki, S. 2007. Living labs, an Open Innovation Concept Fostering Rural Development. Asia-Pacific Tech Monitor, September: 30-38.
Schumacher, J., & Feurstein, K. 2007. Living Labs - the user as co-creator. In Proceedings of the Technology Management Conference (ICE), 2007 IEEE International, 1-6.
Schuurman, D., De Marez, L., & Ballon, P. 2015. Living Labs: a systematic literature review. Open Living Lab Days 2015. DOI: http://hdl.handle.net/1854/LU-7026155
Schuurman, D., De Marez, L., & Ballon, P. 2016. The Impact of Living Lab Methodology on Open Innovation Contributions & Outcomes. Technology Innovation Management Review, 6(1): 7-16. DOI: http://doi.org/10.22215/timreview/956
Ståhlbröst, A., & Bergvall-Kåreborn, B. 2011. Exploring Users Motivation in Innovation Communities. International Journal of Entrepreneurship and Innovation Management, 14(4): 298. DOI: http://dx.doi.org/10.1504/IJEIM.2011.043051
Vargo, S.L. & Lusch, R.F. 2008. Service-Dominant Logic: Continuing the evolution. Journal of the Academy of marketing Science, 36(1): 1-10. DOI: http://dx.doi.org/10.1007/s11747-007-0069-6
Veeckman, C. et al. 2013. Linking Living Lab Characteristics and their Outcomes: Towards a conceptual framework. Technology Innovation Management Review, 3(12): 6-15. DOI: http://doi.org/10.22215/timreview/748
Westerlund, M., & Leminen, S. 2011. Managing the Challenges of becoming an Open Innovation Company: Experiences from living labs. Technology Innovation Management Review, (October): 19-25. DOI: http://doi.org/10.22215/timreview/489
Zwass, V. 2010. Co-creation: Toward a taxonomy and an integrated research perspective. International Journal of Electronic Commerce, 15(1): 11-48. DOI: https://doi.org/10.2753/JEC1086-4415150101