Introduction

We must reconcile the “startup hype” that acts as a rallying cry for new entrepreneurs with the cold reality of the high mortality rates for these startups, which are typically estimated between 67% (CB Insights, 2018) and 75% (Gage, 2012). If we are to achieve – and even exceed – the promised outcomes of this focus on startup activity, we must address the high mortality rates. But how? In this article, we argue that innovation projects are a promising avenue for reducing the startup mortality. The argument is based on the assumption that “getting your first innovation project right, immediately” increases the chances of survival significantly.

Innovation management research aims to unravel the entrepreneurial process by developing frameworks and methods to manage innovation projects. An important literature stream in this domain is Chesbrough’s (2006) notion of open innovation. The open innovation literature tends to focus on the benefits of opening up organizational boundaries. Parallel with the development of open innovation as a research framework, approaches to practically implement open innovation in organizations and in innovation projects have emerged. The majority of the approaches has a clear practitioner focus, and this field is also subject to a lot of sudden “hype” and claims of “radically new approaches” that are sometimes based on single case studies or a limited number of observations. Therefore, we argue for more empirical investigations into the practical implementation of open innovation and innovation management approaches – something which is missing in the current literature.

With this article, we want to focus on two major concepts that, in terms of attention, followers, and publications have clearly outlived their initial hype: the lean startup methodology and living labs. However, despite receiving a lot of attention and devoted followers, there has been little empirical and scientific investigation into the effectiveness and the trade-offs of these two approaches. Although there are some clear similarities and links between them, they have only rarely been mentioned together in studies or publications. Here, we address this gap by first investigating both approaches to identify their similarities and differences. Second, we report on an empirical investigation of 86 living lab projects in terms of outcomes and project characteristics. Third, we develop propositions regarding the living lab versus lean startup approaches and suggest future research to investigate these propositions. Finally, we identify what lessons can be shared across the two approaches.

The Lean Startup Concept

The lean startup is described as a methodology for developing businesses and products that is built upon hypothesis-driven business experimentation, iterative product launches, and validated learning (Ries, 2008; Frederiksen & Brem, 2017). The aim is to shorten product development cycles and reduce market risks by avoiding large amounts of initial funding for big product launches and subsequent failures. The iterative fine-tuning of the innovation based on validated learning from early customer feedback is regarded as the crux of this approach. Thus, in the lean startup methodology, the focus is on the formulation of assumptions related to the end user, the validation of those assumptions, and often their subsequent revision (Ries, 2008; Blank, 2006).

The lean startup methodology was first proposed by Eric Ries in 2008 based on personal experiences with high-tech startups using his personal experiences adapting lean management principles to high-tech startup companies, and was later refined into his seminal book The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses (Ries, 2011). In terms of central ideas and propositions, it is regarded as a follow-up and extension of the customer development idea from Steve Blank’s The Four Steps to the Epiphany (2006). One of Blank’s main points is that organizations were focusing too much on actual delivery and creation of a solution without taking into account consumer demand. Before listening to the customer, these companies spent months or even years perfecting the product without interacting with the customer. As a result, many of these innovations failed to reach uptake by the market because the products were not in sync with actual user needs. This led to an approach where he proposed “going lean” by basing development on iterative cycles of building, measuring, and learning – a process that is based on the principles associated with the terms “failing fast”, “minimum viable product”, “continuous learning”, and “pivoting”.

At the same time, the implied importance of intuition in the lean startup process is a reason for criticism. Often, the validation of assumptions happens in a rather “quick and dirty” fashion, with rapid iterative cycles and pivots. Pivots describe strategic changes of business concepts or products: a course correction to test a new hypothesis (Ries, 2008). One study investigated pivots in the case of 49 software startups and identified as many as 10 different types of pivot and various triggering factors (Bajwa et al., 2017).

Recently, some academic studies have investigated the principles and merits of the lean startup in light of leading theories and empirical evidence from current innovation management academic research. For example, York and Danes (2014) looked deeper into the lean startup methodology and linked it with more established concepts from the innovation management literature. They saw the lean startup as a customer development methodology in the broader theoretical context of new product development. They regarded customer development as an entrepreneurial practice within the context of earlier product development models such as Cooper’s new product development (Cooper 1988, 2008) and Koen’s (2004) new concept model for the “fuzzy front-end”. During the essential phase of hypothesis testing, intuition is seen as having a role in the entrepreneurial process, but the entrepreneur is encouraged to collect information and survey the environment in order to make educated guesses (York & Danes, 2014).

This combination of intuition and more formal processes to reduce uncertainties by iterative and early customer involvement has been advocated by Blank (2006), Maurya (2012), and Cooper and Vlaskovits (2010). York and Danes (2014) summarize the customer development model in four stages: 1) customer discovery: a focus on understanding customer problems and needs, where the goal is to establish a problem–solution fit and develop a minimum viable product (MVP); 2) customer validation: the identification of a scalable and repeatable sales model, where the goal is to establish product–market fit and find a viable business model; 3) customer creation: creating and driving end-user demand; and 4) company building: the transition of the organization from learning and discovery to efficient execution. Stage 1 already includes challenging all assumptions, whereas the product should be launched as soon as possible (i.e., as an MVP) to increase the level of feedback. Subsequently, the lean startup methodology itself can be understood as a set of tools originating from different business development methods. The act of hypothesis testing with potential customers is referred to as “getting out of the building” by Blank (2006), but although the wording implies doing this “outside” or in “real-life”, this actually simply refers to talking to customers, users, and experts.  

Although a large number of incubators and entrepreneurship programmes apply the lean startup methodology, there is still a lack of knowledge regarding its implementation in the real world (Mansoori, 2017). Based on interviews with 11 Swedish technology startups in the setting of a prescriptive accelerator programme, Mansoori (2017) describes vicarious and experimental learning as a means for entrepreneurs to acquire and apply lean startup theory in practice. An empirical approach is also provided by Edison and co-authors (2018), who analyzed different case studies to investigate the use of the lean startup methodology to facilitate software product innovation in large companies. They identified a list of key enablers for success, such as autonomy in decision-making processes or top management support, and inhibitors, often found in complex and bureaucratic business structures that slow down development processes. Finally, a study by Ladd (2016) looked into 250 innovation teams from a cleantech accelerator programme and found out that, in general, the lean startup methodology seemed effective: teams that tested hypotheses about their venture performed almost three times better in a pitch competition (a proxy for success) than teams that did not test any hypotheses. However, the number of validated hypotheses did not show a linear correlation with the success of these teams, which indicates that too much testing can also be detrimental for startup development. Ladd (2016) identified a loss of confidence and too many changes as possible explanations of these results. A recent study by Frederiksen and Brem (2017) investigated the scientific literature in search of antecedents and empirical evidence for the main principles of the lean startup methodology. Their results indicate that, overall, the methods find considerable backing and can be recognized, at least in part, under already established constructs. Heavy use of effectuation logic is evident throughout Ries’ (2011) book, with a clear and explicit emphasis on experimentation over long-term planning, but the main elements and propositions of the lean startup can be at least partly supported by academic research.

Whereas the lean theory is often associated with technology-driven sectors, the methodology is already used in other sectors such as healthcare and communication (Silva et al., 2013). Looking at the ownership structure of lean startups, we mostly see clear management structures that are either team-driven or company-driven, but the scientific literature generally does not elaborate on different stakeholder participation in detail. Nevertheless, Kullmar and Lallerstedt (2017) elaborated on the advantages and limitations of the lean startup approach from the perspective of three different stakeholders: entrepreneurs, business developers, and investors. Although close customer collaboration was considered crucial, the findings also indicated that, when dealing with radical innovation, customer feedback might even be counterproductive for entrepreneurs, as customers tend to focus on the delightful and frustrating aspects of the current offering, whereas radical innovation taps into more latent needs (Thiel & Masters, 2014).

In summary, there is some academic literature that supports the claims of the lean startup methodology, although the evidence is not conclusive. Moreover, the majority of the publications on the lean startup methodology do not include empirical data, but rather rely on spectacular but anecdotal “cases”.

The Living Lab Concept

The concept of the living lab evolved from the notion of long-term field experiments in the 1980s and 1990s, to lab infrastructures aimed at testing innovations in settings aimed at recreating real-life conditions in the 1990s and 2000s, towards an innovation approach based on user co-creation and real-life experimentation in the 2000s and 2010s. Living labs are regarded as complex phenomena where three analytical levels can be distinguished: the organizational level, the project level, and the individual user interactions level (Schuurman, 2015). The living labs literature is very explicative in terms of the participating stakeholders and actors involved. This is apparent at the organizational level (e.g., Leminen, 2013; Leminen et al., 2012) or at the user interactions level (e.g., Dell'Era & Landoni, 2014; Leminen et al., 2014). For this article, we focus on the project level, which is the least discussed level in the living labs literature, as a systematic literature review revealed (Schuurman, 2015).

A living lab project approach is described as a structured approach to open innovation and user innovation (Almirall & Wareham, 2008; Leminen et al., 2012; Schuurman et al., 2016a). Thus, we look at living lab projects from an innovation management perspective. Common elements of living labs are: 1) co-creation, 2) a multi-method approach, 3) multi-stakeholder participation, 4) a real-life setting, and 5) active user involvement (ENoLL, 2018). In terms of methodology, most papers focus on these specific elements without going into further detail about how these elements are combined or linked in a specific methodology. The most concrete are the works of Pierson and Lievens (2005) and Schuurman and colleagues (2016a), who put forward a quasi-experimental design that includes a pre-test, an intervention, and a post-test. This quasi-experimental design – with the elements real-life experimentation, active user co-creation, and a multi-method approach – generates creative tension, according to Almirall and Wareham (2011), where user-led insights are cultivated and tacit, experiential, and domain-based knowledge is surfaced, codified, and communicated.

A key defining aspect of living labs is the real-life context, which allows the dynamics of everyday life to play a vital role in innovation processes. It includes both a regional aspect, such as pushing product tests or needs assessment in cities, rural areas, and real or virtual networks, and an everyday life context in terms of actual user involvement. The lab is anything but a solitary environment. Living labs use multiple methods such as qualitative and explorative research approaches, including, for example, ethnographic methods, co-creation sessions, field tests, and idea scouting. Again, the overall goal is to ensure a continuous, content-based interaction between the lab and its customers. The co-creation aspect and the active user involvement of living labs require strong cooperation and openness towards different actors. The testing and experimentation in real-world circumstances is a defining characteristic of living labs. Nevertheless, the literature fails to acknowledge exactly why this is the case and how it should be realized. It is the “dynamics of everyday life” that are put forward as a reason for not having a systematic or structured approach within living labs. At the same time, multi-stakeholder involvement is a central issue, and a lot of research concerns actor roles in living labs (e.g., Nyström et al., 2014; Schuurman et al., 2016b). In terms of the living lab actors, this task is carried out by the living lab researchers, who engage in a dual role of action researcher as they solve immediate problems while informing (living labs) theory (Logghe & Schuurman, 2017; Ståhlbröst, 2008). Multiple roles lead to divergent interests and an increasing complexity in decision making. However, we do not see these reasons as arguments for not following a clear structure and decision-making process. Especially when looking at the ownership and the business model of living labs, we observe a lack of clarity (Protic & Schuurman, 2018).

The five elements of living labs lead to the assumption that they are able to generate tacit and experiential knowledge that is not obtained in “traditional” innovation approaches. That is why the codifying and communicating suggests that translation of these insights is crucial. In general, we see a great variety of strategies for revenue generation among living labs (Protic & Schuurman, 2018). While some are active in the early stage of innovation processes, others are more likely to serve as test beds or urban development instruments. As Ståhlbröst (2013) describes, labs also offer predefined, fee-based services to their clients (i.e., the “living lab as a service”). In general, these labs tend to have clearer management and ownership structures, as daily operation is very similar to service-driven organizations. We can refer to iMinds Living Labs (now called imec.livinglabs) as an example, as this organization within a larger research institute developed into a service-driven organization after the experience of being part of three funded consortium living labs (see Schuurman, 2015 for a detailed description and analysis). In this living lab as a service organization, projects are carried out for “customers” of the living lab and thus have a clear project owner, whereas in consortium-based living labs, ownership and roles in living lab projects tends to be less clear because of the diverging interests of the consortium partners (Schuurman et al., 2016b).

There are few studies that present concrete results of the outcomes of these living lab projects, and even fewer that compare living lab projects with other innovation projects. Ståhlbröst (2012) puts forward five principles that should guide the assessment of a living lab’s impact. In a follow-up study, Ståhlbröst (2013) assesses these principles in a qualitative way for five micro-enterprises. Nevertheless, the results are rather an application of the principles than an actual impact assessment. Schaffers and colleagues (2012) reported on the results of a European project in which cross-border living lab activities led to new business opportunities and increased revenue, but the sample is also limited. Schuurman and colleagues (2016a) compared 13 projects with a full living lab methodology with 14 projects without a full living lab methodology. The main findings are that the living lab projects seem to foster more actionable user contributions than non-living-lab projects, but that the non-living-lab projects seem to advance faster when going to market, aborting a go-to-market attempt, rebooting with a new innovation project etc., whereas more living lab projects remain in the “in development” stage. Ballon and colleagues (2018) provide the most comprehensive study into impact assessment for living labs and come to the conclusion that impact assessment is difficult and poses severe methodological barriers to be overcome. The paper itself also reports an impact assessment of a sample of living lab projects, focusing on the economic impacts. This study also suggests the added value of a living lab approach and proposes that, although it is difficult to clearly assess impact, this does not mean that no attempts should be made to do it. In this article, we want to assist in filling this gap in research into living labs, open innovation, and user innovation by looking into a larger sample of innovation projects and juxtaposing the findings with the theoretical considerations of both living labs and the lean startup methodology.

Methodology

In the current study, we adopt a mixed design with quantitative and qualitative data. For the quantitative part, we look at all innovation projects carried out by the user research team of imec.livinglabs (previously iMinds Living Labs and iLab.o) from 2011 up to 2018, which makes for a sample of 86 projects. This means all of the projects in our sample are linked to a living lab organization, so we cannot make a comparison with projects that adopted a lean startup methodology. However, the data from this sample allows for the investigation of certain elements of the living lab methodology, which will be contrasted with the lean startup literature in the discussion.

For this sample, we coded the presence of a real-life field trial in the projects based on the project deliverables. We also coded the status of the project in terms of project outcome: “on the market” if the innovation is available for adoption by end users, “abort” if the innovation project is stopped and the team members disband, “reboot” if the innovation project is stopped but the team members continue with a new innovation project based on the insights, and “in development” to indicate that the innovation had not yet been launched. This last category can be regarded as an “in-between state”: over time, these projects will either become available on the market, be aborted, or be rebooted. The data for the initial coding of the projects was taken from a post-assessment interview at the end of each project. However, every year this database is updated based on an online search and a personal follow-up with the project owners to assess changes. The last update of the status dates from May 2018. All of these projects were innovations with a digital component. The majority (58) had a business-to-consumer (B2C) focus, whereas the remaining 28 projects could be labelled as business-to-business (B2B). For an idea of some of the projects, see Schuurman (2015) and Schuurman and colleagues (2016a).

For the quantitative analysis, we simply compared the numbers of the projects with a real-life field trial, which can be considered as living lab projects, with those without a real-life field trial (see Table 1). Because of the relatively small sample size as compared to the outcome categories, no chi-square tests could be performed as the expected cell numbers were less than 5 for more than 20% of the cells. Therefore, here, we simply report the percentages. For the qualitative study, we selected cases from each category (abort, reboot, in development, and on the market) and looked for further evidence related to our literature review.

Results

The main results from the quantitative analysis for the 86 projects are summarized in Table 1. Overall, roughly 1 out of every 4 projects was stopped after the project and almost 1 out of 10 was rebooted based on the project insights, whereas 1 out of 10 are still in development or implementing the lessons learned.

Table 1. Comparison of the outcome of projects with and without real-life trial

Status	Living Lab Projects (With Real-Life Trial)	Innovation Projects (Without Real-Life Trial)	Overall
Abort	10 (28%)	11 (22%)	21(24%)
Reboot	5 (14%)	2 (4%)	7 (8%)
In development	2 (6%)	7 (14%)	9 (10%)
On the market	19 (53%)	30 (60%)	49 (57%)
Total	36 (42%)	50 (58%)	86 (100%)

In this sample of 86 projects, another striking finding becomes apparent. Overall, only a minority (42%) of all projects can be regarded as “real” living lab projects, meaning they contained a proper real-life field trial. These “innovation projects” that lacked a real-life trial were, for example, projects in which testing only took place in a laboratory setting (15 projects) or where user ideation or co-creation took place without an intervention with (i.e., a prototype of) the innovation (33 projects). This can also be explained by the fact that, already in these exploratory stages, the absence of a market need was detected, which was the case for 1 out of 5 of these projects (see also Schuurman et al., 2016a). However, in general, the majority of the projects resulted in the original innovation idea – the one under investigation at the start of the project – being launched on the market at some point. Just over half of the living lab projects with a real-life trial resulted in a market launch, but even 60% of the “innovation projects” also ended up in a market launch. It can be assumed that these entrepreneurs engaged in an innovation project with the living lab organization and either took the “exploratory” learnings from this innovation project to develop a prototype and did the testing themselves or they relied on their intuition and simply launched or aborted the project. However, more investigation would be needed to confirm these assumptions.

The biggest difference between the living lab versus innovation projects is the fact that the majority of all “reboots” occurs in projects with a real-life field trial. The percentage of market launches is slightly lower, and the percentage of “aborted” projects is higher. This seems to support the fact that real-life experimenting indeed surfaces tacit needs. This would allow a decision on whether to continue (and pursue a market launch), or to abort. Moreover, the relatively high number of reboots supports this tacit user need these, as novel tacit needs surface and elicit novel innovation ideas. However, because of the small number of reboot projects, this is again an assumption that needs further validation.

Therefore, to look for further evidence, we performed a small qualitative investigation into the five projects that included a reboot after a living lab project with a field trial. The data from these projects was gathered from the project proposals, the project deliverables, meeting notes and data from a post-assessment survey. Our findings are summarized below:

1. InCitys: This project investigated the potential of a smart city platform for citizens. However, based on a test in a Flemish city, there was low interest from citizens as well as from other actors that would provide content on the platform. However, one use case, in the domain of smart energy, was relatively successful. Based on this finding, the collaboration with the energy provider was intensified and this resulted in a “smart plug” offering being launched on the market.
    
2. Wadify: The objective of this project was to create an online video platform for young people, who would be rewarded for watching advertisements. For the young people, the test was very successful, as they liked the platform very much and showed interest in using it in the future, but the interest from advertisers was too low. However, based on the discussions with the young people and research into their interests, the entrepreneurs made the connection between festivals and smart technologies. This resulted in Playpass, a new direction of the team behind Wadify that focused on smart wristbands for festivals. In this area, they have successfully launched their first product.
    
3. Nazka: This project dealt with the visualization of air quality metrics on maps. During the field trial, the user feedback indicated that the numbers were hard to interpret and that end users were not that interested in this data. This made the company shift from a business-to-consumer (B2C) model towards a business-to-business (B2B) model where they provided the basic infrastructure and opened up their datasets to allow other parties to re-use the data and make sense of it. In this new B2B model, they adopt a licensed-platform approach and no longer interact directly with the end user.
    
4. Veltion: This B2B startup advised companies on the optimization of production and other company processes. They developed an application that could be used by workers to report issues and suggest improvements. Within the living lab test, the application was tested and the experiences of two companies were positive and satisfying. However, interviews with the company managers also revealed that this usage would cannibalize their regular service offering, as it would potentially replace their consulting business. The positive field trial paired with these insights made them change their initial idea, and they now use an adapted form of the application as an “add-on” to their consulting business rather than a standalone offering.
    
5. Planza: This planning tool was initially oriented as a consumer service. The field trial revealed a positive user experience, but indicated a lack of “willingness to pay”. Thus, the original idea behind Planza was deemed not viable, and the team shifted towards a B2B approach. The platform was stripped to keep the functionalities that were of interest in a B2B-setting, and the result was a planning tool for companies.

The above examples indicate that putting the innovation to the test in a proper real-life field trial helps the projects validate critical assumptions and take key decisions regarding the next steps in their innovation development process. One finding that can be abstracted from the cases is that the reboots were not only driven by user insights, but also could be linked to business model insights. This combination of business model research with user research is one of the key assets within imec.livinglabs, but is rarely present in other living labs (see Rits et al., 2015). This is attained by starting all projects with a business model analysis to identify the key uncertainties, which enables the lab to tailor the user involvement activities and real-life tests towards filling these gaps, and by having multi-disciplinary teams of business and user researchers carrying out the projects (see also Schuurman et al., 2018). The experiential learning of user research and real-life field trials seems to provide actionable data that can be used as “evidence” for designing and iterating the business model. Moreover, this multi-disciplinary approach is also an aspect that drives entrepreneurs to use the services of a living lab. First, not all expertise is present in the entrepreneurial team, and time and resources are limited. Therefore, external sourcing of capabilities can shorten development cycles and save effort, as some critical aspects can be outsourced. However, this requires an accurate process of hypotheses building and prioritizing to identify which one should be tackled first. More guidelines and investigation seem necessary in these matters to develop the thinking further.

Discussion and Conclusion

Within this article, we looked into the similarities and differences between two concepts that focus on a practical implementation of open innovation. Both living labs and the lean startup methodology are mainly practitioner-driven and both have an avid base of “believers”. However, for both concepts, there is a lack of quantitative studies that measure impact and outcomes of these approaches in a more systematic manner. Moreover, despite some obvious similarities, both concepts are rarely studied or mentioned together. Building upon lessons learned, Table 2 compares the two concepts in terms of their various stages, their focus and real-life context, the methodology mainly applied, and the ownership structure.

Table 2. Comparison of the lean startup and living lab methodologies  

	Lean Startup	Living Lab
Focus	Clearly positioned as an innovation management approach.	Very explicative in terms of the participating actors and stakeholders, co-creation, and real-life experience.
Stages	Customer discovery Customer validation Customer creation Company building (Cooper & Vlaskovits, 2010; York & Danes, 2014) Combined with “getting out of the building” for assumption testing.	Pre-test Intervention Post-test (Pierson & Lievens, 2005; Schuurman et al., 2016a) Combined with real-life experimentation, active user co-creation and a multi-method approach.
Real-life	Focus on formulating assumptions related to the end user and fast iterations of assumption validations by “getting out of the building” only.	Allow the dynamics of everyday life to play a vital role in the shaping of the innovation.
User role	User feedback used for assumption testing.	The user is continuously involved in the innovation process.
Methodology	The focus is mainly on quantitative methods and metrics.	Emphasizes qualitative and explorative research approaches.
Stakeholder participation	Lack of detailed elaboration on participation by different stakeholders.	Multi-stakeholder involvement as a central topic (e.g., Nyström et al., 2014; Schuurman et al., 2016b).
Ownership	Clear entrepreneur or innovator, or in most cases an innovation team.	Ownership often unclear, unless in the “living labs as a service” model.

Based on the gained insights, we can conclude that both approaches start from customer development as the basis to successful innovation. Whereas the lean startup is more explicitly positioned as an innovation management approach with a clearly different approach compared to the traditional stage-gate new product development process, the living lab approach is very explicative in terms of the participating actors and stakeholders, active user co-creation, and real-life experience. However, in terms of innovation management approach, the living labs literature is underdeveloped.

The four stages of the lean startup offer anchor points for the living lab elements. Especially in the first two stages, a living lab approach seems compatible with the goals of problem–solution and product–market fit. Even the customer creation stage can be tackled with a living lab approach, as long-term user involvement might generate initial user demand and innovation advocates (Almirall & Wareham, 2011; Schuurman, 2015).

The lean startup literature focuses on formulating assumptions related to the end user and fast iterations of assumption validations by “getting out of the building”. While it simply aims to interact with (potential) end users and stakeholders in order to validate assumptions, the living lab approach allows the dynamics of everyday life to play a vital role in the shaping of the innovation. In a way, the use of external sources of knowledge is much more intentional and limited in the case of the lean startup.

Looking at the “methodological toolbox” that is linked to both approaches, the lean startup focuses more on quantitative methods and metrics, whereas living labs also emphasize qualitative and explorative research approaches (such as ethnography, co-creation sessions, etc.). Especially in the first stage of the lean startup process, more qualitative methods seem appropriate, whereas for product–market fit, more quantitative methods seem appropriate.

One of the other major distinctions between both concepts is the ownership of the process. In the lean startup, there is a clear entrepreneur or innovator, or in most cases an innovation team. In living labs, this ownership is less clear, except in organizations offering a “living lab as service”. This leads us to conclude that both approaches are rather complementary to one another. For living labs, the lesson learned from the lean startup methodology would be to incorporate a more structured and iterative process with clear decision making and ownership. Also, the flexibility and the rapid iterations can be valuable principles to structure living lab operations.

On the other hand, lean startups can learn from the multi-stakeholder interactions and the co-creative approach to innovation. The multi-facetted, multi-disciplinary nature of living lab organizations can be of critical value. This allows startups to involve the most needed expertise at the ideal moment, given that most critical assumptions are detected. Moreover, from the discussion above, we can assume that “getting out of the building” in real-life might provide more actionable input than plain and simple user interactions. For living labs, this poses the challenge of being flexible in terms of project set-up and execution, whereas for startups, capturing and prioritizing assumptions is crucial. Therefore, we plead for both approaches to exchange experiences and adopt best practices from one another. For our own part, we are trying to facilitate this exchange through the Living Labs Special Interest Group of the International Society for Professional Innovation Management (ISPIM), where living lab researchers and practitioners meet with general innovation managers practicing the lean startup methodology. Indeed, the lean startup methodology seems like a great do-it-yourself (DIY) toolkit, whereas living lab organizations seem to be able to complement the entrepreneurial team capabilities where necessary and provide multi-stakeholder inputs and real-life experience. By acting this way, we foresee that it becomes possible to learn what customers really want, not what they say they want or what we think they should want.

In terms of limitations and future research, the most important limitation of this study is the absence of data on lean startup projects. This would allow an empirical analysis to grasp differences between both approaches. At imec, all the living lab projects as well as all activities in the imec’s iStart incubation programme are assessed in a similar way at the start and at the end of the project or coaching period, a dataset containing this type of data is being generated at the moment. However, data from multiple organizations is needed for a broader investigation. This will allow more detailed and statistical analyses in the future. We also urge other researchers to try and gather larger datasets in order to move the research from exploration to validation. This would definitely help startups and entrepreneurs make a more rational choice between various options and approaches.

 

Acknowledgements

This article was developed from a paper presented at the ISPIM Innovation Conference in Stockholm, Sweden, June 17–20, 2018. ISPIM – the International Society for Professional Innovation Management – is a network of researchers, industrialists, consultants, and public bodies who share an interest in innovation management.

 

---

References

Almirall, E., & Wareham, J. 2008. Living Labs and Open Innovation: Roles and Applicability. The Electronic Journal for Virtual Organizations and Networks, 10(3): 21–46.

Almirall, E., & Wareham, J. 2011. Living Labs: Arbiters of Mid- and Ground-Level Innovation. Technology Analysis & Strategic Management, 23(1): 87–102.
https://doi.org/10.1080/09537325.2011.537110

Bajwa, S. S., Wang, X., Nguyen Duc, A., & Abrahamsson, P. 2017. “Failures” to Be Celebrated: An Analysis of Major Pivots of Software Startups. Empirical Software Engineering, 22 (5): 2373–2408.
https://doi.org/10.1007/s10664-016-9458-0

Ballon, P., Van Hoed, M., & Schuurman, D. 2018. The Effectiveness of Involving Users in Digital Innovation: Measuring the Impact of Living Labs. Telematics and Informatics, 35(5): 1201–1214.
https://doi.org/10.1016/j.tele.2018.02.003

Blank, S. G. 2006. The Four Steps to the Epiphany. Foster City, CA: Cafepress.com.

CB Insights. 2018. Venture Capital Funnel Shows Odds of Becoming a Unicorn Are About 1%. CB Insights, September 6, 2018.
Accessed December 8, 2018:
https://www.cbinsights.com/research/venture-capital-funnel-2/

Chesbrough, H. 2003. Open Innovation: The New Imperative for Creating and Profiting from Technology. Boston, MA: Harvard Business Press.

Cooper, B., & Vlaskovits, P. 2010. The Entrepreneur's Guide to Customer Development: A Cheat sheet to The Four Steps to the Epiphany. Cooper-Vlaskovits.

Cooper, R. G. 1988. The New Product Process: A Decision Guide for Management. Journal of Marketing Management, 3(3): 238–255.
https://doi.org/10.1080/0267257X.1988.9964044

Cooper, R. G. 2008. Perspective: The Stage‐Gate® Idea‐to‐Launch Process—Update, What's New, and Nexgen Systems. Journal of Product Innovation Management, 25(3): 213–232.
https://doi.org/10.1111/j.1540-5885.2008.00296.x

Dell'Era, C., & Landoni, P. 2014. Living Lab: A Methodology between User-Centred Design and Participatory Design. Creativity and Innovation Management, 23(2): 137–154.
http://dx.doi.org/10.1111/caim.12061

Edison, H., Smørsgård, N. M., Wang, X., & Abrahamsson, P. 2018. Lean Internal Startups for Software Product Innovation in Large Companies: Enablers and Inhibitors. Journal of Systems and Software, 135: 69–87.
https://doi.org/10.1016/j.jss.2017.09.034

ENoLL. 2018. What are Living Labs. European Network of Open Living Labs (ENoLL). Accessed December 12, 2018:
https://enoll.org/about-us/what-are-living-labs/

Frederiksen, D. L., & Brem, A. 2017. How Do Entrepreneurs Think They Create Value? A Scientific Reflection of Eric Ries’ Lean Startup Approach. International Entrepreneurship and Management Journal, 13(1): 169–189.
https://doi.org/10.1007/s11365-016-0411-x

Gage, D. 2012. The Venture Capital Secret: 3 Out of 4 Start-Ups Fail. The Wall Street Journal, September 20, 2012. Accessed December 5, 2018: https://www.wsj.com/articles/SB10000872396390443720204578004980476429190

Kallmar, L., & Lallerstedt, I. 2017. The Lean Startup Method – A Golden Ticket to Success or a Vision Killer? Master’s thesis. Lund, Sweden: Lund University.

Ladd, T. 2016. The Limits of the Lean Startup Method. Harvard Business Review, March 7, 2016. Accessed December 5, 2018:
https://hbr.org/2016/03/the-limits-of-the-lean-startup-method

Leminen, S., Westerlund, M., & Nyström, A.-G. 2012. Living Labs as Open-Innovation Networks. Technology Innovation Management Review, 2(9): 6–11.
http://doi.org/10.22215/timreview/602

Leminen, S. 2013. Coordination and Participation in Living Lab Networks. Technology Innovation Management Review, 3(11): 5–14.
http://doi.org/10.22215/timreview/740

Leminen, S., Westerlund, M., & Nyström, A.-G. 2014. On Becoming Creative Consumers – User Roles in Living Labs Networks. International Journal of Technology Marketing, 9(1): 33–52.
http://dx.doi.org/10.1504/IJTMKT.2014.058082

Logghe, S., & Schuurman, D. 2017. Action Research as a Framework to Evaluate the Operations of a Living Lab. Technology Innovation Management Review, 7(2): 35–41.
http://doi.org/10.22215/timreview/1056

Mansoori, Y. 2017. Enacting the Lean Startup Methodology: The Role of Vicarious and Experiential Learning Processes. International Journal of Entrepreneurial Behaviour and Research, 23(5): 812–838.
http://doi.org/10.1108/IJEBR-06-2016-0195

Maurya, A. 2012. Running Lean: Iterate from Plan A to a Plan That Works. Sebastopol, CA: O'Reilly.

Nyström A.-G., Leminen, S., Westerlund, M., & Kortelainen, M. 2014. Actor Roles and Role Patterns Influencing Innovation in Living Labs. Industrial Marketing Management, 43(3): 483–495.
https://doi.org/10.1016/j.indmarman.2013.12.016

Pierson, J., & Lievens, B. 2005. Configuring Living Labs for a ‘Thick’ Understanding of Innovation. In Proceedings of the Ethnographic Praxis in Industry Conference, 2005(1): 114–127. Hoboken, NJ: Blackwell Publishing Ltd.

Protic, S. M., & Schuurman, D. 2018. Sustainability of Living Labs in a Smart City Context: A Conceptual Framework on Business Model Design. Paper presented at the 35^th World Conference on Science Parks and Areas of Innovation, IASP Isfahan.

Ries, E. 2011. The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. New York: Crown Books.

Rits, O., Schuurman, D., & Ballon, P. 2015. Exploring the Benefits of Integrating Business Model Research within Living Lab Projects. Technology Innovation Management Review, 5(12): 19–27.
http://doi.org/10.22215/timreview/949

Schaffers, H., & Turkama, P. 2012. Living Labs for Cross-Border Systemic Innovation. Technology Innovation Management Review, 2(9): 25–30.
http://doi.org/10.22215/timreview/605

Schuurman, D. 2015. Bridging the Gap between Open and User Innovation? Exploring the Value of Living Labs as a Means to Structure User Contribution and Manage Distributed Innovation. Doctoral dissertation. Ghent University and Vrije Universiteit Brussel VUB.

Schuurman, D., De Marez, L., & Ballon, P. 2016a. The Impact of Living Lab Methodology on Open Innovation Contributions and Outcomes. Technology Innovation Management Review, 1(6): 7–16.
http://doi.org/10.22215/timreview/956

Schuurman, D., Lievens, B., Veeckman, C., De Marez, L., & Ballon, P. 2016b. Open Innovation Networks: Exploring Actor Roles and Network Orchestration in Living Labs. In Open Innovation: A Multifaceted Perspective, Part 1, 207–239. Singapore: World Scientific Publishing.
http://doi.org/10.1142/9789814719186_0009

Schuurman, D., Herregodts, A.-L., Georges, A., & Rits, O. 2018. Innovation Management in Living Lab Projects: The Innovatrix Framework. Paper presented at Open Living Lab Days 2018, Geneva, Switzerland.

Silva, S. E. P., Calado, R. D., Silva, M. B., & Nascimento, M. A. 2013. Lean Startup Applied in Healthcare: A Viable Methodology for Continuous Improvement in the Development of New Products and Services. In Proceedings of the 6th IFAC/ACM Conference on Management and Control of Production and Logistics, 6 (1), 295–299. Fortaleza, Ceara, Brazil.

Ståhlbröst, A. 2012. A Set of Key Principles to Assess the Impact of Living Labs. International Journal of Product Development, 17(1-2): 60–75.
https://doi.org/10.1504/IJPD.2012.051154

Ståhlbröst, A. 2013. A Living Lab as a Service: Creating Value for Micro-enterprises through Collaboration and Innovation. Technology Innovation Management Review, 3(11): 37–42.
http://doi.org/10.22215/timreview/744

Thiel, P., & Masters, B. 2014. Zero to One. Notes on Startups, or How to Build the Future. New York: Crown Business.

York, J. L., & Danes, J. E. 2014. Customer Development, Innovation, and Decision-Making Biases in the Lean Startup. Journal of Small Business Strategy, 24(2): 21–40.