Alone, we go faster. Together, we go further.
Motto of the Living Lab of Foch Hospital,
Suresnes (Paris), France
Living labs have gained increased attention in research and practice as both a practical and theoretical innovation phenomenon that emphasizes co-creation, real-life settings, and user/customer involvement. More recently, living labs have also emerged as a specific approach to open innovation processes in the context of publics across the EU. Nevertheless, it is still not clear how the understanding of living labs can be translated and organized into new sectorial settings, what type of public sector innovation challenges it addresses, and what role citizens and users have. The aim of this article is therefore to explore and analyze how living labs are applied as processes for public sector innovation. Based on a mixed method approach of 21 European living lab cases, the analysis reveals a pattern of three different processes for living lab organizational and actor roles: living labs organized as cross-sectorial collaboration, living labs emerging within the public sector as main initiator and beneficiary, and living labs developed by civil society actors. The findings are presented as three scenarios for implementing living labs, which also acts as a background for the article's final discussion about the potentials and pitfalls of living labs in public sector contexts.
The acknowledged move from traditional public administration (TPA), over to new public management (NPM), then to the current shift towards new public governance (NPG) has spurred an increased awareness on the role of external stakeholders in developing public services, and hence the way public sector innovation takes place (Hartley, 2005; Torfing, 2019). Public sector innovation is now more dependent on joint processes based on cross-sectorial collaboration, which implies that public innovation has become complex and dynamic, since citizens multifaceted needs require several actors to coordinate their efforts. Innovation therefore now takes place in a complex multi-actor context of politicians, policymakers, public managers, employees, users, citizens, civil actors, and private firms. A platform and methodology for such innovation processes are living labs (Leminen et al., 2012; Ruijer & Meijer, 2020). Living labs are defined as collaborative environments for experimentation in and of real-life contexts (Gascó, 2017). Living labs are still, however, somewhat underexplored in the context of public sector innovation, herein how they are organized and with what they contribute (Schuurman & Tõnurist, 2017; Hansen & Fuglsang, 2020).
Therefore, to better understand and learn from existing living labs, the main aim of this article is to investigate and analyze how living labs spur and enact processes of public sector innovation in a European context, and to discuss the potentials and pitfalls of living labs as a way of doing public sector innovation. This leads to the following two research questions: a) How are living labs applied to engage actors in public sector innovation processes?, and b) What promises do such innovation processes hold?
The research is based on a mixed methods design, encompassing 21 case studies of living labs across nine EU countries (Fuglsang & Hansen, 2021; 2022) and a thorough survey of co-creation methods in the public sector, distributed to public managers in six EU countries (Arundel & Es-Sadki, 2021). The paper extends previous research on the societal framing of living labs (Ruijer & Meijer 2020; Fuglsang & Hansen, 2022), and involving methods used in living labs, by presenting experiences from cases of how living labs can organize public sector innovation processes in terms of various scenarios.
The article is structured as follows: first, a short overview of the theory base is presented, followed by an introduction to the methodology applied. Subsequently, key analytical results are accounted for and discussed. Finally, concluding remarks are given, and future research avenues proposed.
Public sector innovation
Innovation as concept may take slightly different meanings across various sectors and research traditions. Yet, most of the literature maintains that innovation encompasses the two intertwined processes of creating something new, and implementing this new creation in practice (Torfing 2019; Fuglsang & Hansen, 2022). The processes that lead to innovation are summarized in terms of, for example, structures and stages of innovation, specific drivers that lead to innovation, such as entrepreneurs or R&D, specific procedures such as design processes, and certain innovation roles. While much emphasis is on the structures and stages of innovation processes, some authors have also conceptualized innovation as a practice-based inherently incremental activity (Fuglsang, 2010), that is, as integrated with work and organizational routines. The practice-based approach is especially evident in innovation processes taking place within everyday work in public service delivery leading to the creation of new knowledge and new behaviors (Fuglsang, 2021).
The acknowledgement of contextual factors has led to the argument that it is important to develop relevant and restricted concepts for public sector innovation (Gault, 2018). Windrum (2008) proposed a useful distinction between six types of innovation found in the public sector: service innovation, service delivery innovation, administrative and organizational innovation, conceptual innovation, policy innovation, and systemic innovation. Hartley (2005) added governance innovation as a special feature of public sector innovation. Governance innovation refers to new forms of citizen engagement in innovation, and rhetoric innovation, which means new language and concepts in a service domain. Hartley also suggested that rather than speaking of types of innovation, such as radical and incremental, governance or rhetorical, it may be more correct to treat innovation, particularly complex innovations, as multidimensional processes since the different types are connected in practice (Hartley, 2005).
Besides the focus on how and with what innovation contributes, innovation processes in a public sector context, especially in settings with a high degree of citizen-employee encounters, is based on the logic of open, co-creational and collaborative innovation (Hartley et al., 2013; Voorberg et al., 2015). Open innovation describes how the knowledge of citizens and other actors external to government organizations is included (Fuglsang, 2008). Resulting from this openness, the knowledge that is created can be heterogeneous in its nature and might also result in beneficial outcomes for the organization due to, for example, organizational learning and increased innovation capability (Mergel, 2015). Co-creation designates processes of co-initiation, co-design and co-implementation of public services with citizens and users (which encompasses both citizens as users and employees) (Voorberg et al., 2015). These characteristics are especially evident in the application of living labs.
Living labs as processes for public sector innovation
The term “living lab” or “innovation lab” stems from information and communication technology (Eriksson et al., 2005; Nesti, 2017; Fuglsang et al., 2021), where it emerged as a phenomenon, and practice, that supported test environments either as lab facilities or as facilities in real-life settings. From the outset, living labs have therefore been platforms for collaboration processes between developers and users. Later, as the use of living labs spread and the approach became conceptualized within an open innovation paradigm, more layers were introduced. Gascó (2017), based on Schaffers and Turkama (2012) defined living labs in a public sector context as:
“settings or environments for open innovation, which offer a collaborative platform for research, development, and experimentation in real-life contexts, based on specific methodologies and tools, and implemented through specific innovation projects and community-building activities” (Gascó, 2017).
Many current living labs refer back to such forms as quadruple (or even quintuple) helix innovation processes (Hakkarainen & Hyysalo, 2014; Baccarne et al., 2016), that engage actors across sectors and from multiple angles of innovation (Tõnurist et al., 2017). Also, living labs are perceived as strategic, structured and deliberate processes of innovation initiated by a primary actor.
At the beginning of the millennium, the living labs phenomenon appeared in the public sector to ensure interactive innovation processes with a distinct focus on employees and/or citizens. The focus on living labs in public sector contexts was also reinforced by founding the European Network of Living Labs (ENoLL), under the Finnish Presidency of the Council of The European Union in 2006. Living labs differ from other open innovation approaches as they are platforms for experimentation, wherein participants, for example, representatives from private sector organizations, the public sector, universities, users, and citizens meet in person to develop innovations together. Thus, the aspect of place/space often matters at the outset as a trigger for doing innovation, since living labs are developing various new workplace practices and services with a goal of channelling innovative knowledge and routines for innovation acquisition into host organizations (Fuglsang & Hansen, 2022). As such the notion of living lab also supports the acknowledged governance shift towards New Public Governance (NPG) (Dekker et al., 2020; Criado et al., 2021). Consequently, the concept’s extension has led to an on-going discussion about living labs as much more than just an instrument or method; as living labs are also perceived as an innovation methodology, or certain mindset within which to potentially frame both new and existing understandings and practices (Dell'Era & Landoni, 2014; McGann et al., 2018). Thus, the application and introduction of living labs in the public sector might itself be seen as a trigger for more inclusive innovation processes alongside increased awareness that gives employees and citizens an active role in development. Living labs are therefore on the one hand seen as an opportunity to include heterogeneous knowledge from different actors to solve problems, but on the other hand the outcome of such processes and the role of the users and citizens are still not very clear (Hansen & Fuglsang, 2020).
To study how living labs are applied as processes for public sector innovation, a multiple case study (Yin, 2014) was conducted to gain insights on how different constructions of living labs appear in different public sector contexts, and by whom they are initiated. Our case study draws together insights from qualitative and quantitative data collected in 2018 and 2019 (see Fuglsang & Hansen, 2022). All data stems from the EU funded project “Co-VAL”, that is seeking new paths to co-creation of value in order to transform public administration services and processes.
The qualitative data are based on 21 in-depth case studies across nine EU-countries, conducted by the authors of the article. Based on an extensive literature review on living labs in a public sector context (Fuglsang & Hansen, 2019), the following sample criteria were chosen:
- Public service characteristics: large-scale services (digitalization, supporting citizen welfare broadly), or “small-batch” services (public administration, elderly care).
- Sectors/actors: public organized (state level/municipal department), civil society (citizens/non-profit organizations), or private (company/entrepreneurs).
- Form of organization: formalized/less formalized, and/or networked/single organization.
- Temporality: initiatives targeting here-and-now challenges, or initiatives targeting long-term challenges. Temporality in this context is related to the notion of public value. Thus, here-and-now refers to current challenges to specific citizen/user groups, while long-term refers to challenges encompassing future generations.
This led to a final sample of the following 21 cases (Fuglsang and Hansen, 2021; 2022) presented in Table 1.
Table 1. Overview of cases and data material per country
We used a shared case study protocol to guide the research across cases. Concerning the unique case contexts, the protocol allowed for adaptions so that the most suitable strategies for data collection could be used. The data collection strategy was based on data triangulation, by a combination of document studies, semi-structured interviews, and observations, with various weightings. Afterwards, the data was subjected to a cross-case analysis, focusing on how each case adds to and reveals insights regarding the overall unit of analysis: living labs in the context of public sector innovation.
In addition to the empirical case studies, we extracted quantitative data concerning co-creation in the context of design firms, innovation labs, and living labs from a comprehensive European survey on co-creation and public sector innovation (Arundel & Es-Sadki, 2021). The overall aim of the survey intended to estimate the prevalence of co-creation methods in the innovation activities of public sector organizations, factors that influence the use of co-creation, obstacles to the use of co-creation, and the effect of co-creation on innovation activities and outcomes.
The survey followed a detailed protocol where the first stage (“pre-survey preparations”), implied the delivery of the questionnaire (paper mail) to 1125 managers in France, Spain, the UK, Hungary, the Netherlands, and Norway. The second stage (“survey implementation”), was conducted over 4 or 5 months in 2019, where 3497 questionnaires in total were sent out (also offering an online survey option), and 1036 total replies obtained, which means a total response rate of 32.7%. The respondents were public administration middle or senior managers representing three geographical levels: small municipalities 32%, large municipalities 32%, and central-state national administrations 33% from different sectors such as health, education, and central public services.
To analyze the specific survey results concerning user contributions to innovation integrated into this paper’s case study, we used a multinomial logit model to reveal: 1) prevalent co-creation methods, 2) main barriers to spur user contributions to innovation, and, 3) drivers of user contributions based on various sources of demand. The question about co-creation methods was asked following a categorial yes/no/don’t-not know response.
The analysis dives into the role of living labs in promoting users’ involvement in co-creation and innovation processes, while also identifying a pattern of three different approaches to establishing living labs in a public sector context. In the next section, key analytical findings are presented together with offering three scenarios for establishing future living labs.
Different approaches to living lab organizing
Our analysis of the 21 case studies revealed three processes of living labs as experimental settings organized to address public and societal challenges through engaging external stakeholders, especially citizens. Also, it became apparent that the propensity of different institutional levels to use living labs partly depend on the types of challenges to address, as well as the degree of stakeholder engagement needed. These findings are summarized in the form of three fictive scenarios. Scenarios used as method are both a way to present a huge amount of data and a tool to be applied in different development phases to review and analyze potential aspects of establishing a living lab (Stickdorn & Schneider, 2011). The scenarios are based on recurring features observed during the case study research, placed alongside theoretical knowledge (Fuglsang & Hansen, 2021); hence they do not exist strictly as presented, yet are to be read as illustrations of key findings and insights from the research.
Scenario 1: Living labs for “grand” challenges
Living labs can structure innovation processes for addressing challenges to society at large - not only in a here and now context, but also for future generations. Living labs for such “grand societal challenges” address major unsolved problems of education, inequality, climate, digitalization, unemployment, and social heritage. These types of living labs are often organized as cross-sectorial networks based on shared “ownership”, and hence they might be positioned either in or outside a formal public sector organization. Users are involved at various stages to help set priorities for challenges and test innovations at an early stage.
Figure 1. Living lab as cross-sectorial collaboration
In cases relevant to this scenario, living labs stress the engagement of actors in a new way of thinking about innovation in the public sector. Also, the creation of a more experimental and inclusive approach to stakeholder engagement leads to a more elaborated and qualitatively stronger network of interpersonal and inter-organizational relations. Regarding future points to give attention, especially the need for funding and political support, reaching beyond project periods was a key concern among informants. This is also linked to a lack of qualitative impact criteria such that relationship-building and networks established become part of the success parameters for organizing living labs. Lastly, an on-going discussion continues about the degree of citizen involvement, since most cases expressed a wish for and urge to give citizens an even more decisive role.
Cases representing Scenario 1 included: Public Intelligence, PWC Experience Center, Kraków Living Lab, GovLab Austria, ERASME, Torino City Lab, and Guadalinfo.
Scenario 2: Living labs for domain specific challenges
Living labs can be organized to address challenges relative to specific public service services and welfare. In this scenario, they are often positioned within the public sector, and while cross-sectorial collaboration is encouraged, the main decision-making power lies with public managers. Moreover, these types of living labs often have public sector employees as either their sole target group, or as important a target group as citizens. Hence, users are the service recipients, and they are involved through using design approaches, such as observation and interviewing.
Figure 2. Living lab “owned” by the public sector
In cases relative to this scenario targeting specific domain challenges, living lab activities lead to solutions that are close to the context in which they are to be practiced. Consequently, the aspects of organizational learning and increased innovation maturity were highlighted by informants to stress that not only a specific outcome, but also the innovation process itself is very valuable. Therefore, there was an articulated need for incentive structures that spur innovation and the innovation capabilities of employees in the public sector. Knowledge sharing across public sector departments and institutions was also a concern to make sure innovation practices and experiences are diffused. Finally, there was among actors a curiosity towards expanding the existing “limits” of user and citizen involvement relative to decision-making, since employees and managers self-critically reflected if the current existing limits are based on legal concerns, or instead primarily cultural and mental barriers.
Cases representing Scenario 2 included: Aalborg Municipality, AUTONOM’LAB, Stimulab, Norwegian Labour and Welfare administration, SIILAB, Living lab of the ministry of economy and finance, GovLab Arnsberg, Wallonia e-health Living Lab, and L.I.V.E.
Scenario 3: Living labs as citizen-led initiatives targeting public value
Living labs can be initiated or led by citizens to identify and address societal challenges, which might reach beyond specific public welfare services. These types of living lab initiatives are often depicted by a strong civil society engagement and collaborate mainly with the public sector to partly ensure funding. As such, they seem to enact an explicit urging within European public sectors to openly engage citizens in developing innovative public services and creating public value. Hence, it seems relevant to better understand how the public sector might support such bottom-up initiatives, where citizens are engaged as both initiators and users of the activities taking place within the framework of a living lab.
Figure 3. Living lab as citizen-led initiatives
The experiences from citizen-led living labs show that the openness towards outcome creates a platform, whereby other actors than the ones initially thought of as beneficiaries get attracted. As such, the living lab ends up offering place/space for the wider public, thus implicating that explicit actor roles become less important, while the boundary between these is diminished. By showcasing that such new ways of collaboration can function, living labs seem to expand and trigger a change in administrative procedures when cooperating with the public sector. The future potential of these citizen-led living labs may rely on getting municipalities to play along, such that evaluation criteria that mirrors the reality of these initiatives may be developed.
Cases representing Scenario 3 included: Verschwörhaus Ulm, INSP, IDES Living lab, Rome Cooperative Heritage Lab, and Library Urban Lab.
The aspect of co-creation in living lab practices
Across the identified different approaches to living labs, they all engage stakeholders, including users, in developing public services. The results of the multinomial logit carried out (Table 2), support this result, and moreover adds insights on the interplay between co-creation methods and innovation potential. The model shows that all the co-creation methodologies investigated (analysis of user data, conversations with users, focus groups, brainstorming, and prototyping) are statistically significant regarding relevancy of the final outcome of innovation processes taking place in the context of innovation or living labs. Citizens and other stakeholders may be engaged through direct and indirect participation, yet the more active and direct forms (for example, brainstorming with users), the more significant. In addition, the case study analysis indicated that how stakeholders are involved depends on who leads the initiative. Except users in citizen-led living labs, it seems difficult to mobilize stakeholders for the whole innovation process. Users are mainly invited to participate in the following stages: upstream in the ideation phase and the rapid prototyping phase, downstream to test prototypes, and in the development phase (further prototyping, tests, returns, and iterations) of innovations that can be implemented in the public sector.
Table 2. Results of the multinomial logit model: users’ contributions to innovation
Notes. The dependent variable is the contribution of users to develop the most important innovation in public sector units. The contributions are ranked according to the level of benefit, distinguishing between “none”, “low”, “medium”, and “high”. With the estimation method using multinomial logit: ** implies 1% significance, * implies 5% significance, ‡ implies 10% significance.
In sum, the quantitative and qualitative findings highlight two key points relative to co-creation: 1) when engaging in or establishing living labs, active user and/or citizen involvement leads to more relevant innovations for the beneficiaries, and, 2) it is key to reflect upon how and where in an innovation process the user and/or citizen are involved, and what the implications are of this involvement in final decision-making processes.
The analytical results highlight how living labs provide support processes for the overall turn towards new interactive and networked forms of governance. In inducing open innovation based on the integration of multiple stakeholders, the focus changes from internal processes towards taking more into consideration how users and citizens are engaged in active and direct co-creation of innovation. But this turn also implies that what is perceived as a success, and to whom, rests on more qualitative parameters, such as, for example, the degree of relation building, the subjective experience of new initiatives and documented organizational learning.
Thus, a concrete challenge of viewing living labs as a public sector innovation method is that the more qualitative aspects, which are the inherent legitimizing factors of living labs, seem to be hard to measure, and hence mainly quantitative performance indicators based on a New Public Management (NPM) logic are typically applied (Ballon et al., 2018; Bronson et al., 2021; Dekker et al., 2021) Moreover, an overall challenge is that the degree of user and/or citizen involvement, which is at the core of living labs, might not be as high as the definition implies. Instead, it seems as if the specificity of living labs is their capacity, and legitimacy, to change the traditional processes of political decision-making processes relative to developing public services (Fuglsang & Hansen, 2022). Thus, living labs are not only a “technology of management”, but also a political choice due to policy makers taking the risk of inviting users into development processes (Olejniczak et al., 2020; Osborne et al., 2020). A pitfall is that organizing public sector innovation as or within living labs does not in and for itself ensure that user and citizens insights and deliberation will be taken seriously by the end of the innovation process (Wegrich, 2019), at least not in the case of living labs, where the public sector itself is, or may be the sole initiator. Also, the living lab approach to open innovation might favour individual concerns and certain “citizen-skills” of engagement. Thus, it could make living labs platforms for certain “voices”, rather than mainly as a democratic outlet for safeguarding “multi-vocality” relative to societal concerns and future public services. Therefore, even though the logic of New Public Governance (NPG) is still prevalent, the instrumental antecedents of the New Public Management (NPM) paradigm may still be apparent: citizens and users are asked to participate in developing future public services, in innovation processes managed by the public sector, nevertheless, the overall potential of deliberation of what constitutes a fair society based on ideas of the common good might be overlooked (Björgvinsson et al., 2012; Hansen & Fuglsang, 2020).
This research study presented how living labs are used to structure open innovation processes aimed at engaging user perspectives in both the indirect and, especially, direct and active co-creation of innovation. Even though living labs viewed as both a format and methodology can be applied as experiments in stakeholder engagement, the evaluative parameters of what constitutes a successful living lab and living lab activities are still underdeveloped. Nevertheless, living labs as an approach to public sector innovation alters the logic of public governance and supports the transition towards interactive and networked governance (New Public Governance), while at the same time disrupting traditional public sector organizations themselves through stressing extrinsic processes of open innovation, which might serve to ensure a more radical approach to user and citizen involvement (Fuglsang & Hansen, 2022).
Previous research has explored how living labs emerge as niches and bear the potential to frame public innovation in radically new ways by creating foundations for policy actions (Dekker et al., 2020; Ruijer & Meijer, 2020; Fuglsang & Hansen 2022). The research presented here extends this previous research through analyzing and specifying practical scenarios for creating future living labs extracted from empirical cases. Besides the conceptual contribution of these patterns of innovation processes, the research also contributes to practice, since the three scenarios provide inspiration and input for establishing living labs, while also identifying points to pay attention to relative to living labs as a legitimizing construct in engaging external stakeholders in the development of future public services. In continuation hereof, a key concern and managerial implication of the scenarios presented are the relational aspects of driving innovation processes based on an open innovation paradigm. This topic also sets the ground for future research on public innovation, which could benefit from applying a multi-actor scenario perspective on such collaborative processes, also integrating citizen/user perspectives. Lastly, further exploration needs to be done on how innovation processes through the living lab framework described at a micro-level scenario can inform innovation policy at a macro-level, both to document existing practices and also outline future potentials.
This project received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 770356. This publication reflects the views of the authors only, and the Agency cannot be held responsible for any use, which may be made of the information contained therein.
Arundel, A., Bloch, C., & Ferguson, B. 2019. Advancing Innovation in the Public Sector: Aligning innovation measurement with policy goals. Research Policy, 48: 789-98. DOI: https://doi.org/10.1016/j.respol.2018.12.001
Arundel, A., & N. Es-Sadki. 2021. Final Report of Survey Results, Deliverable 2.8.EU H2020 Co-VAL. Brussels, Belgium: European Union. http://www.co-val.eu/public-deliverables/
Ballon, P. van Hoed, M., & Schuurman, D. 2018. The Effectiveness of Involving Users in Digital Innovation: Measuring the Impact of Living Labs. Telematics and Informatics, (35): 1201-1214. DOI: https://doi.org/10.1016/j.tele.2018.02.003
Bronson, K., Devkota, R., & V. Nguyen. 2021. Moving Toward Generalizability? A Scoping Review on Measuring the Impact of Living Labs. Sustainability, 13, 502. DOI: https://doi.org/10.3390/su13020502
Criado, J.I., Dias, T.F., Sano, H., Rojas-Martínc, F., Silvanc, A., & Filho, A.I. 2021. Public Innovation and Living Labs in Action: a comparative analysis in post-new public management contexts. International Journal of Public Administration (44): 451-464. DOI: https://doi.org/10.1080/01900692.2020.1729181
Dell'Era, C., & Landoni, P. 2014. Living Lab: A Methodology between User- Centred Design and Participatory Design. Creativity and Innovation Management, 23(2): 137-154. DOI: https://doi.org/10.1111/caim.12061
Dekker, R., Contreras, J.F., & Meijer, A. 2020. The Living Lab as a Methodology for Public Administration Research: a systematic literature review of its applications in the social sciences. International Journal of Public Administration, 43 : 1207-1217. DOI: https://doi.org/10.1080/01900692.2019.1668410
Dekker, R., Geuijen, K., & Oliver, C. 2021. Tensions of Evaluating Innovation in a Living Lab: Moving beyond actionable knowledge production. Evaluation 27(3): 347-363. DOI: https://doi.org/10.1177/1356389021997848
Eriksson, M., Niitamo, V., & Kulkki, S. 2005. State-of-the-art in utilizing living labs approach to user-centric ICT innovation - a European approach. Lulea: Center for Distance-spanning Technology, Lulea University of Technology.
Fuglsang, L. 2008. Capturing the Benefits of Open Innovation in Public Innovation: A case study. International Journal of Services Technology and Management, 9(3/4): 234-48. DOI: https://doi.org/10.1504/ijstm.2008.019705
Fuglsang, L. 2010. Bricolage and Invisible Innovation in Public Service Innovation. Journal of Innovation Economics, 5(1): 67-87. DOI: https://doi.org/10.3917/jie.005.0067
Fuglsang, L., & Hansen, A.V. 2021. Report on participatory stakeholder model. EU-H2020 Co-VAL (770356). Deliverable 5.3.
Fuglsang, L., & Hansen, A.V. 2022. Framing Improvements of Public Innovation in a Living Lab Context: Processual learning, restrained space and democratic engagement. Research Policy, 51(1): 2-12. DOI: https://doi.org/10.1016/j.respol.2021.104390
Fuglsang, L., Hansen. A.V., Rohnebaek, M. & Mergel, I. 2021. Living Labs for Co-Creation of Public Innovation: A literature review and future research agenda, Administrative Science. DOI: https://doi.org/10.3390/admsci11020058
Fuglsang, L., & Hansen, A.V. 2019. Report on cross-country comparison on existing innovation and living labs. EU-H2020 Co-VAL (770356). Deliverable 5.1.
Gascó, M. 2017. Living Labs: Implementing open innovation in the public sector. Government Information Quarterly, 34(1): 90-98. DOI: https://doi.org/10.1016/j.giq.2016.09.003
Gault, F. 2018. Defining and Measuring Innovation in all Sectors of the Economy. Research Policy, 47: 617-22. DOI: https://doi.org/10.1016/j.respol.2018.01.007
Hakkarainen, L., & Hyysalo, S. 2016. The Evolution of Intermediary Activities: broadening the concept of facilitation in living labs. Technology Innovation Management Review, 6: 45- 58. DOI: https://doi.org/10.22215/timreview/960
Hansen, A.V., & Fuglsang, L. 2020. Living Labs as an Innovation Tool for Public Value Creation: Possibilities and Pitfalls. The Innovation Journal: The Public Sector Innovation Journal, 25(3)
Hartley, J. 2005. Innovation in Governance and Public Services: Past and present. Public Money & Management, 25: 27-34.
Hartley, J., Sorensen, E., & Torfing, J. 2013. Collaborative Innovation: A Viable Alternative to Market Competition and Organizational Entrepreneurship. Public Administration Review, 73(6): 821-830. DOI: https://doi.org/10.1111/puar.12136
Leminen, S., Westerlund, M., & Nyström, A.G. 2012. Living Labs as Open-innovation Networks. Technology Innovation Management Review, 2(9): 6-11. DOI: https://timreview.ca/article/602
McGann, M., Blomkamp, E., & Lewis, J.M. 2018. The Rise of Public Sector Innovation Labs: experiments in design thinking for policy. Policy Sciences, 51: 249-267. DOI: https://doi.org/10.1007/s11077-018-9315-7
Mergel, I. 2015. Opening Government: Designing open innovation processes to collaborate with external problem solvers. Social Science Computer Review, 33(5): 599-612. DOI: https://doi.org/10.1177/0894439314560851
Nesti, G. 2017. Living Labs: A new tool for co-production? In Smart and Sustainable Planning for Cities and Regions, Edited by Adrano Bisello, Daniele Vettorat, Rochard Stephens & Pietro Elisei. Cham: Springer: 267-281.
Olejniczak, K., Borkowska-Waszak, S. & Domaradzka-Widła, A.Y. Park. 2020. Policy Labs: the next frontier of policy design and evaluation? Policy & Politics 48: 89-110. DOI: https://doi.org/10.1332/030557319x15579230420108
Osborne, S., Brandsen, T., Mele, V., Nemec, J., van Genugten, M., & Flemig, S. 2020. Risking Innovation. Understanding risk and public service innovation-evidence from a four nation study. Public Money & Management, 40 :52-62 DOI: https://doi.org/10.1080/09540962.2019.1621051
Ruijer, E., & Meijer, A. 2020. Open Government Data as an Innovation Process: Lessons from a Living Lan Experiment. Public Performance and Management Review, 43: 613-635. DOI: https://doi.org/10.1080/15309576.2019.1568884
Schaffers, H., & Turkama, P. 2012. Living Labs for Cross-Border Systemic Innovation. Technology Innovation Management Review, 2(9): 25-30. DOI: https://doi.org/10.22215/timreview605
Schuurman, D., & Tõnurist, P. 2017. Innovation in the Public Sector: Exploring the characteristics and potential of living labs and innovation labs. Technology Innovation Management Review, 7 (1): 7-14. DOI: http://dx.doi.org/10.22215/timreview/1045
Stickdorn, M., & Schneider, J. 2012. This is Service Design Thinking. John Wiley & Sons, Inc. Hoboken, New Jersey
Tõnurist, P., Kattel, R. & Lember, V. 2017. Innovation Labs in the Public Sector: what they are and what they do? Public Management Review, 19: 1455-1479. DOI: https://doi.org/10.1080/14719037.2017.1287939
Torfing, J. 2019. Collaborative Innovation in the Public Sector: the argument. Public Management Review, 21: 1-11. DOI: https://doi.org/10.1080/14719037.2018.1430248
Voorberg, W.H., Bekkers, V.J.J.M., & Tummers, L.G. 2015. A Systematic Review of Co-Creation and Co-Production: Embarking on the social innovation journey. Public Management Review, 17: 1333-1357. DOI: https://doi.org/10.1080/14719037.2014.930505
Wegrich, K. 2019. The Blind Spots of Collaborative Innovation. Public Management Review 21: 12-20. DOI: https://doi.org/10.1080/14719037.2018.1433311
Windrum, P. 2008. Innovation & Entrepreneurship in Public Services. In Windrum, P. and Koch, P. (Eds). Innovation In Public Sector Services: Entrepreneurship, Creativity and Management. Cheltenham and Northampton, MA: Edward Elgar.
Yin, R.K. 2009. Case study research: Design and methods. Los Angeles: Sage.
Keywords: Citizen engagement, co-creation, living labs, Open innovation, Public sector innovation