Follow effective action with quiet reflection. From the quiet reflection will come even more effective action.
Peter Drucker (1909–2005)
Management consultant, educator, and author
Living labs deploy contemporary open and user-centred engagement processes in real-world contexts where all relevant stakeholders are involved and engaged with the endeavour to create and experiment with different innovations. The approach is evidently successful and builds on the perspective that people have a democratic right to have influence over changes that might affect them, such as those brought about by an innovation. In this article, we will reflect on and discuss a case in which end users took part in the development of a method that stimulates learning and adoption of digital innovations in their own homes while testing and interacting with it. The results show that, when end users were stimulated to use the implemented innovation through different explicit assignments, they both increased their understanding of the situation as well as changed their behaviour. Living lab processes are complex and dynamic, and we find that it is essential that a living lab have the capability to adjust its roles and actions. We argue that being reflective is beneficial for innovation process managers in living labs because it allows them to adjust processes in response to dynamic circumstances.
The living lab approach builds, first and foremost, on the position that end users, or people being affected by the technology, are the only real experts regarding their own contexts, goals, and activities and therefore it yields important insights that are beneficial to innovation processes (Leminen & Westerlund, 2016; Schuurman et al., 2016). This approach is evidently successful and has resulted in new product features, new value propositions, and identification of bugs in systems; but, more importantly, it has enabled profound understanding of use contexts and the real-life benefits of innovations (Hakkarainen & Hyysalo, 2013; Ståhlbröst, 2013). It builds on the perspective that people have a democratic right to influence changes that might affect them as a result of an innovation (Bergvall-Kåreborn et al., 2014). The living lab approach is based on the notion that co-creative innovation processes are more effective and result in innovations that create value for their intended end users (Krogstie et al., 2013).
Earlier research has also shown that user participation positively contributes to the success of digital systems, especially in technically complex system (Lin & Shao, 2000). However, involving end users is not always a straight forward process because companies, at times, are reluctant to challenge their existing mode of operation even though they are faced with valuable input from end users (Hyysalo et al., 2016). In innovation processes, it is also rather common that different perceptions about the form and function of innovations lead to tensions and conflicts between stakeholders (Hakkarainen & Hyysalo, 2013). Adding to that, engineering as a field usually lacks the expertise on how to deal with unstructured human situations, and they have limited traditions in understanding the social context that is necessary to shape a digital innovation from a socio-technical perspective (Bilandzic & Venable, 2011). Hence, developing an innovation is not only a process of interaction and a struggle to develop “the right thing”, it is also a process of understanding, learning, and sharing among the involved stakeholders.
So how can living labs ensure that the engaged end users have the opportunity to make their voice heard and influence innovation processes? And how can a living lab create a process that enhances learning and dialogue between end users and other stakeholders (e.g., problem owners, developers, or living lab practitioners) so that their interaction becomes fruitful for all stakeholders? Innovation process managers in living labs strive to balance the power relations between the different stakeholders– including developers, managers, policymakers, citizens, end users, and affectees – by helping them to both listen and be heard in innovation processes and by enabling them to have an actual impact on the innovation being developed (Bergvall-Kåreborn et al., 2015). To be able to do this, the potential end users or affectees need to be well equipped and have a sound understanding of the digital innovation being elaborated and tested; otherwise, their input may be of a general character or be based on (understandably) naïve assumptions and therefore may fail to have an actual impact. Hence, the aim of the research presented in this article is to reflect on and discuss a case in which end users took part in developing a method that stimulates learning and adoption of digital innovations in their own context while testing and interacting with it.
An Approach to Living Lab Research and Innovation
In research-oriented living labs, it is important to make a distinction between the innovation process and the research process even though they might be interwoven and hard to separate at times. In this article we draw upon our experiences with a particular case of a research-oriented living lab project, Apollon, which will be described in greater detail below. In the Apollon project, the research purpose was to develop a method that stimulated learning and adoption of innovations in the end users’ private homes. The innovation process was focused on end users testing and providing insights related to the digital innovations they tested, for instance usability and usefulness issues (Ståhlbröst & Holst, 2016). In this article, we will report on the research process by reflecting on and discussing the effectiveness of the method being developed.
We have applied an action design research approach with the aim of learning from and contributing to both practice and research while being guided by the notion of reflection in action (Schön, 1991) and by participatory action design research (Bilandzic & Venable, 2011). Being a reflective researcher requires a commitment to learn from experience and evidence, rather than to learn a predetermined path of actions (McMahon, 1999). In living lab research, one vital approach is to be open to what is happening in the context and to adjust the process accordingly. It is therefore important that living lab researchers apply a reflection-in-action approach to their research and innovation processes. Related to the action design research approach, our aim in the case being referred to in this article was to test, evaluate, and re-design two digital innovations while informing theory (Sein et al., 2011). In living lab research processes, it is important to distinguish between the innovation process and the research process of reflecting and learning, where the former is related to the innovation as such while the latter is related to the research process and formalizing learning. Hence, living lab research needs to, in the same vein as action design research (Sein et al., 2011), relate problems arising in practice to classes of problems identified in theory and express the learnings in generalized outcomes. In this process, generalizations can be made on three levels (Sein et al., 2011): i) generalization of the problem instance, ii) generalization of the innovation instance, and iii) derivation of design principles for the type of innovation. Hence, the reflection and learning process extends from focusing on building a solution to applying learning to a broader research problem area.
In the case reported on in this article, we carried out continuous reflections on the interaction with the end users, the suitability of the used interaction and stimulation method, as well as the digital innovations´ inherent functionalities and its influence and suitability to contribute to end users’ objectives. This process was designed to: i) increase knowledge on the design of end-user interaction processes that foster learning and understanding (i.e., the research) and ii) gain insights into problems related to the innovations as such (i.e., the innovation). In the reflective process, theoretical knowledge was applied onto the practical situation based on the researchers’ focus, which in our case has been the use of assignments for end users to carry out in their context to stimulate both their learning and adoption of the innovation. Being a reflective practitioner means that the researcher approaches a practical situation as an exclusive state (Schön, 1991) In this approach, the researcher uses their prior experiences as a basis for their actions with the objective to discover and understand the unique characters of the situation and, based on that, defines the scope and process of the research actions. Based on the shaped problem, the researcher conducts experiments with the aim of discovering related effects and challenges. Being a reflective researcher and practitioner also means that unintended changes can be produced in the situation, which might give it new meanings. Hence, there is a constant interaction and reframing between the practitioner and the situation at hand going through the stages of appreciation, action, and re-appreciation. Hence, the understanding of the situation grows as we strive to change it, and it is then changed through our attempt to understand it (Heiskanen & Newman, 1997).
In our case, we also wanted the end users to adopt the technology. In general, technology adoption is a multidimensional process where an individual’s behaviour is influenced by a variety of conditions. These conditions can be learning, social, and technological (MacVaugh & Schiavone, 2010). Learning conditions are individual characteristics of a single user and can be expected to influence the attainment of new competencies needed to use the new technology. Social conditions explain the cultural and relational specifies shared within the communities to which the user belongs. Technological conditions facilitate the explanation of technical features of the exchanging technology (MacVaugh & Schiavone, 2010). Naturally, the importance of each of these conditions differs depending on the context in which the innovation is intended to be used.
The Living Lab Case: Apollon
The living lab case referred to in this article was carried out as part of the Apollon project, which was financed by the European Commission. The case took place between December 2010 and February 2012. Today, the process of facilitating learning and adoption among end users in living lab processes remains an important and unsolved issue that may benefit from reflection on an established case. In addition, many living lab studies are carried out for a shorter period of time, usually a few weeks; hence, reflections and lessons learned from more longitudinal studies are important contributions to the area of living lab research. We have also applied the approach in more recent energy-saving projects, such as the Cassandra project carried out from 2012 to 2014 (Runardotter & Holst, 2014).
The living lab process used in the Apollon project was designed to stimulate knowledge creation and adoption of energy visualization technologies by users by providing them with assignments to carry out in their homes while they familiarized themselves with the technology and increased their understanding of different energy-saving approaches that they could apply in their homes. For this case, we recruited 20 households interested in testing energy visualization technologies while contributing to our research efforts. Ten of them tested a visualization technology called SABER, which measured and visualized real-time consumption of district heating, electricity, and cold and warm water over the course of seven months. The other ten tested a visualization technology called ELIQ, which measured and visualized real-time electricity consumption, again over seven months.
In this case, we applied a rather traditional living lab approach that focused on real-world tests of innovations with end users in their context. The innovations were quite mature and testing was possible in the end users’ homes. In the follow up interviews, we did however add formative questions to find possible suggestions for improvements of the digital innovations. Hence, the innovation process did not focus on co-creation of innovations; rather the co-creative activities were related to the research process, such as the use of assignments and questions to increase the participants’ understanding and knowledge.
After being recruited, the end users were given instructions and support in installing the digital innovations in their homes, which meant that we interacted with the end users in their homes and we used their input and reactions to continuously reflect on and redesign our study and to receive input to re-design the final solution. To support the test process, we wanted to have continuous and controlled interaction with the end users, so we developed a “test storyline”, that is a detailed step-by-step process consisting of seven assignments and questions with clear instruction regarding how and when the assignments should be carried out. Each assignment was designed to be instructive and enable learning about the situation at hand, in this case energy consumption in the families who tested the technologies. When an assignment was to be carried out, a link to an online survey with clear instructions and fields where the end users could fill in their answers was sent to the end users by e-mail. From a research perspective, we wanted to learn how the assignments could be designed to be undertaken by end users on their own. We designed the assignments into micro-tasks that were functionality driven, small, well defined, and easy to perform. The goal was to stimulate usage of the different functions that each digital innovation offered. For example, an assignment could be formulated as follows:
- For five days before installing your Christmas lights, make daily meter readings and enter the summarized readings in the designated field in the survey.
- For five days after installing your Christmas lights, make daily meter readings and enter the summarized readings in the designated field in the survey. Turn your lights on or off as you wish during this period.
- Then, install and configure timers to turn off your lights at night or when you are way from home. For five days after setting up the timers, make daily meter readings and enter the summarized readings in the designated field in the survey.
These assignments were then followed by questions related to the effect on the end user’s consumption, such as the difference before and after installing the lights or the timers. We asked questions related to their experiences and thoughts about a given assignment, and whether they expected or were surprised by the results. We also asked whether they would consider making a change in behaviour based on what they learned. All the answers were gathered in the online survey. The results from the assignments showed that the end users testing the ELIQ technology had difficulties detecting meaningful differences in consumption because the outdoor temperature also influenced their electricity consumption during the days they made measurements. For example, on colder days, they naturally used more electricity to heat the house. This confounding factor made them reflect on what the ELIQ actually showed them and what conclusions they could draw from the visualization.
Focus group interview with test users
As a closing activity, we invited end users from all 20 of the case households to a group interview. The purpose was to support reflection on action regarding their experiences of the test and to facilitate learning from the process while at the same time receiving suggestions for re-designing the digital innovations. Seven end users joined the meeting and we interacted with them for almost two hours. Unfortunately, the event lacked gender diversity: all seven attendees were male. Most of the participants showed significant interest in energy and environmental issues, and some reported having these interest since childhood. The end users’ joint interest in the topic made us reflect on user-recruitment criteria in cases that extend over a long period of time. Here, it is beneficial for living lab cases if the end users have a solid interest in the application area of the innovation because it can help keep them engaged through the whole test period and while carrying out assignments.
Another interesting reflection from this case was that the end users all reported having joined this living lab case quite spontaneously without involving their family in the decision to participate. This unilateral decision became a problem for them during the test period: the rest of their family were not as engaged in energy-saving activities, which led to a situation where the family fathers had to spend a lot of energy convincing their family members to make an effort and contribute to the test and energy savings in the household. Hence, taking part in real-world tests of innovations in the home might not be a one-person effort: it impacts the family as a whole, that is, the social system.
During the interview, we openly discussed the end users’ experiences of being involved in this test process. In these discussions, the end users identified aspects related to the technology and its usability, such as problems in reading the visualization meter and knowing how much each appliance was consuming, for instance, when different electrical radiators turned on or off, or when the freezer turned on or off automatically. Hence, it was difficult for them to know if the difference in consumption was based on their effort in the short term or if it only was based on their appliances turning on and off in a different way. But, even if this was a problem, the end users stated that they learned a lot from the different assignments, from the reflections they had to do when performing them, and in answering the follow up questions.
The interviewed end users also stated that the assignments given to them in the test storyline made them reflect on their energy consumption in new ways, which in turn stimulated them to take actions to change their energy consumption. For example, most of them invested in low-energy lamps or LED lamps. Furthermore, they developed tendencies to turn off lamps when leaving a room, to connect appliances with transformers to sockets that can be turned off, and to using timers on some of their appliances.
Using the test storyline supported the living lab process of conducting long-term tests involving end users in their private context, where researchers have little or no control over the actions being carried out. The test storyline also supported the communication between the living lab and end users to make sure that they understood what was required from them during the test they volunteered to take part in so that they would not get any unwanted surprises and could be in control. The test storyline as a tool also ensured that end users developed a solid understanding of the innovation because it encouraged them to test all functionalities of the digital innovation while, at the same time, it stimulated learning and increased the competence of end users. Through this approach, the end users acquired knowledge on both energy-saving opportunities, they increased their level of competence in using the digital innovation, they provided competent feedback related to the design of the innovations, and they gave competent feedback on the living lab approach with the test storyline.
Reflecting on the Living Lab Approach
The aim of this article was to reflect on and discuss a living lab approach applied in a case aiming to stimulate understanding and adoption of digital innovations among end users in their homes. In this case, we implemented a test storyline consisting of a structured interaction plan with seven instructive assignments for the end users to carry out during the seven-month test period. The results from this case indicate that giving the end users assignments to carry out during the test was a successful way to stimulate use, increase learning, and foster understanding about the research area, in this case energy saving. Reflecting on what other field research methods, such as ethnography, might have been suitable due to the long-term perspective of the case, which continued over a seven-month period. A classic ethnographic study usually requires six months to two years (or more) in the field for the researchers to become acquainted with the context, the culture, and language within the context and its basic structure (Fetterman, 2010). In our case, this approach would have been too time consuming and would have posed problems with the extent of access to the end users’ homes that would have been needed to obtain the required insights. Ethnographic studies usually collect data through observation (Fetterman, 2010), which in living lab cases becomes challenging due to the aim of involving and understanding a group of individual end users simultaneously. In addition, ethnographic studies do not have a design focus; rather, they focus on understanding, describing, and capturing social and cultural phenomena that might embody aspects relevant to designers (Bilandzic & Venable, 2011). Hence, living lab research has many similarities with ethnographic methods. But, researchers in living labs do not merely want to understand and observe a phenomenon; rather, their aim is to actively change a situation by implementing a digital innovation and by empowering end users with different means.
The results from this study also show that, when the end users were prompted to use the implemented innovation through different explicit assignments, they both increased their understanding of the situation (e.g., energy saving), as well as changed their energy consumption behaviour (e.g., lowered their energy consumption by approximately 10% over a one-year period). When it comes to energy consumption behaviour, it is not trivial to determine what the changes are dependent on in this living lab case. When it comes to end users changing their energy consumption behaviour, we know that feedback and visualization technologies are a positive influence (Darby, 2000; Seligman & Darley, 1977). Hence, in this research, we can only refer to the end users’ testimonies that the performance of assignments educated and influenced them, not only the digital innovations (i.e., feedback and visualization technologies) as such. In addition, in this case, we measured energy consumption over a full year, which is a longer period than the specific seven month living lab case. The reason for this was to obtain comparable data between years. This data analysis accounted for variations in outdoor temperature and wind conditions based on the established degree-day calculation method. Hence, we can conclude that the reduced energy consumption by end users reflects underlying changes in behaviour that are likely to persist over the long term.
Reflecting on the process of engagement in the case also raises the question of whether these results would had been reached if the tests had been carried out in a controlled laboratory setting. In this context, it would have been possible to observe how the end users interact with the technology and they could have given inputs on potential design changes of the digital innovations. In such a situation, the end users are providing feedback based on their immediate and intuitive understanding and thoughts related to the innovations as such, not based on their true use experiences. It has been argued in previous research (Yoo, 2010), that end users find it difficult to give relevant feedback on a digital innovation if they have no experience of actually using it. In addition, in a laboratory setting, it would not have been possible to study changes in behaviour related to actual use of the artefact. Hence, in processes where the aim is to reach a deep level of understanding and insights based on real-world experiences, and when a changed behaviour is a desired outcome, a living lab approach is useful.
In addition, the use of the test storyline that stimulated use of the digital innovation also helped the end user experiment with the digital innovation in a safe and comfortable context. Reflecting on this approach, we conclude that end users feel safe being guided by assignments in their homes; the guideance encouraged them to explore and be more acquainted with the innovation without anyone observing them and measuring their performance, which is a common approach in user tests in laboratory settings. Hence, the real-world context with educational assignments did strengthen the confidence of end users with the innovation and thus contributed to end users being better equipped to give valuable feedback on potential re-designs of the innovation. The end users also became aware of the innovations´ suitability for their specific context and goals, and this awareness contributed to their adoption of the artefact in relation to feelings of compatibility. Thus, they knew how to use the innovation and they understood how it answered their needs and existing values (Rogers, 1983). Another key aspect of the test storyline was that it regularly reminded the end users about the technology and encouraged them to use it. Hence, the likelihood for actual adoption of the technology increased as the innovation became a part of their social system.
Reflecting on the assignments reveals some drawbacks that need to be addressed in future studies. For instance, it can be difficult to determine whether the participants involved in the test actually performed the assignments even though they report having done so. To minimize the risk of false reporting, we strived to give them assignments where they had to enter values into the online survey, such as the exact consumption of electricity (KWh) over a specific period. Even so, it was difficult to know whether they had actually done it. Here we can see that research methods such as ethnography (Hammersley & Atkinson, 2007) could have increased the validity of the study. Another drawback with the living lab approach with test storylines is the scalability. In the experiment being studied in this research, we engaged 20 end users, which made it possible to reflect on their feedback during the design of assignments for the next period and co-creatively design them. We found that, even though the scale of the case was rather small, the findings and input gave us deep insights and understanding of the end users’ situation and the challenges they faced during the test period. For instance, the end users could send an email when the digital innovation did not really work properly and they could be given technical support before continuing the test.
Living lab processes are complex, involving multiple stakeholders in real-life contexts. This complexity leads to processes that are difficult to predict given that the innovation, people, and challenges might move in new directions. Consequently, it is essential that a living lab has the capability to adjust their roles and actions accordingly (Hakkarainen & Hyysalo, 2016). We argue that living lab approaches benefit from applying a reflective stance. Being reflective is beneficial for innovation process managers in living labs because it enables them to adjust processes in response to changing circumstances. Living lab researchers benefits from reflective practices through their contributions to learning and theory. Furthermore, end users involved in living lab research and innovation processes also benefit from a reflective approach, both from the perspective that they are involved in an innovation process that fits into their everyday practices, but also because they can reflect on their own knowledge creation and learning from their involvement.
We thank the European Commission for funding the Apollon project, without which this research would not have been possible. We also thank the European Commission for funding the Privacy Flag project and the U4IoT project that funds the extended research on the aspects of living lab and crowd engagement in innovation processes, which contributed to this article. Furthermore, we thank the end users who participated in the pilots and the companies who made it possible: your input and efforts are of great value for our research.
Bergvall-Kåreborn, B., Howcroft, D., & Ståhlbröst, A. 2014. Disregarding History: Contemporary IS Contexts and Participatory Design. Communication of the Association for Information Systems, 34, Article 68: 1319–1332.
Bergvall-Kåreborn, B., Melander-Wikman, A., Ihlström Eriksson, C., & Ståhlbröst, A. 2015. A Model for Reflective Participatory Design: The Role of Participation, Voice and Space. Paper presented at the 21st Americas Conference on Information Systems (AMCIS), August 13–15, Puerto Rico.
Bilandzic, M., & Venable, J. 2011. Towards Participatory Action Design Research: Adapting Action Research and Design Science Research Methods for Urban Informatics. The Journal of Community Informatics, 7(3).
Darby, S. 2000. Making it Obvious: Designing Feedback into Energy Consumption. Paper presented at the 2nd conference on Energy Efficiency in Household Appliances and Lighting, Italian Association of Energy Economists/EC-SAVE Programme, September 27–29, Naples, Italy.
Fetterman, D. M. 2010. Ethnography: Step-by-Step (3rd ed.). Thousand Oaks, CA: Sage Publications, Inc.
Hakkarainen, L., & Hyysalo, S. 2013. How Do We Keep the Living Laboratory Alive? Learning and Conflicts in Living Lab Collaboration. Technology Innovation Management Review, 3(12): 16–22.
Hakkarainen, L., & Hyysalo, S. 2016. The Evolution of Intermediary Activities: Broadening the Concept of Facilitation in Living Labs. Technology Innovation Management Review, 6(1): 45–58.
Hammersley, M., & Atkinson, P. 2007. Ethnography: Principles in Practice. London: Routledge.
Heiskanen, A., & Newman, M. 1997. Bridging the Gap between Information Systems Research and Practice: The Reflective Practitioner as a Researcher. In Proceedings of the 18th International Conference on Information Systems: 121–132. Atlanta, GA: Association for Information Systems.
Hyysalo, S., Repo, P., Timonen, P., Hakkarainen, L., & Heiskanen, E. 2016. Diversity and Change of User Driven Innovation Modes in Companies. International Journal of Innovation Management, 20(02): 1650023.
Krogstie, J., Ståhlbröst, A., Holst, M., Gudmundsdottir, A., Olesen, A., Braskus, L., Jelle, T., & Kulseng, L. 2013. Using a Living Lab Methodology for Developing an Energy Savings Solutions. Paper presented at the 19th Americas Conference on Information Systems, August 15–17, Chicago, IL.
Leminen, S., & Westerlund, M. 2016. A Framework for Understanding the Different Research Avenues of Living Labs. International Journal of Technology Marketing, 11(4): 399–420.
Lin, W. T., & Shao, B. B. M. 2000. The Relationship between User Participation and System Success: A Simultaneous Contingency Approach. Information & Management, 37(6): 283–295.
MacVaugh, J., & Schiavone, F. 2010. Limits to the Diffusion of Innovation: A Literature Review and Integrative Model. European Journal of Innovation Management, 13(2): 197–221.
McMahon, T. 1999. Is Reflective Practice Synonymous with Action Research? Educational Action Research, 7(1): 163–169.
Rogers, E. M. 1983. Diffusion of Innovation (3rd ed.). New York: The Free Press.
Runardotter, M., & Holst, M. 2014. Views on Energy Efficiency: Findings from the CASSANDRA Project. Paper presented at the 20th International ICE Conference on Engineering, Technology and Innovation (ICE 2014), June 23–25, Bergamo, Italy.
Schuurman, D., Baccarne, B., Marez, L. D., Veeckman, C., & Ballon, P. 2016. Living Labs as Open Innovation Systems for Knowledge Exchange: Solutions for Sustainable Innovation Development. International Journal of Business Innovation and Research, 10(2-3): 322–340.
Schön, D. 1991. The Reflective Practitioner: How Professionals Think in Action. London, UK: Ashgate Publishing Limited.
Sein, M., K, Henfridsson, O., Purao, S., Rossi, M., & Lindgren, R. 2011. Action Design Research. MIS Quarterly, 35(1): 37–56.
Seligman, C., & Darley, J. 1977. Feedback as a Means of Decreasing Residential Energy Consumption. Journal of Applied Psychology, 62(4): 363–368.
Ståhlbröst, A. 2013. A Living Lab as a Service: Creating Value for Micro-enterprises through Collaboration and Innovation. Technology Innovation Management Review, 3(11): 37–42.
Ståhlbröst, A., & Holst, M. 2016. Living Lab: Stimulating Adoption of Smart City Innovations. In OpenLivingLabs Days 2016: Research Day Conference Proceedings: 145–162. Brussels: European Network of Living Labs (ENoLL).
Yoo, Y. 2010. Computing in Everyday life: A Call for Research on Experiential Computing. MIS Quarterly, 34(2): 213–231.
Keywords: action research, Apollon, context, digital innovations, end users, innovation process, Living lab, research process