The great secret that all old people share is that you really haven't changed in seventy or eighty years. Your body changes, but you don't change at all. And that, of course, causes great confusion.
Doris Lessing (1919–2013)
Writer and Nobel Laureate in Literature (2007)
In this article, we focus on living lab methods that support the elicitation of emotions – a key success factor in whether a design solution will be accepted and taken up over the long term. We demonstrate the use of emotional goal models to help understand what is relevant for a target user group in the early phases of design. We promote animations and storyboards to envision the context of use and to gain an understanding of how design ideas can integrate into people’s lives. For the evaluation of ideas and to further understand user needs, we show how technology probes facilitate natural interactions with a suggested solution concept. All methods have in common that they enable older adults without design or development experience to participate in the design process and work towards a meaningful solution by helping to communicate feelings and goals that are often hard to define. Lastly, we present a process model that demonstrates our emotion-led design toolkit at various phases of a living lab process.
Each design and research process consists of varied methods that are fundamental to realizing user goals. In this article, we demonstrate how our methods can be used within three generic living lab design phases: exploration, experimentation, and evaluation (Schuurman et al., 2016). We apply our methods in a living lab project to give older adults a strong voice to share and describe their experiences and emotions and to explore how these insights can be captured for design purposes. The project objective was to develop an innovative personal emergency alarm that evokes positive emotions in older adults and reduces the feeling of “being monitored”. We ask the question:
“What methods cater for the goals and emotions of older adults in a co-design process to develop innovative solutions?”
We propose an emotion-led design toolkit with several artefacts: motivational goal models, animations, and technology probes. We argue that using these artefacts at different phases of a living lab project cycle facilitates effective communication between participant stakeholders and contributes to both innovation and service design methodologies. Service design and user-driven design methods are increasingly important aspects of living labs, recognized as two means of increasing user acceptance of innovations (e.g., the FormIT methodology [Ståhlbröst &Holst, 2013] and citizen-driven innovation [Eskelinen et al., 2015; Gray et al., 2014]. This view is in alignment with Muller (2007) who argued a decade ago that user engagement is too often one-directional, creating applications of technology rather than solutions to user problems.
The following sections report on emotions in designing for health, followed by a review of existing living lab processes. We show how our emotion-led design toolkit can support the phases of a typical living lab process, demonstrated through a case study of personal alarm systems.
Emotions in Designing for Health
In the discipline of design, emotions influence decisions about the look and feel of products and services. For a personal topic such as health, people’s emotions play a major role in the success of a technology, and they afford an opportunity to increase compliance (Lo Bianco et al., 2015).
Yet, design of systems in the domain of health services is still functionality-driven rather than emotion-driven, particularly when institutions, as main stakeholders, fulfill government policies and focus on compliance and liability towards patients rather than patients’ feelings. Here, we complement functional-driven design with users’ desired emotions in order to develop innovative products with a high uptake (see Figure 1).
Figure 1. Comparison model showing the difference between a standard process and an emotion-led design process with emphasis on understanding users’ emotions
Personal alarm systems are an example of technology that has high impact potential but neglects the emotional needs of older people (Miller et al., 2015). Personal alarm systems typically have two features: i) a wearable personal device – the user can raise an alarm if they require emergency attention, for example by pushing a button on a wristband or a pendant worn around the user's neck; and ii) a wellbeing check – the user informs the service provider that they are fine, usually on a daily basis, for example, by pushing a button on a base station connected to a telephone line. In the second case, if no indication of wellbeing is received during the specified period, the service provider initiates checks on the user (Pedell et al., 2014). In this article, the term “personal alarm system” will be used to describe both features: the wellbeing check and the personal device.
The Living Lab Phases: Exploration, Experimentation, and Evaluation
Many studies have noted the importance of real-life contexts and the involvement of end users in living lab innovation processes (Almirall et al., 2012; Leminen, 2015; Veeckman et al., 2013). The end users of potential innovations are seen as “co-creators” (Veeckman et al., 2013) in the innovation process rather than subjects of study. Dell'Era and Landoni (2014) highlight the importance of the research-led aspect of living labs while also emphasizing the importance of users as active co-creators and the real-life context as a factor that modifies the users’ needs.
Although individual living labs have different overall approaches (Almirall et al., 2012), there are many similarities. The FormIT methodology (Ståhlbröst & Holst, 2013) illustrates a typical approach, which emphasizes user involvement in the innovation lifecycle from ideation through to eventual commercialization. Schuurman and colleagues (2016) have identified three generic phases that are common to many living labs: exploration (idea/concept), experimentation (prototyping), and evaluation (pre-launch, launch, and post-launch). Their study also shows that the more closely a living lab approach follows this “ideal” approach, including multi-method user involvement, the greater the positive impact on the final outcome. Leminen and colleagues (2015) note four different user roles within a living lab – informant, tester, contributor, and co-creator – although they indicate that each user can perform multiple roles. The informant contributes an understanding about the users’ life, problems, and needs. The tester evaluates innovations in the environment. The contributor collaborates with the other stakeholders in the development of a service. Finally, the co-creator creates and develops actual solutions with the other stakeholders. Leminen and colleagues (2015) show that the first three roles are most common in living lab projects. Co-creative methods have been established in research as “people who are not professional technology designers may not be able to define what they want from a design process, without knowing what is possible. A process of mutual learning for both designers and users can inform all participants' capacities to envisage future technologies and the practices in which they can be embedded” (Robertson & Simonsen, 2012). However, recent literature has recognized the challenge of actively engaging older adults in design processes and have come up with methods to do this (Edlin-White et al., 2012; Lindsay et al., 2012; Vines et al., 2012; Waycott et al., 2012; Waycott et al., 2013). What we contribute here is a means to specifically integrate emotions in a co-creative living lab process. Table 1 summarizes our emotion-led methods, aligned with the three typical phases of living lab methodologies identified by Schuurman and colleagues (2016).
Table 1. A process table outlining the phases in which emotion-led methods are employed during a standard living lab process, in context of a personal alarm living lab project
Phase 1: Exploration
Our case study focuses on the use of methods to enable the exploration and evaluation of emotions with older adults around personal alarm systems and their wider context of use as a means of innovating both the function and the service offering. In the conceptual phase, we report on the development of an early goal model, with emotions captured from initial exploration and in-depth interviews where the users are operating in the role of an informant.
Participants, data collection, and analysis
Twelve in-depth interviews were conducted, categorized into three groups: i) older people who lived alone (with one exception) and who either currently have or previously have had a personal alarm system installed in their home; ii) family members of older adults who either currently have or previously have had a personal alarm system installed into their home; and iii) older people who never have had a personal alarm system installed in their home. The interviewees in the first group were older than the interviewees in the last group: those who had experience with personal alarm systems ranged from 85 to 91 years of age, whereas those who did not have experience with personal alarm systems ranged from 66 to 79 years of age.
The interviews explored three key questions: What should an alarm technology do (functions)? How should it be (qualities)? and How should it feel (emotional response)? We transcribed the data using content analysis according to Patton (2002) and derived common themes from the data.
Results: Emotions around personal alarm system use
Our interviews revealed that some older people perceived that their feelings were not being taken into consideration. They viewed the wearable pendants as “cowbells” forced onto them:
“She always would joke about her cowbell, and complain about it. ‘Look at what my kids are making me do,’ kind of comment, a slight resentfulness about it. And it was kind of against her independence.” [Participating relative]
The pendants were perceived by the wearers as having a “stigma” attached to them – a perception that others believe the wearers are no longer are able to care for themselves, resulting in the pendants not being worn. The considerations and social environment around the decision of whether or not to wear the pendant were complex and loaded with emotions. The complexity is expressed well by the nephew of one of the users:
“So we did have the discussion and she sort of admitted that she didn't want to wear it and she didn't think that she should and she understood the risks and she was prepared to take the risks and that she didn't want to upset me and she didn't want me to feel like she wasn't cooperating with me. And so she said [mimicked aunts voice] 'so at least I wore it some of the time'. You know these times when she was wearing it was when someone was there and she didn't really need it. But for her, that was her compromise.” [Participating relative]
It became clear that personal alarm systems need to consider more than just safety aspects. The pendant limits mobility in that the alarm only works in the owner's house, so that the wearer might be hesitant to leave their home. Interestingly, most pendants have an effective range of 300 metres, but despite the strong feeling of being confined to a small space, older users are told that the maximum range is about only 70 metres in order to better pinpoint a wearer’s location in an emergency. The limitations this information poses to the older person in their everyday life apparently are not considered by service providers.
The wellbeing check (the second component of the system) requires the user to remember to push a button each day. Otherwise, the service provider calls to check upon the client, which leads many older people to feel they are a burden, despite paying for the service. Others feel that they are perceived by their families as suffering from memory loss:
“And no matter what system I try [claps with hand on his knee in frustration and enforcement several times] I still manage out of 10 days that I miss out 2 or 3 times by completely forgetting and that is what ANNOYS [emphasis] me.” [Older user]
Pressing the button on the wellbeing check base station does not convey any meaning to the older person and is therefore forgotten. Additionally, the wellbeing check provides no feedback indicating whether the button has been pressed on a particular day. Pressing the button on the wellbeing check a second time on the same day initiates an inquiry to the service providers and is perceived as a signal of an emergency. Hence some older people do not feel confident using the system. Further, the wellbeing check is not easily configurable, for example, users cannot adjust the time of day when the wellbeing button should be pushed, which leaves users feeling that they are not in control of the system.
The older people we interviewed indicated a desire for the personal alarm system, in particular the wellbeing check, to evoke feelings of independence, safety, being in touch with other people, control, and integration. Most importantly, they wanted to feel cared about. Table 2 provides an overview of the captured emotions.
Table 2. Overview of current and preferred emotions surrounding personal alarm systems
Based on the emotions we captured, we integrated the emotional goals into a motivational goal model according to the notation of Sterling and Taveter (2009), extended by Marshall (2014), as shown in Figure 2. In this model, emotions (hearts) are attached together with desired qualities (cloud shapes) to functional goals (parallelograms). The emotions were used as high-level specifications in the following phases of the design process to develop a prototype and the final design of the personal alarm system.
In order to ensure a smooth transition from the exploration to the experimentation stage, we suggest that the outcomes of the exploration phase, namely the emotional goal model, should be validated in its context of use with the prospective end users. The goal models are intended as high-level specifications for designers and developers to create a prototype.
Figure 2. Mapping functional, quality, and emotional goals with stakeholder roles and system interactions
Phase 2: Experimentation
In technology development, we face the challenge of anticipating how technology will be adopted and integrated into people’s lives. Technology itself changes our lives — how we perceive and handle situations (described by Carroll and Rosson (1992) as the task-artefact-cycle). Although user-centered design and experience design (Buxton, 2007) help us envision future use by better understanding users’ lives, designers still face a problem of validating future use scenarios before creating a solution. Future users themselves often have no clear understanding of the implications new technologies will have on their lives and are thus limited in giving input into design decisions. When changing existing situations into preferred ones (Simon, 1982), a designer with input from end users should strive to understand contexts, issues, relationships, environments, and emotions where a design problem is situated. Scenarios one of many useful tools in a designer’s toolkit (Loke et al., 2005) and are used to understand the complexities of the design context (Iacucci et al., 2002). Here, we use scenarios, or imagined stories of events, during the experimentation phase as means to explore design options, anticipate future problems, and describe contexts of user experiences with products (Lim & Sato, 2005; Mathews & Heinemann, 2012). We therefore develop a new approach to animate current and future use scenarios with emphasis placed on the early emotional goal model described in the exploration phase. In doing so, we aim to envision and visualize future technology use. We suggest that animated scenarios can be used as a tool to co-evaluate earlier insights of user research with participants, particularly surrounding sensitive issues such as feelings and personal life goals. We also expect that, by co-evaluating insights from the conceptual phase using animated scenarios, participants can express their own emotions using personas and those emotions can be better expressed in animations rather than written or sketched scenarios.
Scenario design: Creating the animations
We created three animations (Figure 3) based on the existing interview data, including emotions around personal alarm system use, from the exploration phase. These animations were used to co-evaluate the goal model and validate the barriers and reasons older adults do not use current personal alarm systems.
Figure 3. Screen captures of animated scenarios. Left: “I forgot”, middle: “Cow Bell”, right: “Dress Code”
Our three aims for the use of the animations for co-evaluation as a means to explore future scenarios were:
- To determine whether the problems presented in the scenarios were identified and interpreted in similar ways by the participants reflecting on their own situation.
- To determine if the animated scenarios reflected a realistic story (context) of a user involved with personal alarm systems, with particular focus on both operating the pendant and conducting daily wellbeing checks.
- To encourage participant feedback to help redesign a new personal alarm system once a shared understanding about the relevance of the scenarios was established.
In a co-design workshop with four older people, we discussed these animations. According to Massimi and colleagues (2007), a participant number of four was considered to be suitable due to the personal nature of the topic, creating a familiar environment of “having tea together”. People with and without alarm pendants discussed the scenario in pairs, including feelings about the personas, intervention points in the scenario, and design ideas to improve feelings and living situation of the people depicted in the scenarios. In this case, users act as both informants, who correct understanding of the situation, and contributors, who collaborate to develop the service design.
Results using animated scenarios and storyboards
When shown the animations, participants were engaged with the plot of the story. This engagement became particularly clear after participants commented on the animations after viewing them, because they related feelings of the animated personas to their own life situations. We confirmed that the three scenarios and the depicted emotions were perceived as realistic and something the participants could relate to in their own lives, but we were also able to create an atmosphere of openness that provided a foundation for engaging the participants in co-creative design activities. Using printed storyboards (Figure 4), the participants identified and commented on aspects of the animations in which personas needed to be better understood by their relatives and service providers with implications for design.
Figure 4. An example of an animation storyboard with annotations from workshop participants
Design ideas were directly put into the context of the scenario. The ideas were adjusted until the scenario reflected a true reality for a personal alarm user, as shown by the annotations in Figure 4. For example, the size of the pendant was not problematic for participants, but merely its appearance. Figure 5 shows an example of one of the ideas generated in the workshop: the pendants could be redesigned to be worn as a piece of jewellery.
Figure 5. A user wearing an alarm pendant (left) and a workshop participant showing an item of her own jewellery as a pendant redesign idea (right)
We found that the animated scenarios helped us to better include older people in the decision and design processes and to validate some of the imagined everyday scenarios of use. The older adults’ emotions and their motivations were key in this process. Animations are well suited to expressing such emotions. The storyboards were a good way to capture comments and ideas generated by these animations.
Phase 3: Evaluation
Probes are particularly suited to investigating people’s everyday life in situations difficult to reach with traditional social science methods such as questionnaires, interviews, focus groups, or participant observations. Rather than relying on the presence and intervention of the researcher, probes are designed to encourage and empower subjects to collect data themselves (Arnold, 2004). The participants use probes to provide some insight, at their discretion, about their daily lives. Often, challenges and opportunities are only discovered when the technologies are used and evaluated with users in real-world settings (Doyle et al., 2010; Waycott et al., 2012). Personal information and story generation are two important benefits that we see here in the use of probes as artefacts contributing to users’ point of view. Due to the logging functionality, technology probes ensure that participation of a user is visible and re-countable (Graham & Rouncefield, 2007).
The technology probe was seen as instance of the goal model and had logging capabilities (as is typical of technology probes) to monitor and record the use of the application. At the beginning of the field study, none of the researchers, designers, or older adults had a clear idea about how the final personal alarm system technology would look. It was particularly important to first engage the participants in simple technology use so they could confidently handle the interaction with their family members. Future design is thus grounded in a thorough understanding of users’ experiences, requirements, and preferences (Lindsay et al.,2012).
Generally, technology probes can collect data about use to inform a better understanding, not so much about how to improve the technology but rather about actual needs in supporting specific activities (in our case activities evolving around building and maintaining interactions of older adults and their relatives to communicate wellbeing) (also see Hutchinson et al., 2003). Hence, technology probes are conducted prior to actual prototyping of the future system. In these latter phases, the users are acting as testers of the technology probe as well as collaborators influencing the evolving design as the functional and emotional goal models are refined through the supporting interview processes.
Prototype development: Technology probe for the wellbeing check
The technology probe for the wellbeing check was motivated by the goal model (see Figure 1) and facilitated the involvement with the user. The technology probe focused on the daily wellbeing check, rather than the pendant. We collaborated with a software company that followed an agile development approach. The emotional goals were communicated to the company and they defined their development goals in alignment with the emotional goals and mirrored their daily progress on each of them. The technology probe development and communication about alternatives was driven by the emotional goals. The technology shown in Figure 6 was developed and implemented in nine households.
The prototype in Figure 6 enabled relatives to send photos with captions. The user had could then scroll through photos and send messages back to their relatives. Only when no interaction takes place over a defined period of time does the app ask the user to indicate their wellbeing (e.g., “You haven’t been in touch. Are you ok?”), and the user responds by pressing a button in the app. Thus, the wellbeing check in this prototype is the existence or absence of this “ping” as monitored by the backend systems of a service provider.
Figure 6. IPad with picture app used as base station for wellbeing check
Evaluation: Results from implementing the technology probe for the wellbeing check
Older people and relatives (nine older participants and five relatives) were interviewed after a four-week trial with the implemented technology probe for the wellbeing check. Overall, the participants liked the social and personalized aspect of the application and the feedback was positive. One relative commented:
“It’s really fantastic. Because it’s not masquerading or trying to pretend it’s something that it’s not. It’s harnessing that activity, or harnessing that interaction to mine it for really useful data, so it’s not […] the teddy bear with the hidden camera in it monitoring what’s going on in the room, it’s not presenting in that way. It’s very upfront.” [Relative]
However, although one participant was happy with the social aspect, they were not comfortable with the monitoring aspect of the alarm and still felt that control was taken away from her:
“I wouldn’t want to have any automatic checking on me. I want to be in control of whether someone is coming. I want to make a conscious decision. Last year, I had really high blood pressure and I went to bed and thought ‘either I will wake up or not and that is fine’.” [Older person]
The technology probe for the wellbeing check, coupled with interviews, enabled us to view the goal qualities in the light of the user activities.
However, it was very difficult for us to meet the expectations of some older people to truly feel “cared about”. The following quote summarizes the different expectations:
“I think that that’s the conflict because, for me as a relative and a carer, the assuredness was related to the functional aspects of the device, whereas for the user, their assurance isn’t related to that at all. Their assurance is much more around the emotional ideas and that idea around the connectivity. And I think that was the clash, in that what I emotionally needed was very different to what my aunt emotionally needed.[…] and the reason that we implemented the system was... I mean to put it really bluntly was farming out a task.” [Relative]
In our solution, we expect a certain commitment of relatives and carers to spend some time in communicating with the older person. It was difficult to find people in this trial that would send a photo to the older person every day. Although we try to meet the emotional goals of older people, we are aware that we rely on other people whose emotions or time allowance might not be in alignment with those of the older person.
The Emotion-Led Design ToolKit
Here, we summarize the use of the different emotion-led methods used in the three phases of the living lab process. The first phase of the process (exploration) begins with the designer conducting user research around a design problem or theme. Our design problem focused on personal alarm systems for older adults. Collections of insights are formed during this phase and then are translated into a goal model with a focus on emotions (Figure 7).
Figure 7. A process flow model illustrating the emotion-led design toolkit to supports the living lab process from exploration (Phase 1), to experimentation (Phase 2), and then to evaluation (Phase 3)
Emotions gathered during the exploration phase are represented in the goal model and are crucial in moving towards a designed outcome. It is important for researchers to evaluate these emotional goals to ensure that they reflect the true nature of the design problem. During the exploration phase, the researcher begins to create individual scenarios that show the emotional goals of the user and the functional goals of the system in context. It is presented to end users in the format of animations to co-evaluate initial user research from the conceptual phase to ensure the goals reflect the key concerns and emotions of the users that need to be addressed in the design solution. In our case, we focused on the older adults as the main users of the system. We suggest that it would be useful to repeat the co-evaluation with other main stakeholders, such as relatives and carers. Evaluation is the last phase of the working model. It is here, after the scenarios have been refined and themes and insights have been developed and evaluated with the user, that a designer can work towards a designed outcome.
In this case study, our approach conforms to an overall model that is typical of living labs. We take a similar iterative developmental approach from exploration, to experimentation, and then to the early stages of evaluation. Our users took on the typical living lab roles of informants, testers, contributors, and co-creators and were involved in multi-method approaches throughout the lifecycle. Our study proposes an emotion-led design toolkit that can be added to similar approaches in other living labs. This approach captures the emotional and quality goals of older adults (in this case) and translates them into actionable requirements that sit alongside functional goals. We show that motivational goal models are a suitable way to express field data derived from interviews – in particular emotions of users. These models are part of a development methodology and can be combined with scenarios to express user’s emotions, motivations, and roles (Marshall, 2014; Sterling & Taveter, 2009), each of them describing and providing context of the domain. The goal models provide a place where abstract design concepts can be collected and represented (Pedell et al., 2009), but it will also be possible to evaluate final solutions against these goals. They are a lens through which use activities can be analyzed and recorded and then discussed among researchers and older adults. In reflection of our initial aim to give older adults a strong voice in the design process, the technology probes facilitated natural interactions between family members and yielded useful insights into how they used the newly designed system. Data gathered using technology probes are fragmentary and unstructured, which makes the process of translation from field data to abstract generalization for development difficult. A process of combining technology probe data collection and motivational goal models allowed us to talk about intangible outcomes with users that can be surprising, complex, and subtle.
To conclude, we emphasize the contribution of this research, which is a demonstration of how goal models, animations, and technology probes can be used to refine, link, and strengthen the transitions from different phases of a living lab process.
We would like to thank both the Smart Services CRC and the Australian Research Council (DP160104083) for partial funding of the research. We also extend a big thank you to all our participants for sharing their experiences and providing input into the design process.
Almirall, E., Lee, M., & Wareham, J. 2012. Mapping Living Labs in the Landscape of Innovation Methodologies. Technology Innovation Management Review, 2(9): 12–18.
Arnold, M. 2004. The Connected Home: Probing the Effects and Affects of Domesticated ICTs. In A. Bond, A. Clement, F. de Cindio, D. Schuler, & P. van den Besselar (Eds.), Proceedings of Participatory Design Conference (PDC 2004): 183–186. Pala Alto, Ca: CPSR.
Buxton, B. 2007. Sketching User Experiences: Getting the Design Right and the Right Design. San Francisco, CA: Morgan Kaufmann.
Carroll, J. M. & Rosson, M. B. 1992. Getting around the Task-Artifact Cycle: How to Make Claims and Design by Scenario. ACM Transactions on Information Systems (TOIS), 10(2): 181–212.
Dell´Era, C., & Landoni, P. 2014. Living Lab: A Methodology between User-Centred Design and Participatory Design. Creativity and Innovation Management, 23(2): 137–154.
Doyle, J., Skrba, Z., McDonnell, R., & Arent, B. 2010. Designing a Touch Screen Communication Device to Support Social Interaction amongst Older Adults. In Proceedings of the 24th BCS Interaction Specialist Group Conference: 177–185. Swindon, UK: BCS Learning & Development Ltd.
Edlin-White, R., Cobb, S., Floyde, A., Lewthwaite, S., Wang, J., & Riedel, J. 2012. From Guinea Pigs to Design Partners: Working with Older People in ICT Design. In P. Langdon, J. Clarkson, P. Robinson, J. Lazar., & A. Heylighen (Eds.), Designing Inclusive Systems: 155–164. London: Springer-Verlag.
Eskelinen, J., Robles, A., Lindy, I., Marsh, J., & Muente-Kunigami, A. 2015. Citizen-Driven Innovation: A Guidebook for City Mayors and Public Administrators. World Bank and European Network of Living Labs.
Graham, C., Rouncefield, M., Gibbs, M., Vetere, F., & Cheverst, K. 2007. How Probes Work. In Proceedings of the 19th Australasian Conference on Computer-Human Interaction: Entertaining User Interfaces: 27–37. New York: ACM.
Gray, M., Mangyoku, M., Serra, A., Sánchez, L., & Aragall, F. 2014. Integrating Design for All in Living Labs. Technology Innovation Management Review, 4(5): 50–59.
Hutchinson, H., Mackay, W., Westerlund, B., Bederson, B. B., Druin, A., Plaisant, C., Beaudouin-Lafon, M., Conversy, S., Evans, H., Hansen, H., Roussel, N., & Eiderbäck, B. 2003. Technology Probes: Inspiring Design for and with Families. In Proceedings of the SIGCHI conference on Human factors in Computing Systems: 17–24. New York: ACM.
Iacucci, G., Iacucci, C., & Kuutti, K. 2002. Imagining and Experiencing in Design: The Role of Performances. In Proceedings of the 2nd Nordic Conference on Human-Computer Interaction: 167–176. New York: ACM.
Leminen, S. 2015. Q&A. What Are Living Labs? Technology Innovation Management Review, 5(9): 29–35.
Leminen, S., Nyström, A.-G., & Westerlund, M. 2015. A Typology of Creative Consumers in Living Labs. Journal of Engineering and Technology Management, 37: 6–20.
Lim, Y.-K., & Sato, K. 2006. Describing Multiple Aspects of Use Situation: Applications of Design Information Framework (DIF) to Scenario Development. Design Studies, 27(1): 57-76.
Lindsay, S., Jackson, D., Schofield, G., & Olivier, P. 2012. Engaging Older People Using Participatory Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: 1199–1208. New York: ACM.
Lo Bianco, M., Pedell, S., Renda, G., & Kapoor, A. 2015. HCI Methods for Empowering Discussion on Person-Centered Fall Prevention with Older Adults. In Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction: 255–263. New York: ACM.
Loke, L., Robertson, T., & Mansfield, T. 2005. Moving Bodies, Social Selves: Movement-Oriented Personas and Scenarios. In Proceedings of the 17th Australia Conference on Computer-Human Interaction: Citizens Online: Considerations for Today and the Future. Canberra, Australia: Computer-Human Interaction Special Interest Group (CHISIG) of Australia.
Marshall, J. 2014. Agent-Based Modelling of Emotional Goals in Digital Media Design Projects. International Journal of People-Oriented Programming (IJPOP), 3(1): 44–59.
Massimi, M., Baecker, R. M., & Wu, M. 2007. Using Participatory Activities with Seniors to Critique, Build, and Evaluate Mobile Phones. In Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility: 155–162. New York: ACM.
Matthews, B., & Heinemann, T. 2012. Analysing Conversation: Studying Design as Social Action. Design Studies, 33(6): 649-672.
Miller, T., Pedell, S., Lopez-Lorca, A. A., Mendoza, A., Sterling, L., & Keirnan, A. 2015. Emotion-Led Modelling for People-Oriented Requirements Engineering: The Case Study of Emergency Systems. Journal of Systems and Software, 105(C): 54–71.
Muller, M. J. 2008. Participatory Design: The Third Space in Human-Computer Interaction: Development Process. Mahway, NJ: Erlbaum.
Patton, M. Q. 2002. Qualitative Evaluation and Research Methods. Thousand Oaks, CA: Sage Publications, Inc.
Pedell, S., Miller, T., Vetere, F., Sterling, L., Howard, S., & Paay, J. 2009. Having Fun at Home: Interleaving Fieldwork and Goal Models. In Proceedings of the 21st Annual Conference of the Australian Computer-Human Interaction Special Interest Group: Design: Open 24/7: 309–312. New York: ACM.
Pedell, S., Lopez-Lorca, A. A., Miller, T., & Sterling, L. 2015. Don't Leave Me Untouched: Considering Emotions in Personal Alarm Use and Development. In M. Tavana, A. Ghapanchi, & A. Talaei-Khoei (Eds.), Healthcare Informatics and Analytics: Emerging Issues and Trends: 96–127. Hershey, PA: IGI Global.
Robertson, T., & Simonsen, J. 2012. Participatory Design: An Introduction. In J. Simonsen & T. Robertson (Eds.), Routledge International Handbook of Participatory Design: 1-18. London: Routledge.
Schuurman, D., De Marez, L., & Ballon, P. 2016. The Impact of Living Lab Methodology on Open Innovation Contributions and Outcomes. Technology Innovation Management Review, 1(6): 7–16.
Simon, H. A. 1982. The Sciences of the Artificial (2nd edition). Cambridge, MA: MIT press.
Ståhlbröst, A., & Holst, M. 2013. The Living Lab Methodology Handbook. Luleå, Sweden: Luleå University of Technology and Centre for Distance-Spanning Technology (CDT).
Sterling, L., & Taveter, K. 2009. The Art of Agent-Oriented Modeling. Cambridge, MA: MIT Press.
Veeckman, C., Schuurman, D., Leminen, S., & Westerlund, M. 2013. Linking Living Lab Characteristics and Their Outcomes: Towards a Conceptual Framework. Technology Innovation Management Review, 3(12): 6-15.
Vines, J., Blythe, M., Lindsay, S., Dunphy, P., Monk, A., & Olivier, P. 2012. Questionable Concepts: Critique as Resource for Designing with Eighty Somethings. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: 1169–1178. New York: ACM.
Waycott, J., Pedell, S., Vetere, F., Ozanne, E., Kulik, L., Gruner, A., & Downs, J. 2012. Actively Engaging Older Adults in the Development and Evaluation of Tablet Technology. In Proceedings of the 24th Australian Computer-Human Interaction Conference: 643–652. New York: ACM.
Waycott, J., Vetere, F., Pedell, S., Kulik, L., Ozanne, E., Gruner, A., & Downs, J. 2013. Older Adults as Digital Content Producers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: 38–48. New York: ACM.