A Taxonomy of Factors Influencing Drop-Out Behaviour in Living Lab Field Tests

The concept of a “living lab” is a relatively new research area and phenomenon that facilitates user engagement in open innovation activities. Studies on living labs show that the users’ motivation ...


Publisher
The Technology Innovation Management Review is a monthly publication of the Talent First Network.

Overview
The Technology Innovation Management Review (TIM Review) provides insights about the issues and emerging trends relevant to launching and growing technology businesses. The TIM Review focuses on the theories, strategies, and tools that help small and large technology companies succeed.
Our readers are looking for practical ideas they can apply within their own organizations. The TIM Review brings together diverse viewpoints -from academics, entrepreneurs, companies of all sizes, the public sector, the community sector, and others -to bridge the gap between theory and practice. In particular, we focus on the topics of technology and global entrepreneurship in small and large companies. We welcome input from readers into upcoming themes. Please visit timreview.ca to suggest themes and nominate authors and guest editors.
timreview.ca are aligned with these benefits. The authors discuss the implications of their findings for the value proposition literature and for companies evolving their business models for servitization.
Finally, Michael Neubert from the International School of Management in Paris examines how lean global startups develop new foreign markets more rapidly due to digitalization. By interviewing 73 senior managers of lean global startups, the author gained insights into how digitalization allows lean global startups to increase decision-making efficiency and to optimize strategies and processes for evaluating international markets, thereby enabling them to internationalize faster.
For future issues, we are accepting general submissions of articles on technology entrepreneurship, innovation management, and other topics relevant to launching and growing technology companies and solving practical problems in emerging domains.
We have also recently issued a call for papers (tinyurl.com/ y7fv8crv) for a special issue on Technology Commercialization and Entrepreneurship with guest editors Ferran Giones from the University of Southern Denmark and Dev K. Dutta from the University of New Hampshire, USA.
Please contact us (timreview.ca/contact) with potential article topics and submissions, and proposals for future special issues.

Chris McPhee, Editor-in-Chief
Welcome to the May 2018 issue of the Technology Innovation Management Review. The authors in this issue share insights on the factors that erode engagement among users in living labs, the types of innovation instruments living labs can use to promote co-creation, how to develop value propositions for servitization, and how digitalization can help lean global startups internationalize faster.
In the first article, Abdolrasoul Habibipour, Anna Ståhlbröst, and Birgitta Bergvall-Kåreborn from Luleå University of Technology in Sweden and Botnia Living Lab, along with Annabel Georges and Dimitri Schuurman from imec.livinglabs in Belgium, examine the reasons why some users drop out of living lab field tests. Based on 14 interviews with living lab experts, they develop a taxonomy of the factors that influence drop-out behaviour in living lab field tests and propose a unified definition of "drop-out" in living lab field tests. The resulting taxonomy of 44 key factors that influence dropout behaviour can help organizers anticipate potential problems and keep users motivated and engaged throughout the innovation process.
Next, Lotta Haukipuro and Satu Väinämö from the University of Oulu in Finland and Pauliina Hyrkäs from the Northern Ostrobothnia Hospital District examine three innovation instruments used by Oulu Urban Living Labs to co-create needs-based solutions. The innovation instruments are applied in three different environments -a school, a hospital, and an airport -in which 12 SMEs and startups developed solutions based on predefined needs of customer organizations and with the participation of users from stakeholder organizations. Based on the results, the authors propose a new, generic model for using innovation instruments to facilitate co-creation for the development of needsbased products and services in different service domains.
Then, Kwesi Sakyi-Gyinae and Maria Holmlund from the Hanken School of Economics in Finland examine how to create value propositions in a servitization context by focusing on the customer perspective. Their findings demonstrate how customers articulate the benefits (or "value in use") of a selected offering, which can be used to develop value proposition elements that

Introduction
Studies on open innovation have increasingly emphasized the role of individual users as collaborators in the innovation processes, and users are now considered one of the most valuable external sources of knowledge and a key factor for the success of open innovation (Jespersen, 2010). One of the more recent approaches of managing open innovation processes are living labs, where individual users are involved to co-create, test, and evaluate digital innovations in open, collaborative, multi-contextual, and real-world settings (Bergvall-Kareborn et al., 2009;Leminen et al., 2012;Ståhlbröst, 2008). A major principle within living lab research consists of capturing the real-life context in which an innovation is used by end users by means of a multi-method approach (Bergvall-Kåreborn et al., 2015;Schuurman, 2015). The process of innovation development in the living lab setting can happen in different phases, including exploration, design, implementation, test, and evaluation (Ståhlbröst, 2008). Nevertheless, testing a product, service, or system as one of the key components of living labs has been more focused than other phases of innovation development (Claude et al., 2017). Although we have not found any clear description or definition for the term "field test" (nor for the term "field trial", which has been used interchangeably in some literature); Merriam -Webster Dictionary (2008), says that the aim of conducting a field test is "to test (a procedure, a product, etc.) in actual situations reflecting intended use". In a living lab setting, a field test is a user study in which test users interact with an innovation in their real-life everyday use context while testing and evaluating it (Georges et al., 2016). What distin-The concept of a "living lab" is a relatively new research area and phenomenon that facilitates user engagement in open innovation activities. Studies on living labs show that the users' motivation to participate in a field test is higher at the beginning of the project than during the rest of the test, and that participants have a tendency to drop out before completing the assigned tasks. However, the literature still lacks theories describing the phenomenon of drop-out within the area of field tests in general and living lab field tests in particular. As the first step in constructing a theoretical discourse, the aims of this study are to present an empirically derived taxonomy for the various factors that influence drop-out behaviour; to provide a definition of "drop-out" in living lab field tests; and to understand the extent to which each of the identified items influence participant drop-out behaviour. To achieve these aims, we first extracted factors influencing drop-out behaviour in the field test from our previous studies on the topic, and then we validated the extracted results across 14 semi-structured interviews with experts in living lab field tests. Our findings show that identified reasons for dropping out can be grouped into three themes: innovation-related, process-related, and participant-related. Each theme consists of three categories with a total of 44 items. In this study, we also propose a unified definition of "drop-out" in living lab field tests.
Many of life's failures are people who did not realize how close they were to success when they gave up. Thomas Edison (1874-1931 Inventor and industrialist " " timreview.ca

A Taxonomy of Factors Influencing Drop-Out Behaviour in Living Lab Field Tests
A. Habibipour, A. Georges, A. Ståhlbröst, D. Schuurman, and B. Bergvall-Kåreborn guishes living lab field tests from the traditional field tests is that the commercial maturity of the prototyped product, service, or system in traditional field tests are higher than in living lab field tests. On the other hand, living lab field tests are usually conducted in an open environment, in contrast to traditional field tests, where the testing is undertaken within a controlled situation. As digital innovations are one of the key aspects of living lab activities (Bergvall-Kåreborn et al., 2009), in this study, we focus on digital products, services, or systems as the focus of living lab field tests.
Involving individual users in the process of developing IT systems is a key dimension of open innovation that contributes positively to new innovations as well as system success, system acceptance, and user satisfaction (Bano & Zowghi, 2015;Leonardi et al., 2014;Lin & Shao, 2000). Although, when it comes to testing a digital innovation, it is recognized that keeping users motivated is more challenging than motivating them to start participating in a project in the first place (Ley et al., 2015;Pedersen et al., 2013;Ståhlbröst & Bergvall-Kåreborn, 2013). Consequently, users tend to drop out of a field test before the project or activity has ended, as the motivations and expectations of the users change over time (Georges et al., 2016). The reasons for dropping out might be due to internal factors relating to a participant's decision to stop the activity or external environmental factors that caused them to terminate their engagement (O'Brien & Toms, 2008). These factors influence participants during all phases of the innovation process, from contextualization to test and evaluation .
A number of studies have acknowledged the importance of sustaining user engagement during living lab activities (Hess & Ogonowski, 2010;Leonardi et al., 2014;Ley et al., 2015). However, to the best of our knowledge, there are no studies investigating the drop-out rate in living lab field tests. Despite this, within the process of system development in a general level, the dropout rate has usually been reported more than 50% (De Moor et al., 2010;Hess et al., 2008;Sauermann & Franzoni, 2013), which might have negative consequences for both the project outcome as well as the project organizers. Given that participating users already have a profound understanding and knowledge about the activity or project (Hess & Ogonowski, 2010), they are able to provide more useful and reliable feedback compared to the users who join the project when it is already underway (Ley et al., 2015). Moreover, once a project is underway, a trustful relationship between the users and developers has (presumably) already been es-tablished and this trust has been shown to be positively associated with project results (Carr, 2006;Jain, 2010). Also, having users drop out of projects is costly both in terms of time and resources as the developers need to train new users and provide them with adequate infrastructure, such as hardware, software, and communication technology (Ley et al., 2015). Finally, the issue of drop-out is important to the extent that Kobren and colleagues (2015) assert that, after dropping out, a participant provides no additional value for the project or activity.
Despite the above-mentioned consequences that dropout has for the projects or activities, the literature lacks theories describing the phenomenon of user drop-out within the area of field tests in general and living lab field tests in particular. But, before such theories can be developed, we must define, categorize, and organize the factors that may influence drop-out behaviour. Such a taxonomy can form the basis of a theoretical framework in the area of this study. Accordingly, the aims of current study are: i) to provide an empirically grounded definition of a "drop-out" in living lab field tests; ii) to develop an empirically derived, comprehensive taxonomy for the various factors that influence drop-out behaviour in a living lab setting; and iii) to understand the extent to which each of the identified items influence the drop-out behaviour of participants in living lab field tests. To achieve this goal, we first extracted findings from our previous work on the topic to identify the factors that influence participant drop-out behaviour, and then the results were validated across 14 semi-structured interviews with experts in living lab field tests.
The article is organized as follows: After presenting the theoretical framework in the next section, we outline the methodology and research process we used to derive the taxonomy, followed by a summary of our previous work on this topic, from which we extracted an initial list of factors. After that, we present our definition of "drop-out" in living lab field tests. Then, we discuss the most influential factors on drop-out behaviour and present the taxonomy we developed to categorize drop-outs in living lab field tests. Finally, we discuss the implications and limitations of the study, and we offer some concluding remarks.

A Theoretical Framework
In this section, we develop a framework to identify and categorize various factors that influence participant drop-out behaviour in living lab field tests. There are 7 timreview.ca

A Taxonomy of Factors Influencing Drop-Out Behaviour in Living Lab Field Tests
A. Habibipour, A. Georges, A. Ståhlbröst, D. Schuurman, and B. Bergvall-Kåreborn different users, and technical, social, cultural, and organizational components (Pilemalm et al., 2007). When it comes to involving individual users in socio-technical systems, all technical features of the system, the social interactions supported by the system, and other sociotechnical aspects influence how the users perceive and interpret their experiences and subsequently how they behave (Di Gangi & Wasko, 2009). In a study of participatory design for the development of socio-technical systems, Pilemalm and co-authors (2007) highlighted the importance of active user participation throughout the whole process of socio-technical system design and development, and they argued that this topic deserves more research.
The integration of social structures and perspectives with technical functions is the central problem in the design of socio-technical systems (Herrmann, 2009). In order to tackle this problem and integrate the impacts of socio-technical theory within the area of IT-system development, we found the technology-organization-environment (TOE) framework (Depietro et al., 1990) suitable because it has been developed to link information system innovation with contextual factors, and it enables us to address the development process of IT innovations in open systems (Chau & Tam, 1997). In addition, the TOE model has broad applicability and possesses explanatory power across a number of technological, industrial, and national/cultural contexts (Baker, 2012). Furthermore, it can be extended to settings for examining and explaining different innovation modes (Song et al., 2009).
Another benefit of using the TOE framework is that it is highly flexible and generic and, instead of explicitly specifying different variables in each category, it allows us to include different sources of influence on system design and development process (Zhu & Kraemer, 2005). Accordingly, the TOE framework provides a more holistic view of all three main aspects of a sociotechnical system (i.e., the social, technical, and sociotechnical aspects) and helps us to better meet the needs and expectations of the various involved stakeholders throughout the design and development process (Herrmann, 2009;Nkhoma et al., 2013).

A Taxonomy of Factors Influencing Drop-Out Behaviour in Living Lab Field Tests
A. Habibipour, A. Georges, A. Ståhlbröst, D. Schuurman, and B. Bergvall-Kåreborn In this model, technology is associated with the technical aspects of a socio-technical system, which might be related to the platform, innovation, infrastructure, etc. Environment reflects more on the social aspects of a socio-technical system such as the real-life everyday use context, the personal context, and so on. And, finally, organization is associated with the socio-technical aspects of a socio-technical system in ways such as organizing the research, communication between different stakeholders, designing the processes, etc. Figure 1 shows the theoretical framework for this study.

Methodology
In order to better understand drop-out behaviour of field test participants, a detailed and systematic study needs to be conducted in the relevant natural setting using a qualitative approach (Kaplan & Maxwell, 2005).
In contrast with a typology in which the categories are derived based on a pre-established theoretical framework, the taxonomies are emerged empirically within an inductive approach and are developed based on observed variables (Sokal & Sneath, 1963).
In order to develop a taxonomy for factors influencing drop-out behaviour, we used various qualitative data collection methods to gather information about the reasons participants drop-out of living labs field tests.
In this study, we collected qualitative data in two major steps. First, we extracted from our previous studies on the topic possible reasons for participant drop-out in living lab field tests. Second, these findings were validated by interviewing experts in living lab field tests to increase and ensure the validity and trustworthiness of the collected data to build a taxonomy for drop-out. Figure 2 summarizes the research process for this study, which is explained in detail below.
In the first major step, we explored documented reasons for participant drop-out in field tests. As recommended by Strauss and Corbin (1998), when a research field still lacks explicit boundaries between the context and phenomenon, reviewing previous literature can be used as a point of departure for further research. Accordingly, this phase of data collection followed the results of our earlier literature review on the topic . Through this process, we extracted 29 items (or factors) that influence participant drop-out behaviour. In addition, we identified other possible factors that may influence participant dropout based on our results from four different field tests: three with imec.livinglabs (www.imec-int.com/en/livinglabs) in Belgium (Georges et al., 2016) and one with Botnia Living Lab (tinyurl.com/y8nf4lcg) in Sweden (Habibipour & Bergvall-Kåreborn, 2016). In these field tests, the data was collected by conducting an open-ended questionnaire as well as direct observation of drop-out behaviour. This also resulted in 42 items. After eliminating redundant or similar items, we ended up with 53 items.
In order to promote stronger interaction between research and practice and to obtain more reliable knowledge, social scientists recommend that studies should include different perspectives (Kaplan & Maxwell, 2005). This approach is in line with Van de Ven's (2007) recommendation to conduct social research as "engaged scholarship", which they define as: "...a participative form of research for obtaining the different perspectives of key stakeholders (researchers, users, clients, sponsors, and practitioners) in studying complex problems. By involving others and leveraging their different kinds of knowledge, engaged scholarship can produce knowledge that is more penetrating and insightful than when scholars or practitioners work on the problem alone." Thus, in the second round of data collection, we conducted 14 semi-structured, open-ended interviews with experts in living lab field tests. Eight out of 14 interviewees were user researchers or panel managers from imec.livinglabs in Belgium and six of them were living lab researchers from Botnia living lab in Sweden. These experts were selected because they were not only familiar with living lab studies in general, but also because they had extensive work experience in relation to conducting living lab field tests. Although interviewing dropped-out participants could also provide us valuable information, their point of view is usually limited to one or two field tests, in contrast to the experts that have been involved in various field tests in different contexts. Moreover, in many cases, it was not feasible to ask them to be interviewed given that they had already dropped out of a previous research project, which is their right as voluntary participants.
The aim of these interviews was to validate the findings of the first data collection wave with the researchers, which enables us to find an initial structure for the proposed taxonomy. The results from this step were analyzed separately in two groups in each living lab (i.e., Botnia and imec.livinglabs). Accordingly, in this study, we used data, method, and investigator triangulation to increase the reliability as well as the validity of the results and greater support to the conclusions (Benbasat et al., 1987;Flick, 2009

A Taxonomy of Factors Influencing Drop-Out Behaviour in Living Lab Field Tests
A. Habibipour,A. Georges,A. Ståhlbröst,D. Schuurman,. Research process for this study timreview.ca

A Taxonomy of Factors Influencing Drop-Out Behaviour in Living Lab Field Tests
A. Habibipour, A. Georges, A. Ståhlbröst, D. Schuurman, and B. Bergvall-Kåreborn The topic guide of the interview consists of two major parts. First, the interviewees were asked open questions about living lab field tests, drop-out, and components of drop-out (e.g., definition, types of drop-out, main drop-out reasons, and when they consider a participant as dropped out). In the second part, we used the results of our previous studies as input for developing the interview protocol and, thus, the interviewees were given 53 cards, each one showing an identified factor. We asked the interviewees to put each of these cards into one of three main categories -not influential at all, somewhat influential, or extremely influential -according to their perceived extent of influence on participant drop-out in the living lab field tests they were involved in. They also were provided with some empty cards in case they wanted to add other items that were not presented in the pre-prepared 53 cards. This rating procedure was done to help us to understand the degree of importance of each item. Then, they were asked to group extremely influential items into coherent groups with a thematic relation. This helped us to identify the main categories of drop-out and enabled us to develop our taxonomy.
To analyze the data, we used qualitative coding because it is the most flexible method of qualitative data analysis (Flick, 2009) and allows researchers to build a theory through an iterative process of data collection as well as the data analysis (Kaplan & Maxwell, 2005). In this regard, developing a taxonomy is the first step in empirically building a theoretical foundation based on the observed factors (Stewart, 2008). This approach facilitates insight, comparison, and the development of the theory (Kaplan & Maxwell, 2005) and enables us to identify key concepts in order to develop an initial structure for the taxonomy for drop-out in living lab field tests. The coding was done in three major steps. First, all suggested categories by the interviewees assigned a unique code (e.g., "1" for interaction, "2" for timing issues, etc.). Second, redundant or similar categories were combined and assigned the same code (e.g., "timing" and "scheduling", "interaction" and "communication", etc.). Finally, considering our theoretical framework, all remaining categories were grouped into three main meaningful themes that represented the social, technical, and socio-technical aspects.

Building on Previous Studies
Our previous studies show that keeping users motivated and engaged is not an easy task as they may tend to drop out before completing the project or activity (Georges et al., 2016). However, to the best of our knowledge, there are few studies addressing reasons for participant drop-out in living lab field tests.
In Habibipour, Bergvall-Kareborn, and Ståhlbröst (2016), we carried out a comprehensive literature review to identify documented reasons for drop-out in information systems development processes. We identified some influential factors on drop-out behaviour and classified them into technical aspects, social aspects, and socio-technical aspects. When it comes to technical aspects, the main reasons that lead to drop-out are related to the performance of the prototype or interactions with it such as task complexity and usability problems (e.g., instability or unreliability of the prototype). Limitation of users' resources, inadequate infrastructure, and insufficient technical support are other technical aspects. Regarding the social aspects, issues related to the relationship (either between users and developers or between participants themselves), lack of mutual trust, and inappropriate incentive mechanisms are the main reasons. In considering the socio-technical aspects, wrong user selection and privacy and security concerns were further highlighted in the studies. However, in the abovementioned study , the authors did not focus on a specific phase or type of activity, and extracted the drop-out reasons for all steps of the information systems development process such as ideation, co-design, or co-creation, and, finally, test and evaluation.
In Georges, Schuurman, and Vervoort (2016), we conducted a qualitative analysis within three living lab field tests to find factors that are related, either positively or negatively, to different types of drop-out during field tests. The field tests were carried out in living lab projects from iMinds living labs (now imec.livinglabs). The data in this study was collected via open questions in post-trial surveys of the field tests and an analysis of drop-out from project documents. The results of this study show that several factors related to the innovation, as well as related to the field trial setup, play a role in drop-out behaviour, including the lack of added value of the innovation and the extent to which the innovation satisfies the needs, the restrictions of test users' time, and technical issues.
We have also attempted to present a user engagement process model that includes the variety of reasons for drop-out (Habibipour & Bergvall-Kåreborn, 2016). The presented model in this study is grounded on the results timreview.ca

A Taxonomy of Factors Influencing Drop-Out Behaviour in Living Lab Field Tests
A. Habibipour, A. Georges, A. Ståhlbröst, D. Schuurman, and B In total, we extracted 29 items from the first article , 27 items from the second article (Georges et al., 2016), and 15 items from the third article (Habibipour & Bergvall-Kåreborn, 2016). By removing redundant items, we ended up with 53 factors that influence drop-out behaviour. In this study, we build on these studies by addressing the need for a clear definition of "drop-out" as well as a taxonomy of possible reasons participants drop-out.

Proposed Definition
Defining the key concepts is the first step in constructing a theoretical discourse. Our finding also supports the argument put forward by O'Brien & Toms (2008), who stated that user disengagement might be due to an internal decision of the participant to stop the activity or external environmental factors that caused them to terminate their engagement before completing the assigned tasks. Accordingly, the drop-out decision can be made consciously or subconsciously by the participants, but is characterized by the fact that they do not notify the field test organizers. For instance, an interviewee made a distinction between dropped out users and a defector which is someone who notifies the project that they will stop testing but will still give feedback: "If you stop testing and you keep on filling in the surveys (participating in research), you are not a dropped-out user. You need to make a distinction between those who stop testing the application and those who stop filling in the surveys..." What is common in all of the above-mentioned arguments is that the participants showed their initial interest to participate in the field test but they stopped performing the tasks before the field test has ended. Thus, we propose this definition for drop-out in living lab field test as: "A drop-out during a living lab field test is when someone who has signed up to participate in the field test does not complete all the assigned tasks within the specified deadline." Within this definition, there are three important elements: 1. The dropped-out participant signed up to participate. This element implies that the potential participants must be aware of what is expected of them.
2. The dropped out participant did not complete all the assigned tasks. Depending on the type of field test, timreview.ca

A Taxonomy of Factors Influencing Drop-Out Behaviour in Living Lab Field Tests
A. Habibipour, A. Georges, A. Ståhlbröst, D. Schuurman, and B. Bergvall-Kåreborn this could be the act of using/testing the innovation, but could also refer to participating in research steps (e.g., questionnaires, interviews, diary studies). This distinction was noted by Eysenbach (2005) in his law of attrition (drop-out attrition and non-usage attrition).
3. The drop-out participant has not completed the tasks that were assigned to them within the specified deadline that was agreed upon.

Proposed Taxonomy
Taxonomies are useful for research purposes: they can help leverage and articulate knowledge and are fundamental to organizing knowledge and information in order to refine information through standardized and consistent procedures (Stewart, 2008). As mentioned in the methodology section, the taxonomy we developed through this study is grounded by the results of a literature review article  as well as the results of four living lab field tests (Georges et al., 2016;Habibipour & Bergvall-Kåreborn, 2016). The findings of the previous steps were validated across 14 semi-structured interviews. This triangulation of the data strengthens the validity of the presented taxonomy and makes our results stronger and more reliable (Benbasat et al., 1987). The interviewees were asked to group the items that are extremely influential on participant dropout into coherent groups. Our goal was to identify the categories most frequently suggested by the interviewees. Table 1 shows the categories of items that they initially suggested: B1 to B8 refers to the interviewees in imec.livinglabs in Belgium and S1 to S6 refers to the interviewees in Botnia Living Lab in Sweden. In some cases, an item can belong to different categories because the same item was interpreted differently by the interviewees. For example, two interviewees mentioned privacy and security concerns as "personal context" while six of them considered it under the category of "participants' attitudes". Thus, we decided to put the privacy and security concerns under the "participants' attitudes" category.
An important outcome of this study was a refinement of the initial list of items that was extracted from our previous studies. During the interviews, we asked the interviewees to express their feelings about each item and add any comments or explanations. By doing so, we eliminated some items that were similar and combined the items that were very closely related. In this study, we were also interested in discovering other factors influencing drop-out behaviour that we were not aware of. Some of the interviewees also added additional items to

A Taxonomy of Factors Influencing Drop-Out Behaviour in Living Lab Field Tests
A. Habibipour, A. Georges, A. Ståhlbröst, D. Schuurman, and B. Bergvall-Kåreborn our original list. As a result, we ended up with a revised list of items (44 items), which we used to develop the final taxonomy, which is shown in Table 2.
According to the results of the 14 interviews and based on the number of overlaps in the categories, we determined that nine categories was the most meaningful way of organizing the factors influencing drop-out behaviour in living lab field tests. The identified categories were grouped under three main themes: innovation-related factors, process-related factors, and participant-related factors. In the sub-sections that follow, we discuss each of these themes in detail.

Innovation-related factors
The categories under this theme are directly related to the innovation itself and reflect the technical aspects when it comes to socio-technical systems. Technological problems, perceived ease of use, and perceived usefulness were the categories that were most frequently suggested by the interviewees. The main innovation-related items (perceived usefulness and perceived ease of use) are in line with the technology acceptance model (Davis, 1985;Venkatesh et al., 2000). Whereas in the technology acceptance model the perceived usefulness and perceived ease of use are the main drivers of adoption, within our model, these two items can be related to drop-out behaviour.
• Technological problems: As the results of the interviews revealed to us, technological problems are among the most important innovation-related factors that play a role in drop-out behaviour. This category of items may be associated with, for example, trouble installing the innovation, a lack of flexibility or infrastruc-ture compatibility issues, as well as issues with the stability and maturity of the (prototype) innovation.
• Perceived usefulness: This category highlights the importance of user needs. When the innovation does not meet the user's needs, it might be difficult to maintain the same level of engagement throughout the lifetime of a field test. Also, a participant who is voluntarily contributing in a field test must be able to see the potential benefits of testing an innovation in their everyday life.
• Perceived ease of use: The complexity of the innovation might negatively influence participant motivation. When the innovation is too complex to use or is not easy to understand, participants may become confused or discouraged. Moreover, when the innovation is not sufficiently mature, it is difficult to keep the participants enthusiastically engaged in the field test.

Participant-related factors
Some of the suggested categories were directly related to the individuals and their everyday life contexts. This theme mainly reflects social aspects and environment when it comes to socio-technical systems. The participants' attitudes or personalities, their personal contexts, and their resources can be classified under the participant-related theme.
• Participants' attitudes: There are a number of items that can be subsumed under the category of participants' attitudes. For example, this category includes situations in which the participants forget to participate, when the innovation does not meet their expectation, when they do not want to install something new on their device, when they do not like the concept or

A Taxonomy of Factors Influencing Drop-Out Behaviour in Living Lab Field Tests
A. Habibipour, A. Georges, A. Ståhlbröst, D. Schuurman, and B. Bergvall-Kåreborn

A Taxonomy of Factors Influencing Drop-Out Behaviour in Living Lab Field Tests
A. Habibipour, A. Georges, A. Ståhlbröst, D. Schuurman, and B. Bergvall-Kåreborn idea, and when they have concerns about their privacy or the security of their information.
• Everyday context: In a living lab approach, the users usually test innovations within in their own, real-life setting, therefore, challenges they face in their personal lives -unrelated to the testing activity -can negatively influence their motivation and may cause them to drop out of a field test.
• Participants' resources: Limitations in participants' resources can also influence the likelihood that they will drop out. They might either have not had enough time to be involved in the field test, or the project may place too many demands on their resources, such as requiring them to drain their own mobile batteries or consume part of their Internet data quota.

Process-related factors
These factors relate to the process of organizing a field test in a living lab setting where the socio-technical aspects are in focus. The three categories under this theme were associated with task design, interaction with the participants, and the timing of the field test.
• Task design: The results showed that there are various factors related to the design of the field test. For instance, when the tasks during the field test were not fun to accomplish, participants tend to drop out before completing the test. The interviewees also considered items such as a long gap between the field test's steps or a lengthy field test as influential factors that might be associated with the task design in the field test.
• Interaction: Interaction and communication with the participants was considered as one of the most important categories of items that influence a participant's decision to drop out. Unclear guidelines on how to do the tasks, lack of an appropriate technical support, and insufficient triggers to involve participants are some examples of the items in this category.
• Timing: Inappropriate timing of the field test (e.g., summer holiday) and too strict or inflexible deadlines are the most influential factors on drop-out behaviour in this category. When the participants are not able to participate in a field test at their own pace, they would prefer to not test the innovation any longer.
The developed taxonomy based on the resulted themes and categories is shown in Figure 3. The numbers in parentheses indicate the number of items under each category. The items under each of the themes and subcategories are shown in Table 2.

The Most Influential Factors on Drop-Out Behaviour
In this study, we were also interested in knowing the extent to which each of the identified factors influences the drop-out behaviour of participants in living lab field tests. As mentioned in the methodology section, we asked the interviewees to group the items into three categories: not influential at all on drop-out behaviour (=1 point), somewhat influential on drop-out behaviour (=2 points), and extremely influential on drop-out behaviour (=3 points). They chose and categorized the items based on their previous experiences with various living lab field tests and, therefore, these results are from their own perspective. Next, we summed the item scores and sorted them from highest to lowest, as shown in Table  3. Using this method, the minimum possible total for a given item is 14 (14 x 1), and the maximum possible total is 42 (14 x 3). Our results show a range from 18 to 40, with the top-10 items having totals of 35 or higher.
Of the top-10 items in Table 3, seven are related to the innovation itself. Problems related to installing the innovation; compatibility issues; the complexity, stability, and functionality of the innovation; usability; and ease of use are examples of items identified by the interviewees as the most influential innovation-related factors on participant drop-out behaviour. The implication of these findings is that, first and foremost, building sustainable user engagement in a living lab field test depends on careful consideration of issues that might emerge due to technological problems, perceived usefulness, and perceived ease of use. When the innovation does not work as promised, when it is not compatible with the participants' device, when it is technologically complex, and when it doesn't meet participants' needs and expectation, it is very difficult to keep the users enthusiastically engaged in the living lab field test. Accordingly, participants may drop out in the very early stage of the field test without even having the opportunity to fully test the innovation.

Conclusion
In this study, our aim was to provide a definition for "drop-out" in living lab field tests; to develop an empirically derived, comprehensive taxonomy for the various influential factors on drop-out behaviour in a living lab field test; and to understand the extent to which each of the identified items influence participant drop-out behaviour. To develop a theoretical discourse about dropout in field tests, there is a need to define, categorize, and organize possible influential factors on drop-out behaviour. Accordingly, we first identified factors influencing drop-out in the field tests from our previous research on the topic and then interviewed 14 experts who are experienced in the area of field testing in a living lab setting.
According to our definition, a dropped out participant in living lab field testing is someone who has signed up to participate in the field test but does not complete all the assigned tasks within the specified deadline. Our presented taxonomy revealed that the most influential reasons participants drop out were mainly related to the innovation, with additional factors being related to the process of the living lab field test and the participants themselves. Considering our suggested framework, each of the main three themes reflects a specific element of TOE framework. Technical aspects (i.e., technological problems, perceived ease of use, and perceived usefulness) are the group of items that are associated with technology in which the innovation plays the central role in this theme. When it comes to social aspects, environmental context such as participants' everyday context and their resources are more influential on their drop-out behaviour. Accordingly, social aspects are more related to the participants and their personal context. Regarding the socio-technical aspects, the way of organizing the research, communication and interaction between different stakeholders, designing the tasks, and timing also influence drop-out behaviour. This group of factors is associated with the organizing the processes when it comes to TOE framework.
Our results also illustrate that the innovation-related items have greater influence on drop-out behaviour. We do not wish to imply that the process-related and participant-related items are not important. What we are arguing is that, when the innovation is not stable or is not sufficiently mature, or if it is not compatible with the participants' device, or when it is technologically complex, the participants are not able to continue participating in the living lab field test even if they do not want to drop out. Reflecting on the argument made by O'Brien & Toms (2008) that drop-out might be due to an internal decision of the participant or external factors that caused them to drop out, our findings showed that external factors (technological, environmental, etc.) exert greater influence on participant drop-out behaviour. Our suggestion is that the innovation should be as stable, easy to understand, and easy to use as possible and, if it is not possible to sufficiently simplify the field test, it should be divided into subtests. Moreover, the organizers of a living lab field test timreview.ca

A Taxonomy of Factors Influencing Drop-Out Behaviour in Living Lab Field Tests
A. Habibipour, A. Georges, A. Ståhlbröst, D. Schuurman, and B. Bergvall-Kåreborn

A Taxonomy of Factors Influencing Drop-Out Behaviour in Living Lab Field Tests
A. Habibipour, A. Georges, A. Ståhlbröst, D. Schuurman, and B. Bergvall-Kåreborn must make the participants aware and well-informed about the whole process of the field test by providing them clear, accessible, and comprehensible guidelines before and during the field test.
The presented taxonomy can be put to work in several ways. For instance, we believe that there is a need for practical guidelines that describe what the organizers of a living lab field test should do and how they should act in order to keep participants motivated and reduce the likelihood of drop-out throughout the innovation process. This taxonomy can be used as a framework to develop such practical guidelines for the field test organizers. As another example, this taxonomy might be used as the basis to develop a standard post-test survey to identify the reasons for drop-out in various field tests in different living labs.
However, our study has limitations. One limitation was that the drop-out reasons was extracted based on the field tests in two living labs (namely, Botnia Living Lab and imec.livinglabs). Therefore, we might not be aware and well-informed about the way that other living labs set-up, organize, manage, and conduct their field tests, and consequently, the drop-out reasons could be different in those field tests due to many reasons such as cultural factors. Furthermore, drop-out behaviour might be associated with other influential factors such as degree of openness, number of participants, level of engagement, motivation type, activity type, and longevity of the field test. As an example, fixed and flexible deadlines to fulfill the assigned tasks might have resulted in different drop-out rates in a living lab field test (Habibipour et al., 2017). Therefore, these findings are tentative and might not be possible to generalize in different situations.
We also acknowledge the limitation of our study regarding the degree of influence of each factor on dropout behaviour. On the one hand, although the initial list of these factors were extracted from the dropped out participants viewpoint in our previous studies, the degree of influence of each factor was only evaluated by the experts in the area of living lab field tests based on their real experiences and views. On the other hand, the total scores for the influential factors were quite close to each other and even overlapped for some items. Therefore, due to the small sample size of respondents, the results might be changed slightly if one more or one fewer respondent were included. In future studies, one way to overcome this limitation would be to use 5-point scoring in order to gain greater resolution of differences and to show averages instead of total score. Finally, future iterations of this work should triangulate our data by including the perspective of dropped-out participants in a more longitudinal study by utilizing different data collection methods and techniques (e.g., interviewing the dropped out users and even those who have completed the test). The limited number of interviews (14 interviewees) can also be considered as another limitation of this study, and further interviews would have made the information even richer.
This study also opens up avenues for future research.
As O'Brien and Toms (2008) have introduced re-engagement as one of the core concepts of their user engagement process model. An interesting topic for further research would be to clarify how and why user motivation for engaging and staying engaged in a living lab field test differ. Moreover, it is important to study how the organizers of a field test can re-motivate dropped-out participants in order to re-engage them in that field test and to examine the benefits of doing so. Another opportunity for future research is to understand patterns of reasons that lead to drop-out behaviour, and thus different types of drop-outs. This would, however, require more respondents by using a more quantitative approach, given that such a large number of items scored by a small number of respondents might not provide robust results. Our hope is that the presented definition and the taxonomy can be used as a starting point for a theoretical framework in the area of this study.

A Taxonomy of Factors Influencing Drop-Out Behaviour in Living Lab Field Tests
A. Habibipour, A. Georges, A. Ståhlbröst, D. Schuurman, and B. Bergvall-Kåreborn About the Authors Abdolrasoul Habibipour is a PhD student in Information Systems at Luleå University of Technology in Sweden and is a User Engagement Expert at Botnia Living Lab, Sweden. His research focuses on user engagement in living lab research, with a particular emphasis on users' motivations and needs. Abdolrasoul has previously worked in information technology (IT) projects for more than 13 years as a project leader and project manager as well as software designer and developer. He is currently involved in international innovation and research projects such as UNaLab project, U4IoT project, as well as Privacy Flag project, all of which are financed by the European Commission. Birgitta Bergvall-Kåreborn is Vice Chancellor and Professor in Information Systems at Luleå University of Technology, Sweden. Birgitta's research interests concern design-oriented research focused on participatory design in distributed and open environments; human-centric and appreciative methodologies for design and learning; value-based information systems development; the increasing overlap between stakeholder participation and labour sourcing; and its consequences for value creation and value capture. She has published several articles within these areas, and she has participated in a large number of national and international research projects.

Introduction
Co-creation has been an emerging trend in the business development of companies in the 21st century (e.g. Sanders & Stappers, 2008). Adopting the principles of the lead user method (von Hippel, 1986(von Hippel, , 2005, the concept of co-creation was originally developed and popularized by Prahalad and Ramaswamy (2004). According to those two scholars, co-creation is the value that is generated together by a company and their customer -the customer co-constructs the service experience to suit their own context (Prahalad & Ramaswamy, 2004). Sanders and Stappers (2008) further elaborate cocreation in relation to design development process and see co-creation as an act of collective creativity shared by two or more people. They state that, through codesign, collective creativity can be applied across the entire design development process: co-design is an explicit instance of co-creation where the creativity of designers and ordinary people meet and work together. Thus, co-creation can be seen as "a way of working" rather than as a set of certain methods (Sanders & Stappers, 2008), and the co-creation experiences of the consumer become the very basis of value context (Lusch & Vargo, 2006;Prahalad & Ramaswamy, 2004). In this article, co-creation refers to the way of working to develop new solutions together with end users right from the early stages of development.
According to a recent report, 58% of businesses have piloted co-creation projects to help them innovate, 54% of businesses say that co-creation has helped improve their social impact, and 49% of businesses work with consumers on a regular basis (Hitachi, 2018). Although it seems that everyone is co-creating, actual success in co-creation depends on selecting and properly using appropriate methods and processes, because they can significantly affect project outcomes (Piller et al., 2010;Steen et al., 2011). Furthermore, small and mediumsized enterprises (SMEs) often do not have the resources, or all the needed competencies, to carry out the innovation activities (e.g., Ståhlbröst & Holst, 2013).
These challenges highlight the importance of "living labs" as recognized providers of innovation tools and methods. As a key element of the living lab approach, co-creative innovation processes are effective and result in innovations that create value for end users (Krogstie et al., 2013;Ståhlbröst & Holst, 2013). Hakkarainen and Hyysalo (2016) also underline the role of the organizer of co-creation, arguing that the success of real-life collaboration depends on how the co-design This multiple case study focuses on co-creation facilitated with innovation instruments in three different environments -a school, a hospital, and an airport -in which 12 SMEs and startups developed solutions based on predefined needs of customer organizations, and where stakeholders actively participated through user involvement methods facilitated by a living lab. The article provides new knowledge regarding the benefits of the co-creation, user involvement, and use of the living lab approach within different contexts. Our findings show concrete benefits of co-creation for stakeholders such as companies, customer organizations, and end users. Based on our results, we propose a new, generic model for using innovation instruments to facilitate co-creation for the development of needs-based products and services in different service domains.
What could be better than listening to an end user telling a potential customer about the need for our solution?
CTO of a participant company " " timreview.ca

Lotta Haukipuro, Satu Väinämö, and Pauliina Hyrkäs
process has been orchestrated, facilitated, and managed. Yet, according to, for example, Leminen and Westerlund (2017) research into innovation tools and methods within the living lab approach is scarce.
The living lab approach, relying on the user innovation (von Hippel, 1976) and open innovation (Chesbrough, 2003) paradigms, has been adapted to an increasing extent in the development of new products and services. However, in the living lab literature, there is a broad variety of definitions (Leminen, 2015). We rely on Ståhlbröst's definition of a living lab as "an orchestrator of open innovation processes focusing on co-creation of innovations in real-world contexts by involving multiple stakeholders with the objective to generate sustainable value for all stakeholders focusing in particular on the end users". The common elements of a living lab include a multi-method approach, end-user engagement, multi-stakeholder participation, a real-life setting, and co-creation. The living lab approach strives for mutually valued outcomes that are results of all stakeholders being actively engaged in the process from the early phases (Malmberg & Vaittinen, 2017). Leminen and Westerlund (2017) point out that, by using appropriate tools, living labs can significantly foster the emergence of innovation. They categorize living labs by the usage of innovation tools, and identify a living lab type that relies on both iterative, nonlinear innovation processes and customized tools. This type of living lab has prior experience and knowledge of innovation activities but wishes to keep the innovation activities flexible, which increases the likelihood of fruitful outcomes.
The aforedescribed research areas together combine a larger whole to which this article aims to contribute. The purpose of this article is to explore an innovation instrument as "a way of working" to create new solutions for the needs from different service domains. We seek to contribute to the discussion of innovation challenges by shedding light on the benefits of a facilitated innovation process with the living lab approach. Through empirical findings from three innovation instruments, we aim to show how diverse companies' solutions are efficiently co-created in different contexts, yielding improved solutions of the companies and accelerated innovation processes based on the needs of the customer organizations. Here, innovation refers here to the innovation definition comprised by Skillicorn (2016): "Executing an idea which addresses a specific challenge and achieves value for both the company and customer." By instrument, we mean "a means whereby something is achieved, performed, or furthered" (Merriam-Webster, 2018), thus innovation instrument means furthering in-novation in a facilitated process. In particular, by innovation instrument, we mean a facilitated process during which a selected group of SMEs and startup companies co-create new solutions for the specific needs of the customer organizations.
In this study, the co-creation was facilitated by Oulu Urban Living Labs (OULLabs; oullabs.fi/en) in Finland, which has provided user-centered development services for the local innovation ecosystem since 2010 (Anttiroiko, 2016;Haukipuro, 2014Haukipuro, , 2016 and therefore has established long-term collaborations, for example, with the City of Oulu. Besides the living lab specialist services and face-to-face user involvement methods, a digital user community and user involvement tool was used for co-creation. The three cases were selected because they provide new knowledge of the improvement of innovation culture through co-creation from diverse environments and, through comparative analysis, enable forming a basis for a generic innovation instrument model. We focus on how the stakeholders, such as companies, customer organizations, and end users, perceive the benefits of co-creation through innovation instruments and how the co-creation should be facilitated.

Research Design
This study follows a multiple case study design to examine three distinct innovation instruments -"Agile Piloting", "IdeaSprint", and "Innovation Path" -which were developed in the national Six City Strategy program (6Aika, 2015) -across 12 separate company cases as research subjects. The multiple case study design enables the analysis of data within each case but also across different situations, aiming to understand the similarities or differences of the cases and increasing the validity through multiple sources of evidence (Baxter & Jack, 2008;Gustafsson, 2017;Stake, 1995;Yin, 2003). The three individual innovation instruments are applied in within the same geographical area but focus on different service domains: education (Agile Piloting), aviation (IdeaSprint), and healthcare (Innovation Path). The instruments were targeted at SMEs and startups (hereinafter companies) to develop new solutions for the needs of the three customer organizations in Oulu, Finland: the local school, the local hospital, and the national airport operator). The domains were selected by the Six City program (6Aika, 2015), and each particular instrument was designed for its particular domain. IdeaSprint is suitable for companies as customers, Agile Piloting for a public sector customer, and Innovation Path for a healthcare customer due to a need for intensive and longer-term co-creation. timreview.ca

Lotta Haukipuro, Satu Väinämö, and Pauliina Hyrkäs
The development of the companies' solutions to meet the needs of the customer organizations and end users focused on co-creation facilitated by Oulu Urban Living Labs. OULLabs is a non-profit living lab founded in 2010 at the University of Oulu, Finland, which aims to provide a diverse environment for innovation, research, development, and testing of new applications and services in an authentic environment with real users and thus to expedite competitiveness of the companies. More than 100 living lab projects have been conducted in the OULLabs environment. In particular, user involvement through different methods provided by the living lab enabled the co-creation in each innovation instrument. For example, a user involvement method used in all three innovation instruments was adapted from the World Café method introduced in 1995 by Brown and Isaacs. The idea of the method is to create a café-like setting, which enables groups of people to participate in evolving rounds of dialogue to bring forth new insights (Brown & Isaacs, 2005). Given that the target group of each innovation instrument was a group of companies, the World Café process was modified by the living lab to ensure the utmost benefit for the devel-opment of the companies' solutions through parallel, systematic, end-user involvement in a joint event, as described in detail in the next sections.
The specific elements of the three innovation instruments are depicted in Figure 1, but all three instruments share four overall phases: 0. Preparation, 1. Selection, 2. Co-Creation, and 3. Piloting. In this article, we focus on the co-creation phase.

Participant companies and customer organizations
The 12 companies were distributed across the three innovation instruments as follows: four companies were selected to Agile Piloting, three companies to IdeaSprint, and five companies to Innovation Path. The customer organization/company decided the selection criteria and how many SMEs were selected to participate.

Lotta Haukipuro, Satu Väinämö, and Pauliina Hyrkäs
The three customer organizations and their development goals are described in Table 2.

Innovation instrument 1: Agile Piloting
The original agile piloting concept (Mustonen, 2015) was developed by a national organization and tested in urban development projects in Finland. The aim of the concept is to provide companies an opportunity to pilot prototypes and services in an authentic environment with real end users. In the Agile Piloting instrument referred to in this article, the aim was to find and test new solutions for a school environment by a faster process compared to the original concept. The development needs of the selected pilot school (i.e., the customer or-ganization) were identified in consultation with the school's teachers, pupils, and parents. Based on the identified needs, a public tender was opened for companies. In total, 15 tenders that described the companies' solutions were submitted. Next, an evaluation committee ranked the tenders and selected the four promising ones to enter the program. These four companies (see Table 1) were given an introduction to cocreation and user involvement, after which the companies, together with the living lab, planned the methods and tools to be used in the experiments at the school. During the process, the companies were supported to independently collect the first end-user experiences in the school. At the end of the Agile Piloting, a

Innovation instrument 2: IdeaSprint
In recent years, a city-owned enterprise providing innovation services for local companies in Oulu has organized rapid ideation events or "sprints". The aim of these sprints is to find new solutions for real problems/needs of customer companies who share their needs in the form of challenges for developers, for example, SMEs and startups who pitch their ideas and receive immediate feedback on them. In this study, we focus on a particular instance of this event in which the co-creation phase organized by a living lab was included for the first time. It represents our second innovation instrument, IdeaSprint, which proceeded as follows.
During the event, local companies were presented with a challenge through which representatives of the national airport operator (i.e., the customer organization) introduced their development needs regarding airport services for local companies. To address this challenge, the companies ideated and planned solutions that were then discussed in a "sparring session" with the customer organization. In a pitching event, eight companies presented their solutions, of which the three best (see Table 1) were selected by the customer organization. Following the baseline mapping conducted for these three companies, the living lab supported them in the planning of end-user involvement activities. The preliminary user experiences were collected through surveys in an online tool provided by the living lab. At the end of the IdeaSprint, a joint user workshop was organized at the local airport. In the workshop, the World Café method was applied: two companies further ideated their concept with mixed user groups formed from 10 end users/passengers and four customer company representatives. One company solution of the three was mature enough to be tested in practice, enabling user-product interaction and the collection of user experiences.

Innovation instrument 3: Innovation Path
The third innovation instrument was a one-year pilot project in the healthcare sector, hereinafter Innovation Path. The objective was to create new healthcare solutions for the pre-defined needs of the local hospital (i.e., the customer organization), which would be achieved by developing and testing a process through which companies and healthcare professionals together would cocreate new hospital services. During the preparation phase, the needs were collected from two service areas of the hospital. Based on the identified needs, a call for solutions was opened to companies and developers. Altogether, 24 applications from 15 companies and two developers were received. An evaluation team of 65 professionals from healthcare, information technology (IT), and business areas was formed to rate the applications using an online tool provided by the living lab.
Five applications -two idea-level and three conceptlevel solutions (see Table 1) -were selected to enter the Innovation Path to be further developed through different co-creation methods and tools tailored on the basis

Lotta Haukipuro, Satu Väinämö, and Pauliina Hyrkäs
of their maturity and needs. Co-creation activities were organized in cooperation with the hospital innovation personnel to engage suitable healthcare professionals to participate in the co-creation activities. The detailed needs of the hospital and the aim of co-creation were introduced to the selected companies, after which the co-creation with professionals took place in form of different activities. More than 60 healthcare and IT professionals participated in the co-creation. First, a conversational workshop was organized, during which the modified World Café method was used to collect initial feedback from the groups of healthcare and IT specialists. Next, each concept was further developed through online methods such as a survey, online discussion, or user diary, depending on the nature of the solution. Finally, individual user-testing sessions were carried out by pairs of users interacting with the companies' prototypes in a hospital testing environment. In the final evaluation event, each company presented their solutions to an evaluation group of over 20 healthcare professionals from different fields, who then made a decision on further cooperation.

Data Collection
Research data were collected through multiple methods, which is typical for qualitative case studies (Eisenhardt, 1989). Table 3 provides an overview of the primary and ancillary research data used in this study. The primary data mainly consists of the key informants' semi-structured in-depth interviews regarding each innovation instrument: the company representatives of the 12 case companies, the customer organization representatives and professionals, and the facilitators of each innovation instrument. The interviews were audio recorded and transcribed. The transcripts were explored multiple times to develop in-depth understanding. In addition, the living lab researchers were involved in the processes of all three innovation instruments by observing and taking notes. Ancillary data comprise various materials such as companies' applications, feedback discussions, and questionnaires used for different purposes. The qualitative cross-case analysis was conducted for comparison of the cases to increase the explanatory power of the study (Eisenhardt 1989;Halinen & Törnroos 2005) and to enable the triangulation of data in order to increase the reliability of the study (Denzin 1973;Eisenhardt, 1989). The principle in the data analysis was to systematically seek connections, recurrences, and alterations from the primary and ancillary data and draw out patterns. Table 4 summarizes the methods of user involvement used in the three innovation instruments. The two methods used in all three cases were i) discussion/sparring, which was conducted in individual meetings with each participant company in the beginning of the process of each case, and ii) the modified World Café method used in the workshops. The other methods, such as online methods, user testing, and the focus group were not used in all cases, mainly due to the different durations of the cases but also due to the nature and maturity of the companies' solutions. In each case,

Lotta Haukipuro, Satu Väinämö, and Pauliina Hyrkäs
the methods were tailored to the groups of companies (e.g., a joint workshop) and for each company separately depending on the maturity level and nature of the solution, for example. The goal was to find the most appropriate way to enhance the further development of the solutions. For example, in the joint workshops, each company applied a different type of user involvement method, such as user testing, a questionnaire, or a focus group discussion according to the plan. Moreover, online methods (e.g., surveys, online discussions, and user diaries) were generally aimed at concept-level solutions based on earlier experiences (Haukipuro et al., 2016).

Findings and Discussion
In general, the participant companies perceived the cocreation activities provided by the living lab beneficial, frequently describing the overall process as "easy" and "well organized". According to the findings, the innovation instruments were particularly beneficial for B2B companies as they enabled direct contact with the large and desirable customer organizations, which would otherwise have been difficult for them to reach. Moreover, the IdeaSprint companies highlighted the commitment of the customer organization and appreciated that there was "an atmosphere of innovating together" during the whole process. The findings are presented in more detail in the subsections that follow.

Co-creation activities
The modified World Café method was a liked and suitable way to collect feedback from end users, regardless of the solution's maturity level. The method was used in the workshops organized in different phases: in the Innovation Path, the workshop was held already in the beginning of the process, when the companies did not have concrete plans or prototypes to discuss. Therefore, the workshop was rather a conversational, interactive event allowing the hospital professionals from different fields to meet with each company in groups. In the IdeaSprint and Agile Piloting, similar workshops were organized in the later phases of the process, when companies had a prototype or a testable product. Regarding the IdeaSprint workshop organized at the airport (Figure 2), both the companies and the customer organization thought it as a success. Particularly valuable for them were the mixed groups: in each group, there were end users/passengers and airport operator representatives, which enabled direct communication between the company, service provider, and end users. As one company representative put it: "What could be better than listening to an end user telling a potential customer about the need for our solution?" (Company #9) In another example, a representative of the IdeaSprint customer organization did not have particularly high expectations about the workshop organized at the airport it; rather, they had a skeptical attitude based on earlier experiences of workshop outcomes. However, the successful workshop totally changed the representative's view about the end-user involvement. Another representative also regarded the workshop as a great event, pointing out the importance of involving the end users:

Lotta Haukipuro, Satu Väinämö, and Pauliina Hyrkäs
"It was the part of the process that exceeded the expectations most, and it was a good session, really worth participating in." (Customer organization representative #1) In the Agile Piloting, given that there were user-involvement features built into the companies' solutions, the co-creation activity organized by the living lab focused on the workshop implemented with the modified World Café method at the local school ( Figure 3). In addition, each company applied their own type of methods at their booths (Table 5). The findings show that companies regarded the workshop outcome as useful and end users also enjoyed participating in the workshop.
In the Innovation Path (Figure 4), the initial feedback was collected after various co-creation events using feedback forms and, later, during interviews. Overall, the hospital professionals were enthusiastic about new ways of enabling them to provide their expertise for the development of the companies' solutions. According to the professionals' feedback, the different events were successful, with typical descriptions being "great" and "inspiring". In particular, the conversational nature of the events was considered very beneficial. Although the strict hierarchy of the hospital environment was apparent in the Innovation Path process, the co-creation workshop participants, regardless of their work role, felt they could speak freely and express their real opinions on the matters: "Although it demands courage to participate, it is important -otherwise, we nurses cannot be heard." (A hospital professional) The professionals emphasized their involvement in the early phase as a way to ensure that the development of the solutions is on the right track. The most liked activity among the hospital professionals was the testing session as, for the first time, they saw concrete prototypes and could test them in practice. The user testing of each company solution was organized as paired testing with four to five pre-selected pairs of professionals in a hospital test environment ( Figure 5). The living lab user researcher guided the session with a precise assignment,

Lotta Haukipuro, Satu Väinämö, and Pauliina Hyrkäs
but observed in the background during the actual testing. The paired testing setup worked well and, according to the participants, the testing situation was "interesting" and "felt realistic".

Online methods
An online user involvement tool was used in two cases (Table 4). In the IdeaSprint, all three finalist companies conducted a survey about their concept using the online tool. The aim of the survey was to find out the preliminary thoughts of the end users. According to one company, the online tool was very beneficial as the survey results formed the basis for discussions in the workshop at the airport. Two companies appreciated the combination of online tools and traditional workshops, whereas one company considered user testing as the best way to collect user experiences. Considering the Innovation Path instrument, the findings revealed an adoption barrier to online tools, which were not that familiar to all hospital professionals. The online tool was used two times during the process: for the evaluation of the companies' applications and for the evaluation of the companies' concepts through a survey, an online discussion, or a user diary. There was also a clear division in how the companies experienced the online tool. The online involvement revealed that one company's solution had not developed towards concept phase but was rather still an idea, which greatly affected the online feedback collection. Thus, the professionals' feedback gathered through online methods was not so beneficial for the development of this solution. The other companies collected professionals' feedback through the online discussion and surveys at the online tool, but their experiences ranged from "quite useless" to "beneficial".

Impact of the end-user feedback on the solutions
All IdeaSprint companies could name concrete impacts of the end-user feedback. They pointed out that the user feedback confirmed their previous thoughts or plans, brought up issues that they had not detected earlier:

"We found out what (features) our solution should include and what the entity the offer for piloting would comprise." (Company #9)
Also, all the Innovation Path companies reported having made some concrete changes in their solutions based on feedback from the hospital professionals (both medical and IT) . The feedback on one solution in particular had a tremendous impact: the original plan was totally changed, including the size, appearance, and use logic of the product. The companies also reported the impact being tangible through small changes in solutions and, for example, the discovery of use scenarios related issues during the testing session:

Lotta Haukipuro, Satu Väinämö, and Pauliina Hyrkäs
In the Agile Piloting, the few companies that by themselves actively collected feedback from end users from the earliest phase benefited the most and could utilize this feedback to improve their solutions during the process. For the rest, the impact of the feedback confirmed the companies' previous thoughts. However, the overall experiences of the Agile Piloting companies regarding the whole process were positive. They reported having gained valuable user experience and customer understanding to be exploited in their future development. The fast, non-bureaucratic experiment enabled fast decision making about whether to continue the development of the solution or to quit it:

"We learned that the experiment is worth it -and that end users know better what they want than the company that develops the solutions." (Company #8)
Role of the facilitator Based on the empirical findings, the role of the living lab as organizer, manager, and facilitator of the co-creation in all the cases was crucial. The participant companies valued the support from the living lab in the preparation and organization of the co-creation activities. According to several companies, the process was made "smooth" and "easy" for them because all practical and resource-demanding arrangements such as participant recruitment were taken care of by the living lab. Thus, the stakeholders considered the workshops with the World Café method as a "fruitful" and "pleasant", mainly thanks to successful pre-arrangements and an "encouraging" and "inspiring" atmosphere. As one company representative put it: "We could focus on substance because the other aspects were taken care of by the organizer." (Company #4) Moreover, the expertise of the living lab researchers was highly appreciated. For example, according to the hospital professionals, it made the co-creation events effective and meaningful. In particular, the hospital professionals regarded the arrangements and facilitation of the user-testing sessions as "great", pointing out that there were no co-creation procedures or related expertise in their organization. The living lab's support in the sparring and in preparing the end-user involvement (e.g., online) was also mentioned as important because the small companies did not have the resources or competences to properly organize such activities. Overall, the companies and customer organizations estimated that, given the support and services provided by the liv-ing lab, they "saved a lot of resources". From the point of view of the customer organizations, the co-creation facilitated by the living lab was the most important element to achieve successful results.
The overall outcomes obtained by each company in each three innovation instruments are described in Table 5.

The model of innovation instrument with co-creation
Based on the findings, a generic model of innovation instrument with co-creation was synthetized from the three innovation instruments explored in this study. Figure 6 depicts the new model, which includes three phases: preparation, co-creation, and piloting. The preparation phase contains the selection of a customer organization. In selecting a customer organization, the context should be interesting and the promise should be sufficiently attractive -it must be more than simply a promise of further discussions, as was the case with Innovation Path. For example, in IdeaSprint, the companies were promised a paid pilot. Based on the findings, the co-creation phase and facilitated user involvement has been extended to cover the needs collection with the end users of the customer organization. In addition, findings show that co-creation should also cover the selection of SMEs in terms of interactions between stakeholders to ensure the mutual understanding of the development need and the maturity of the solution. The maturity level of each company's solution is assessed, and the suitable "innovation path" and methods tailored to the companies' needs. Tailoring requires strong user involvement knowledge from the facilitator. The co-creation activities are conducted for the group companies whose solutions are at the same maturity level, which saves the resources of the all stakeholders involved and accelerates the innovation process, however, taking into account the individual needs and fit of the method for each company. At the end of the co-creation phase, the company solutions are tested with end users, after which they should be ready for further steps such as piloting in customer organizations or procurement. Ideas and concept-level solutions require more effort and co-creation activities compared to early prototypes. Stakeholders involved for each phase vary from customer organization and facilitator in the preparation and piloting phases to the broader group of stakeholders in the co-creation phase. In particular, the role of the facilitator (e.g., the living lab) is crucial from the collection of the development needs to the piloting to ensure the proceeding and successful outcome of the process. timreview.ca

Conclusion
The purpose of this article was to increase knowledge of the benefits of co-creation in a facilitated process in different contexts. Our empirical findings indicate that cocreation can have a significant role in the development of needs-based solutions. Through a variety of living lab user involvement methods tailored to the needs of the companies (Leminen & Westerlund, 2017), promising results were achieved both for the companies and customer organizations as well as for the end users. In this study, the co-creation was implemented in an efficient manner through innovation instruments for the groups of selected SMEs and startups, instead of separate time-and resource-consuming activities. Moreover, the suitability of the co-creation activities was ensured by tailoring them according to the maturity level of the 12 solutions. The companies were guided through the facilitated process in each innovation instruments. The outcomes indicate the potential of innovation instruments with co-creation in order to efficiently develop new solutions that meet the end-users' needs. Hence, a generic model for innovation instrument with co-creation was formed.
In line with previous research, for example by Steen and colleagues (2011), our findings show that, by involving end users in the early stage of the solution development, companies receive concrete benefits such as valuable insight regarding their ideas and concepts, which they can then take into account in the development. In the later phase, user testing provided information of the usage and revealed issues to be considered in the further development of the solutions. The findings based on the three different innovation instruments show that co-creation activities, irrespective of the nature of the development environment (e.g., school, hospital, or airport) were regarded as an effective way to develop user-friendly solutions that meet the needs of the customer organization. Although the use of online methods distributed the opinions, the combination of the online and face-to-face methods was seen as fruitful. Overall, the findings indicate the significance of the living lab as the organizer and facilitator of co-creation activities, largely due to the lack of resources and competences in companies (Sanders & Stappers, 2008;Ståhlbröst, 2013). Due to the varying timeframe of the innovation instruments and the amount of the co-creation activities, the most powerful impact was obtained in the longest innovation instrument, the Innovation Path. However, the two shorter innovation instruments yielded promising results as well.
To summarize, the main benefits of innovation instruments with co-creation facilitated by the living lab for participant companies were: i) an easy, tailored, and low-resource-demanding, multi-method co-creation process; ii) a co-operation opportunity with a desirable Pauliina Hyrkäs, BHSc (Health Management), Northern Ostrobothnia Hospital District (NOHD), Finland, has worked in local, national, and international healthcare sector projects as a project manager, a coordinator, and a designer. With her project team, she developed nationally applicable innovation process for Finnish University Hospitals as well as the Development and Innovation Process for the NOHD. Currently, she is starting up the innovation activities at the Oulu University Hospital (OYS), designing the innovation process for the Future Hospital OYS 2030 and for the innovation ecosystem around the OYS as well as the healthcare co-creation process to be utilized at the European Union level. Lotta Haukipuro, Satu Väinämö, and Pauliina Hyrkäs customer; iii) direct interaction between company, customer and end users during the process; iv) obtained knowledge and experience of the impact of co-creation and end-user involvement; v) improved, co-created products meeting the needs of the end-users and customers; and vi) a valuable reference and use case to support new business of the companies. Continued cooperation in terms of piloting or other activities with the customer organization was an outcome in many cases, but other cases resulted in the cancellation of product development, which may have prevented future loss of investment by the companies. From the perspective of the customer organizations, an accelerated innovation process of pre-defined products and services was gained, resulting in company solutions that meet the needs of customer organization and their end users. Keywords: innovation instrument, innovation process, living lab, co-creation, SME, startup, user involvement

Introduction
Modern-day manufacturers expand their product lines via implementation, maintenance, upgrades, and a lifecycle approach, offering not just a product or equipment, but an outcome. New technologies, such as Internet of Things (IoT) devices, sensors, and big data are making it easier for manufacturers to monitor, analyze, and manage their products on the market, thus further driving the servitization trend. The term "servitization", used to describe the transformation journey of a manufacturer, was first invented in the 1980s (Vandermerwe & Rada, 1988). Its origins can be traced to the 1960s when Rolls-Royce created its "power by the hour" concept, whereby the use of a fully-maintained aircraft engine was sold by the hour rather than by the unit. Servitized firms are increasingly offering this type of service provision for several reasons, ranging from the need to identify a new competitive source or avoid price competition, to the desire to add value to traditional manufactured products while competing in an increasingly globalized market. Servitized firms are also seeking to innovate and sell solutions that meet customers' needs more comprehensively to avoid competing solely on a cost basis.
Regardless of its touted potential, servitization often produces mixed, underwhelming results in practice (e.g., Suarez et al., 2013). Mixed results are fundamentally attributable to the challenges in developing and implementing service-oriented business models (Gebauer, 2009;Gebauer et al., 2005;Kowalkowski & Kindström, 2013;Martinez et al., 2010). Bearing in mind that the challenges associated with developing and implementing a servitization business are the fundamental reasons for underwhelming servitization results, this study seeks to address one foundational element of such businesses, namely value propositions Payne & Frow, 2014;Storbacka, 2011). It is prudent to focus specifically on value propositions because it has been demonstrated that they have been successfully developed and communicated by less than 10% of companies , indicating the extent of their untapped potential. The development of value propositions is associated with innumerable benefits, especially in a servitization setting that demands new capabilities and management practices. In practice, servitization is fundamentally about changing seller-and product-based value propositions to customer-and service-based proposals.
This study was conducted in response to calls from the research community and industry for a greater empirical exploration of value propositions. It uses customer valuein-use as a starting point and employs empirical data on value propositions in a servitization context. The findings demonstrate how customers articulate the value-in-use, or benefits, of a selected offering. These results are subsequently used to develop value proposition elements that are aligned with these benefits. The implications for the value proposition literature and for companies in a servitization situation are discussed.
Your customers are the judge, jury, and executioner of your value proposition. They will be merciless if you don't find fit!

What Do Business Customers Value? An Empirical Study of Value Propositions in a Servitization Context Kwesi Sakyi-Gyinae and Maria Holmlund
There is a need, when developing value propositions, to understand what customers consider to be valuable. Smith, Maull, and Ng (2014) similarly assert that the "customer's value of a product could lie in the benefits they attain from the product instead of product ownership, suggesting that the provider could shift focus from the means of achieving such benefits (the product) to the benefits themselves." This study takes this customer perspective as the starting point for value propositions and uses Macdonald, Wilson, Martinez, and Toossi's (2011) customer-based definition for such value-in-use: "The benefits that accrue to customers and enable them to achieve their own business goals, purposes, objectives and/or priorities as result of engaging their resources with a provider's offering." A customer's conception of these benefits or value-in-use may be unknown to the seller or may differ from the seller's (Strandvik et al., 2012).
Indeed, many manufacturers assert that they are customer oriented but this might only apply superficially, while not probing deep enough to uncover how a customer really acts and thinks. Examples of this gap include a company having a product rather than a value-in-use perspective, not having an interest in or being unable to understand customer thinking regarding decisions and priorities, or not grasping differences in different reasoning approaches held by customers. Recognizing the customer's view will enable the manufacturer to develop value propositions that resonate with the customers' conception of value. This can then enable mutual value for the manufacturer and the customer and thereby ultimately foster servitization.
Using customer value-in-use as a starting point, the purpose of this study was to contribute knowledge to research and practice in which empirical data are utilized for value propositions in a servitization context. The study was conducted in response to calls from the research community and industry for a greater empirical exploration of value propositions. The case study findings demonstrated how customers articulated the valuein-use of the selected offering and are described herein. These results were subsequently used to develop value proposition elements that are aligned with customer value-in-use. The outcome of this matching is presented in this article. The case corporation, called ABC Global (for the purpose of confidentiality in this article), operates in the manufacturing industry. This successful company, established almost 100 years ago, is stock listed today and has almost 2,000 employees worldwide. The company is typical for manufacturing companies pursuing a servitization strategy in that it was established as an original equipment manufacturer (OEM) but has in the last few decades pursued a gradual transitioning into service business in most of its business units.

Literature Review
Although the call for studies that address value propositions based on empirically founded customer value-inuse in a servitization context has surged in recent times, surprisingly few studies have taken up this call.
The most relevant studies are those by Ng, Parry, Smith, Maull, and Briscoe (2012), Smith, Maull, and Ng (2014) and Macdonald, Wilson, Martinez, and Toossi (2011). Ng and colleagues' (2012) study on a selected case reported 11 value-creating attributes and calculated efficient bundles from the perspective of the seller's resources and costs. Smith and colleagues (2014) conducted a study of an equipment manufacturer and reported four nested value propositions: asset, recovery, availability, and outcome. Arguably, that study was more seller oriented because the interviews were conducted among the seller case company's employees. Macdonald and colleagues (2011) interviewed buying groups and suggested a model for assessing customer value that looks at elements such as usage process quality, relationship, service quality, and value-in-use.
Value propositions have sometimes been equated with a "silver bullet" statement asserted by the provider to the customer (Yu-Lee & Haun, 2006). However, this statement-only approach is incomplete. Previous literature (e.g., Anderson et al., 2006;Ballantyne et al., 2011;Barnes et al., 2009) contends that robust, well-crafted value propositions comprise three key elements that will be used in this study: value points, value statements, and value substantiation. According to Anderson and co-authors (2006), the value point of customer value propositions can be: points of parity (similar elements that yield the same functionality or performance as the next market option); points of difference (unique elements that make providers' offerings stand out in the market as superior or inferior); and points of contention (elements that customer and provider disagree on in terms of functionality when compared to the market). After the manufacturer has undertaken customer research to understand value-in-use, it is then imperative to understand these value points in the offering. A value statement is a "clear, compelling and credible expression of experience that the customer will receive from a supplier's measurably value-creating offer" (Barnes et al., 2009). It is a concise way for manufacturers to express their value-adding intentions to the client. A supplier's value statement succinctly articulates timreview.ca

What Do Business Customers Value? An Empirical Study of Value Propositions in a Servitization Context Kwesi Sakyi-Gyinae and Maria Holmlund
the technical, economic, service, or social benefits that deliver value to customers (Anderson et al., 2006). This creates an understanding of how future conditions will differ from present conditions -for example, higher operational efficiency, greater rate of revenue generation, or lower cost incurrence (Yu-Lee & Haun, 2006).
There are tried and tested approaches to value substantiation. The three most relevant approaches in the context of this study are value equations, value case histories, and value calculators (Anderson et al., 2006;Barnes et al., 2009). Value equations express in words and simple mathematical operators how to assess differences in functionality or performance between a supplier's offering and the next-best alternative. Value case histories document the cost savings or value added that reference customers have actually received from their use of the supplier's offering. Value calculators are spreadsheet software applications that are used to demonstrate the potential value that customers will derive from a supplier's offering. While some of these approaches may be suitable for transaction-oriented services, others are preferable for more complex service offerings (Barnes et al., 2009). Also, these elements could cover technical, social, and emotional benefits among others (Anderson et al., 2006).
Based on the essential elements of customer value-inuse and value propositions found in the literature review, two research questions were formed to guide the empirical study: • What benefits does a product offering provide to customers?
• What value propositions can be developed that resonate with these benefits?

Method
A case study is defined as "a strategy for doing research which involves an empirical investigation of a particular contemporary phenomenon within its real-life context" (Robson 2002). Such an approach enables a deeper understanding of elements expedient for developing value propositions in a servitization context (Saunders et al., 2009). In this study, the case company is suitable because of its disposition along the continuum of servitization from product oriented to use or result orientated and thus presents a fruitful opportunity for investigating value propositions and how they can be developed based on the customer's perspective. The data were generated primarily through customer interviews. Nonethe-less, as a foundation for the customer interviews, an extensive familiarization and understanding of the case firm was conducted through more than 60 hours of internal expert interviews, reviews of more than 300 pages of company material (including annual reports, offering brochures, service strategy documents, and benchmarking study reports), and product demo videos. Six customer firms (termed Alpha, Beta, Gamma, Delta, Epsilon, and Zeta) were selected to represent customer group profiles that exist in the business unit. This selection was based on the customer's industry of operation, size of operational capacity, geographical distribution, and the ownership of the case firm's current product system. Profitability was not considered in the selection. Ten interviews were conducted with informants who were facility managers, information technology (IT) program managers, operations engineering managers, directors of business process, or in charge of quality assurance. The sampling strategies are consistent with Patton's (2005) criterion sampling approach wherein the selection of companies and informants is based on predetermined criteria of importance. The interview topics covered the informant and their background and experience; the customer company's goals, priorities, and requirements for procurement and suppliers; and offering-related issues such as use and benefits. The interviews were conducted in English and lasted an average of 45 minutes.

Empirical Findings
The company ABC Global bundles the hardware and equipment that it offers not just with additional support services, such as maintenance, calibration, and repair, but also with a software component that is an integral part of the offering.
The aim of the current empirical study was to illustrate customer-articulated benefits in relation to a selected case. Therefore, a data-driven inductive approach was used for the analysis to record the benefits communicated by customers to be of value to them. Initially, various concrete benefits were extracted from the data. These were considered to be value-in-use drivers because they constituted concrete advantages that the customers considered to be invaluable having previously accessed the product offering. Altogether, 20 value-inuse drivers were identified from the interviews. Thereafter, the identified benefits were further classified into separate, broader value dimensions that were given labels and definitions. The outcome of this stage of the analysis was six value-in-use dimensions: system, infrastructure, integration, usage, relationship, and price. timreview.ca

What Do Business Customers Value? An Empirical Study of Value Propositions in a Servitization Context Kwesi Sakyi-Gyinae and Maria Holmlund
Each value-in-use dimension is summarized in Table 1 and includes the defined data, value-in-use drivers, and representative quotations from the interviews with customer firms.
Three key value points are applied in this study: points of parity, points of difference, and points of contention. Based on the value-in-use drivers and dimensions below, an analysis of ABC's current offering was conducted: • System: The seller and the customers currently disagree on the functionality compared to competing systems on the market (implying a point of contention).
• Infrastructure: The seller and the customers agree that the effort currently needed is less than for other options on the market and that this is a feature that makes an offering stand out as superior or inferior (implying a point of difference value point).
• Integration: The seller and the customers agree that the initializing effort currently required is less than that of other options on the market and that this is a feature that makes an offering stand out as superior or inferior (implying a point of difference value point).
• Usage: The seller and the customers agree that the performance of the ongoing operation of the offering is similar to other industry offerings (implying a point of parity value point).
• Relationship: The seller and the customers agree that the handling of customer problems is better with the seller than with other options on the market and that this is a feature that makes an offering stand out as superior or inferior (implying a point of difference value point).
• Price: The seller and the customers agree that the pricing model of service contract fees, as opposed to a one-off fee plus other variable costs, is better for the seller than other options on the market and that this is a feature that makes an offering stand out as superior or inferior (implying a point of difference value point).
Given that the structure of the newly proposed service offering is novel to the market and very few competitive offerings exist, competitive offering considerations (as suggested by Barnes et al., 2009) were not relevant and were not incorporated in the identification of value points. Of the three most relevant value substantiation techniques suggested by Barnes and colleagues (2009) and Anderson and colleagues (2006), the one that was considered most suitable for the purposes of this case study was value equations. Value equations are expressions in words and simple mathematical operators that demonstrate how to assess differences in functionality or performance between a supplier offering and the next-best alternative. In this case study, value equations were considered more suitable than value case histories and value calculators. Value case histories were deemed not realistic in this case because there no current customers for the new offering, meaning that there are not yet any reference customers. In the context of this study, value calculators were more appropriate in the latter stages of service development, when sales people need to demonstrate the value-selling approach to potential customers (Anderson et al., 2006). In order to achieve a robust value proposition, value substantiations have been aligned with the value points and value statements. The quantification was done to reflect the standalone customer value-in-use and was not compared to next-best market alternative. The three vital elements (value point, value statement, and value substantiation) were applied to this case, and each element was aligned with the value-in-use findings from the customers, as summarized in Table 2.

Conclusions
This research demonstrated how data from customers on their value-in-use of a selected offering could be used to develop value proposition elements to align with these insights. In so doing, it exemplifies the original value proposition elements described by Anderson, Narus, and Van Rossum (2006), namely value points, value statements, and value substantiation. Previous scholarly work has focused on one or two aspects (e.g., Barnes et al., 2009;Macdonald et al., 2011;Yu-Lee & Haun, 2006) but has not explored and demonstrated the coherence that can emerge when these elements are aligned with customer value-in-use.
A central premise of this study was the need to incorporate the customer value perspective into servitization and value proposition research; therefore, the new value-in-use dimensions warrant a supplementary discussion. We found that the benefits that accrue to customers from product functionality (i.e., the system dimension) influence customers' business priorities and goals. A provider's product functionality or performance is a significant resource in the customers' value-creating process and must be carefully considered in any value assessment framework in industrial timreview.ca

What Do Business Customers Value? An Empirical Study of Value Propositions in a Servitization Context Kwesi Sakyi-Gyinae and Maria Holmlund
markets. The manner in which customers use the provider's offering to achieve their own goals is a key value driver for the customer in terms of their transition to purchase the new offerings. Hence, understanding customers' IT resources in light of the provider's service transition strategies (particularly with a hosted solution) is a key consideration. A lack of understanding in this direction can hinder the crafting of value propositions that resonate with customers.
Usage and relationship were found to offer some benefits to the customers and enabled their own creation of value. In a significant way, quality, as articulated by the customers in this study, is typified by the ease of application use and sustained functionality. In the technology acceptance literature, perceived ease of use refers to the extent to which a person accepts that using a particular method would not have a cost and believes that using the technology will be effortless. In this sense, the ease by which software upgrades are executed is crucial.
In a servitization process, identifying the relationship dynamics that will enable the customer to achieve their own goals is important. Findings from our study show that the speed at which issues on the customer interface are resolved by the provider has a positive repercussion on the achievement of the customer's own goals and priorities. Examples here include quicker response time to problems and faster application troubleshooting. From a value-proposition perspective, this will enable providers to highlight enhanced gains in "faster" or "quicker" operations, which ultimately means monetary gains for the customer.
The findings in this study show a lucid distinction between infrastructure and integration as value-in-use dimensions. The former relates to the physical resources needed to utilize the provider's offerings, whereas the latter is concerned with the operational processes required to execute and initialize the system. An understanding of the importance of the value-in-use dimension of integration in servitization guides the provider to reflect on which optimal solution provider role to take on, either as systems sellers or systems integrators. Understanding pricing preferences and how they support customers' business goals is important for the manufacturers, particularly as servitization tends to move them from the traditional mode of a cost-plus pricing model to a value-based pricing model.
For managers, the findings of this study have three important implications. First, in their quest to evolve their business model in order to harness servitization potential, manufacturers need to thoroughly understand what customers value both in their present and proposed offerings. The six value-in-use dimensions reported in this study provide a springboard that can be used by managers to understand customer value. In themselves, these six dimensions remind managers that product and price considerations alone are not enough when ascertaining customer value; other value-in-use dimensionsthose relating to customers' own resources (i.e., infrastructure) and the way in which they are able to effectively combine (i.e., interaction) resources in an efficient and interactional (relationship) manner to achieve the goals that matter most to them -are also important.
Second, an important implication from this study for managers is the need to craft value propositions from three inter-aligned elements: value points, value statements, and value substantiation. Servitization (in relation to a result-oriented typology) often proves to be a new path for industry stakeholders -one that many customers will grudgingly journey along. Thus, managers must understand how each element of their business model value propositions can be aligned to provide a coherent message for customers. Any savings for the customer will mean a level of burden incurred by the provider in the form of key resources, activities, or partners; this burden will need to be assessed and the most profitable and sustainable cost structure determined. Servitization in manufacturing is about value and the measurable benefits that will accrue to help the customer achieve their own business goals.
Finally, communication will be imperative, particularly when considering the transition from one-off product sales to service contract terms. For example, value substantiations will form and justify parts of the terms in the service contract, and this sales channel should also be used as an awareness creation or educational document not only for pre-contract negotiations but also post-subscription support. Other possible key channels are the sales team, who need to replace their push approaches with value-driven pull techniques to enable customers to create their own value. Internal communication via channels such as the intranet is also critical and should be used to drive the customer value language of the servitization offering. timreview.ca

About the Authors
Kwesi Sakyi-Gyinae is an MSc graduate from the Hanken School of Economics in Helsinki, Finland, and he currently works in the United States helping technology startups to create and implement strategic sales processes to maximize revenue.

Maria
Holmlund is a Professor of Marketing at the Hanken School of Economics in Helsinki, Finland, where she also received her PhD. Her research interests include methodological and conceptual issues related to service and customer-oriented management in business-to-business and businessto-consumer markets. timreview.ca

Introduction
Digitalization describes the integration of digital technologies into any aspect of daily life that can be digitized (Gray & Rumpe, 2015;Khan, 2016). However, Gartner (2018) defines digitalization with a more business-oriented focus: "Digitalization is the use of digital technologies to change a business model and provide new revenue and value-producing opportunities; it is the process of moving to a digital business." Digitalization is based on the availability of large amounts of external and internal (and often cloud-based) data (Gray & Rumpe, 2015) from different sources, and data mining and machine learning techniques to use it for decision-related purposes, such as the identification of a business opportunity or predictions of future market and client behaviour (Witten et al., 2016).
Depending on their business model, firms with digitalized market development processes can acquire clients from foreign markets without investing in a local production or sales force using, for example, their website traffic to identify market opportunities or online marketing tools to acquire a global client portfolio while remaining in their home market (Coviello et al., 2017). The use of digital technologies allows for a higher speed of internationalization because they help firms learn more quickly about new markets and to develop local networks (McKinsey, 2016), resulting in faster local product adaption and client interaction (Autio & Zander, 2016;Coviello et al., 2017).
As new international ventures that create new market niches using innovative technologies and new business models (Rasmussen & Tanev, 2015;Tanev, 2017), Lean global startups need to internationalize early and fast. The digitalization of new foreign market development helps them to more efficiently identify new market opportunities in global markets. With this approach, they are saving resources while developing the most attractive markets. This article examines how lean global startups develop new foreign markets more rapidly due to digitalization. Thus, the aim is to understand the impact of digitalization on speed of internationalization of lean global startups. The study addresses a gap in the scholarly literature and a practical need to evaluate new foreign markets and business opportunities more quickly and more regularly and to understand what helps lean global startups react more quickly to opportunities and threats with respect to changing market attractiveness. Furthermore, it outlines why and how digitalization is important throughout the internationalization process. The research followed a multiple case-study design using different sources of evidence, including 73 interviews with senior managers of lean global startups. The findings reveal that digitalization allows lean global startups to increase decision-making efficiency and to optimize strategies and processes for evaluating international markets. The findings suggest that lean global startups can benefit from the use of digital technologies by applying a more efficient foreign market development process with regular reviews and a reduced workflow, by faster mediation between local market realities and strategic goals, by analyzing all foreign markets instead of just a sample of them, and by optimizing decision-making processes including the ability to make long-term, strategic decisions due to better market information.
Digital is worth its weight when all parts of the choir sing their respective parts in harmony to achieve a higher purpose and make a unique impact.
Pearl Zhu Author of Digital Maturity " " timreview.ca The Impact of Digitalization on the Speed of Internationalization of Lean Global Startups Michael Neubert it may be particularly helpful for lean global startups to use digitalized technologies to collect and analyze data about international markets and client feedback to speed up decision-making processes, because they depend on iterative, incremental product development cycles. The impact of digitalization is especially high in foreign markets, because a lean global startup needs to create knowledge, for example, about client needs, and it needs to develop client networks to operate successfully. The speed of creating this intellectual and social capital determines the speed of internationalization (Vahlne & Johanson, 2017). Digitalization in the sense of predictive analytics using, for example, artificial intelligence algorithms helps lean global startups to predict the future attractiveness of foreign markets. Better predictions will increase the efficiency and quality of market selection decisions and the opportunity to participate in future market growth. Because lean global startups often are pioneers and global market leaders in small, global market niches (Neubert, 2017a;Tanev, 2017), early and fast internationalization is necessary to be competitive. Pricing is one example. Lean global startups need to understand their own costs, as well as market prices and their products' value, to select correct prices and pricing models (Neubert, 2016a;2017b).
The purpose of this study is to show that the use of digital technologies in market development processes increases the speed of internationalization in lean global startups. The rationale is to close a gap in the literature (Coviello et al., 2017;Vahlne & Johanson, 2017) to better understand the impact of the use of digital technologies in market development processes on the speed of internationalization. The article reports the modelling of a process for evaluating and selecting international markets and outlines why and how digitalization is important throughout the whole process. It aims to expand the study of international management by including a deeper and broader range of the use of digital technologies in international market development using the example of lean global startups.
Calls for research on the impact of digitalization on internationalization by Coviello, Kano, and Liesch (2017), Manyika and colleagues (2016), and Vahlne and Johanson (2017) provided impetus for this study. Merkert, Mueller, and Hubl (2015) also stress the need for further research about the usefulness of digitalization for decision-making purposes. Their findings suggest that the advantages of machine learning in decision-support systems are higher effectiveness and reduction of manual work. Digitalization is instrumental in facilitating earlier and faster internationalization through digit-alized knowledge, network creation, and decision-making processes (Coviello et al., 2017). Therefore, it should be addressed through qualitative research methods, for example, multiple case studies (Vahlne & Johanson, 2017) to explore and better understand the perceptions of lean global startups about digitalization of foreign market development.
The article is structured as follows. First, the literature on the lean global startup model, international market development processes, and the digitalization of international market development is reviewed. Then, the research methodology, including the sampling strategy, is described. Next, the findings about the impact of digitalization on foreign market development activities of lean global startups are presented. Finally, the article concludes with a list of key findings and recommendations.

Literature Review and Theoretical Framework
The theoretical framework of this study is based on a review of the literature regarding lean global startups (Neubert, 2017a;Tanev, 2017) and the digitalization of development processes for international markets.
The lean global startup model In 2015, Rasmussen and Tanev introduced the lean global startup as a new type of firm. A lean global startup can be considered a new international venture that creates a new market niche using innovative technology and a new business model (Tanev, 2017). A lean global startup implements a business plan in incremental and iterative product cycles (Tanev, 2017), developing modest, viable products that are tested in the most attractive markets (Neubert, 2017a). Because of the immediate client and market feedback in this process, products and services can be quickly adapted to their needs (Tanev, 2017). Lean global startups often start their global operations through up-stream activities, such as the developing and patenting a medical application, before engaging in downstream activities, such as sales, pricing, and export (Neubert, 2017b).

International market development processes
According to the Uppsala internationalization process model, firms use an establishment chain to develop to new foreign markets (Vahlne & Johanson, 2017). As shown in Figure 1, an establishment chain in form of a market development process can be described as a series of four steps: i) market evaluation and selection phase; ii) market preparation; iii) market entry; and iv) timreview.ca The Impact of Digitalization on the Speed of Internationalization of Lean Global Startups Michael Neubert market growth and development (Neubert, 2011(Neubert, , 2013. In the first step, firms collect and analyze data to understand the current and future attractiveness of a foreign market (Neubert, 2013). In the following steps, firms might use data analytics to prepare for decisions about market entry strategies, market entry modes (Ahi et al., 2017), distributors or joint venture partners, marketing campaigns, or market opportunities. Given that markets change rapidly, this market evaluation and selection process needs to be repeated regularly for every market and might lead to different outcomes, such as a market exit (Sapouna et al., 2018;Neubert, 2011) or a new product launch. Structured market development processes should increase the efficiency and the speed of internationalization for firms of different size, industry, and market, as studied in Europe (Neubert, 2016b), Africa, Asia, and Latin America (Neubert & van der Krogt, 2017).
Due to the large amount of existing data and a steady stream of new data due to a fast-changing and complex environment, the basic assumption of the current study is that the preparation of decisions and decision-making processes in international market development would benefit from digitalization, thereby reducing the evaluation and selection workflow from three tasks (Neubert, 2011) to just one task and increasing the limited number of analyzed countries (Neubert, 2013) to the whole population. The digitalization of the entire market development process might increase the speed and quality of decision-making processes, if users (e.g., finance managers, international marketers, or business developers) understand the benefits and limitations of digitalization. The faster and better a firm understands a market and is able to develop local networks, the higher the speed of internationalization (Neubert, 2016b).

Digitalization of international market development
Digitalization describes the integration of digital technologies into any aspect of daily life that can be digitized (Gray & Rumpe, 2015;Khan, 2016). However, Gartner (2018) defines digitalization with a more business-oriented focus: "Digitalization is the use of digital technologies to change a business model and provide new revenue and value-producing opportunities; it is the process of moving to a digital business." Digitalization is based on the availability of large amounts of external and internal, often cloud-based data (Gray & Rumpe, 2015) from different sources, and data mining and machine learning techniques to use it for decisionrelated purposes, such as the identification of a business opportunity or predictions of future market and client behaviour (Witten et al., 2016).
One important development of digitalization is the creation of online platforms and exchanges involving economic (e.g., Alibaba and Amazon) and social (e.g., LinkedIn) transactions (Coviello et al., 2017) to efficiently identify sales opportunities in new foreign markets. These platforms and exchanges offer tools and information (e.g., logistical support, export insurance, export documentation, and financing) to execute these sales opportunities.
Digitalization also increases the effectiveness of decision-support processes and reduces the amount of associated manual work (Merkert et al., 2015). Traditional data-driven and fact-based decision-making processes increase the productivity and profitability of companies by 4-7% compared to their competitors (Bohanec et al., 2017;Müller et al., 2018). Companies using prescriptive, analytics-based, machine-learning algorithms have been shown to increase their revenues by more than 15% on average (Kawas et al., 2013).
Knowledge about foreign markets increases international performance (Stoian et al., 2017). Digitalization creates social data (market networks) and intellectual data (market knowledge) about foreign markets earlier and faster than other methods, while also improving firms' attractiveness, decision processes, and capabilities of decision makers (Clark et al., 2018). Although decisions are often based on historical data or on experiences from other markets, a new market entry is a long-term investment in the future attractiveness of an untested foreign country (Neubert, 2017a). If a firm The Impact of Digitalization on the Speed of Internationalization of Lean Global Startups Michael Neubert decides to enter a new foreign market in a given year, the decision is often based on historical market data (e.g., from two or more years earlier), but the effects of the decision (e.g., significant new clients and sales revenues) will take place approximately two years later. Therefore, predictive algorithms should be used to assess future markets' attractiveness. Although predictive algorithms cannot eliminate uncertainty, they can improve allocation of resources and prioritization of projects. International managers must manage digitalization carefully by mediating between local market realities and corporate goals and understanding the limits and benefits of digitalization (Ransbotham et al., 2015). Early warning systems that once would have taken years to create now can be rapidly developed and optimized from real-world data. To assess the usefulness of prediction models, we must evaluate them not on their ability to recapitulate historical trends, but instead on their accuracy in predicting future events.

Theoretical framework and research questions
Drawing on the review of the literature, the theoretical framework developed for this study (Figure 2) is based on the notion that digitalization -through the application of a market intelligence and analysis software -will improve learning (Neubert, 2016a) and networking abilities (Neubert, 2016b) of lean global startups, which, in turn, will lead to more efficient decision-making processes, which will ultimately increase the speed of internationalization.
The purpose of this study has brought up the following three research questions: 1. What are the perceptions of lean global startups about the digitalization of foreign market development activities?
2. What are the perceptions of lean global startups regarding the expected impact of digitalization on the speed of internationalization? ?
3. What are the perceptions of lean global startups about the factors that determine the use and the selection of software products to support digitalization?

Research Methodology
This study uses a multiple case-study research design to answer the exploratory research questions (Yin, 2015). In contrast to an experimental design or a survey, a multiple case-study method offers greater flexibility, allows an in-depth analysis of a complex research problem (Yin, 2015) within a highly-contextualized environment, and allows for a comparison between different cases. This research design helps answer the research questions because it allows the use of the replication logic as a possibility to obtain external and internal validities as well as to analyze pattern-matching properties between theories and cases (Yin, 2015).
This study used different sources of evidence to derive robust conclusions and to achieve construct validity. Therefore, we applied the triangulation concept to the data collection phase to guarantee that different sources of evidence were used to collect data from each case. The primary source for data collection comprised qualitative, semi-structured, individual interviews with 73 senior managers of lean global startups. Other sources of evidence were firm and product flyers and brochures, corporate website, internal documents provided by the subject matter experts, and other secondary data. The data were collected in October and November 2017. The reliability criteria were met by using the same questionnaire, the same study protocol, The Impact of Digitalization on the Speed of Internationalization of Lean Global Startups Michael Neubert and the same data structure in the data collection phase. The duration of the interviews was, on average, 30 minutes. There were 73 interviews although some interviewees did not answer all questions; therefore, some results are based on fewer than 73 responses.
The data analysis followed a logical sequence starting with an individual analysis of each interview, followed by a cross comparison of the results to identify differences and similarities between the answers provided by the different firms, and finally a theoretical and literal replication using a pattern-matching approach. The main goal of this approach is to increase the possibility to transfer and generalize the findings to other contexts.
This study uses a purposive case-selection strategy.
After drawing a random sample from a database of 1,475 lean global startups, typical cases from the sample were selected. According to Yin (2015), this sampling strategy produces a statistically representative sample. Data saturation was achieved with 73 interviews (81% response rate). This relatively high sample size allows for a better triangulation of data and helps to strengthen the results of the whole study (Yin, 2015). The case studies in this research project are classified as lean global startups in the sense of new international ventures that create a new market niche using innovative technology and a new business model (Tanev, 2017). They are high-technology firms mainly from the IT, medical technology, and biotechnology industries. They have developed patented technologies, including prototypes of products, before starting with marketing and sales activities. The lean global startups in this sample are focused on a small global market niche using a pioneer strategy and a lean internationalization process (Neubert, 2011(Neubert, , 2013 to internationalize early and fast (Neubert, 2015).

Digitalization of Foreign Market Development Activities
The findings of this multiple case study are presented to answer the first research question: What are the perceptions of lean global startups about the digitalization of foreign market development activities?
Only six of the lean global startups (8%) use digitalized international market development processes with country market data and predictive analytics in the form of market studies to evaluate the attractiveness of foreign markets. Most of the other lean global startups remark that they acquire and collect market data only occasion-ally and for a single purpose, for example, during a market entry project. Such data is therefore not entered into their enterprise resource planning systems, nor is it updated regularly. Another six (8%) lean global startups are currently planning to digitalize their international market development processes. According to the lean global startups, the limited use of big data and predictive analytics in strategic decision making can be explained by the incipient stage of the use of business intelligence due to a relatively limited theoretical knowledge and missing practical experience. Our data confirm this statement. Only 20 (27.4%) of the lean global startups consider their theoretical knowledge and only 14 (19.2%) rate their practical experience as high or very high (Table 1). However, 86% of the SMEs indicate interest in the topic of digitalization and willingness to explore its benefits and applications for their organization.
The lean global startups indicate that macroeconomic data are mostly available, but that there are very limited sector-specific data available in industry and services sectors. This statement is very important, because digitalization, and especially big data and predictive analytics, depend on the availability and the quality of data. On the one hand, the lean global startups argue that digitalization is therefore still in a premature stage and is therefore difficult to use to support business objectives. On the other hand, lean global startups admit that they do not have a complete overview of the existing physical and digital data in their organization and on the market. One lean global startup mentioned that "we don't have the knowledge and experience to identify and to prepare our existing data". Finally, the majority of the lean global startups mention that this challenge is further complicated by a lack of data analysis and interpretation capacity in their organization.
The applications of big data and predictive analytics with the highest economic benefit are according to lean The Impact of Digitalization on the Speed of Internationalization of Lean Global Startups Michael Neubert global startup perceptions "lead generation", "client acquisition", and "client development" in the sense of the identification of cross-, re-, and up-selling potentials in existing client portfolios in global markets (Figure 3). "Sales channel optimization" follows next and describes, according to lean global startups, the selection of the appropriate market entry mode, for example (in)direct export, licensing, or a wholly-owned subsidiary. This includes the acquisition and development of licensees, distribution, and joint venture partners in steps 2 to 4 (= more efficient network creation), and the evaluation and selection of new foreign markets in step 1 (see Figure 1) based on a more efficient creation of market knowledge.
Lean global startups expect a high economic benefit of digitalization because, currently, decisions about international market development activities are often based on gut feeling, business experience, a lack of available data and without a clear understanding of the future market attractiveness. They assume that digitalization will support them in creating knowledge and domestic networks faster and with higher quality, thus reducing client acquisition costs by increasing the efficiency of marketing and acquisition activities (e.g., higher conversion and client retention rates as well as higher cross-selling, reselling, and up-selling ratios). Lean global startups expect that this will reduce the time to enter a new foreign market and therefore increase the speed of internationalization. Lean global startups understand digitalization as an improved usage of internal and external data to analyze their current situation and to predict the future attractiveness of foreign markets. This helps them analyze future market developments, make faster and better-informed decisions, allocate resources more efficiently to different markets, and react more quickly to market changes.
Lean global startups perceive the additional economic benefit of digitalization for strategic and support processes like strategic planning, controlling, marketing channel selection, or pricing as less of a priority -but still important. This judgement is mainly based on the fact that strategic and support processes already use data analytics to prepare decisions (e.g., market research).
The main finding of the first research question -What are the perceptions of lean global startups about the digitalization of foreign market development activities?is that only very few lean global startups have theoretical knowledge about digitalization and practical experience with digitized international market development processes. However, they expect high economic benefits, especially in their international networking ability, for example, with lead generation, client acquisition, and client development, where they currently use no digitized processes.

Expected Impact of Digitalization on Speed of Internationalization
In this section, the findings of this multiple case study are presented to answer the second research question: What are the perceptions of lean global startups regarding the expected impact of digitalization on the speed of internationalization?
The findings provide detailed insights into the perceived impact of digitalization in terms of big data and predictive analytics on sales performance in foreign markets. One lean global startup sees "the main impact in the acquisition of potential clients" due to a decreasing efficiency of sales processes. As another lean global startup states: "We need too many leads to acquire a new client" and "our client acquisition costs are increasing". Lean global startups indicate that pre-dictive data can assist mainly in the acquisition of new clients and in the estimation and the identification of new opportunities in global markets. To a lesser extent, it can also assist in retaining clients, predicting prices, and in competitive and risk analysis ( Table 2).
The lean global startups expect a strong impact on productivity, profitability, and sales revenues of international operations (Table 3). Among them, 64.7% expect a 6% or greater in productivity, especially in "the acquisition of new clients", as one senior manager remarks; 66.7% expect a 6% or greater increase in profitability; and 68.6% expect a 6% or greater increase in sales revenues. The expected impact on cost reductions is substantially lower. Only 41.2% expect 6% or greater cost reductions, 43.1% expect between 1-5%, and 15.7% do not expect any impact on cost reductions. Table 2. Applications of big data and predictive analytics, as reported by senior managers of lean global startups Table 3. Expected impact of digitalization on internationalization, as reported by senior managers of lean global startups timreview.ca

The Impact of Digitalization on the Speed of Internationalization of Lean Global Startups Michael Neubert
Next, we wanted to understand which lean global startups expected the greatest positive impact. Therefore, we divided the firms into four clusters depending on the level of digitalization in their international market development activities (Table 4).
The four clusters show that only six (8%) lean global startups currently digitalize their international market development activities. The majority of the lean global startups is interested (42%) or is planning (30%) to digitalize their international market activities. Only 14 (19%) of all lean global startups are not interested in digitalizing their international market development activities, because they do not see any concrete benefits or use cases.
When analyzing the results from Table 3 (here: impact area productivity) based on the clusters of Table 4, we see that the lean global startups in Cluster 4 typically do not expect significant impact from digitalization ( Table  5). The highest impact is expected by those lean global startups planning to digitalize (Cluster 2), followed by the lean global startups of currently digitalizing (Cluster 1) and interested in digitalizing (Cluster 3). This enthusiasm, with expectations higher than actual results, often is caused by digitalization projects still in the implementation and optimization phase, which do not exploit the full potential of international digitalization yet.
Further analysis of the data shows that lean global startups in Cluster 1 have the greatest experience and knowledge about international digitalization, followed by Clusters 2, 3, and 4, thereby confirming an expected positive correlation between knowledge, experience, and application.
The main finding of the second research question "What are the perceptions of lean global startups regarding the expected impact of digitalization on the speed of internationalization?" is that lean global startups expect a significant impact of digitalization on productivity, profitability, and sales revenues, especially through a higher efficiency of all international learning and networking activities in foreign and domestic markets due to the application of big data and predictive analytics.

Use and Selection of Software to Support Digitalization
In this section, the findings of this multiple case study are presented to answer the third research: What are the perceptions of lean global startups about the factors that determine the use and the selection of software products to support digitalization?
When considering the use of a business intelligence service in terms of big data and predictive analytics in international market development, the lean global startups are inhibited by a diverse set of assumptions. Lean global startups are concerned about a lack of support in configuration and training (39%), data protection (33%), and an unclear selection, processing (41%), and evaluation of data (i.e., a "black-box effect") (44%).
In particular, the black-box effect prevents many lean global startups from investing in digitalization, as justified by one lean global startup in the following way: "How can I trust the results of the software if I don't understand the underlying algorithms?". "It is a real dilemma?", asks another. On the one hand, lean global startups have difficulties in trusting the software. On the other hand, they are barely able to make important decisions due to "missing basic information", as several lean global startups indicate.
The main benefit, expected by 82% of the lean global startups, is a higher efficiency in their international learning and networking activities. One lean global startup states: "We need permanent and current market feedback to adapt as early and as fast as possible to Table 4. Clusterings of case firms depending on the level of digitalization of international market development activities Table 5. Expected impact of digitalization on productivity of internationalization timreview.ca The Impact of Digitalization on the Speed of Internationalization of Lean Global Startups Michael Neubert changing markets and client needs". This includes big data and predictive analytics for long-term investment planning as well as "alerts" for short-term, tactical management to react to market opportunities or crises.
To select a business intelligence platform to support internationalization, the lean global startups will mainly consider a solution that responds to the expected problems mentioned above. Thus, 72% of the lean global startups would choose based on the price-performance ratio, meaning a price that reflects the perceived value. The second important requirement, mentioned by 66% of the lean global startups, is the ability to integrate into existing enterprise resource planning (ERP) applications, such as customer relationship management (CRM) systems. Lean global startups will also consider regular updates with current data and improved usability (58%), efficient set-up and testing procedures (53%), and references from existing clients (51%). Fewer lean global startups are concerned about the size and reputation of the provider (21%), the quality of the client service (38%), and additional services as professional training and consulting services (43%).
In combination with the findings that address our first research question -What are the perceptions of lean global startups about the digitalization of foreign market development activities? -the main challenge for providers of digitalization services is to overcome potential deficits in theoretical knowledge and practical experience of users and to transparently demonstrate the benefits and the added value of the application.
The main finding of the third research question -What are the views of lean global startups about the factors that determine the use and the selection of software products to support digitalization? -is that lean global startups need digitalized international market development processes to increase the efficiency of their business model. Transparency about benefits, data collection and analysis, the ability to integrate into the existing ERP systems, and an attractive price-performance ratio are the main decision criteria of potential users.

Conclusions
The key findings of this survey are: • Most lean global startups have limited theoretical knowledge about digitalization and lack practical experience with digitalized international market development processes.
• Most lean global startups expect significant economic benefits in their international networking ability -for example, in lead generation, client acquisition, and client development -where they currently do not use digitized processes.
• Lean global startups expect a significant impact of digitalization on productivity, profitability, and sales revenues, especially through a higher efficiency of all international networking and learning activities due to the application of big data and predictive analytics.
• Lean global startups select software to support digitalization based on transparency about benefits, data collection and analysis, the ability to integrate in the existing ERP systems, and an attractive price-performance ratio.
• The main objections to selecting and using a software tool to support digitalization include lack of support in configuration and training; data protection; and unclear selection, processing, and evaluation of data (i.e., the "black-box" effect).
The findings show that lean global startups expect a significant impact of digitalization on internationalization. This result confirms the findings of other studies (Bohanec et al., 2017;Kawas et al., 2013;Müller et al., 2018). Digitalization allows lean global startups to create knowledge and networks at a faster rate, which increases the efficiency of decision-making processes, which ultimately increases the speed of internationalization. In detail, digitalization increases the speed of internationalization using internal and external data to predict future market development, allowing lean global startups to act on several levels: • applying a structured and disciplined internationalization process with regular reviews and a reduced workflow (one instead of three market evaluation steps) • mediating between local market realities and strategic goals • analyzing permanently all foreign markets instead of just a sample of them • optimizing decision-making processes and the ability to make long-term, strategic decisions due to better market information The findings of this study are relevant for scholars, researchers, managers, and policy makers who support timreview.ca

About the Author
Michael Neubert is a Professor at the International School of Management in Paris, France, where he obtained his PhD and is now also Chair of the Strategic Management Committee. He teaches international business, intercultural communication, doing business in foreign markets, and international finance. His research interests concern the internationalization of high-tech startups. Michael is a member of the Academy of International Business, and he is a partner of a private equity firm that invests in hightech startups and supports them in the development of new foreign markets. Michael is also the CEO of C2NM (www.c2nm.com), a Swiss consulting firm specializing in the field of international and intercultural management.

The Impact of Digitalization on the Speed of Internationalization of Lean Global Startups Michael Neubert
activities that promote digitalization, to better understand the impact of digitalization on internationalization speed in lean global startups, to increase the efficiency of decision-making processes, and to optimize international market evaluation strategies, processes, training, and (financial) support projects.
While offering new ideas for future research, a multiple case study research design has several limitations in scope and size. Future scholarly work should include quantitative assessments of the perceptions of lean global startups with qualitative and quantitative data to provide greater clarification of the statistical significance of the study's variables, replicate it with other case-study firms belonging to different countries or industries and at different stages of development, and going from expectations to actual tests of the factors of this study.