March 2015 Download this article as a PDFAbstract

Lately, the concept of smart cities has been changing from a top-down and mostly technological-driven approach, towards a bottom-up process that facilitates participation and collaboration among city stakeholders. In this latter respect, the city is an ecosystem in which smart applications, open government data, and new modes of participation are fostering innovation. However, detailed analyses on how to manage bottom-up smart city initiatives, as well as descriptions of underlying challenges and barriers, are still scarce. Therefore, this article investigates four collaborative smart city initiatives in Europe to learn how cities can optimize citizen involvement in the context of public sector innovation. The analytical framework focuses on the different stakeholder roles in the ecosystem and the civic capacities to participate in the innovation process. The findings illustrate how more inclusive citizen involvement can be realized by providing different tools that align with the specific capacities and skills of the citizens. Furthermore, through specified workshop formats and peer learning, citizens lacking technical skills were also enabled to participate in the evolution of their cities, and to generate solutions from which both the city and everyday urban life can possibly benefit.


The roll out of high-bandwidth connectivity and the growing adoption rate of mobile technologies such as smartphones and tablets are said to be transforming the public realm and the way we live and interact in urban areas. These and other digital technologies, such as wireless sensor networks and network-based applications, have begun to cover the city and have started to form the backbone of a large, intelligent infrastructure (Schaffers et al., 2012). Through these rapidly advancing technological capabilities, citizens are increasingly able to access real-time information about the city environment anytime, anywhere they want. However, at the same time, many cities are confronted with a wide range of challenges such as the environmental pollution, traffic jams, governance, etc. More specifically, city governments seem to struggle to meet the demands for improvement in public service delivery associated with the quality of urban life – while facing the prospect of ever-diminishing resources (Gudeman, 2008). In this regard, new technologies can help to map and understand information about the city dynamics and to deliver more effective services.

Furthermore, bottom-up processes are being increasingly considered for sensing the dynamics of cities based on the participation of citizens. Citizens are becoming actively encouraged to see the city as something they can collectively "tune", in a manner that it is efficient, interactive, adaptive, and flexible (Arup, 2010). By performing a multiple case study analysis of four collaborative smart city initiatives in Europe, namely Ghent (Belgium), Issy-les-Moulineaux (France), Manchester (UK), and Athens (Greece), we seek to yield insights into how bottom-up processes within smart city initiatives can be facilitated, with a particular focus on the role of the different stakeholders in the ecosystem and the civic capacities to participate.

To reach this objective, the article first discusses the smart city concept and the civic capacities to engage in the public domain, followed by an introduction of the living lab framework as a possible facilitator of bottom-up innovation. Next, the research design is presented with some additional information about the four cases. Finally, we discuss the interplay between the living lab methodology and the development of the toolkit, and how these were aligned with the capacities of the citizens.

The Smart City through Open Data and Mobile Apps

Over the past few years, many smart city projects and initiatives have popped up as a seeming answer to challenges that cities are facing (Pallagst et al., 2009). Challenges such as traffic jams, environmental pollution, etc., are demanding new and innovative ways to manage urban life and are pushing cities to invest in the necessary information and communication technology (ICT). In this context, the European Union (EU) funding programs such as Horizon 2020 are an important driver to promote and support the development of smart cities throughout Europe. The smart city concept is relatively new and evolving, and many different definitions have been proposed. The mapping study of smart cities in the EU by the European Parliament showed that “Smart Cities come in many variants, sizes and types. Every city is unique, with its own historical development path, current characteristics and future dynamic. The cities which call themselves ‘Smart’, or are labelled as such by others, vary enormously” (European Parliament, 2014). The local development path, the interpretation of the concept, and place-specific characteristics can thus explain the various implementations of smart cities.

Among these different definitions and implementations, we see that, on the one hand, ICT plays a dominant role in becoming more intelligent, interconnected, and efficient (e.g., Hall et al., 2000), while on the other hand, a broader perspective with social and economic factors is incorporated in the definition of the smart city concept. In this article, we follow the definition of Caragliu and colleagues (2009) as it balances between economic and social demands, and links up to democratic processes. According to the authors, a city may be labelled smart “when investments in human and social capital and traditional (transport) and modern (ICT) communication infrastructure fuel sustainable economic growth and a high quality of life, with a wise management of natural resources, through participatory government” (Caragliu et al., 2009).

Nowadays, the ICT layer underpinning the smart city concept relates to smart embedded devices ranging from smartphones to sensors, smart meters, and other instrumentation that sustain the intelligence of the city (Schaffers et al., 2011). Data coming from these sensors, or integrated networks, can provide citizens with real-time and location-based information. For example, sensors can monitor the air quality or detect patterns of movement of people in the city. These data, and information stemming from these datasets, can help governments in better understanding the city environment (e.g., improving urban planning) and in creating and delivering new effective services. Additionally, we see that more and more government entities are opening up their data, meaning “data produced or commissioned by government or government controlled bodies, which can be freely used, reused and redistributed by anyone” (Open Government Working Group, 2015). These data are made available at no cost to the public, so that, for example, (citizen) developers or startups can add relevance and value to the information and develop a service based on the data. In this respect, de Lange and de Waal (2013) consider cities as information-gathering systems in which data commons arise: “As these data are being aggregated, they may become a ‘data commons’: a new resource containing valuable information for urban designers”. The availability of data and access to it, along with the skills of citizens to use the data in a meaningful way, are hereby two preconditions to establish a data commons (de Lange & de Waal, 2013).

In this context, urban competitions on open data, or hackathons, are increasingly being organized to stimulate the development of mobile applications. For example (Baccarne et al., 2014) illustrated that the goal of these hackathons is to stimulate both citizens and professionals to work with open government data, with the belief that it will result in more efficient and user-centric applications.

Smart city applications thus form a new digital layer of the city, in which citizens are not only invited to participate in the data collection (e.g., crowdsourced information about air quality), but also in the actual ideation and development process of the services. In this view, the services are not only thought to make the city smarter, but also to serve the mobile citizen in a better way (Hielkema & Hongisto, 2013).

Participation, Citizen Involvement, and Civic Capacity

In the literature, some authors (e.g., Baccarne et al., 2014; Schaffers et al., 2011) are still rather hesitant about the value of ICT-enabled smart city solutions, while others clearly express their beneficial use (e.g., European Parliament, 2014; Hancke et al., 2012). According to the latter view, the use of ICT makes a city a "smart" city, because it improves the efficiency and effectiveness of the city processes, activities, and services. Despite these clear-cut opportunities, there is also the belief that, without engaging citizens about the role and impact of technology in their cities, the smart city vision will fail (FutureEverything, 2013). If cities want to reinvent themselves, solely pushing out highly technical solutions will not work, because new forms of digital divide can be created. Instead, a good balance between bottom-up processes (i.e., including the voices of citizens), and the technology push is desirable (Pallot et al., 2011). The mapping study of smart cities by the European Parliament (as referred to earlier) showed that one of the success factors for smart cities is “people”, or the involvement of citizens in the creation and realization of the smart city vision (European Parliament, 2014). This form of participation shifts the role of the citizen from a mere passive subject into an engaged actor (Schaffers et al., 2012) and promotes the view of a “participatory governance”, or as it is also called, “empowered participatory governance” (Abers et al., 2003). This democratic reform is called participatory because it relies “upon the commitment and capacities of ordinary people to make sensible decisions through reasoned deliberation” and it is empowered because it attempts to tie “action to discussion” (Abers et al., 2003).

However, meaningful participation will largely depend on the specific capacities and skills of the citizens (Wagemans, 2002). In this regard, Saegert  (2004) speaks of civic capacities or “the ability to participate in public life with the result of more democratic governance at various scales”. Moreover, Stembert and Mulder (2013) speak of different “participation parameters” to facilitate participation and co-creation between citizens and local governments. In their study, they focused on three parameters to investigate citizen participation in the public domain: ability, motivation, and satisfaction. The first parameter, ability, stresses the importance of guiding and supporting the users in a positive and obstructive way. Not everyone has the ability to easily express themselves or to imagine a proposed solution. Therefore, the authors’ advice is to communicate in a "common language". For example, generative tools reveal a "new" language that is predominantly visual and they make use of a large set of components that together form "creative toolkits" that people can use to express their thoughts, feelings, and ideas (Sanders, 2000). These toolkits help to bridge the gap between developers and users. Besides providing the right tools and techniques, the users’ motivation is another crucial parameter. Malone and colleagues (as cited in Stembert & Mulder, 2013) relate motivation to the goal users pursue: “money, love, and glory”. However, public governments cannot reward participants with money generated by taxes and would be better off triggering citizens with “the motivator of love or glory in the form of creativity” (Leadbeater, 2006). Last, satisfaction refers to how the participation process is perceived as satisfying by the user.

These different parameters should thus be taken into account when seeking citizen involvement in the public domain. Furthermore, participation will always lead to some unintended consequences; there will be always some citizens that will be included, while others will be excluded (Turnhout et al., 2010).

The City as Living Laboratory: An Ecosystem to Foster Innovation

One way to organize bottom-up processes within smart city initiatives is by applying the living lab approach. Living labs can be regarded as “physical regions or virtual realities where stakeholders form public–private–people partnerships (4Ps) of firms, public agencies, universities, institutes, and users, all collaborating for creation, prototyping, validating, and testing of new technologies, services, products, and systems in real-life contexts”  (Westerlund & Leminen, 2011). The living lab concept appeared in academic discussion in the 1990s, but really took off in 2006 when the European Commission initiated projects to advance, coordinate, and promote a common European innovation system (Dutilleul et al., 2011). According to (Pallot et al., 2011), living labs are a good way to bridge the gap between technology push (i.e., solution developers) and application pull (i.e., user communities), because they bring the necessary combination of digital skills, creativity, and innovation methods together. Coenen and colleagues (2014) describe living labs following a “meet in the middle philosophy”, an approach “for involving both the voice of citizens and local grassroots organizations to represent the bottom-up perspective and the voice of government and companies to represent the top-down view”. Schaffers and colleagues (2012) take one step further, stating that “cities are becoming a living lab itself, a playground of innovation and transformation”, exemplified by the emerging ways of collecting and using urban data. Living labs can thus be regarded as an effective means to facilitate bottom-up processes within smart city initiatives, as they promote multi-stakeholder collaboration and consider users as innovators (von Hippel, 2005).

Living labs can have different thematic focuses and interests, such as focusing on innovation in health, media, smart grids, etc. In this article, we focus on urban living labs that specifically involve citizens in city development to make urban areas better suited to their needs (Juujärvi & Pesso, 2013). Obviously, the goal of urban living labs differs fundamentally from more ICT-oriented living labs, which tend to be rooted in commercial contexts; here, the generated public value will be more of concern than the economic value (Baptista, 2005).

Regarding the key participants and their roles, Juujärvi and Pesso (2013) found that the role of citizens in urban living labs is more comprehensive than in other types of living labs. They discovered that citizens can have multiple roles in urban living labs, ranging from a mere informant to tester as well as contributor and co-creator in the development process. Furthermore, the motivation to participate can also be different, because citizens can have a natural motivation to participate in shaping their environment through a “sense of place”, “a sense of being at home in a town or a city” (Horelli, 2013). Last, the role of the city can be described here as the “enabler” or “mediator” in the ecosystem, bringing everyone to work together effectively (Ratti & Townsend, 2011).

Table 1 provides more information about the different actor roles in living labs (Leminen & Westerlund, 2012) and specifies the role for each stakeholder in urban living labs (Juujärvi & Pesso, 2013).

Table 1. Actor roles in urban living labs (Juujärvi & Pesso, 2013; Leminen & Westerlund, 2012)

Common Actor Roles in Living Labs

Actor Roles in Urban Living Labs


Organization that provide supportive technology, virtual or physical space, and other necessary resource for use by participants

City representatives as enablers:

Create a vision, allocate resources, provide strategic leadership, and promote networking


Seeks efficiency gains and new knowledge, and wants to learn new practices to boost their innovation processes

Firms and local service providers as utilizers:

Create suitable products and services, set small-scale objectives, and produce place-based knowledge


Public and private company or organization that provides the network with their product portfolio

Educational institutions as providers:

Engage students as innovators, provide innovative R&D methods, and augment knowledge systematically


Potential customer of products and services from a provider or other actor

Residents as users:

Participate in experiments, empower citizens through co-creation, and produce place-based user experience

Research Approach

As part of the Citadel on the Move project (Box 1), this research was conducted by the iMinds-SMIT research organization at the Vrije Universiteit Brussel in Belgium. The pilot project initially focused on a network of four smart city initiatives in Ghent (Belgium), Issy-les-Moulineaux (France), Manchester (UK), and Athens (Greece), where citizens were engaged to participate in the design of a toolkit to build mobile applications. At the same time, the four cities were opening up their data and transforming it into a publicly usable format. Citizens were invited to provide suggestions for new datasets or to convert the dataset by themselves.

In this article, a multiple case study analysis of the four smart city initiatives is described. In the analysis, we focus on the following levels: i) the actor roles in the ecosystem and ii) the required civic participation capacities. These specific dimensions were chosen to provide an overview of the different stakeholders, to analyze the role they play, and to reveal how participation and collaboration is set up between the stakeholders in order to involve citizens.

In these different cases, Ghent and Issy-les-Moulineaux mainly focused on the delivery of better services within the tourism domain, whereas Athens and Manchester sought new services within the transportation domain to enable citizens to overcome health challenges and adopt more active lifestyles. In the latter two cities, sensor networks to measure air quality were also installed. By tapping into the innovation potential of citizens and by facilitating collaboration, these cities were interested in gaining better insights into citizens’ needs and establishing a better communication with the citizen.

In early facilitated workshops, five main themes were identified from the discussions: i) environmental information, ii) parking in the city, iii) events in the city, iv) points of interest in the city, and v) crowdsourced information. Based on these themes, so-called mobile "templates" were created that citizen (developers) could use to quick-start the mobile application development process. The source code of the templates, together with guides, were made available on the project platform and GitHub.

For our analysis, we used the user feedback collected from the living lab experiments of the four cases. These experiments were set up in an iterative and gradual approach, which aligned with the maturity of the mobile application development toolkit. In total, four iterative testing cycles were set up involving self-reporting methodologies (e.g., diaries), participatory methodologies (e.g., design charettes), and observational methodologies (e.g., participant observation in the city). By deploying this multi-methodological approach, feedback about various aspects of the toolkit was collected from the early stages of the project until the eventual self-governance of the toolkit. The chosen methodological approach was designed to test, evaluate, and co-create the toolkit with the citizens. These findings were used to investigate how bottom-up processes can be set up between the city and its citizens, and how hurdles can be tackled concerning the civic capacities of the participants.

Box 1. The Citadel on the Move project

The Citadel on the Move project ran from 2012 to the beginning of 2015, with the objective of uniting local governments, living lab practitioners, ICT specialists, and citizens to harness the power of open data and user-driven innovation to develop mobile applications that can be easily shared across Europe. The project helped local governments to open up and share their data through a common architecture and usage of standards, and it helped citizens to take part in the application development process through different provided tools and workshops. By the end of the project, Citadel had helped more than 120 cities across Europe to open up their data and create over 600 basic applications.

The project was funded by the European Commission's Information and Communication Technologies Policy Support Programme (CIP-ICT-PSP.2011.5.1).


In this section, we first provide an overview of the different stakeholders in the innovation ecosystem, together with a role description. Next, the user feedback of the four cases is discussed along the different living lab testing cycles to formulate conclusions on how citizen involvement can be optimized.

Actor roles in the innovation ecosystem

According to the typology of Leminen and Westerlund (2012), we identify the following roles in the ecosystem: the city as enabler, the citizens as users, and the research organization as provider. The role of utilizers is not present within this ecosystem, because the scope of the initiative is more oriented towards generating public value. Figure 1 illustrates the different stakeholders and exemplifies the role they play within this particular ecosystem.

Figure 1

Figure 1. Actor roles in the innovation ecosystem

The four local governments play the role of enabler in this ecosystem as they set out the smart city objectives, provide the necessary resources, and bring the different stakeholders of the living lab network together. In all cases, the city promoted the networking among citizens, the developer community, students, small and medium-sized enterprises, etc. to increase awareness about open data and to enable cooperation among the different stakeholders. During the living lab experiments, these stakeholders were brought together to both brainstorm and build around new ideas for applications with help from the provided toolkit. 

A research organization (iMinds) was also involved in the ecosystem to provide innovative research and development methodologies. The organization had previous experience in the design and implementation of user-driven methods in living labs, and could thus accumulate knowledge over the long term. In this instance, the research organization did not have direct contact with the living lab participants, because such interaction would have a negative impact on the community building and citizen–government relationship. Instead, the research organization provided diverse protocols and guidelines to the cities on how to set up the living lab experiments. Afterwards, both the city and the research organization collaboratively assessed the results.

The last role is that of users, who were invited to provide feedback and participate in the co-creation processes. In this ecosystem, users were defined as citizens, (citizen) developers, and professional developers who were interested in using and creating innovative applications in the domain of tourism and transport. These different groups were segmented into different categories based on their level of skills and technical knowledge: none, limited, or high. The following sections show the importance of categorizing users based on skill level.

Testing, evaluating, and co-creating mobile (template) applications

In this section, the results are presented from the early user requirements workshops until the last iteration cycle of the mobile application templates. The findings show how cities organized the bottom-up processes and how civic involvement was accomplished.

In the summer of 2012, a first workshop was organized in each city to gather preliminary thoughts and expectations about how the creation of applications in the transportation and tourism domains could be facilitated. Various stakeholders were invited to these workshops to discuss new ideas based on some predefined paper mock-ups. These mock-ups described some basic application features, and mostly served to define the first user requirements.

In the next phase, the user requirements were taken into account to develop a first version of the application templates. Based on the stakeholder feedback, five mobile application templates were created, focusing on the following aspects: i) environmental information, ii) parking in the city, iii) events in the city, iv) points of interest in the city, and v) crowdsourced information. These templates were working mobile web applications based on HTML5 and PHP. JavaScript and JSON were also used to enhance the user experience and allow the communication with the application’s back-end and data respectively. By providing these templates, the cities facilitate mobile application development, as anyone is able to download the source code from the platform. This way, citizens are able to personalize the application templates in order to meet their needs. For example, citizens are able to combine multiple templates, add or remove features and datasets, etc. Figure 2 shows a first version of the templates.

Figure 2a   Figure 2b

Figure 2. Screenshots of the parking and points-of-interest application templates

Through these standard templates, cities are providing an easy way for citizens to start creating their own public services, and it makes the development processes less time-consuming and more cost-effective. Furthermore, when citizens can easily access open data, the innovative potential of citizens becomes stimulated as citizens themselves can determine the mobile applications they want and need.

To gather user feedback and iterate the development, the applications templates were launched into the living labs networks of the four cities. In total, four iteration cycles took place in order to optimize the use of the templates.

In the first testing cycles, the cities agreed to only recruit "citizen developers", because these are the citizens who have some development skills as well as innovative ideas for new applications. In total, 25 citizen developers were carefully selected and tested the first version of the application templates. Feedback was collected from interviews, focus groups, and journals. This latter method could foster the self-reporting of citizen developers about the experiences and activities with the toolkits. Also, logging provided substantial information about the number of downloads, error information, etc.

After two testing cycles, the results showed that about half of the citizen developers had been intensively adapting the templates over a period of one or two days. The parking application and the crowd-sourcing template were perceived as most interesting, whereas the urban planning template was perceived as rather useless due to a lack of data. In general, the citizen developers found the application templates easy accessible. Because the templates had been developed using cross-browser HTML5 technology, there were no problems in using these templates on different types of mobile devices or operating systems. The user interface was rather well received and many suggestions were made to improve it.

Despite this positive feedback, none of the citizen developers actually started developing their own application, even after many technical difficulties were resolved after the first iteration. Furthermore, it became clear that the feedback differed depended on the skill level of the citizen developers. Some citizen developers perceived the download and installation processes as rather easy:  “I found and downloaded the app files fairly easily and had to make some changes to the config file, which I am used to doing” (Manchester, December 2012). On the other hand, it seemed that some citizen developers were not familiar with these processes and stopped using the templates after downloading them: “We should have something very basic, like for example, the framework of Wordpress, where you find some boxes to fill in – some drag-and-drop elements. This is clearly what I expected to see, not some coding lines” (Issy-les-Moulineaux, December 2012).

In practice, less experienced citizen developers did not succeed in installing the templates, even with the help of others or when consulting the documentation. Instead, they evaluated the templates through the online demo website and stopped using the templates. In contrast, more experienced citizen developers were able to install and customize the templates. Based on this feedback, the cities and the research organization decided to implement a different approach based on the skill levels of the users.

After gathering feedback and iterating two testing cycles, none of the recruited citizen developers had created their own application. To increase usage and improve participation (regardless of skill level), additional tools were developed. The application templates would still remain available to the more skilled citizen developers and professional developers, however, a new tool, called the “App Generator Tool” (Figure 3) was made available to ordinary users. With this tool, citizens with limited-to-no technical knowledge could participate more easily in the application development processes. This way, cities guarantee that every citizen, including those lacking specific capacities, is able to become involved and be heard.

Figure 3

Figure 3. Screenshots of the App Generator Tool

The role of the App Generator tool is to allow users to combine various datasets of a city and build an application online without having to write a single line of code. In order to generate a new application, users simply need to fill in a form. Several fields should be filled in, for example, to select a city and (one or more) dataset(s), to define the theme colour and fill in a title for the application. When the application is created, a unique identification number is assigned, and the application can also be shared with others.

Besides creating a more accessible tool, a separate evaluation track based on the level of skills was set up by the research organization. This step was necessary because, in the upcoming testing cycles, not only citizen developers were involved, but also a larger number of ordinary citizens. Therefore, separate surveys were programmed: one evaluating the application templates through the demo website for non-technical participants and one survey that guided the more experienced users through the download and installation processes. Furthermore, a participatory design workshop was organized in which different stakeholders (e.g., citizens, professional developers, data enthusiasts, thematic experts) were invited. To bridge the "gap" between technical and non-technical participants, simple and creative communication tools were used. In this way, people could easily express themselves by using visual aids, drawings, and so forth. At the end of the sessions, some paper mock-ups were presented that were based on several scenarios. These paper mock-ups were given as an inspiration for the (citizen) developers to start developing new applications (Figure 4).

Figure 4

Figure 4. Paper mock-ups of mobile applications from Athens and Issy-les-Moulineaux

This tailor-made approach was proven very successful: more citizens were being able to participate and to provide custom feedback. To further engage citizens in the development process, specific “Apps4Dummies” workshops were also organized. In these workshops, a demonstration was given of the different tools, and knowledge was shared about open data and coding in general. At the end of the living lab experiments, 80% of the key users stated that they had learned something new about creating applications in general, and half of them expressed that they are eager to learn more about the topic (e.g., data formats and conversion of datasets). The main conclusion was that the transfer of knowledge and skills proved to be more empowering that just the provision of tools.


This article discussed the findings of four smart city initiatives in Europe, with a specific focus on citizen engagement and the capacities to participate in the public domain. In conclusion, we identify three key lessons learned through this study:

1. The living lab approach facilitates participation

After describing the various roles in the ecosystem, it became clear that the living lab approach played a central role in bringing different stakeholders together. By facilitating collaboration, stakeholders came together to jointly create new services, citizens made contact with their administrations, and mutual understanding was created. At the end of the testing cycles, citizens clarified that they better understand the challenges their city is facing, and that they would like to further contribute to the process of opening up data and building applications.

The living lab approach also entailed iterative testing and feedback. In the beginning of the project, citizen developers tested and positively evaluated the application templates, although none had the actual intention of developing their own application. This outcome did not fulfill the expectations of the cities: they had hoped to stimulate application development by providing standardized building blocks through the templates. The analyses of the user feedback showed that the users’ motivations and the abilities to participate were not fully satisfied. At first, the less skilled users were excluded from the development process, because they did not have the proper skills. Therefore, the cities, as well as the research organization, decided to develop different tools and a more targeted user recruitment and evaluation methodology to optimize the citizen involvement. This targeted approach seemed successful, given that all target groups started using the tools to create new applications. The user feedback was also more satisfactory: the tools were perceived as easier to use and more useful. Furthermore, after the end of the living lab experiments, two-thirds of the users were still using the tools to explore open data or to further improve their application idea. Here, we see that the iterative living lab approach not only proved its beneficial use in bottom-up co-creation, but also in validating the evaluation methodology and monitoring the participation parameters.

2. Co-creation processes can both include and exclude

Next, it became clear that facilitating co-creation processes between citizens and government entities could include some citizens and exclude others. This result was also found in (Turnhout et al., 2010), as one of the unintended consequences of participatory governance. Although it is impossible to involve everyone, the results here showed that, if different tools are aligned with the specific capacities and skills of the users are provided, more chances are created for users to become heard and take part. Interplay could be detected between the collected user feedback and factors influencing the civic capacities to participate. First of all, the iterative testing cycles made it possible to quickly respond to some technical issues and develop a better solution in the next phase. Participants are often frustrated when technical issues occur, and this frustration could evoke a decreasing interest in the long term (with possible drop-outs). But, more importantly, listening to the user feedback and taking the users’ abilities and motivations into account, overcame possible failures or low-usage intentions in relation to the technical solutions provided. The development of the App Generator Tool enabled ordinary citizens to easily create applications, and technical skilled users no longer dominated the development processes.

To optimize the involvement, it was also necessary to develop a separate evaluation track for each of the targeted user groups. By making specific questions that matched the profiles of the citizens, the data collection methods were perceived as rather adequate, and not too easy or too difficult to respond to. The creative tools in the participatory design workshops were also very successful in creating "a common language" for communication between the different stakeholders.

3. The approach empowers citizens

One of the most important outcomes for the cities is that, by providing and co-developing the toolkit, citizens were given the opportunity to contribute to the opening-up process of data and to the building of service applications. What in advance was limited to only a few, can now be done by anyone. Citizens acknowledge that, by participating in the diverse evaluation activities and workshops in their cities, they have learned new skills and knowledge, and they can now independently create an application. For the more skilled citizens, the hackathons and other application competitions provided opportunities to network, to disseminate their work, and to exchange experience. For this target group, the motivation of "playfulness" and the opportunity to showcase their expertise and creativity prevailed.

Last, the organization of workshops and peer-learning activities in the community were also vital in supporting the citizens. In the beginning, we observed users who only used the App Generator Tool and then, along the testing cycles, acquired more skills and started to execute more advanced operations. On the platform, the more skilled users also helped the less experienced ones. Catalyzing this mutual support and connecting people with different perspectives thus strengthen civic engagement and the opportunities for creating innovative solutions.



This study was a joint effort funded under CIP-ICT-PSP.2011.5.1 "Open Innovation for Future Internet-Enabled Services in ‘Smart’ Cities" (no. 297188). Our gratitude goes to the entire Citadel on the Move consortium. In particular, we would like to thank the pilot cities for their efforts and support.

An earlier version of this article was presented at the 2014 International Conference on Engineering, Technology, and Innovation (ICE), which was held from June 23rd to 25th in Bergamo, Italy. The ICE conference discusses systems engineering as a socio-technical task, with a focus on design of products and services, and the entrepreneurial innovation process for its adoption in society and the economy.



Abers, R. N., Fung, A., & Wright, E. O. 2003. Deepening Democracy: Institutional Innovations in Empowered Participatory Governance. London: Verso.

Arup. 2010. Smart Cities - Transforming the 21st Century City via the Creative Use of Technology. London: Arup. Accessed March 1, 2015:

Baccarne, B., Merchant, P., Schuurman, D., & De Marez, L. 2014. Urban Socio-Technical Innovations with and by Citizens. Interdisciplinary Studies Journal, 3(4): 143–156.

Baptista, M. 2005. E-Government and State Reform: Policy Dilemmas for Europe. The Electronic Journal of E-Government, 3(4): 167–174.

Caragliu, A., Del Bo, C., & Nijkamp, P. 2009. Smart Cities in Europe. Serie Research Memoranda 0048. Amsterdam: Vrije Universiteit, Faculty of Economics and Business Administration.

Coenen, T., van der Graaf, S., & Walravens, N. 2014. Firing Up the City? A Smart City Living Lab Methodology. Interdisciplinary Studies Journal, 4(3): 118–128.

De Lange, M., & de Waal, M. 2013. Owning the City: New Media and Citizen Engagement in Urban Design. First Monday, 18(11).

Dutilleul, B., Birrer, F. A. J., & Mensink, W. 2011. Unpacking European Living Labs: Analysing Innovation’s Social Dimensions. Central European Journal of Public Policy, 4(1): 60–85.

European Parliament. 2014. Mapping Smart Cities in the EU. European Parliament: Policy Department, Economic and Scientific Policy. Accessed March 1, 2015:

FutureEverything. 2013. Smart Citizens. Manchester: FutureEverything. Accessed March 1, 2015:

Gudeman, S. 2008. Economy’s Tension: The Dialectics of Community and Market. New York: Berghahn Books.

Hall, R. E., Bowerman, B., Braverman, J., Taylor, J., Todosow, H., & von Wimmersperg, U. 2000. The Vision of a Smart City. Report no. BNL--67902; 04042. Upton, NY: Brookhaven National Lab.

Hancke, G. P., de Carvalho e Silva, B., & Hancke Jr, G. P. 2012. The Role of Advanced Sensing in Smart Cities. Sensors, 13(1): 393–425.

Hielkema, H., & Hongisto, P. 2013. Developing the Helsinki Smart City: The Role of Competitions for Open Data Applications. Journal of the Knowledge Economy, 4(2): 190–204.

Horelli, L. 2013. New Approaches to Urban Planning. Insights from Participatory Communities. Helsinki, Finland: Aalto University.

Juujärvi, S., & Pesso, K. 2013. Actor Roles in an Urban Living Lab: What Can We Learn from Suurpelto, Finland? Technology Innovation Management Review, 3(11): 22–27.

Leadbeater, C. 2006. The User Innovation Revolution: How Business Can Unlock the Value of Customers’ Ideas. London: National Consumer Council.

Leminen, S., & Westerlund, M. 2012. Towards Innovation in Living Labs Networks. International Journal of Product Development, 17(1/2): 43–59.

Open Government Working Group. 2015.What Is Open Government Data? Open Government Data. Accessed March 1, 2015:

Pallagst, K., Aber, J., Audirac, I., CunninghamSabot, E., Fol, S., Martinez-Fernandez, C., Moraes, S., Mulligan, H., Vargas-Hernandez, J., Wiechmann, T., & Wu, T. 2009. The Future of Shrinking Cities: Problems, Patterns and Strategies of Urban Transformation in a Global Context. Berkeley, CA: Berkeley Institute of Urban and Regional Development.

Pallot, M., Trousse, B., Senach, B., Schaffers, H., & Komninos, N. 2011. Future Internet and Living Lab Research Domain Landscapes: Filling the Gap between Technology Push and Application Pull in the Context of Smart Cities. eChallenges e-2011 Conference Proceedings: IIMC International Information Management Corporation.

Ratti, C., & Townsend, A. 2011. The Social Nexus. Scientific American, 305(3): 42–48.

Saegert, S. 2004. Community Building and Civic Capacity. New York: Aspen Institute Roundtable for Community Change.

Sanders, E.-N. 2000. Generative Tools for Co-Designing. In S. A. R. Scrivener, L. J. Ball, & A. Woodcock (Eds.), Collaborative Design, 3–12. London: Springer.

Schaffers, H., Komninos, N., Pallot, M., Trousse, B., Nilsson, M., & Oliveira, A. 2011. Smart Cities and the Future Internet: Towards Cooperation Frameworks for Open Innovation. In J. Domingue et al. (Eds.), The Future Internet, 431–446. Springer.

Schaffers, H., Ratti, C., & Komninons, N. 2012. Special Issue on Smart Applications for Smart Cities - New Approaches to Innovation: Guest Editors’ Introduction. Journal of Theoretical and Applied Electronic Commerce Research, 7(3): II-VI.

Stembert, N., & Mulder, I. J. 2013. Love Your City! An Interactive Platform Empowering Citizens to Turn the Public Domain into a Participatory Domain. Delft, Netherlands: Technical University Delft.

Turnhout, E., Van Bommel, S., & Aarts, N. 2010. How Participation Creates Citizens: Participatory Governance as Performative Practice. Ecology and Society, 15(4): 26.

von Hippel, E. 2005. Democratizing Innovation: The Evolving Phenomenon of User Innovation. Journal Für Betriebswirtschaft, 55(1): 63–78.

Wagemans, M. 2002. Institutional Conditions for Transformations: A Plea for Policy Making from the Perspective of Constructivism. In C. Leeuwis & R. Pyburn (Eds.), Wheel Barrows Full of Frogs Social Learning in Rural Resource Management, 245–256. Assen, Netherlands: Van Gorcum.

Westerlund, M., & Leminen, S. 2011. Managing the Challenges of Becoming an Open Innovation Company: Experiences from Living Labs. Technology Innovation Management Review, 1(1): 19–25.

Share this article:

Cite this article:

Rate This Content: 
No votes have been cast yet. Have your say!

Keywords: citizen involvement, Living lab, open government data, smart city, toolkit