Introduction 

It has been claimed that Artificial Intelligence (AI) carries enormous potential for service and product innovation. In this paper, the term AI-based innovation refers to new and improved products and services that are based on the use of AI-technologies, rather than to the use of AI as a tool for innovation management. Examples of AI-based innovation include new monitoring tools that use the automatic identification of objects in a video stream from learned data, new services based on speech recognition, or new optimization techniques for improved logistics based on automated knowledge acquisition using historic data. These innovations use AI technologies (definition to follow below) in one of its many forms, such as deep learning from sequences of data, knowledge-based decision making, or complex pattern recognition. With a history of more than half a century, AI technologies can no longer be called “new”. However, recent advances in data processing tools, falling prices for computation and data storage, and a pervasive sensorization and digitization of our environment, have led to a new surge in AI-enabled products and services.

There are numerous opportunities for new and improved services and products arising from AI technology, many of which are based on the fact that the technology often relies on learning from data. Such an approach is very different from traditional IT system design, and can result in systems that deliver entirely new functionality or improved quality features (for example, recognition rates of pattern recognition systems). However, AI systems may often be more difficult to explain than conventional software systems, as they employ statistical techniques that are not easily explainable using everyday (that is, non-mathematical) language. Also, such systems often require large amounts of data, either for training or for large knowledge bases, which also may impair easy and straightforward explanation of its actions. Since historic data, learning, and evaluation are of central importance in the design and construction of AI-based systems, their development can be very different from conventional systems. Similarly, questions of user interaction and user acceptance can be very different from traditionally developed IT systems, for example, as regards explainability or ethics. Finally, learning systems involve important issues of data acquisition, quality of data, and responsible use of personal data. All of these characteristics pose the question of whether or not managing AI-based innovation implies challenges that are specific to the development of AI-based solutions.

There is little published empirical work on AI innovation management challenges to date. This contrasts with many studies, including those published by large multinational consulting firms, proclaiming enormous potential for AI technologies. Although the visionary dimension of these studies is often inspiring, they often use broad and general projections about AI technology and its potential benefits. In order to avoid both the fear and hype surrounding AI, real-world data about the status quo of AI-based innovation is necessary for evidence-based innovation policy making. Such factual evidence is even more important for specific approaches to AI-based innovation management, in order to provide an early understanding of actual real-world coming challenges, and to develop management and policy answers to those challenges. Consequently, the aim of this study is to present empirical data from Austrian companies on specific challenges of AI-based innovation.

The main aim of this paper is to provide empirical evidence for specific innovation management needs of companies using AI, based on a broadly defined group of economic entities. This breadth was chosen with the purpose of supporting evidence-based policy making for AI-based innovation. The long-term perspective of this study aims to help design a national AI strategy, along with specific support measures for AI-based innovation. The paper concludes with recommendations for AI-based innovation management to meet the needs of policy makers interested in supporting AI-based innovation.

Existing work and context

Smart technologies are considered as major drivers of innovation (Lee & Trimi, 2018; Makridakis, 2017) and knowledge for innovation (Fischer & Fröhlich, 2001). A broad range of policy papers (Agraval et al., 2019; Dutton, 2018) and marketing studies from consulting companies have argued for the innovation potential and economic benefits of AI (PAICE, 2018; Li et al., 2017). However, little empirical data on specific practical challenges of AI-based innovation exists.

The study in this paper was part of a larger exercise to estimate the economic footprint of Austrian AI companies, and current international strategies to support an AI environment conducive for innovation. The study design therefore included a larger-scale estimation of AI technology application in various sectors of the Austrian economy. For this, data from multiple innovation and research project databases was analysed. The resulting information was placed in the context of economic statistical data, in order for the Austrian government to understand the size of the overall importance of AI technologies already deployed. Expert interviews were part of the exercise. Here, we report only on these interviews in the context of innovation policy and innovation management. From a more general point of view, this study provides an example of technology-related innovation management challenges, that is, specific challenges for innovation management that are contingent upon a technology, cf. (Prem, 2015). 

Defining Artificial Intelligence

The current lack of empirical data is aggravated by the lack of a commonly accepted definition of AI. Many including the European Commission (EC, 2018) define AI based on the objective of creating human-like behaviour in machines for perception, reasoning, and action. Another possibility is to define AI entirely based on their ability to learn, that is as learning systems. Although this includes a vast amount of applications and sectors, it excludes more (symbolic) rule-based systems, for example, in so-called diagnosis system applications or in other systems that require predictable and understandable behaviour. A definition solely focused on learning would exclude many traditional AI systems in natural language translation. Expert systems, or case-based reasoning systems and other types of rule-based reasoning systems would also be excluded.

An appropriate definition of AI can also be based on the various academic subfields of AI as a field in computer science and engineering. These subfields include: reasoning (logic), learning (neural networks), machine perception (understanding of speech, text, images, & videos), and autonomous behaviour (driving, robotics). Note that this is a mixture of technology-related aspects (learning) with more application-oriented ones (machine perception).

For analysis in this paper, we use the latter characterization based on various academic and engineering AI subfields. Such a definition is well aligned both with the organization of AI research, and also with classification schemes of funding agencies. Industrial robotics was excluded for this reason as it is more a field of automation and production engineering, while autonomous robotics (such as autonomous vehicles from lawn mowers to self-driving cars, etc.) was included as a field of AI. In addition, focusing only on machine learning, as seems to be a current emerging trend, would exclude the field of rule-based AI that has a decades-long tradition, and is comparatively strong in many countries including Austria.

Methodology

Focus and selection

The focus of our study is on Austrian companies using AI technologies for innovative services and products (AI-based innovation). We report on results from 11 interviews with experts, both producers and users of AI technologies as innovative products and services. The selection of potential AI-innovators was based on a keyword list (in English and German) to identify AI technologies belonging to academic AI subfields. The list includes topics in machine learning, knowledge representation and reasoning, autonomous robots (including autonomous driving), machine learning, pattern recognition, and natural language processing. For example, it includes “neural network”, “deep learning”, and “connectionism” to discover innovation and research projects in machine learning. The list is based on IT expert knowledge and existing classification schemes such as the ACM classification often used by innovation agencies. Potential companies were identified using innovation agency databases, industry data, and job search data related to artificial intelligence.

Our interviews were with employees of private research institutes creating AI applications. In most cases, the persons interviewed were CEOs, CTOs, or department heads of these companies. The set includes both small-and-medium sized enterprises, as well as large industry players. All companies in our set deploy or develop AI solutions with the aim of creating innovative services or products. The interviews were performed following a structured interview process about company characteristics, activity sectors, core competencies, innovative AI applications, motivations for using AI, technologies used, the role of start-ups, business models, main customers, barriers, and obstacles.

Results

The study resulted in a rather coherent picture of the current state-of-deployment involving AI technologies. This means that there was broad agreement between the experts on aspects such as general opportunities for innovation involving AI, the current state of its deployment, and on many of the challenges and problems which companies that aim to innovate by using AI are facing today.

Sectors and application areas

The selected company experts covered a range of sectors, with added focus on automotive and other machining industries, that are traditionally strong areas of the Austrian economy with many innovative SMEs and also large industry.  They included people at dedicated AI companies that address broad economic sectors including the service sector. The experts were asked about their core competencies to better distinguish consultants and AI application developers from other enterprises that internally develop their own AI-based solutions. The interviews listed the following areas:

• Analytics, Text Mining, Information Capture 
• Enterprise Content Management
• Privacy protection
• Transport and mobility
• Automotive
• General AI
• Sign language
• Natural language understanding

Although experts from only 11 companies were interviewed in detail, the number of developed AI applications discussed in these interviews was more than 35. They include a broad range of AI application areas, from online sentiment analysis to trend and incident analysis in documents, autonomous driving, intelligent searches to identify experts, predictive maintenance for industrial applications, rolling stock optimization in the transport domain, software defined network management, intelligent travel agency, sign language translation, financial risk management, and AI assistants for human resource management. The applications can be roughly classified in the following area categories with examples (see Table 2):

The main motivations for using or developing AI for innovative products and services include automation, process optimization (adaptation, acceleration), improved efficiency (with respect to costs or personnel), increased flexibility, complexity management, and knowledge management. AI technologies used include machine learning, data analysis and prediction techniques, natural language processing, image analysis, deductive systems, and knowledge graphs.

Technologies

Table 3 provides an overview of the concrete AI technologies that experts mentioned in their interviews, along with the corresponding AI field.

Table 3. Technology field and concrete technologies mentioned by the experts.

Source: Expert interviews (right column) and author’s classification (left).

The role of start-ups and new AI business models

An opinion prevails among those interviewed that start-ups have a vital role to play in both the application and deployment of AI innovations. They are considered the main leaders and competence carriers in AI technology and are praised for their flexibility compared to large industry actors. Specialized start-ups are also believed to invest more in the development of novel AI methods compared with the large software industry. Also, the interviewees consider their solutions to be more straightforward for deployment in comparison with the more complex environments of comprehensive framework providers. On the downside, AI start-ups can be difficult to identify and learn from as they are small and often still developing their value propositions for various sectors.

Regarding AI solution business models, the respondents suggest that these are not fully clear and still being investigated, as the focus is often on quality improvements rather than new business models. It is expected that price planning and dynamic pricing may become a more important aspect of AI applications. AI-as-a-service has already emerged as a specific case and there is a trend towards licensing per service, per application case, or based on usage volume. In addition, there is a trend to shift the development of solutions to the customer given the emergence of more mature training tools for data-driven AI solutions.

AI-applications and AI-development are central to many consulting activities in the domain. Indeed, it is sometimes difficult to clearly delineate consulting companies from AI application developers. There are even indications that a new profession of “AI trainer” is emerging: experts in the computer application domain with competencies in data analytics, where the former is often considered more important than the latter.

Many interviewed experts were convinced that sooner or later no company (at least in a technical domain) will be able to achieve success without a certain degree of automation and, hence, autonomy. This will make AI a general computing method with a strong focus on data-driven approaches to system creation, and also automation.

Challenges and barriers

From an innovation management perspective, the lack of IT and AI experts was the biggest challenge in our interviews. This concerns general IT-experts, but also IT-staff with dedicated AI expertise: AI generalists, AI specialists in neural networks, AI software engineers, and data scientists. The interviewed experts also pointed out that currently even graduates from technical universities, including computer science graduates, may not have acquired sufficient AI expertise during their curriculum. Another main barrier is the cost of creating the required know-how for innovations; AI techniques often require many trial-and-error cycles during the development process. This implies long development times and an inherent difficulty in predicting development time. In addition, it was pointed out that robotic technologies are often costly because of hardware requirements and human effort needed for building or developing robotic systems.

The respondents also identified a current lack of knowledge about AI in the sense that there is insufficient general awareness and knowledge in their own company, including among C-level executives. This often results in people having unrealistic expectations about AI. Managing AI-based innovation is thus a huge challenge for experts when there is not even an agreed definition of AI. Today’s lack of AI knowledge also reduces the credibility of AI solutions. There are many claims from marketing professionals that cannot be confirmed in practice, which unfortunately also results in a lack of acceptance of failures during the innovation process. This comes on top of the recognized challenge that many solutions based on machine learning cannot easily give explanations for their own behaviour. The lack of clear regulation and legislation is a related problem, for example, involving responsibility in the health sector, with control applications or in other engineering fields.

Small and medium-sized enterprises (SMEs) trying to apply AI are often hesitant because of these uncertainties. In addition, they are challenged by the fact that they may be lacking data in terms of volume or quality. Innovation managers often have difficulties estimating the realizability of AI-based innovation projects, in particular when using statistical techniques such as neural networks.

Table 4 provides an overview and classification of the barriers and challenges mentioned in the interviews.

Table 4. Barriers and challenges and some examples (interviews).

Source: Expert interviews (right column) and author’s classification (left).

Discussion

Many Austrian companies have by now come to recognize AI as an important technical enabler of innovation. Although there has been research in AI technology for more than 50 years, there is nevertheless still a sense of novelty today that seems to be driving experimentation. Many aspects of this technology are still emerging, and companies are trying to understand the technology’s possibilities, what their own capabilities are, and where the benefits really lie for innovation.

Innovation characteristics of AI 

Although AI has been studied for more than half a century, it still rapidly developing. There is broad agreement that it is not even fully clear which methods, approaches, or techniques should be included in its definition. The concept of “AI” often describes features of a desired application; this means that the term is defined as making computers do what so far only humans can do. This meaning comes with how the term originated around the 1960s in the US. It should be obvious now that this characterization of AI is really an oxymoron, as it implies that it is a definitional moving target: it emphasizes technological abilities that are somehow not yet demonstrable by computers. This also seems to mean that whenever a technology that originated as a result of AI research matures, it then becomes part of the standard repertoire of computer science and is no longer considered as being “proper AI.” Examples of this include search methods studied for chess computers and early feature detection for image recognition. These techniques eventually became part of the canon of computer science curricula, rather than being framed as due to AI research. Such revisionary history is one of the reasons for the difficulties in clearly defining the term.

The general aim of AI is to make computers smarter for the aid of human perception, decision-making, and action. In this sense, AI systems do not necessarily always have to outperform humans. In many new application areas, AI systems are developed with the aim of achieving automated perception, decision-making, and action with less-than-human degrees of precision. In many useful application scenarios, the AI system can add round-the-clock performance, or simply a way to deal with very large amounts of data. Examples are image recognition and classification applications that may not be always 100% correct, but which nevertheless help to pre-sort cases for human inspection. Other applications of AI may actually target improved quality, for example, in AI-based medical image classification or high-precision robotics applications. These examples point to general value propositions of AI technologies regarding potential innovation, including attempts at AI-based innovation ranging from performance and quality gains, to radically new features that would not be achievable without AI technology, for example, in cases of learning from historic data where no explicit parallel human knowledge is available.

The motivations listed in the interviews about why to use AI are broad and often overlap. Automation and process improvements are a big driver. Another area is management of complexity including knowledge management. Other obvious motivations are increased efficiency regarding technical parameters, personnel resources, and costs.

Besides these qualitative innovation targets, the use of AI promises to deliver technical solutions in areas that could not previously be solved by computer applications. For example, AI learning systems trained on large amounts of data can be used for automated video classification. This will enable previously unavailable solutions in security applications that help to improve quality and reduce costs. Again, this underlines that AI-based innovation is both incremental, and also often an enabling technology where no automated system with satisfactory performance was previously available.

In summary, there are a broad range of innovation promises for AI; from mere improvements to enabling completely novel product and service offerings. And indeed, the innovation examples provided in the interviews clearly range from incremental innovation (for example, quality production improvements using AI for error detection) to “new to the world” innovation (automation of sign language translation). The emphasis in the interview examples was generally on incremental improvements, with some examples given of process automation that could not have been done without the application of AI.

AI in engineering

There are good reasons why many companies in Austria innovating with AI operate in the engineering domain. AI learning systems in many cases require large amounts of training data. Such data is usually difficult to create, unless it is already provided by a company’s technical systems, such as digital production systems, plant control systems, or other technical systems that continuously monitor, and often control operation. Engineering environments (in electronics, automotive production, or machining) therefore appear as prime candidates to roll out novel AI services, simply due to the availability they have of sufficient amounts of data. It became clear in our interviews that indeed the very existence of data is a major driver of experimentation with AI-based innovations. This “data-push” combines with a “technology-push” from current AI development tools, which are now widely available, often at rather low costs, or even for free online.

In addition, engineering companies are more likely to have the required skillsets in-house with regard to computer engineers and data scientists, for example, compared to the service sector. People with these skillsets experiment with novel technologies and typically have a mindset adjusted to technological competitiveness. 

Experimentation, resources and capabilities

The focus on experimentation in the interviews had both a technological and a company dimension. The relative novelty of AI for most companies means that they are in ongoing exploration of their AI innovation resources and capabilities (Tidd & Bessant, 2014). This includes functional capabilities in particular, such as experienced personnel, but also resources, specifically data. Other potentially limiting technical aspects include computational requirements for AI training or AI application.

There is a second dimension inherent in the technological characteristics of AI, at least for learning systems and data-driven systems. At the current state-of-the-art, developing AI systems is a process of trial-and-error. While there are of course often situations in which novel technical solutions require an iterative approach, the situation is exacerbated in the case of AI because of the inherently statistical nature of many AI solutions. For such statistical (learning) systems it is often not fully clear if a solution is possible at all. In addition, the process of tuning a learning system requires several stages of training and test cycles. The lack of technical predictability becomes an important challenge for innovation management if there are inflated expectations about AI’s technological possibilities. Many company experts warned about the danger of disappointment that may arise from very high expectations, followed by only mediocre or modest performance from an AI solution. The resulting disillusionment could eventually mean that companies refrain or delay too long from exploring potentially promising solutions.

It is particularly interesting for AI innovation management that companies may not fully understand their data resources to the extent necessary for AI solutions. Small companies may lack the kind of long and consistent data sets that are typically required for deep learning solutions. More critically, the interviews suggested that this is a specific problem for smaller companies and that it is very hard for most of them to know precisely what information is in their data, for what time periods that data is reliable, etc. A new kind of “metadata expertise” could therefore become essential for assessing the technical viability of an AI solution, and for designing an AI system and an efficient development process.

From the perspective of innovation management, data is an interesting case as it represents both a technical and a historic dimension. The usability and value of any given data set will depend on the technical characteristics of the precise AI technology, for example, deep learning, case-based reasoning, or a symbolic expert system. In addition, data carries an element of history that is typically not well described in explicit metadata information. Rather, this data history requires competent interpretation by human domain experts in order to understand any potential limitations or opportunities. In the interviews, this aspect of domain knowledge in combination with a proper appreciation and understanding of the available data was mentioned as a current shortcoming. Here, some of the experts we spoke with suggested the potential future job profile of an ‘AI training’ expert. These AI training experts are knowledgeable in how to develop data-based AI systems and they also understand important limitations of AI systems. However, they are not necessarily experts in the application domain.

In summary, there are at least three specific aspects of AI innovations that require consideration for innovation management at the level of business innovation: technologies, resources, and capabilities. 

This suggests that the successful development of AI-based innovations at an early (pre-market) stage may already require three different types of experts: AI experts, domain experts, and metadata experts. The current lack of experts in science and engineering in many OECD countries also underlines the importance of proper policies for human resources in these areas.

Value creation

On the demand side, customers consider obvious criteria such as the cost and value proposition of an innovative solution. Less obvious AI-related aspects are trust and understandability, as well as the ability to explain and predict system behaviour. These aspects are closely linked. In engineering domains, it is particularly important that solutions (for example, involving control, but also maintenance, automation, etc.) are reliable. In many cases where AI solutions promise improvements over traditional approaches this comes at the price of reduced clarity and predictability. This is not necessarily only true for statistical learning systems, however. Even large-scale rule-based systems may easily become practically untraceable and extremely difficult to predict. The related and specific challenges for AI-based innovation have already become an important subject in research policy, and also in AI research itself. Interestingly, the focus in public discussion is often on explainability, which is a rather different concept. In any case, the typical iterative development and necessity to assess quality through testing is a challenge for AI-based innovation in engineering as many potential customers express concerns about the reliability of innovative AI solutions even where they may outperform existing systems.

For the case of Austria there is a further key aspect to consider. As mentioned before, the strong machining, electronics, and automotive industries can build applications based on historic data. However, they may also have very tight requirements or expectations regarding predictability, reliability, and explainability of systems. For the AI innovation manager, this may imply a preference for solutions that exhibit these characteristics. And in areas that are less regulated or where there is no hard requirement for full predictability and explainability, it means a focus on testing, evaluation, and demonstration to gain the necessary trust.

Following our interviews, the question about AI-specific business models remains an interesting open issue. Many of the respondents did not see such a model emerging just yet. The mention of change in the business model, such as shifting from products to services, is more in line with typical business model innovation following digitization (Prem, 2015b). The more interesting case is AI-as-a-service, where it may be necessary to distinguish online creation of AI systems (for example, training neural network models), from online use of already trained AI systems. Issues such as data ownership, dynamic service pricing, and intellectual property rights of AI models could become AI-specific innovation challenges and methods.

Conclusion

The data collected from expert interviews regarding AI-based innovation identifies key challenges for innovation management. Some of these challenges are specific to AI-based solutions. In the context of recently published AI strategies, the interviews suggest that significant emphasis needs to be put on human factors, including training and communication involving AI techniques. Successful AI innovation management also needs to address the availability of high volumes of good-quality data, especially in SMEs. Of particular importance is human expertise in the AI and application domain, as well as for historic and semantic aspects in the case of statistical techniques that rely on past data.

The study aimed to inform the development of an Austrian national AI strategy. The data may also be useful for innovation managers seeking to understand both the opportunities and challenges of companies aiming to deploy innovative AI solutions. For researchers, the data suggests potential new focus topics of further research, for example, AI-related business model development, proper management of expectations in AI-related innovation processes, and further insights into the constraints emerging from the historic aspects of data, along with required metadata expertise.

For policy makers interested in supporting AI-based innovation, the results suggest focussing on human resources such as AI experts, as well as developing further emerging new job profiles such as “AI-trainers” who are proficient in training AI systems without necessarily developing novel AI techniques. In addition, research policies should support investment in technologies for explainable and trustworthy AI. Regulatory aspects concern the freedom to work with new business models and the development of a clear and reliable regulatory framework for AI-based innovation.

References