Research is four things: brains with which to think, eyes with which to see, machines with which to measure and, fourth, money.
Albert Szent-Györrgyi (1893–1986)
Hungarian Nobel Laureate in Physiology or Medicine (1937)
In developed countries, a large share of R&D work is performed in universities, but the real significance of their contribution is larger, because they conduct most of the fundamental research. In this article, we examine one aspect of the academic sector that is visible to most outsiders, a field that requires usually the most resources as well: the research infrastructure. Hungary is currently in the process of forming its own National Infrastructure Roadmap. We present the results of a nation-wide survey carried out in 2014 by the National Innovation Office in support of the National Infrastructure Roadmap. The results represent a good starting point for developing measures and setting up goals for scientific fields. With the identification of research infrastructure usage by industry, this method might provide a best practice for other countries to undertake similar evaluations for their respective infrastructures.
Business–academia collaborations are nowadays viewed as key factors in bringing R&D results to companies, through the universities “third role” of supporting economic development and the supporting of the national competitiveness (Ambos et al., 2008; Etzkowitz, 2003; Rasmussen et al., 2006). These collaborations between industry and universities lead to more intense R&D (Bozeman, 2000) and also to an increase in licensing activities, and through them an increase in R&D’s impact on innovations for the business sector as well (Bonaccorsi et al., 2014). Regardless of the innovation model we examine, be it science-push or the (relatively) new networked model, the core of these theories is the major role of academia in innovation. All models conclude – as is logical – that basic R&D has an impact on innovation, although they differ significantly on how exactly this happens (Caraça et al., 2009; Kline & Rosenberg, 1986). We can assume that it is true that basic research has an impact on innovation. But in this article, we examine one aspect of “how” and try to answer the question “to what extent”.
Governments and industry increasingly perceive universities as “a major agent of economic growth”: the knowledge factory, as it were, at the center of the economy. In such an economy – one in which ideas and the ability to manipulate them count for more than the traditional factors of production – the university is seen as an increasingly useful asset. It is not only the nation's R&D laboratory, but also the mechanism through which a country augments its “human capital” to better compete in the global economy. A large share of R&D work, about 25 to 35 percent, is performed in universities (Eurostat, 2016), but the real significance of their contribution is larger, because they conduct most of the fundamental research.
Some authors analyze the relationship between universities and industry on the basis of case studies (e.g., Meyer-Krahmer & Schmoch, 1998); various publications dealing with the problem of how to improve the technology transfer from universities to industry have conducted broad surveys at universities regarding their industrial contacts (e.g., Chapple, 2005; Guerrero et al., 2015; O’Kane et al., 2015).
In this article, we examine one aspect of the academic sector that is visible to most outsiders, a field that requires usually the most resources as well: the research infrastructure. Research infrastructure usage is one of the most logical and apparent usages of academic resources besides research contracts with scientists and their institutions. The role of research infrastructure is widely considered as important as basic R&D for innovation, if not more important. It can also be used as an “indicator” for understanding science and technology policy (Jacob & Hallosten, 2012). Still, it has only been partially studied, and literature on it is limited (Hallonsten & Heinze, 2012).
In this article, we share the results of a survey conducted in 2014 among the Hungarian research infrastructure owners: it is our attempt to define the cooperation levels between industry and academia. First, we examine the role of research infrastructure. Then, we describe the context of the survey: the development of a National Infrastructure Assessment and Roadmap in Hungary. We next describe the survey itself and present the results before finally offering conclusions and discussing the implications of the work.
The Role of Research Infrastructure
The problem of deriving value from research infrastructure has a long history dating back to at least the 1940s, and the approaches range from basically giving lots of money to research infrastructure to demanding income from them (Hallonsten & Heinze, 2012). Most countries spend huge sums to upkeep, build, or upgrade their research infrastructures in order to provide the necessary equipment for scientists. And some fields of science, such as physics, require relatively large amounts compared to other fields, such as social sciences. Given that spending on R&D for the academic sector comes from governments, it is politically important to make people understand what comes out of this spending. One of the explanatory factors is the usefulness of research infrastructure to industry and therefore its ultimate impact on the economy.
The usefulness and importance of research infrastructure is emphasized through various initiatives, such as the European Strategic Forum for Research Infrastructures (ESFRI) roadmap, a European Commission forum for research infrastructures (ESFRI, 2010). The roadmap aims to identify new research infrastructures of pan-European interest corresponding to the long-term needs of the European research communities, covering all scientific areas, regardless of possible location. Economic importance is not a key factor in selecting the infrastructures for the roadmap – which is fully acceptable, because these infrastructures in almost all cases support basic research, and their industrial relevance is not a priority. Although it is not a factor in selecting the infrastructures to the roadmap directly, the evaluation process and the connecting application the research infrastructures (buildings, lab equipment, etc.) have to show their relevance to industrial users. The industrial aspect arises mostly from the political side – governments and their citizens wish to see a return on their investment, and not through scientific achievements that are poorly understood by the general public. Take, for example, the lack of general understanding about the Higgs boson despite a simple explanation being called for and provided to make the concept more comprehensible. Rather, citizens wish to see the impact of such investment through products and technologies that boost industry. Many of those responsible for making science policy prefer to view innovation in the spirit of the science-push model, or the linear model at best. Although the linear model is obsolete by now, because it draws a single direct line between basic research and innovation (not considering the organic nature of the process) and there are many new models trying to take its place – such as the multi-channel interactive learning model or the revisited contingent effectiveness model (Bozeman et al., 2015) – its simplicity gives it an advantage over the other models.
Nevertheless, looking either of the above-mentioned models, we find that the importance of the academic sector and higher education is undoubted, but still, the public has to be convinced of this fact from time to time. In the case of research infrastructure, one interesting example is that of a major infrastructure under construction, the European Spallation Source (ESS), a multi-disciplinary research centre based on the world’s most powerful neutron source. Currently under construction in Sweden, this new facility will enable new opportunities for researchers in the fields of life sciences, energy, environmental technology, cultural heritage, and fundamental physics. A key factor in the decision for building the ESS in Sweden was “to explain the purpose and usefulness of the facility and the research” (Agrell, 2012). However, the linear innovation model leaves a very strong and not very positive mark on public science communication, which can be summed up as “the assumed ‘unexplainable’ nature of advanced scientific projects and activities” and “the power of catchwords and compelling non-scientific arguments” (Agrell, 2012). This situation sometimes results in decisions that are suboptimal, not only from the scientific side but also from the economic side. For instance, certain studies indicate that the decision to build ESS in Sweden was much more of a political decision than one that was based on scientific evidence on the optimal location (Hallonsten, 2014). This decision has a component that is interesting from the industry–science cooperation side as well – before the decision was made, the idea of public–private partnership was brought up so that it would boost Swedish industry partners’ potential to become partners for the ESS completion, but it was found that their added value would be doubtful. This fact was not taken into consideration during the final decision making either.
The overall situation in “big science policy” is the logical consequence of policy change over time from “justifying investment in basic science by reference only to the utility of basic research” (Elzinga, 2012). With the financial restrictions appearing after the Cold War was over, the “old arguments” (or the old communication panels) could no longer be used by scientists, who admitted that “OECD represents the economic and political interests of its members, not the intellectual interests of scientists” (Elzinga, 2012). From about the late 1990s, it has become a more and more demanding question to see how science contributes to the economy and to society as a whole. Although there is a certain danger to the academic sector in the cooperation with industry, namely the delaying or even the suppression of scientific publications (Banal-Estanol et al., 2015), the expected gain from using these infrastructures for applied research outweighs scientific reasoning.
Nowadays, the arguments on science’s business orientation include greater cost consciousness, flexibility, and efficiency (Barzelay, 2001). The result is higher education acting more and more as a private company from a public relations view: institutions hire managers to oversee scientific budget and projects, form profit centres and build “brands”. One prominent example in the case of “big research infrastructures” is their use of acronyms to “code” their infrastructures so that they are easy to say and remember, such as ALLEGRO, FAIR, ALICE, CLARIN, VIRGO, CESSDA, PRACE, and so on.
Science (and research infrastructures) face the dilemma of how to commercialize their knowledge and show their usefulness to the public (Huzair & Papaioannou, 2012). The usefulness of science is usually shown through open days and various events to the public, but they also have to prove to decision makers that the science they do is important for the economic actors as well.
This importance is hard to measure, however. What is the desirable level of cooperation with the industry? If we ask a policy maker, then the answer will be likely “as much as possible”. But, until now, there has been no attempt to define what “as much as possible” really means. By developing a robust dataset, we seek to define the current and expected amounts of cooperation for each science field’s level of industrial cooperation.
Hungary's National Infrastructure Assessment and Roadmap
Hungary is currently in the process of forming its own National Infrastructure Roadmap, which would be a natural addendum to that of the ESFRI. In 2014 a nation-wide online survey was carried out by the National Innovation Office within the framework of the National Infrastructure Assessment and Roadmap project (known in Hungary by the acronym NEKIFUT). The survey targeted the owners of research infrastructure to gather data on their scientific relevance, demand for improvements, openness for usage by researchers, and so on.
The online survey was completed by 450 infrastructure owners, from which a scientific board selected the ones that could be considered as “research infrastructure”. Infrastructures that were of scientific importance but were not research-oriented were omitted from the analysis; for instance, we did not include infrastructure used for educational purposes only. The selection process was guided by the following definition of research infrastructures:
“Those facilities or families of facilities, live and physical material repositories, data repositories, as well as information systems and services which are indispensable for scientific research activities and for the dissemination of the results. Those human resources which are necessary for the professional operation, use and services of research infrastructures are considered to be an integral part of Research Infrastructures.”
The structure and size of research infrastructures depend largely on the specificities of the given scientific field, as well as the needs of the research community using it. The entire process was carried out in broad cooperation with the scientific community. The project was led by a steering committee, while the three main academic branches (physical and engineering sciences, life sciences, and social sciences and humanities) were examined by separate working groups (with a total of 83 members). Overall, the project contacted several thousand researchers.
This process has resulted in numerous valuable outputs, including the development of indispensable tools and methodologies for the governmental research infrastructure development programme; the definition of various infrastructure categories with an internationally unique system for their classification; and the assessment and classification of existing research infrastructures. It has further resulted in IT development for the register itself.
After the evaluation of the online survey results, 328 infrastructures were taken into the Register of Research Infrastructures and their data are currently used to provide background information for the national roadmap. This number of research infrastructures can be considered as the vast majority of Hungarian research infrastructures, considering that there are 44 Academic (Hungarian Academy of Sciences) Research Institutes including all scientific fields and 12 higher education units (universities and faculties) involved in basic research in Hungary.
Our ability to compare this volume internationally is currently limited. However, there is one survey on research infrastructure at the European level: the Mapping European Infrastructure Landscape (MERIL). The MERIL portal gives open access to an inventory of "research infrastructures of more-than-national relevance in Europe across all scientific domains", including the humanities and social sciences. One main goal of MERIL is to “allow policy-makers to assess the state of research infrastructures throughout Europe to pinpoint gaps or duplications and make decisions about where best to direct funding”, therefore it can be considered a policy-making tool as well. From 27 European countries, it lists 495 operational research infrastructures, 26 of which are Hungarian. If we compare our figure to MERIL’s figures, the Hungarian database can be considered a robust one – to our knowledge, no other national or international database exists containing this number of research infrastructures.
Analysis of National Research Infrastructure
The online survey was filled out mostly by universities and academic research institutes, giving us a good overview of the division of research infrastructure across the various scientific disciplines. All research infrastructure were categorized by their main discipline; interdisciplinary work was not taken into account even though there are certain fields that regularly use interdisciplinary approaches. According to the survey design, each infrastructure was asked to provide its main discipline only; respondents were not obliged to describe connections with other disciplines, and the detail provided by respondents varied widely in this regard.
Natural Sciences made up more than half of the examined infrastructures (Figure 1), which is not surprising given that this branch requires the most research infrastructure. Engineering Sciences come second; this branch has a strong connection to applied research and has a relatively high need for a diversity of research infrastructures. Medical Sciences and Agrarian Sciences also have connections to applied research, but each has fewer research infrastructures than Engineering Sciences. The number of research infrastructures devoted to Social Sciences and Arts & Humanities is less than 10% of the number devoted to Natural Sciences and represents only 5% of total research infrastructures in Hungary.
Figure 1. Distribution of scientific branches among national research infrastructures in Hungary (n=328)
From these figures, it can already be seen that, the biggest need for “stand alone” research infrastructure comes from the Natural Sciences. As we “shift” towards more and more applied research areas, the demand for a dedicated research infrastructure lessens – medical infrastructure is usually used for actual medical practice as well, agrarian infrastructure is usually used for actual agrarian processes, and infrastructure in engineering is used for production and development besides basic research. The case of Social Sciences and Arts & Humanities is somewhat special because the low amount of infrastructure means that there are only a few infrastructures (in this case databases) dedicated to these fields. They require fewer databases, but the databases must be more comprehensive and mostly international.
The above analysis provided us with evidence on the characteristics of each branch. Common sense also tells us that basic research has a bigger infrastructural need in the Natural Sciences, whose research activities involve basic research more often than those branches with other possible applications. The problem is that, until now, no attempt has been made (mainly because the lack of data) to assess the current and expected amount of usage of these infrastructures beyond basic research.
This matter can be answered by looking at the cooperation levels of discipline fields with companies. We can assume that the usage of a research infrastructure by companies provides a good indicator for infrastructure usage beyond basic research. Cooperation with companies usually takes the form of applied research or experimental development; only seldom does basic research come into the picture. Applied research and experimental development in optimal cases result in a new or advanced products and thus the cooperation will have an economic impact as well. With the usage of data gathered from the survey, we can measure current levels of cooperation with industry for each branch (OECD, 2015).
Among the many other data asked from the research infrastructures’ owners, we use the following equation to calculate a scientific branch cooperation index, which measures the levels of cooperation with industrial partners and desired partnership intensity:
SCI = scientific branch cooperation index
CU = company utilization of research infrastructure (%)
TU = total utilization of research infrastructure (%)
N = number of infrastructures in scientific branch
For instance, research infrastructures in Physics have an average scientific cooperation index of 7.3% for 45 infrastructures, containing figures as high as 86% of total usage and 40% of company usage. However, some infrastructures in the same scientific branch are not used by companies at all.
Other data were considered for use in the determination of the scientific cooperation index, but were later rejected upon testing. For instance, the actual number of researchers was originally thought to provide a good weighting number for the infrastructure usage. This figure, however, was found to have no impact on the industrial usage. In most cases, industrial users do not directly use the infrastructure, but rather ask for its usage and the additional knowledge of the scientists, because they simply do not have the skills to use, for instance, a spectrometer. A scientist can cooperate in various projects at any given time, or may not get involved in any project at all; therefore, the total number of scientists at a research infrastructure is not taken into consideration in calculating the scientific cooperation index.
The data from the 328 infrastructures were used, divided among disciplines after the data consolidation. Figure 2 shows the results comparing each of the branches. The results of the analysis are not surprising in the sense that they support the expectations of industrial partnership levels in the scientific branches. However, with the exact level of cooperation defined, we can provide a good basis for any further expectations for industrial usage in certain scientific branches.
Figure 2. Levels of cooperation with industry by scientific branch, as measured by the scientific cooperation index (SCI)
The overall extent of cooperation between industry and research infrastructure is very low, with an average scientific cooperation index of 6.8% (Figure 2). Thus, cooperation, with slight differences among the scientific branches, is an exception rather than a rule. In the case of the Natural Sciences, the index is 7.3%; this above-average score can be considered good performance given that the majority of the examined research infrastructures came from this branch. With this score, Natural Sciences are second in cooperation levels with industry; however, this figure also suggests that, despite policy’s demand for more and more industrial usage and income generation, the cooperation levels are still very low. Given that the costs of infrastructure upkeep or improvement in the Natural Sciences are among the highest of all the branches, it is expected by policy makers that these infrastructures should “overperform” – performing better by 7.1% than the overall (as seen, already very low) average is certainly not the expected score.
Within the Natural Sciences branch, Earth & Environment Sciences perform very well, and not surprisingly, the discipline with the strongest orientation towards basic research – mathematics – lags behind with an index score of only 2.2%. (Because we weighted the infrastructures with their numbers, this latter figure has little influence on the overall score – deducting it, the 7.3% of usage still remains firmly in place.)
Engineering Sciences definitely take the lead in this comparison, with an index of 10.1%, which is by almost 50% better than the average. We can assume that these infrastructures are designed (though perhaps not consciously) to be used not only for basic research but for research into applied science questions as well. This design results in a closer relationship to industrial partners and a more effective usage of the infrastructure. The usage model of engineering infrastructures should be examined in more depth, because this higher level of cooperation could be used to boost industrial usage in other disciplines’ infrastructures as well.
Agrarian Sciences underperform, though one would expect that the index should be higher because of its relatively close relationship with applied research. It is important to note that this field has two main parts: crops and livestock. These fields perform very differently, with crops reaching an index of almost 12%, whereas the index for livestock infrastructure is only 2%, and the number of sample units are almost equal. In Hungary, livestock numbers have decreased in recent years, and it is obvious that not much research has been done in this field. On the other hand, crops remain a key factor in Hungary’s GDP, as can be seen in its R&D involvement – and through it in the research infrastructures’ cooperation levels as well.
Medical Sciences and Social Sciences and Arts & Humanities range around the same modest levels of cooperation, though the reason for this is likely to be different. In the case of Medical Sciences, although the total utilization of the research infrastructures is high, the company usage is low. On the one hand, these infrastructures are mostly used for actual medical practice; on the other hand, these infrastructures are dedicated solely to basic research – other infrastructures that are used not only for basic research are used in most cases in applied medicine (mainly through measurements). Therefore, only a small part of the “dedicated” basic research infrastructure can be used for company research, and it can be assumed that companies would rather use infrastructures that are closer to applied medicine.
Social Sciences and Arts & Humanities have very low levels of cooperation with industry – in this case, the reason is that these disciplines mostly use either databases that are international or have a strong national characteristic (e.g., linguistic databases). In the case of company cooperation, these databases are usually not directly used by the companies; the added value of the scientists for the data plays a key role in the collection and evaluation of the gathered data.
In general, research infrastructure usage in Hungary is quite low, but the question remains, compared to what? This study provides a good starting point for making measures and setting up goals for each scientific field. Also, we hope similar assessments and surveys will be made by other countries, thereby making international comparisons possible. The exact cause of the “underperformance” of research infrastructure in Hungary has yet to be identified. Nonetheless, our results are based on a robust dataset and lead us to some conclusions to form a realistic picture of the level of cooperation demand for the discipline categories.
First, it would be wise to agree on a level of expected industry–infrastructure cooperation between the infrastructure’s stakeholders. It has been shown that the “old model” of financing these infrastructures cannot be maintained for various reasons (e.g., communications, politics); however, the other extreme, namely the demand for all-industrial usage of infrastructure designed for basic research, can cause more harm than good. When determining the desired levels of cooperation, it always has to be taken into account which discipline is using the infrastructure. Nowadays, decision makers put demands based mainly on building or upkeep costs of the infrastructure, which generates unrealistic demands.
Taking the above figures into consideration, it might be a fair expectation that infrastructures designed primarily for basic research should reach at least 5% company usage as a starting point, whereas those that can be used more for applied research should reach an industrial usage of 10%.
Second, in certain disciplines (Medical Sciences and Social Sciences and Arts & Humanities), it would be useful to drop demands for industrial cooperation – the existence of some basic research infrastructure makes it possible to form company cooperation, though not necessarily directly linked to the infrastructure itself. Also, we can assume that infrastructures that are used and designed primarily for basic research can be used for applied research with certain limits. Although licensing is taken into account, the actual company usage of it is not always clear to either of the stakeholders. There is a gap between scientists and company managers, and neither of them realizes the possible potential or results of such cooperation. A possible solution for this issue would be use of technology transfer officers at each research infrastructure, and, if possible, the “redesigning” of research infrastructures to better serve the identified needs of business users.
After determining the “desired level” of cooperation, certain innovation methods should be put into practice, much like the forming of technology transfer offices at the universities. Without these, no cooperation strategy can be built and the gap between science and industry will not close. Although the survey described here did not ask whether research infrastructure has dedicated management staff, this is a critical question and might be added to similar future surveys. However, we now have data on the services provided by the research infrastructures, which is a good starting point to have the research infrastructure more open towards the business sector.
This article provides a basis for assessing research infrastructure by estimating the desirable level of research infrastructure involvement in industry, which is also a level for their likely maximum involvement. With the identification of research infrastructure usage by industry, the usage of this method might provide a best practice for other countries to undertake similar evaluations for their respective infrastructures.
An earlier version of this article was presented at the 2016 ISPIM Innovation Forum in Boston, United States of America, March 13–16. ISPIM – the International Society for Professional Innovation Management – is a network of researchers, industrialists, consultants, and public bodies who share an interest in innovation management.
Agrell, W. 2012. Framing Prospects and Risk in the Public Promotion of ESS Scandinavia. Science and Public Policy, 39(4): 429–438.
Ambos, T., Mäkelä, K., Birkinshaw, J., & D'Este, P. 2008. When Does University Research Get Commercialized? Creating Ambidexterity in Research Institutions. Journal of Management Studies, 45(8): 1424–1447.
Banal-Estañol, A., Jofre-Bonet, M., & Lawson, C. 2015. The Double-Edged Sword of Industry Collaboration: Evidence from Engineering Academics in the UK. Research Policy, 44(6): 1160–1175.
Barzelay, M. 2001. The New Public Management. Berkeley: University of California Press.
Bonaccorsi, A., Colombo, M., Guerini, M., & Rossi Lamastra, C. 2014. How Universities Contribute to the Creation of Knowledge-Intensive Firms: Detailed Evidence on the Italian Case. In A. Bonaccorsi (Ed.), Knowledge, Diversity and Performance in European Higher Education: A Changing Landscape: 205–230. Cheltenham: Edward Elgar.
Bozeman, B. 2000. Technology Transfer and Public Policy: A Review of Research and Theory. Research Policy, 29(4-5): 627–655.
Bozeman, B., Rimes, H., & Youtie, J. 2015. The Evolving State-of-the-Art in Technology Transfer Research: Revisiting the Contingent Effectiveness Model. Research Policy, 44(1): 34–49.
Chapple, W., Lockett, A., Siegel, D., & Wright, M. 2005. Assessing the Relative Performance of U.K. University Technology Transfer Offices: Parametric and Non-Parametric Evidence. Research Policy, 34(3): 369–384.
Caraça, J., Lundvall, B., & Mendonça, S. 2009. The Changing Role of Science in the Innovation Process: From Queen to Cinderella? Technological Forecasting and Social Change, 76(6): 861–867.
Elzinga, A. 2012. Features of the Current Science Policy Regime: Viewed in Historical Perspective. Science and Public Policy, 39(4): 416–428.
Etzkowitz, H. 2003. Innovation in Innovation: The Triple Helix of University-Industry-Government Relations. Social Science Information, 42(3): 293–337.
Eurostat. 2016. Eurostat Statistics Explained: R&D Expenditure. Eurostat. Accessed July 1, 2016.
Frascati Manual. 2015. Guidelines for Collecting and Reporting Data on Research and Experimental Development: The Measurement of Scientific, Technological and Innovation Activities. Paris: OECD.
Guerrero, M., Cunningham, J. A., & Urbano, D. 2015. Economic Impact of Entrepreneurial Universities' Activities: An Exploratory Study of the United Kingdom. Research Policy, 44(3): 748–764.
Hallonsten, O. 2014. Unpreparedness and Risk in Big Science Policy: Sweden and the European Spallation Source. Science and Public Policy, 42(3): 415–426.
Hallonsten, O., & Heinze, T. 2012. Institutional Persistence through Gradual Organizational Adaptation: Analysis of National Laboratories in the USA and Germany. Science and Public Policy, 39(4): 450–463.
Huzair, F., & Papaioannou, T. 2012. UK Biobank: Consequences for Commons and Innovation. Science and Public Policy, 39(4): 500–512.
Jacob, M., & Hallonsten, O. 2012. The Persistence of Big Science and Megascience in Research and Innovation Policy. Science and Public Policy, 39(4): 411–415.
Kline, S., & Rosenberg, N. 1986. An Overview of Innovation. In R. Landau & N. Rosenberg (Eds.), The Positive Sum Strategy: Harnessing Technology for Economic Growth: 275–305. Washington, DC: National Academy Press.
Meyer-Krahmer, F., & Schmoch, U. 1998. Science-Based Technologies: University–Industry Interactions in Four Fields. Research Policy, 27(8): 835–851.
O’Kane, C., Mangematin, V., Geoghegan, W., & Fitzgerald, C. 2015. University Technology Transfer Offices: The Search for Identity to Build Legitimacy. Research Policy, 44(2): 421–437.
Rasmussen, E., Moen, Ø., & Gulbrandsen, M. 2006. Initiatives to Promote Commercialization of University Knowledge. Technovation, 26(4): 518–533.