In memory of Björn Gustavsen

 

Introduction

In the varied, not to say confusing, discussion of quality in participatory and action research, the use of reliability and objectivity as notions to understand the topic is largely missing (Herr & Anderson, 2015; Reason, 2006). In a companion article that forms the first part of this conceptual review and development of quality concepts for participatory and action research (Lindhult, 2019), I focused on validity, which has some presence in the literature and debate. This is less the case for the concepts of objectivity and reliability. But, is objective and reliable scientific activity not relevant or not appropriate in participatory and action research? Why are the concepts missing or rejected?

One reason is the dominance of positivist and empiricist scientific paradigms and ideologies and a spectator view of knowledge (Dewey, 1929), which motivate efforts to faithfully mirror the lawful patterns of reality (Rorty, 1979). Such views make participatory and action research look too “subjectivist” and “activist” to be objective and reliable, and they lead participatory and action researchers to look for what they see as more viable and appropriate standards. Some have followed the qualitative camp, turning towards more lenient versions in the context of more dynamic, changing, and contingent circumstances, such as reviewability (i.e., enabling a reviewer to follow the trail of research in the documentation) instead of reliability. Qualitative researchers tend to play down objectivity in face of the important role of subjectivity in unavoidable interpretative dimensions of social science.

The purpose of this article is to address the following question: Can the concepts of reliability and objectivity be reconceptualized and reappropriated to enable understanding of scientific excellence in participatory and action research? I believe it is fruitful to take a step further to consider the “subjective” and active role of researchers as vital in enabling scientific objectivity and reliability.

Robust research practices: Rethinking objectivity and reliability

Inquiry needs to be trusted to consistently develop and secure knowledge claims with limited risk of failure and too much deviance from acceptably trustworthy outcomes. Here, reliability and objectivity as quality concepts have important guiding roles in inquiry. Reliability is predominantly conceptualized as reaching the same outcomes in repeated use of research instruments and operations, often seen as human senses empowered, or even better, replaced, by measurement tools. If humans are more or less part of instruments and operations of research, for example as interpretative philosophies are arguing is necessary in interpretative research operations, standardization of instruments (e.g., interview protocols and procedures) is recommended to avoid variation and the corrupting influence of subjectivity. Objectivity is impartiality and being (to a satisfactory degree) free from bias of inquirers that can risk trustworthiness. It is traditionally conceptualized as being free from any influence coming from the subjects of inquiry (that is, free from “subjectivity”), something that is properly achieved by distancing the researcher from any participation in the research domain and from any interaction with the research object.

I believe discarding objectivity and reliability as many qualitative as well as participatory and action researchers tend to do is unfruitful and based on a too-limited view on the meaning and use of the concepts. It is taking its point of departure in less fruitful conceptualization out of an overzealous desire to root out any presence of actors of inquiry influencing the processes and methods. It is often influenced by assuming a spectator view of knowing and a positivistically influenced understanding of inquiry. Through appropriate clarification and rethinking of the concepts, including its ontological and epistemological assumptions, they can become open for involving actors and their characteristics in ways that can enhance, not only compromise, research quality.

Objectivity

A core issue is the role of the “subjective” in making science robust and valid for producing high-quality knowledge. Objectivity is traditionally seen as erasure of subjectivity, for example as “self-distancing” and “disinterestedness” or, more explicitly, “the holding in abeyance, or erasure, of the individual mind’s desires, interests, assumptions, and intents while that mind is in the process of knowing the material world” (Solomon, 1998). Objectivity is typically associated with reality and truth and has the general connotation of solidity, trustworthiness, accuracy, impartiality, etc. The general connotation for many uses of subjectivity includes unreliability, bias, an incomplete (personal) perspective, etc., typically indicating the possibility of error. Objective knowledge, knowledge of reality as it is, is what is attained when all subjective factors are rooted out. But science can only be performed through subjects, implying that fully objective knowledge in this view is impossible for us humans. As Kant (1998) said, pure objectivity, knowledge of the thing in themselves (Das Ding an sich), is impossible because we necessarily sense the world through basic categories of the mind such as space and time. We always wear our blue-tinted spectacles and we cannot take them off, as he eloquently expressed it. The implication is that our subjective, mentally ingrained space–time categories hinder us from attaining full objectivity in seeing reality as it truly is. The spectator, mirroring image of knowledge (Dewey, 1929; Rorty, 1979) is quite evident, but our mental mirrors are inherently flawed.

The role of the subject in advancing science

There are other points of departure for understanding objectivity. We can build on cues for rethinking from Francis Bacon as an early originator of the visions and ideology of modern science. Influenced by humanist scepticism, Bacon “was entirely preoccupied with the specific problems he perceived as hindrances to the advancement of knowledge and the implementation of objectivity in the sciences” (Zagorin, 2001). His main contribution was his identification and elaboration of “idols” – fallacies of the human mind – and the outline of reasonable practices (invention of arts of “induction” in developing notions to interpret nature) for the orderly advancement of learning from experience to successively more reliable axioms with wider import (Bacon, 1960). Approaching objectivity means tempering and minimizing the influence of our idols and using our capacities in applying reasonable norms and practices of inquiry for creating high-quality knowledge.

Bacon had a more practice-based, pragmatic view of knowledge creation in line with Dewey (1939a). Knowledge production is less understood as a logical exercise or “God’s eye”, fly-on-the-wall unobtrusive registration of sense impression mirroring a given reality. It is rather an active, creative, and organized achievement of inquiring actors in developing and securing more reliable connections between notions and experience, better value and workability in practice, and new or enhanced resolution of problematic situations. To know is to be able to think and do in richer, novel, and improved ways, thus scientific research is enabling and empowering for betterment of human condition, as Bacon emphasized. A point in participatory and action research is that research actors and participants can significantly support reliability and objectivity, not only corrupt it by their “subjective” influence. Making claims valid, reliable, and objective in relation to arguably worthwhile purposes often requires significant skills, competence, and virtues of the inquiring subjects. Scientific production is based on relevant competencies and capacities to maintain norms of good inquiry by inquiring actors and their communities.

Threats to objectivity: Bacon’s idols

Objectivity requires actors and includes their effort to diminish the influence of factors that may undermine good norms and practices of inquiry. Here, Bacon’s (1960) idols, fallacies of the human mind, are important to deal with in freeing the inquirer from these obstacles to the trustworthy advancement of scientific learning.

The first type of idol, the idols of the tribe, is common to the human species. It is a range of limitations and deficiencies of the human mind rooted in an anthropocentrism of projecting will, passion, and experience into distorted, reified images of things. People are creating “wishful sciences” by more readily believing to be true what they wish were true, and sticking to these opinions despite countervailing evidence, something today reinvented as “knowledge resistance”. Their senses are often weak, incompetent, and erring in face of the authority of established superstitions and seemingly self-evident principles. Like Dewey (1939a), he saw an experimental attitude and practice as the way to properly establish the authority of experience and both freeing and disciplining the mind for securing advancement of learning.

The second type of idol, those of the cave, are errors due to the peculiarities and variety of impressions of each particular individual, every one of whom is dwelling in his own cave of interests and opinions. The information revolution, starting with the technology of printing that Bacon took early advantage of, has been, no doubt, a vehicle for general learning and enlightenment. But, surprisingly, it has also led to the construction of new caves, today pinpointed in newly invented notions such as “filter bubbles” and “information silos”. Through modern “intelligent” technology, Internet “trolling”, and social media, personalized information is created that is reproducing one-sided and even false opinion where people can, as Bacon say, be obedient to their own fancies.

The third type of idol is the most troublesome, according to Bacon: the idol of the forum (or market place), where “commerce and consort” can create and spread false or ill-defined notions and language that can twist understanding away from experience. The development of the interpretative sciences has, since Bacon, developed science as an enterprise not only for the interpretation of nature but also for the interpretation of culture where words and language form an important dimension of knowledge. But, the challenge of the “deceits” of language seem even more pronounced today in contemporary, more relativist (e.g., post-modern) philosophies questioning that there is something outside the dynamics of language and discourse that can stabilize linguistic expressivity. The development of post-truth cultures is enlarging relativism by embedding it in social fields. People in these settings do not really care about linking words and narratives to experience and evidence but instead are twisting words and narratives according to their personal interests. Bacon´s suggestion that notions need to be successively invented and carefully applied through experimentation to order experience is an important part of his proposed inductive method, but this seems like a weak remedy. However, it may be in accordance with established scientific practices today as well as with the logic of experiential and reflective learning (Kolb, 2015; Schön, 1983) often referred to in participatory and action research.

The fourth type is the idol of the theatre, which consist of the different dogmas coming from false systems of philosophy or deficient principles in the sciences which gives rise, like stage plays, to fictions and unreal worlds. The positivist philosophy of science and the spectator view of knowledge, with its mirroring fiction that I have been criticizing, is an illustration of this type of fallacy of which Bacon had the lengthiest discussion.

Developing the subject to improve objectivity

Although Bacon was optimistic in inventing new inductive practices for the advancement of science, he did not believe this would be accomplished by doing away with the subjective fallacies. He believed that “the first two classes of idols were hard to eradicate and the other two classes could not be eradicated at all. The most that could be done with the latter…, was to point them out so that their insidious effect on the mind could be identified and overcome” (Zagorin, 2001). The optimistic belief that enlightenment, education, science, and the information revolution should free the human mind from its fallacies has not been generally confirmed, but instead seems today to be more in line with Bacon´s more wary view. Thus, Bacon did not argue for avoiding subjectivity, which still is a dominant ideal in today’s objectivity understanding. Objectivity, instead of neutrality, stamping out subjectivity, and detachment from research context is based on developed forms of subjectivity that is able to temper Baconian idols and maintain norms of good inquiry. It is a move from naïve to critical subjectivity (Herr & Anderson, 2015; Reason, 1994) that enables one to, through self-reflexivity, critically examine one’s own biases and put in place procedures, such as peer reviewing by critical friends, which can support more impartial and truth seeking, that is, objective, inquiry. Bacon himself is also proposing critical procedures as a remedy. For example, for cave fallacies and today’s filter bubbles, which tend to lock people into information sources that strictly conform to their own world view and allow one-sided views to thrive, Bacon suggests; “follow the rule that whenever their minds seized upon something with special satisfaction, they should consider it suspect and take special care to keep their minds balanced and clear” (Zagorin, 2001).

To be objective is to develop and make satisfactory claims to knowledge that are not dependent on who is making the claim or on peculiarities of the organization of inquiry. It has a moral dimension of inquiry as fair, impartial, and unbiased. It is in accordance with Gustavsen´s (2003) image of a referee in a football match in the introductory quotation, where the referee has to maintain objectivity in face of an often-biased situation. Objectivity is crucial in maintaining the norms and rules of the game, and the trust of the actors in the fairness of the game. This is equally important in science. One way to deal with subjectivity is to include more subjects in inquiry (Westbrook, 1995), like the team of referees in a football game. Objectivity is then recognized as intersubjectivity (Lindhult, 2008), something that accrues when there is consonance among subjects emerging from an interaction between them. This is supported by social, communal, and practice-based epistemology and inquiry orientation often used in participatory and action research (Coghlan, 2016). It is social in the sense of recognizing the distribution of expertise and inquiry capacities in society, and through its important communal dimensions. We are standing on the shoulders of others (building on their insights as well as misconceptions!), and we are interacting indirectly or directly with others in the co-production of knowledge. Through active and collaborative research roles, the best available expertise of various forms of knowledge and research practices and skills of inquiry can be mobilized. It is also social in the sense of creating spaces and furthering norms of questioning and thinking and acting differently from established knowledge, authorities, and traditions through collaborative critical reflection, as well as integrating contributions through public discussion (Foucault, 1984; Kant, 1991). This is expressed in the motto Sapere aude! (Dare to be wise!) that Kant saw as epitomizing Enlightenment ideology.

Knowledge is practice-based in the sense of good, scientific inquiry being based on the skillful and competent performance of effective and efficient research practices and methods appropriate in context and for chosen purposes. Moreover, in the sense that inquiry requires action to change and transform situations and problems to develop, test, and validate knowledge claims and resolve issues (Dewey, 1939a). Here participatory and action research is closely allied to science of design, the science of the artificial (Simon, 1996), opening up for a participatory worldview (Heron, 1996) where we are creators of knowledge to approach human purpose, not only discoverers of existing patterns in reality. Simon emulates Dewey’s (1939a) definition of scientific inquiry in his oft-cited definition of a designer as everyone “who devises courses of action aimed at changing existing situations into preferred ones” (Simon, 1996). Like Bacon, Simon argues that knowledge of nature and knowing in human operation combine in the goal-oriented design of artifacts as resolutions to issues or realization of imagined opportunities. In this way, science constitutes the empowering of people for development of human betterment. But, while Bacon focused on “interpretation of nature”, design sciences related to various professions and disciplines, such as engineering, management, creative arts, innovation and design research, and social reform (Dewey, 1929; Schön, 1983; Simon, 1996), are devoted to imagining, constructing, and innovating physical and social artifices that can realize human visions and goals. Here the “wishful sciences” that Bacon condemned have a scientific status on par with other sciences to the extent that they advance the sciences of design.

Objectivity through practical wisdom

I believe objectivity, here, fruitfully means the development of good judgment, Aristotelian phronesis (practical wisdom), a capacity for considered, balanced judgment in situations mediating and integrating different considerations (e.g., different interpretations and stakeholder concerns) when participating in research contexts (Lindhult, 2004). Phronesis is, according to Aristotle, experience-based judgement: “a true and reasoned state of capacity to act with regard to the things that are good or bad for man” (Aristotle, 1980; see Eikeland [2008] for an Aristotelian clarification and defence of action research). It involves determining ends and the means of attaining them in particular situations. As Lewin says in a seminal article outlining action research (Lewin, 1946), without consideration of ends, it is like a boat without navigation moving round and round in circles. Wisdom as practical integrates intelligence with virtue in acting according to norms of good judgment and inquiry. Instead of seeing the only alternative to the traditional focus on neutrality and value freedom as being value laden and partial, objectivity here means impartiality in balancing and integrating different values, interests, and purposes in context of research situations with various parties and stakeholders.

A strength in participatory and action research is that objectivity as good judgment and practical wisdom is more a mutual and common accomplishment, not judgment by one or a few “wise guys”, where distributed expertise and judgment is pooled and balanced through discussion (Eikeland, 2006; Gustavsen, 1986; Pålshaugen, 2002). Thus, objectivity is based on good organization and institutions of inquiry involving broader collaboration of inquirers. It is a kind of political skill in the Aristotelian sense of public discussion (see also Habermas [1989] and the tradition of discursive/deliberative democracy) with an action orientation towards common advantage, where truth seeking is one important good. Objectivity requires also that people are open to adopting a scientific attitude to life. An inclusive and participatory orientation on who is allowed to partake, share in, and contribute to the scientific community might temper the post-truth cultures evident today and reinforce the unity of science that Dewey saw as a cooperative alliance among engaged professionals and citizens (Dewey, 1938). For example, the focus on Responsible Research and Innovation and Inclusive Innovation is today furthering such an orientation in order to make science more responsive to societal challenges (Owen et al., 2012; Schillo & Robinson, 2017). Here, democratic dialogue as a leading element and infrastructure in inquiry to pool and develop practical wisdom is a suitable norm (Gustavsen, 1992).

Reliability

Let us turn to reliability as a quality concept. Whereas objectivity is focused on dealing with different contextual factors of inquiry, reliability ensures that the process of inquiry is on a safe road to valid outcomes – “truths”. Reliability refers to research processes, practices, and methods that have a degree of stability in outcomes according to different reviewers and assessment practices, and in varied research contexts. It is an achievement of good practices of inquiry in that it enhances the degree of trustworthiness of knowledge claims. It is more than replication logic. A more general and adequate understanding of reliability is robustness and consistency in the process of inquiry so that inquiry operations can lead to expected, intended, or desired outcomes with limited risks of unexpected negative consequences.

The ideal of research is not only observing and mirroring from a distance. This metaphor hides the researchers own active and necessary conceptual and practical construction of research situations and objects in order to create knowledge. It means constructing research situations and performing practices where ideas operationalized can be connected, through these practices, to consequences assessed based on expectations and purposes in inquiry. For example, to arrange research situations to be unobtrusive of subjects of research often requires considerable constructive capacities not just passive, distant mirroring. Creating relationships with subjects such that they feel at ease with the presence of researchers or research instruments requires communicative skills, ethical behaviour to build trust, and authorization by parties involved. Arranging, conducting, and analyzing interviews are basic research practices that require entrepreneurial, reliable, and skillful action from problematization, sampling, access, and creation of scientific social relation to validation and publication of findings on the clarification and resolution of problem academically and (for some, but not all, research approaches) practically.

Research objects are, in important ways, constructed and reconstructed in the transaction between the inquirer and the objects of inquiry as Dewey (1929) points out. Careful formation of notions was central in Bacon´s invention of the scientific art of induction. In science as design, not only notions but also many other artifacts as objects of knowledge are constructed where reliability is not only based on stable causal structures and conditions but on how actors cause things to happen and bring them about through creativity and skillful “directed operation” (Dewey, 1929). That is, reliability is an accomplishment through control of consequences and stabilizing, skillful practices so that a high-quality service, an accurate weather foresight, an open, democratic meeting, or an organization free from bullying and racism can be continuously constructed. Susman and Evered (1978) see here a scientific role for participatory and action research in developing “practics” – know-how and action principles or guides for dealing with different situation – for example, in solving problems, enable organizational learning, or mediating and progressively integrating between parties (Follett, 1930).

Reliability is also relevant in dynamic research processes that are difficult to replicate but that are understood as reviewability in tracking the process from initiation to outcomes. Like an auditor, a reviewer can authenticate the findings by following the process and decision trail of the inquiry (Lincoln & Guba, 1985), for example by checking if the findings are consistent with reported data. (Lincoln and Guba [1985] call this dependability.) This calls for good process documentation, for example, through journalling (Coghlan & Brannick, 2014). This does not discard replication logic but adapts and weakens it to be applicable to more context- and subject-dependent knowledge areas. It also turns more to process rather than results (replication of outcome) in understanding reliability, still with a focus on what external reviewers can do to assess and enhance reliability.

Furthermore, qualitative as well as participatory and action researchers argue for a degree of reliability in various research models, methods, procedures, and tools designed to overcome, or at least temper, Baconian fallacies in human understanding. This view is embedded, for example, in the wealth of recommendations for doing good interview research, and in the common action research cycle model (i.e., plan, act, observe, and reflect, then repeat). The point of research methodologies is to make fullest use of our capacities for understanding, learning, and inquiry by offering more generic methodological knowledge and providing support for reasonable decisions for appropriate and skillful use in different contexts. Continuous improvement aids inquiry to become more competent in achieving worthwhile purposes through sifting out comparatively better practices and making better situational choices. This is in line with Bacon´s methodological proposal he called induction, but with a stronger emphasis on construction, invention, and creation in skillful organization of inquiry.

Reliability and learning

Reliability is not only inherent in methods, but also in the organization of learning (Lindhult, 2008). If knowledge is to be created, somebody, often a number of interacting individuals in a network or community, needs to learn and express this learning in communicative form. Reliability as a particularly distinctive feature of participatory and action research lies in the organization of inquiry as a mutual and interactive learning process (Svensson et al., 2007). Learning and knowledge production support the need for continuously occurring emergent issues and problems to be dealt with and resolved in the inquiry process, and in relation to purposes and their achievement (process validity according to Herr and Anderson [2015]). Purposes are approached and realized, implying consistency in aims (a dimension of what Herr and Anderson [2015] call catalytic validity).

Reliability is also supported through learning in relation to purposes, assumptions, and changing research situations so that inquiry is adaptive and self-correcting – a form of double-loop learning (Argyris & Schön, 1978). It can also be supported by the participatory process in the sense of emergent, evolutionary inquiry as a dynamic process of interaction that is navigated and stabilized based on coordinated understanding and practical agreements. The most robust guide for scientific advancement is practicing open democratic dialogue as operational directives (Gustavsen, 1992). It means openness for contribution of all on equal terms, a democratic aspect of quality of inquiry (Herr & Anderson, 2015; Lindhult, 2015). It gives freedom to experiment and try out different lines of inquiry, where dialogue is the medium for critical assessment and sifting out the most trustworthy claims to knowing among the proposed candidates in dialogical contestation and argumentative review (Habermas, 1984). It is also a medium for tempering the idols of our forceful, but fallible, human understanding. Anyone can get it wrong or be unable to see the full picture of an issue or a proposed solution. In line with the social epistemology mobilized in participatory and action research, experience, knowledge, reasoning capacity, perspectives, and creativity are distributed, where open democratic dialogue provides opportunities for pooling resourcefulness and capacities of many through a kind of practical, interactive, communicative, and expansive rationality (Habermas, 1984; Hatchuel, 2002; Schön, 1983; Simon, 1996). Inquiry as open, emergent, creative, and dialectical processes among inquirers can no doubt be a challenge for reliability and objectivity in the short run, but dialogical and participatory procedures has a capacity for learning and self-correction in a longer time perspective.

Table 1 summarizes the shift and reconceptualization of objectivity and reliability proposed in supporting quality in participatory and action research.

Table 1. Reliability and objectivity reconceptualized and expanded

	“Normal” Research	Reconstruction in Qualitative Research	Action Research / Participatory Research
How researchers can create reliability	Instrumental reliability Accuracy of methods/tools Rigour in process	Documentation of process and choices (e.g., project log) Data is consistent with results (dependability) Critical reflection Standardization and training for minimizing subjectivity	Adaptive, goal-seeking reliability as dynamically regulated process Effective organizing interactive and participatory learning processes
How stakeholders /external actors can contribute to reliability	Replication of the study	Process review Auditability (e.g., audit trailing)	Contribute to learning and correction in inquiry as (extended/actual) participant
Objectivity	Neutrality Non-biased Detached position	Reflexivity Description of subjects / pre-understanding Confirmability of findings	Critical subjectivity Intersubjectivity Practical wisdom Impartial norms of inquiry Open democratic dialogue

Reliability and validity

Reliability and validity are related in the sense that they both support trustworthiness. Reliability as consistency and stability in a process of inquiry is normally an indicator of validity and can be seen as a validity dimension in itself. On the other hand, striving for reliability can diverge from or even corrupt validity. It is not only the common Baconian fallacy of focusing on what can be reliably measured despite its questionable validity. Consistency, like in the temperature measurement of a thermometer, might hide systematic and systemic deviances, such as technical errors in the instrument, limited skills in using it, or ignorance of situational factors such as windiness, which leads to lower validity in results. In a social context, power structures repressing some views and voices, negative feedback dynamics in a system reproducing unwanted conditions, or isomorphisms in an organizational field driving conformity might lead to spurious trustworthiness. A dominant (positivist) belief in the existence of and the scientific search for universal, Newtonian laws of human behaviour given in reality supports an ideological interest in status quo (e.g., the quite universal lower status of women compared to men can be trustworthily “proven”), instead of an interest in transformation, emancipation, and betterment of human conditions through scientific and collaborative, constructive efforts (e.g., a more equal society). Critical approaches to social science urge us to recognize a fundamental choice of scientific engagement between supporting status quo (with much better chances to attain reliability and objectivity according to traditional standards) or emancipation from structures and relation of domination restricting individual and social improvement and transformation (requiring reconceptualized understanding of scientific excellence).

Furthermore, less reliability in method and process might open up possibilities for greater validity in results in the sense of varied and rich understanding of an issue and possible ways to deal with it (Merriam & Tisdell, 2016). For example, participatory and action research frequently contribute to quality by opening up opportunities for inquiry into new, unheard, sometimes silenced, voices, experiences, and viewpoints adding to (or disrupting) the dominant learning in a field or context. It might create a more unstable, even conflictual and chaotic, process, but it might also lead to important research qualities and enhanced validity. In addition, courage to break with the urge for rigour might encourage creativity by thinking and acting differently, leading to more interesting and valuable research results in relation to knowledge interests and purposes of inquiry (Alvesson & Sandberg, 2013).

Quality through enabling and organizing learning, dialogue, and democracy

All research processes start with some “subjective” factors as it is necessarily humans involved and driving knowledge-creation projects. Humans are both knowledgeable and ignorant, and they necessarily use their pre-understanding for interpreting and constructing a knowledge object. It provides pre-judgment as a resource and at the same time prejudice, as Gadamer (1975) is integrating in the German word Vorurteil. But, in line with Bacon, it does not mean to wholly root out subjective factors such as passion, imagination, skills, creativity, and feelings (Brown, 1996), but rather to use them properly. Scientists need to be passionate, entrepreneurial, and disciplined in establishing good norms (and enabling effective and efficient practices) of inquiry that can purposefully and trustworthily advance human learning. In this sense, all science scholarships must be engaged and use persuasive powers and expertise to initiate, enable, manage, and navigate research processes and organize inquiry to objectify and ensure the reliability of the processes and validate their outcomes.

The reconceptualization of the quality concepts opens up different ways for all participants in participatory and action research to contribute to quality and have a shared responsibility for process and results, for example, by contributing to good interaction and dialogue, reviewing emergent findings, or providing adaptivity to aims. The goodness of inquiry processes seems to be particularly emphasized in participatory and action research with its focus on participation, good dialogue, learning, and experimentation. It is the character of the inquiry process that gives this type of approach its distinctive qualities. It is assumed that these dimensions both support more participatory, inclusive, and democratic science and also increase validity and quality in its operation and outcomes. For example, Herr and Anderson (2015) emphasize “process validity” as an important quality criterion, and most of their other suggested criteria, such as dialogic, democratic, and catalytic validity have important process, and thus reliability, dimensions.

Florin and Lindhult (2015) identify nine norms of excellence as shared responsibility in collaborative inquiry: focused on freedom of inquiry, dialogic and democratic processes, shared value, participatory governance, extended epistemology and capacity, self-reflection, building common ground, quality and viability, and efficient methodology. These norms of excellence are constitutive features of inquiry, guidance for and indicators of the successful performance of it, as well as a basis for fair/ethical treatment of involved parties.

There are rich and varied suggestions for reliable research practices and methods that can enable these norms of excellence in the expanding research literature on participatory and action research. It can be seen as an extension and refinement from mainstream methodology literature, which often is deficient or displaying a lacuna in supporting participatory and active research roles. For example, interviewing, often seen as scientific when one party is extracting data in a distant and passive mode for research from other parties, can be shifted in power balance to more mutual and active roles for both parties where interviewing is expanded to reflective conversation, dialogic workshops, and collaborative experimentation with new frames and practices. It becomes a collaborative learning process where all share and all contribute, approaching the vision and ideal of creative democracy (Dewey, 1939b). There is ample room for integrating and further developing the existing repertoire, for example, of meeting and workshop designs for scientific inquiry that can integrate academic, problem solving, and practice improvement in different phases of the processes from problematization to collaborative problem solving, innovation, and transformation. This serves as a collaborative call in response to Gustavsen´s (2017) view that “there is a need for a thorough working over of research practices” to bring participatory and action research further. It resonates with Susman and Evered’s (1978) proposal of participatory and action research as an enabling science for developing “practics” that lead to action guides for releasing human potential, creating human artifacts, and transforming social systems that we are part of in line with envisioned and agreed purposes.

Conclusion

This article, and its companion (Lindhult, 2019), has showed how scientific excellence in participatory and action research can be fruitfully understood through the basic quality concepts used in mainstream science, when these concepts are clarified and reconceptualized. The aim is integrational: to advance the inclusion of participatory and action research not only as a peripheral school in the family of sciences, but also, as Gustavsen argues, as a main school through “an ability to deal with the traditional tasks of research in a way which is superior to other schools of thought” Gustavsen (1992). I have made an effort to show the way that participatory and action research has comparatively advantageous ways to realize the traditional quality standards of validity, objectivity, and reliability in aiming for scientific excellence. I hope this can build understanding and confidence in the community of participatory and action researchers and spur further discussions on quality in the research community at large.

I have focused on conceptual reconstruction. The advantage of using established concepts for describing and assessing quality is that the discussion can integrate and be part of long existing discourses on research quality. The disadvantage is that the diversification in perspective and approaches in science developed since the 1970s has also questioned the continued use of these traditional concepts, where some would argue that it is better leaving these traditional quality concepts behind and talk about quality in other more appropriate ways. It is easy to be co-opted by traditional views on science when using traditional concepts through which scientists describe what they do. Better then, it is argued, to change to another quality vocabulary. My argument is that the reconstruction above is made in a way that it can accommodate different views on science, and it particularly incorporates the important action and practice dimensions of participatory and action research. All types of scientific approaches need to confront the fundamental issues that these quality concepts are dealing with: trustworthiness through a satisfactory degree of validity, objectivity, and reliability.

On the other hand, institutionalized use of traditional concepts means the discourse on quality is not on the same terms but often is a kind of guerilla war on dominant conceptual understanding. In trying to understand the traditional concept in new more suitable ways for participatory and action research, the risk is that it is confusing and difficult to understand because of the existing ingrained understanding. In the experience of this author, traditional textbook understanding can easily make people impregnable from other understandings of quality. For example, the role of participatory and action research in basic textbooks like Bryman (2016) is rudimentary, or near to awkward. Probably because of such textbook authors, participatory and action research can be seen as peripheral, deficient, or awkward from the point of view of their conception of excellent research. On the other hand, another widely used introductory textbook in the qualitative research field, by Merriam and Tisdell (2016), has included in its new fourth edition a short introduction to action research, which it describes as approaches “that have become increasingly popular in the last few years”.

In making conceptual moves in traditional discourses on scientific excellence, I have tried to clarify in what ways participatory and action research, in comparison with other research approaches, has certain quality opportunities and benefits, and also limitations and shortcomings. It can expand the repertoire of strategies for promoting validity and reliability developed in the qualitative research field (e.g., see the eight strategies identified by Merriam and Tisdell [2016]) when more active and participatory stances are recognized as enabling scientific excellence. At the same time, one should not conceal the fact that, many times, it is not so easy to realize them in a concrete research practice. There is also a need to pay attention to inherent limitations and deficiencies within participatory and action research, for example, that participation is time consuming, sometimes frustrating, and can create political tensions, and that, in many contexts, it is not the most suitable type of research approach. Collaborative and action research takes into account a broader spectrum of knowledge forms and value dimensions, but it increases the complexity of the research role and research work. In addition, attempts to satisfy the knowledge needs of several parties can often lead to more work and more dilemmas.

In order to fully integrate participatory and action research in the scientific community, there is a need to recognize the valuable role of the researcher as learner, as collaborator, and as participant in knowledge creation and improvement. To is a need to recognize the role of collaborative, active, and participative dimensions in science and learning in general in the scientific inquiry process. There is also an important need to go on reconceptualizing science as well as doing science in different ways, showing that other understandings and practices of science and ways of being scientific are worthwhile and fruitful (Eikeland, 2012). Collaborative and action research in the process of mainstreaming may opt for integration and hybridization with other research approaches, designs, and methods in many different ways in the motley tapestry of science. Through the mainstreaming of participatory and action research, it will be less of a black sheep in the family of social science methodology, and maybe also somewhat less a “saving angel” bringing relevance and democracy to an ivory-towerish and elitist academic community. But participatory and action research can start to feel more at home and welcomed in the academy as one among a plurality of relatives, be it close sisters and brothers or more distant cousins. The choice of participatory and action research will be more based on situated judgement in each research program and project on the comparative advantage of participatory and action research as research design, often in the grey zone of relative degrees of participatory, action, and practice orientation.

 

---

References

Alvesson, M., & Sandberg, J. 2013. Constructing Research Questions: Doing Interesting Research. Los Angeles: SAGE.
https://dx.doi.org/10.4135/9781446270035

Argyris, C., & Schön, D. 1978. Organizational Learning: A Theory of Action Perspective. Reading, MA: Addison Wesley.

Aristotle. 1980. The Nicomachean Ethics. Oxford: Oxford University Press.

Bacon, F. 1960. The New Organon. Indianapolis, IN: Bobbs-Merrill.

Brown, J. R. 1996. The I in Science. Oslo: Scandinavian University Press.

Bryman, A. 2016. Social Research Methods (5th ed.). Oxford: Oxford University Press.

Coghlan, D. 2016. Retrieving a Philosophy of Practical Knowing for Action Research. International Journal of Action Research, 12(1): 84–107.

Coghlan, D., & Brannick, T. 2014. Doing Action Research in Your Own Organization. London: SAGE.

Dewey, J. 1929. The Quest for Certainty: A Study of the Relation of Knowledge and Action. New York: Minton, Balch & Company.

Dewey, J. 1938. Unity of Science as a Social Problem. In J. A. Boydston (Ed.) (2008), John Dewey, The Later Works, Volume 13. Experience and Education, Freedom and Culture, Theory of Valuation, and Essays: 271–280. Carbondale, IL: Southern Illinois University Press.

Dewey, J. 1939a. Logic: The Theory of Inquiry. London: George Allen & Unwin Ltd.

Dewey, J. 1939b. Creative Democracy: The Task Before Us. In J. A. Boydston (Ed.) (2008), John Dewey, The Later Works, Volume 14: 224–230. Carbondale, IL: Southern Illinois University Press.

Eikeland, O. 2006. Validity of Action Research and Validity in Action Research. In K. A. Nielsen & L. Svensson (Eds), Action and Interactive Research: Beyond Practice and Theory: 193–240. Maastricht: Shaker Publishing.

Eikeland, O. 2008. The Ways of Aristotle: Aristotelian Phronesis, Aristotelian Philosophy of Dialogue, and Action Research. Berlin: Peter Lang.

Eikeland, O. 2012. Action Research: Applied Research, Intervention Research, Collaborative Research, Practitioner Research, or Praxis Research? International Journal of Action Research, 8(1): 9–44.

Florin, U., & Lindhult, E. 2015. Norms and Ethics: Prerequisites for Excellence in Co-production. Paper presented at HSS-2015, SIRA/SPARC Track on Participatory Research, May 28–29, 2015, Kalmar, Sweden.

Follett, M. P. 1930. Creative Experience. London: Longmans, Green and Co.

Foucault, M. 1984. What is Enlightenment? In P. Rabinow (Ed.), The Foucault Reader: 32–50. New York: Pantheon Books.

Gadamer, H.-G. 1975. Truth and Method. New York: Crossroad.

Gustavsen, B. 1986. Social Research and Participative Dialogue. In F. Heller (Ed.), The Use and Abuse of Social Science: 143–156. London: SAGE.

Gustavsen, B. 1992. Dialogue and Development: Theory of Communication, Action Research and the Restructuring of Working Life. Assen/Maastricht: Van Gorcum.

Gustavsen, B. 2003. New Forms of Knowledge Production and the Role of Action Research. Action Research, 1(2): 153–164.
https://doi.org/10.1177/14767503030012003

Gustavsen, B. 2017. Action Research and the Promotion of Democracy. International Journal of Action Research, 13(2): 101–111.

Habermas, J. 1984. The Theory of Communicative Action, Vol I. London: Polity Press.

Habermas, J. 1989. The Structural Transformation of the Public Sphere: An Inquiry into a Category of Bourgeois Society. Cambridge, MA: The MIT Press.

Hatchuel, A. 2002. Towards Design Theory and Expandable Rationality: The Unfinished Program of Herbert Simon. Journal of Management and Governance, 5(3/4): 260–273.
https://doi.org/10.1023/A:1014044305704

Herr, K., & Anderson, G. L. 2015. The Action Research Dissertation: A Guide for Students and Faculty. Thousand Oaks, CA: SAGE.

Heron, J. 1996. Co-operative Inquiry: Research into the Human Condition. London: SAGE.

Kant, I. 1991. Political Writings. Cambridge: Cambridge University Press.

Kant, I. 1998. Critique of Pure Reason. Cambridge: Cambridge University Press.

Kolb, D. A. 2015. Experiential Learning (2nd ed). Upper Saddle River, NJ: Pearson Education.

Lewin, K. 1946. Action Research and Minority Problems. Journal of Social Issues, 2(4): 34–46.
https://doi.org/10.1111/j.1540-4560.1946.tb02295.x

Lincoln, Y., & Guba, E. 1985. Naturalistic Inquiry. Newbury Park, CA: SAGE.

Lindhult, E. 2004. Achieving Quality in Action Research: A Matter of Judgment. Paper presented at the 20th EGOS Colloquium, July 1–3, 2004, Ljubljana, Slovenia.

Lindhult, E. 2008. Att bedöma och uppnå kvalitet i interaktiv forskning. In B. Johannisson, E. Gunnarsson, & T. Stjernberg (Eds.), Gemensamt kunskapande – den interaktiva forskningens praktik. Växjö, Sweden: Växjö University Press.

Lindhult, E. 2015. Towards Democratic Scientific Inquiry? Participatory Democracy, Philosophy of Science and the Future of Action Research. In E. Gunnarsson, H. P. Hansen, & B. Steen Nielsen (Eds.), Action Research for Democracy – Intervening in the Current Crisis: New Ideas and Perspectives from Scandinavia. London: Routledge.

Lindhult, E. 2019. Scientific Excellence in Participatory and Action Research: Part I. Rethinking Research Quality. Technology Innovation Management Review, 5(9): 6–21.
http://doi.org/10.22215/timreview/1237

Merriam, S. B., & Tisdell, E. J. 2016. Qualitative Research: A Guide to Design and Implementation. San Francisco, CA: Jossey-Bass.

Owen, R., Macnaghten, P., & Stilgoe, J. 2012. Responsible Research and Innovation: From Science in Society to Science for Society, With Society. Science and Public Policy, 39(6): 751–760.
https://doi.org/10.1093/scipol/scs093

Pålshaugen, Ö. 2002. Discourse Democracy at Work: On Public Spheres in Private Enterprises. Concepts and Transformation, 7(2): 141–192.
https://doi.org/10.1075/cat.7.2.03pal

Reason, P. 1994. Three Approaches to Participatory Inquiry. In N. Denzin, & Y. Lincoln (Eds.), Handbook of Qualitative Research: 324–339. Thousand Oaks, CA: SAGE.

Reason, P. 2006. Choice and Quality in Action Research Practice. Journal of Management Inquiry, 15(2): 187–203.
https://doi.org/10.1177/1056492606288074

Rorty, R. 1979. Philosophy and the Mirror of Nature. Princeton, NJ: Princeton University Press.

Schillo, R. S., & Robinson, R. M. 2017. Inclusive Innovation in Developed Countries: The Who, What, Why, and How. Technology Innovation Management Review, 7(7): 34–46.
https://doi.org/10.22215/timreview/1089

Schön, D. 1983. The Reflective Practitioner: How Professionals Think in Action. Aldershot, UK: Avebury.

Simon, H. A. 1996. The Sciences of the Artificial (3rd ed). Cambridge, MA: MIT Press.

Solomon, J. R. 1998. Objectivity in the Making: Francis Bacon and the Politics of Inquiry. Baltimore, MD: Johns Hopkins University Press.

Susman, G. I., & Evered, R. D. 1978. An Assessment of the Scientific Merits of Action Research. Administrative Science Quarterly, 23(4): 582–603.
https://doi.org/10.2307/2392581

Svensson, L., Ellström, P.-E., & Brulin, G. 2007. Introduction – On Interactive Research. International Journal of Action Research, 3(3): 233–249.

Westbrook, R. 1995. Action Research: A New Paradigm for Research in Production and Operations Management. International Journal of Operations & Production Management, 15(12): 6–20.
https://doi.org/10.1108/01443579510104466

Zagorin, P. 2001. Francis Bacon’s Concept of Objectivity and the Idols of the Mind. The British Journal for the History of Science, 34(4): 379–393.
https://doi.org/10.1017/S0007087401004411