Showing posts with label Method. Show all posts
Showing posts with label Method. Show all posts

Tuesday, May 22, 2018

On the Difference Between Method and Methodology




"What is the difference between methods and methodology in research?" This question was very recently asked in LinkedIn's "Research, Methodology, and Statistics in the Social Sciences" Group, which currently has 137,052 members. A number of answers were given, including one from me. I've chosen to share mine below, preferring to start with Methodology: 

Methodology is the study or theory or interrogation of a range of Methods of scientific/scholarly research in terms of their relative/respective appropriateness to solving specified types or sets of research problems. Methods, on the other hand, are the actual techniques (not 'tools', which are a different kettle of fish) of data collection and analysis, respectively, that we use in particular studies. 


In principle, the Method(s) we use in a given study is/are determined to be fit for the (stated) purpose -- the problem statement -- in terms of the Methodology or Methodologies they are couched in and, moreover, broadly accepted by the Gatekeepers (our peers and revered precursors) in our respective disciplines. 

It is common knowledge that, nowadays, Methodologies and Methods cut across various disciplines and sub-disciplines.

Thursday, June 09, 2016

Opening Remarks at a Commemorative Symposium on Prof. H. Odera Oruka


["The written word does not forget" ~ MY]

This symposium, dedicated as it is to the Father of Philosophic Sagacity, makes pertinent the question: What is the overriding problem of method in Philosophy today? In his research on Sage Philosophy, Odera Oruka appears to abandon his discipline's patent method (or is it patent tool?): the armchair -- or, if you prefer, introspection/contemplation. He immerses himself, with little time for apologies, in a method which in an earlier time might have been called ethnographic, but which is more trendily labeled, nowadays, qualitative research.

But is this an abandonment or in reality an enriching departure from the very norm which creates false boundaries of intellectual space? And if an enrichment, are field interviews, such as Odera Oruka conducted, to be seen as a form of dialogue -- or as observation? Or are they, from another view, a necessary component of what now passes for scholastic interrogation? And if interrogation, is that, as practice, a throwback to subconsciously remembered/emulated encounters with agents of the police state? [As an aside: the other day I encountered one thinker's sardonic (or was that sadistic) commentary on numbers, which went something like this: "if you torture numbers long enough, they will tell you anything you want".]

CLICK HERE TO READ THE FULL TEXT OF THE OPENING REMARKS





FURTHER READING
Azenabor, Godwin (2009) "Odera Oruka's Philosophic Sagacity: Problems and Challenges of Conversation Method in African Philosophy" Thought and Practice: A Journal of the Philosophic Association of Kenya. Premier Issue, New Series, Vol. 1 No. 1, June 2009, pp. 69-86

Hapanyengwi-Chemhuru, Oswell (2003) "Odera Oruka's Four Trends in African Philosophy and their Implications for Education in Africa" Thought and Practice: A Journal of the Philosophical Association of Kenya. New Series, Vol. 5 No. 2, December 2013, pp. 39-55.






NOTE: I made these opening remarks on December 10, 1999 during my tenure as Dean, Faculty of Arts, UoN

Sunday, November 08, 2015

CSO 302: Nine Key Standards of Scientific Research


INTRODUCTION TO NINE STANDARDS OF SCIENTIFIC RESEARCH

Certain broad standards, principles or criteria have been established for determining what scientific research is all about, and what it is not. That is, the standards help us to determine in some detail whether, or the extent to which, a proposed or completed study meets the expectations of the relevant scientific community -- in the present case, Sociology. They all apply, to a greater or lesser degree, to quantitative or qualitative research, or a combination of the two. The identity or image of Sociology as a discipline, or of any other discipline for that matter, does not somehow disappear just because one wishes to do qualitative, or indeed quantitative, research.

We can clearly identify nine such standards or criteria. These are: universality, replicability, control, measurability (Leedy, 1980: 46); validity, reliability, objectivity, ethics (Comte, 1853; Durkheim, 1982; Weber, 1964: 126-130; Giddens, 1993: 114-116) and representativeness (Sarantakos, 1994: 18-26; Weber, 1964: 110). The cited authors are only examples of scholars who have engaged in conversations about these standards. 

I enthusiastically take the opportunity to share with the reader my own understanding and critique of each of the standards, and of related conversations by other scholars, on separate blog pages. Links to the respective blog pages are embedded in the list below, and readers are invited to click on each at their convenience:

3. Control
4. Measurability
5. Validity
6. Reliability (Pending)
7. Objectivity (Pending)
8. Ethics 
9, Representativeness.


Let me, as I conclude this brief piece, invite debate. The more robust the debate, the better all around.

P.S: An earlier version of this introduction was published in early November 2015, but was quickly withdrawn ('reverted to draft') to allow for more of the nine key standards to first appear. Six standards have now been published. The seventh, "Ethics", has taken more time than previously anticipated -- as the filling of one gap or other in the narrative revealed the exciting need to do yet another, and so on -- but is nearly done. The remaining two will follow soon after it. The release of this updated introduction today (January 25th, 2016) thus appeared as appropriate as it was ever going to be, and so here it is.


References
American Sociological Association, ASA (1999) Code of Ethics and Policies and Procedures of the ASA Committee on Professional Ethics. Washington, DC: ASA

Becker, Lawrence C. and Charlotte B. Becker, Eds (2003) A History of Western Ethics. Second Edition. New York and London: Routledge.

British Sociological Association, BSA (2002) Statement of Ethical Practice for the British Sociological Association. Durham: BSA (consider the PDF version provided)

Comte, Auguste (1853) The Positive Philosophy of Auguste Comte. Volume I. Trans. Harriet Martineau. New York: D. Appleton and Co.

Cooper, John M. (2003) "Classical Greek Ethics", pp. 9-18, in Lawrence C. Becker and Charlotte B. Becker (2003) A History of Western Ethics. Second Edition. New York and London: Routledge.

Durkheim, Emile (1957) Professional Ethics and Civic Morals. Trans. Cornelia Brookfield. Longon and New York: Routledge.

Durkheim, Emile (1982) The Rules of Sociological Method and Selected Texts on Sociology and Its Method. Edited with an Introduction by Steven Lukes. Trans: W.D. Halls. New York: The Free Press

Giddens, Anthony (1993) New Rules of Sociological Method: A Positive Critique of Interpretative Sociologies. Second Edition. Stanford: Stanford University Press

International Sociological Association (2001) Code of Ethics. Madrid: ISA

Jones, Robert Alun (1986) Emile Durkheim: An Introduction to Four Major Works. Beverly Hills, CA: Sage Publications, Inc. (CLICK HERE: Excerpts from Jones' book focusing on Durkheim's The Rules of Sociological Method)

Kahn, Charles H. (2003) "Presocratic Greek Ethics", pp. 1-8), in Lawrence C. Becker and Charlotte B. Becker (2003) A History of Western Ethics. Second Edition. New York: Routledge.

Leedy, Paul D (1980) Practical Research: Planning and Design. Second Edition. New York:  Macmillan Publishing Co.


Sarantakos, Sortirios (1994) Social Research. London: The Macmillan Press

Weber, Max (1964) The Theory of Social and Economic Organization. Edited with an Introduction by Talcott Parsons. New York: The Free Press

Measurability: A Key Standard of Scientific Research

Nine key standards, or principles, of scientific research can be gleaned from texts on research method. As I see them, these are:


3. Control
4. Measurability
5. Validity
6. Reliability (Pending)
7. Objectivity (Pending)
8. Ethics 
9. Representativeness.

I say something about each of the nine standards in separate posts, as shown in the links above. What I want to briefly talk about here is the measurability standard. 

Measurability 

Measurability means susceptibility to measurement. It means, in other words, that what is to be measured must embody quantities and/or qualities -- or have features, dimensions or characteristics -- that can be subjected to appropriate or 'true' measurement. Concerning the defining characteristics of a measurement, Bell (1999) notes that:
"A measurement tells us about a property of something. It might tell us how heavy an object is, or how hot, or how long it is. A measurement gives a number to that property. Measurements are always made using an instrument of some kind. Rulers, stopwatches, weighing scales, and thermometers are all measuring instruments. The result of a measurement is normally in two parts: a number and a unit of measurement, e.g. ‘How long is it?... 2 metres." 
The measurability rule is anchored on the above conceptions, and so requires that the variables around which the researcher intends to collect data should be measurable, or susceptible to acceptable ‘measurement’ (Leedy, 1980: 46)[1]. This is easier done in the natural sciences than in the social sciences; in quantitative studies than in qualitative. Still, one must, in the social sciences too, endeavor to quantify, measure and evaluate. Indeed, the guiding principle of the measurability rule -- its corollary, in other words -- is this: "What can be measured must be measured." Thus, not measuring what can be measured is not an option allowed anyone.

In quantitative research, the measurability standard features prominently in all methodological procedures that revolve around, or build up to, hypothesis testing. The testability of a hypothesis is indeed a function of the measurability of the variables that constitute it. Such measurability is, in turn, dependent on the indicators chosen to represent them; but one is not entirely free to choose just any indicator(s). They must be such as most peers or reviewers or supervisors or sponsors can be persuaded to accept as valid and reliable. Still, one has some leeway in choosing the indicators one wishes to use, so long as the reviewers, supervisors or funding entities are prepared to see or accept one's findings “in the light" of the chosen indicators.

Viswanathan (2010: 285-210) has interrogated in depth the challenges encountered in "Measure development procedures," particularly in connection with quantitatively-oriented social science research. Among these challenges, which he believes are not sufficiently appreciated, is the underpinning assumption, which he faults, that "a construct can be [readily and invariably] isolated and examined." Here is his line of thinking:
"By measuring or manipulating individual constructs, relationships between constructs are studied and substantive hypotheses about these relationships are tested. The very notion that numbers can be assigned to attributes of people, objects, or events is presumed on being able to study attributes or constructs separate from other constructs. After all, measurement relates to rules for assigning numbers to attributes of people, objects and events. [But] this is not the case for many phenomena. A complex network of constructs may influence a phenomenon and may not be separable into individual constructs for purposes of measurement." 
Measurability procedures implemented in qualitative -- that is, 'non-numerical' -- research attempt, not always successfully, to overcome or circumvent such intractables as Viswanathan has pointed out. These procedures include, for example, “comparative judgement (arranging factors in a hierarchy of importance)” or  “scaling (correct versus incorrect responses to a given set of questions)” (Leedy, 1980: 46). Other ‘non-numerical’ strategies that have been used, according to Worsley (1992: 113)[2] include: “careful classification[3], organizing and combining of field-notes [which] requires patient checking and cross-checking and the application of systematic techniques.” But he acknowledges that “numerical methods are sometimes applied to field-data too” (Worsley, 1992: 113). Indicators are often used as proxies/surrogates for more fuzzy or abstract concepts, such as status (Worsley, 1992: 96). Such indicators are inevitably numerical. 

One may also, in connection with the nominal level of measurement (the simplest level), and even with the ordinal level of measurement, calculate frequencies/percentages, and cross-classify in qualitative research. Scaling, moreover, is not just about correct v. incorrect. Incorporating comparative judgement, one may also use scales of: high v. low, do v. don’t, very high to very low, very good to very bad, strongly agree to strongly disagree.

To be fair to everyone, no one is quite attempting stubbornly to square the circle in this measurability conversation. Still, the challenges of measuring require, and will surely be tempered with, constant and open-minded vigilance -- as well as an abundance of capacity to deal with critiques and contrary views, including (so be it) one's own self-criticism. Bell (1999: 1), like others before her, subsumes this conversation under the theme "uncertainty of measurement". Thus:
"Uncertainty of measurement is the doubt that exists about the result of any measurement. You might think that well-made rulers, clocks and thermometers should be trustworthy, and give the right answers. But for every measurement - even the most careful - there is always a margin of doubt. In everyday speech, this might be expressed as ‘give or take’ ... e.g. a stick might be two metres long ‘give or take a centimetre’."



[1] Sarantakos talks of the need for precision in measurement (pp. 18-26)
[2] Peter Worsley, ed. 1992. The New Introducing Sociology. Revised Third Edition. London: Penguin Books.
[3] Another word for classification, given by Worsley (1992: 97) is categorization; and both, he argues, are an often overlooked “form of measurement” – even if they are “the simplest level” (that is, the nominal level) of measurement. There are in total four levels of measurement, the others being, in an ascending order of complexity: Ordinal (or rank-order), interval and ‘ratio’ scale (Worsley, 1992: 97-98).



REFERENCES

Bell, Stephanie (1999) Measurement Good Practice Guide: A Beginner's manual to Uncertainty of Measurement. (No. 11, Issue 2). Taddington: National Physics Laboratory

Leedy, Paul D. (1980) Practical Research: Planning and Design. Second Edition. New York:  Macmillan Publishing Co.,

Sarantakos, Sartirios (1994) Social Research. London: The Macmillan Press

Viswanathan, Mathu (2010) "Understanding the Intangibles of Measurement in the Social Sciences," pp. 285-312, in Geoffrey Walford, Eric Tucker and Mathu Viswanathan, Eds. (2010) The SAGE Handbook of Measurement. Los Angeles: SAGE 

Worsley, Peter, ed (1992) The New Introducing Sociology. Revised Third Edition. London: Penguin Books


Saturday, November 07, 2015

Universality: A Key Standard of Scientific Research

Nine key standards, or principles, of scientific research can be gleaned from texts on research method. As I see them, these are:

1. Universality
3. Control
4. Measurability
5. Validity
6. Reliability (Pending)
7. Objectivity (Pending)
8. Ethics 
9. Representativeness.

I say something about each of the nine standards in separate posts, as shown in the links above. What I want to briefly talk about here is the universality standard. 

Universality: 

The tern universality has roots in the broader conception of a pervasive universe, of which we are all inescapably a part. More specifically, it is anchored in the related notion of universals. Thus:
"Universals are features (e.g., redness or tallness) shared by many individuals, each of which is said to instantiate or exemplify the universal...The metaphysical issue is whether or not these features exist independently of the particular things that have them:  realists hold that they do; nominalists hold that they do not; conceptualists hold that they do so only mentally"(A Dictionary of Philosophical terms, in Garth Kemerling (2011) The Philosophy Pages. available online).
Thus, moreover, "evolutionary universals", as Talcott Parsons argues, are those similitudes, those patterns of resemblance, which scholarship detects -- recognizes -- in the evolution of human society.

Armstrong (1986, 1989) understands that universals do have particulars, the very constituents or components which make universals possible in the first place. Moreover, the particulars of any one universal resemble one another. As resembling constituents of universals multiply within and across universals, what we end up with is identity, which, to repeat, is borne out of resemblance, and which in the end it morphs out of. And so, one can see, universals are made of common particulars.Thus:
"If we consider ordinary, first-order, particulars, then . . . two things, while remaining two, can resemble exactly. At least exact resemblance is possible (assuming that the Identity of Indiscernibles is not a necessary truth). In the limit, resemblance of particulars does not give identity. But now consider the resemblance of universals. As resemblance of properties [monadic universals] gets closer and closer, we arrive in the limit at identity. Two become one. This suggests that as resemblance gets closer, more and more constituents of the resembling properties are identical, until all the constituents are identical and we have identity rather than resemblance." [This quote is from Armstrung (1989) as found in Pautz (1977). The original article is yet to be accessed]  
So universality is ultimately "standardized" via the metamorphosis of its resembling parts into a more holistic identity recognized by the, or at the very least a, scientific community. The universality standard requires that any research project should be designed and planned in such a way that any competent researcher, not just the one(s) who wrote the proposal,  should be able to successfully undertake it. (Leedy, 1980: 46). In this sense, the research project has a life of its own, independent of any particular researcher, within the confines of the relevant discipline or scientific community.

Universality demands a high level of discipline and transparency in the research habits of all who do research. In a sense, too, universality makes all scientific discoveries part of a common, with standard operating procedures and the common ownership of the discoveries claimed and accumulated across geographies, disciplines and time.

READ: Universal (metaphysics)

ALSO READ: Professor JeeLou Lin "D.M. Armstrong, Universals: An Opinionated Introduction"


To partially sum it all up, let's see in this quotation how the International Council for Science (ICSU) defines the universality of Science:
"The universality of science in its broadest sense is about developing a truly global scientific community on the basis of equity and non-discrimination. It is also about ensuring that science is trusted and valued by societies across the world. As such, it incorporates issues related to the conduct of science; capacity building; science education and literacy; access to data and information and the relationship between science and society..." (see ICSU)
Furthermore, a noteworthy point -- on the importance of doubt and criticism (including self-criticism) in all claims to, or efforts to affirm, scientific discovery -- is made by Michel Paty (2001: 8) who, inspired by Descartes's Discourse on Method, argues that:
"...the idea of of universality, as well as ideas of reason and of demonstrative (and even objective) science, with which it has a constitutive link, carries with it the requirement of its own criticism...[Thus]... the only true knowledge is that knowledge that, for every thinking subject, overcomes the obstacles [posed] by doubt."

REFERENCES:
Armstrong, D.M. (1986) "In Defense of Structural Universals", Australasian Journal of Philosophy 64 (1986) pp. 85-88.

Leedy, Paul D. (1980) Practical Research: Planning and Design. Second Edition. New York:  Macmillan Publishing Co.,


Parsons, Talcott (1964) "Evolutionary Universals in Society" American Sociological Review, Vol. 29, Issue 3 (June, 1964), pp. 339-357


Paty, Michel (2001). "Universality of Science : Historical Validation of a Philosophical Idea." in Habib, S. Irfan and Raina, Dhruv. Situating the history of science : Dialogues with Joseph Needham, Oxford India Paperbacks, p. 303-324, 2001.  [see p. 7 of the pdf version given in this link]

Pautz, Adam (1977) "An Argument Against Armstrong's Analysis of the resemblance of Universals"




~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
CSO 302

Sunday, October 18, 2015

Replicability: A Key Standard of Scientific Method Now Found to be Under Existential Threat

Nine key standards, or principles, of scientific research can be gleaned from texts on research method. As I see them, these are:
2. Replicability
3. Control
4. Measurability
5. Validity
6. Reliability (Pending)
7. Objectivity (Pending)
8. Ethics
9. Representativeness.

I will say something about each of the nine standards in later posts. What I want to briefly talk about today is replicability. It turns out that, highly valued as it is, replicability is easier 'said' in conversations about methodology -- defined as the science or theory of method -- than 'done' in actual empirical research. This is what we pick from a study captured in Jessica Firger's account, the link to which I give in the References section below.

First, though, let us recapitulate the accepted meaning and procedures of replicability. Then, second, we will turn to the new study to see how its findings shake the foundations of the scientist's faith (now 'blind faith'?) in the utility and robustness of replicabiliy as a verification process.

First, then, about the centrality of replication to scientific method and to science's "theory of evidence": As a concept of methodology, replicability calls for precision in measurement and accuracy of procedure in all scientific endeavors. That is to say, it calls for the doing of any research project in such a way as to ensure its step-by-step, beginning-to-end, repeatability or 'reproducibility' -- which is what replicability means -- in any subsequent study, by any competent researcher, intended to attest to the veracity of its discoveries or claims. To repeat, it should be repeatable in its exact original form, and should yield the very same results. Thus:
“The research should be repeatable. Any other competent researcher should be able to take your problem and , collecting data under the same circumstances and within the identical parameters as you have observed, achieve results comparable (sic) to those you have been able to secure” (Leedy, 1980: 46; see also Seale, 2004). 
That emphasis on "the very same results", as opposed to 'comparable' results, highlights the ultimate basis for the confirmation or validation of scientific claims. However, the confounding truth, which subverts such a neatly articulated metric, is that several important factors may be suggested as underpinning the likelihood of not achieving such results in the real world which social science, including/particularly sociology, routinely deals with. This is the real world of conscious, querying, skeptical (and often over-researched) human beings unsure of the risks, or real value, of availing themselves and telling everything that researchers wish to be told. And that's even before one factors in variability, for various reasons, in the ways respected scientists implement even widely agreed-upon procedures -- or the procedures that should, in the name of reason and perhaps common sense, be so agreed -- for data capture, processing, analysis and synthesis.

But why all this fuss about replicability or reproducibility to which I believe all seasoned interrogators of methodology have been paying attention for years? Firger, I think, captures the underlying sentiment as well as anyone can or has:
"Most rational humans hold some faith in science. We trust that scientists, through their hard-won expertise, are well equipped to conduct studies that provide proof of why things are the way they are, help solve problems and explain mysteries. Much of that trust is built on an implicit belief that research findings are concrete truths—in other words, that if given the same set of parameters, they would be easy to reproduce. This type of replication is essential to science because it validates key discoveries and helps scientists make progress in their fields of research" (Firger, 2015).
So replicability is a standard worth respecting, implementing and defending. But what if it turns out that in practice we cannot, or have not been doing so? What are we to make of our scientific truths, then? That, at least implicitly, is for me the most fundamental, if implicit, question driving the University of Virginia study, as we will now see.

Second, then: what was the purpose of the new study and what does it tell us? We can discern that purpose and related method from this portion of the study's problem statement, as seen in the abstract:
"Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available"  (Open Science Collaboration, 2015)
What about the results? The key finding of the study was that the results of less than 50% of the studies subjected to replication were corroborated; that is, independently validated. That is a huge shortfall, as elaborated below:
"Replication effects (Mr = .197, SD = .257) were half the magnitude of original effects (Mr = .403, SD = .188), representing a substantial decline. Ninety-seven percent of original studies had significant results (p < .05). Thirty-six percent of replications had significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and, if no bias in original results is assumed, combining original and replication results left 68% with significant effects" (Open Source Collaboration, 2015; see also PSA, September 2015).
Firger (2015) notes problem in this regard: that the threat to replicability may be exacerbated by the existential pressure, perceived by scientists seeking recognition and success (presumably via promotion and tenure, or higher levels of funding, or 'all of the above'), to dump rigor, and all its 'yokes' and promise, in favor of building the capacity to "weave a memorable story out of tenuous science" -- and so to achieve such success.

In the end, perfect replicability, certainly of studies involving conscious human subjects, may not be feasible, even when driven purely by the dictates of scientific rigor, ahead of the capacity for time-travel back to the chronological 'moment' at which the original study of interest was conducted. Incidentally, contrary to Drummond's (2009) claim, this is not to say that 'reproducibility' is a more viable posture for the scientist. The attributes he assigns to reproducibility are in fact, precisely, those of replicability, as widely understood by those who routinely use the latter term (see, for example, the clarification offered by the Replicability Research Group, 2015). What Drummond wrongly sees as the weaknesses of replicability are its ideals and strengths, and what he touts as the strengths of reproducibility are its weaknesses. Reproducibility, defined as he so facilely does, would be a recipe for widespread intellectual fraud.

So what is one to do in the meantime? We will just have to make do with such rigors and caveats as we are able to muster and agree upon. But the rationale for replicating scientific work remains as great and as persuasive as it ever was, and its utility for science and human progress just as enormous. All that we have discovered is its disappointingly low success score, certainly in psychology -- which is a human science. We have also found, alas, that the pressure for recognition and success makes the scientist as much an opportunist as the proverbial self-sacrificing servant of truth. We can work with sharp focus on these personal/human 'frailties' of the scientist -- not necessarily of science -- to raise replicability's score. Replication is (natural) science's ultimate audit tool, which we will abandon only at society's and civilization's own great peril.

Still, we will also have to more consciously and more robustly develop other methods of arriving at the truth -- and the natural scientist, in particular, will have to respect with greater humility and sharper awareness of that potentially debilitating soft underbelly, just discovered, of his/her cherished branch of science. The other methods I have in mind are those which social science and the humanities have been grappling with, self-critically (and against the backdrop of harsh and even dismissive criticism by often sanctimonious natural scientists), for decades, and even more than a century, now. Many of these fall under the qualitative label, as opposed to the quantitative, and include versions of such broad and cross-cutting, timeless and a-disciplinary methods as: observation (naturalistic, participant or device-assisted), comparison, deduction, induction, abduction, focus-group conversations, dialectics, hermeneutics, and, particularly a la Foucault (1973: x-xxiv), genealogy and even archaeology.

REFERENCES:

Drummond, Chris (2009) "Replicability is not Reproducibility: Nor is it Good Science." Proceedings of the Evaluation Methods for Machine Learning Workshop at the 26th ICML, Montreal, Canada

Firger, Jessica (August 28, 2015) "Science's Reproducibility Problem: 100 Psych Studies Were Tested and Only Half Held Up"  Newsweek

Foucault, Michel (1973) The Order of Things: An Archaeology of Human Sciences. New York: Vintage Books.

Leedy, Paul D. (1980) Practical Research: Planning and Design. Second Edition. New York:  Macmillan Publishing Co.,

Open Science Collaboration (2015) "Estimating the Reproducibility of Psychological Science"  Reproducibility Project: Psychology. University of Virginia

PSA - Psychological Science Agenda (September 2015) "Science Paper Shows Low Replicability of Psychology Studies" APA

Replicability Research Group (2015) "Replicability vs Reproducibility." Tel Aviv University, Department of Statistics and Operations Research

Seale, Clive (2004) "Replication/Replicability in Qualitative Research" in The SAGE Encyclopedia of of Social Science Research Methods (Michael S. ewis-Beck, Alan Bryman and Tim Futing Liao, Eds.)

van Rijn, Hedderik and Sabine Scholz (2014) "Replication of Experiment 3 of 'Tracing Attention and the Activation Flow in Spoken Word Planning Using Eye Movements' by A Roelofs (2008,


POSTSCRIPT:

All that I have said above is based on the assumption, not necessarily correct, that:
1. The original study to be subjected to replication provided all the details necessary to do so.
2. Each scientist involved in the replication exercise is/was fully competent to do so, and that the replicating work is itself fully open to replication.

It is also important to observe that the scientist seeking to do replication work may have to come to terms with three potential failure nodes:
1. The failure to replicate a study whose methodological procedures and related tools were not detailed enough to permit faithful or full retracing.
2. The failure to replicate because the researcher did not quite have the requisite competency or capacity -- intellectual, resource, temporal -- to replicate.
3. The failure to replicate because the original work is/was of such a type (partially or totally qualitative, for example) that it is/was not fully, adequately or in any other meaningful way replicable.


END NOTE: Paper updated October 19-20, 2015

Sunday, April 13, 2014

CSO 302 Qualitative Research Methods, Final Examination Papers, 21 December 2004 to 11 January 2014

To skip the text below and go straight to the examination papers, click here 

As is to be expected, the past papers included in the set I am providing in this blog post reflect the subject-matter of qualitative research methods taught under course code CSO 302 -- a third-year course at the University of Nairobi. The course underscores the contribution of qualitative methods to knowledge-generation, and to the deeper understanding of the social forces operating in 21st century society. 

We can trace the origins of qualitative methods back to the deep, millennia-long, past of human existence -- the ontological/ existential/proto-naturalistic past -- which preceded even the first stage in Auguste Comte's Law of Three Stages; that is, the "theological stage." Similarly, of course, we can trace the very roots of the natural (and social) sciences as we have come to know them back to that same past.

There is clearly a determination in the course to distinguish between methods of data collection and methods of data analysis, and to go for detail. And even though (a) similar methods often appear under a variety names; and, (b) similar names are or seem to be used, on occasion, interchangeably for both data collection and data analysis, you are challenged to spot the random disguises (under a), and to both extricate 'analysis' from 'collection' and articulate the paired differences (under b). 

All this puts a high premium on conceptual clarity and empirical example. Designing an efficient taxonomy of methods, which should proceed from such clarity and example(s), remains an ever-present challenge, which you are challenged to overcome any imaginative way you can, without 'brutally' violating Bateson's (1987: 284-287) classification principles embodied in his theory of logical types. 


Read More >>> Past Papers

Reminder: To access the CSO 302 examination papers, click here 

Tuesday, September 17, 2013

On the Foundations and Evolution of Research Method and Knowledge-Creation ~ CSO 302, CSO 501, CSO 589

(Or The Origins of the Methods by which we Acquire the Knowledge of what we Know)

By Mauri Yambo

[Note: What follows is an excerpt of a much longer manuscript] 

There are, throughout history, essentially four foundations – four origins –of the methods by which humans have acquired knowledge of what they know – or believe they know. Let us first mention them, and then briefly interrogate their historical connections. The four foundations are:

1.     Naturalistic/Ontological/Existential
2.     Theological (Religio-Cultural Belief Systems)
3.     Metaphysical/Philosophical/Speculative
4.     Positivist/Scientific.
Anchored on three of these foundations are what one may call the Holy Grails of scientific knowledge generation (the fourth foundation, the theological, is excluded on rationalistic grounds):
From ‘mainstream’ Philosophy:
i.                 Deductive and Inductive Reason/Methods
ii.               Dialectics
iii.             Rationalism/Reason.
From Marx: Materialistic Conception of History (or Dialectical Materialism).
From Positivist (or Social) Philosophy and Natural Science: Positivism-cum-Empiricism -- that is, the Scientific Method.
What is the basis of the four classes of origins listed above? It is, certainly for us in Sociology, the helping hand of Auguste Comte (1798-1857). There must be other routes back to the same origins, of course. Talcott Parsons (1902-1979)[1], for example, has more recently interrogated the evolutionary universals presumably characteristic of all societies, but more on this a little later. And Comte himself owed a great debt of scholastic gratitude to his precursors in the natural sciences – particularly physics.

Widely known as a/the founding father of Sociology, Comte developed a Law of Three Stages. This Law stated that all knowledge, all scientific disciplines, societies as a whole, and even the individual mind, pass through three immutable stages as they grow from infancy to maturity, namely: Theological, Metaphysical and Positivistic. Rather obliquely, the law suggests one more thing: that the three stages also track the dialectical progression of method as we know it generically to-day. But the law ignores the first of the four stages that I have listed above. This is something the more ‘evolutionist’ Parsons is in principle not guilty of, in view of his emphasis on continuities between modern humans and the ancestors of the homo-sapiens family. That evolutionary connect is sufficient for present purposes to posit the first stage which Comte did not see, but which materialist Marx was all too aware of: the naturalistic/ontological/existential stage.

We can add this, however: if dialectics be the template for the evolution of method, and for what Foucault termed “The Order of Things”, then, contrary and superior to Comte’s arrangement, ontology is the antithesis of nothingness, and theology the synthetic result. Ontology thus predates theology, and is the first foundation of knowledge and method Theology is then followed by the metaphysical stage, and the metaphysical by the positivistic. Thus, there are not three stages/foundations in the development of method, but four.

Parsons’ notion of evolutionary universals suggests that societies evolve through a fairly consistent pattern of stages marked by distinct sets of universals. The first set – which, following his own reasoning, I have rearranged in terms of dialectical priority of emergence as: kinship, technology, communication (through language) and religion – is consistent with the notion, already touched on, that ontology precedes theology. This is particularly so when we consider the likelihood of a considerable time/“cultural” lag between the last two. Doesn't Aristotle’s ontology, too, portray being qua being as precedent to spiritual life?

  1. Naturalistic/Ontological/Existential Foundations:
Naturalistic or ontological or existential foundations of research method, and of knowledge, evoke the circumstances of that earliest period of human existence when all that could be known was known, and all the data that could be gathered was gathered, only (and only directly) through the senses – because nature gave no other options, and the technology with which to extend the reach of the senses was still in the unimagined future. In view of how human nature and culture were to evolve in the millennia that followed, it was truly a paleo-empirical world -- in which the sense of sight, “the gaze”, was already sovereign (to repeat Foucault’s emphasis). As Parsons (1964: 340) has imagined, “Vision, whatever its mechanisms, seems to be a genuine prerequisite of all the higher levels of organic evolution.” However, the other senses were also crucial to evolution, and most probably helped to trigger the language gene[2]’[3].

We can extend the commentary a little more, to say this. Philosophy did not create the reality of things which ontology speaks to. It merely gave it a name with, the wisdom of empirical hindsight The ontological basis of method – and the ontological content of human knowledge – were both, in the beginning, weaved out of truly elemental, down-to-earth, presences, dispositions and ‘acts’ of disparate individuals and groups, along the following lines:
 
1.1.Observing (‘Sense Perception’)
At the start, reality – nature, people, made-up-things, stumbled-upon things, things seen, things heard, things touched, things tasted, and things smelt (before anyone could “smell a rat”) – was and could be ‘apprehended’ only because, and insofar as, it was and could be seen, heard, touched, tasted and/or smelt – with the ‘naked’ sense-organ. It could be ‘apprehended’ because it could be observed – naturalistically. But such ‘apprehending’ was without doubt little more than intuitive, instinctive and sub-conscious in the beginning. Yet that was a vital start. It took a substantial while for the apprehension to evolve a narrative structure (that is, form and content), and for the narrative to become, in the language and the thought-process in which it was couched, coherent. [Marx and Engels (2000: 3) couch this ontological imperative of the human condition in terms of "The first premises of all history" and the "first fact"]

So what we now call ‘naturalistic observation’ is clearly the front-runner and pathfinder among the observational methods which humans have used throughout the ages to gain knowledge. It has evolved variants such as astronomy, participant observation (or ethnography), surveillance, monitoring (including CCTV monitoring), surveying and “intelligence”.

1.2.Doing
Observing could go, and typically went, hand-in-hand with doing. One could observe the doing – on the go. And one could likewise do what one observed in another’s or others’ doings. Those tentative, prehistoric acts were the precursors of practices and habits and experimentation – as well as the ancestors of the habits, customs, cultures, professions and disciplines – that have made the motley world what it has become. Doings are the building-blocks of culture and social structure. Ontologies, doings and happenings are thus the starting points of all our narratives, all our hermeneutic postures, and all history.

1.3.Experiencing (Feeling)
Experience is, at root, existential. We experience states of being and emotional states , happenings and doings, which we or others come to narrate in whatever way for immediate purposes, and sometimes for deep posterity – which will thus know us but which we will really never know. Feeling happy, sick, angry, fearful/afraid, safe, good – these are emotions and conditions we are driven to take stock of, observing clumsily or with varying degrees of clarity, depth, understanding, and even eloquence. Experiencing pain, grief, defeat, victory – and breakthrough – is all ingrained in the human condition, which we try to make sense of and articulate lessons from.


1.4.Telling
Telling is a re-enactment or codification in narrative form of that which we or others have observed, done and/or experienced. It is thus the summation, of sorts, of all of ontology’s four manifestations, including itself, and of all that ontology means and makes possible. Often, we interrogate in order to narrate, and do tell (and thus affirm) by way of an interrogation. A good telling is a “closing of the loop”, and a hard thing to do. Fast-forward and you realize that, in a sense, ethnomethodology is a wonderful kind of telling. So is thick description. So is mythology and poetry and drama and the novel. History is a telling, in its dialectical as well as chronological renditions and traditions. The epic and all the sagas of the world are a telling. Music and painting are a telling, and so is sculpture. Monuments are a telling, too. And thus all stories are by definition already told. 
2. ‘Theological’ Foundations

When was the theological stage predominate? From the moment in antiquity that the memes of the maker (of things out of ontological things) and the creator (of Being out of Nothingness, if I may rephrase Marcuse) entered and sufficiently seized the human mind, all the way up to the Age of Reason (the Enlightenment Age) in the 16th to 18th Centuries. This brief answer adds to the overall weight of evidence which, as earlier indicated, shows that Comte’s Law of Three Stages did not capture the sequence of stages of interest as discretely (or correctly) as it so emphatically suggested. In other words, his three ‘stages’ displayed, and continue to display, considerable and even overwhelming temporal overlaps – and sequence distortions. ‘Theology’, or more correctly religion, did emerge prior to philosophy, but has had a long history of co-presence, and a rich symbiotic relationship, with philosophy. It has not ceased to be, even in the positivistic stage. So it is safer to see the stages as more-or-less localized periods of dominance, even though the term dominance can itself be disputed on certain persuasive grounds.

The crucial point here is this: During a very long period of human existence, the template or prism through which humans observed, interpreted and explained their experiences in the world was theological. That is to say, God (and the gods) were the ultimate causality – the necessary and sufficient source of all happenings (as well as all causes and all effects), and the anchor of all explanations and all unquestionable beliefs.  The dominant world-view was dogmatic. Over time, however, contrarian narratives and reason emerged, fuelled by the desire for a more persuasive truth – a desire born in part at least by accumulating contradiction between ontology or experience and the body of knowledge that one was called upon to believe unquestioningly.


3.Philosophical/Metaphysical Foundations
This stage overlapped in at least two phases with the Theological Stage. The first phase represents the age of the great Greek philosophers and their successors – from around 470 BC to 322 BC – which found itself implanted in a still overwhelmingly religious, Delphi-leaning,  epoch . Thus:
n  Socrates: 470-399 BC
n  Plato: 428/6-348/7 BC
n  Aristotle: 384-322 BC.

The second phase – the Age of Reason or The Enlightenment – ran from the 16th Century to the 18th. This was the age of Rousseau, Voltaire, Hegel and their contemporaries. It was the age of the French Revolution. Religion might have already begun to cede ground, in a very palpable way, to Reason, but could not be shaken off.

Leedy (1980: 41), who is by no means a giant in the field, makes the memorable observation that:
“In all of mankind’s long history, we have devised only two ways to seek the unknowable [I prefer to say, “the unknown”]. One of these is by means of deductive logic, the other is by means of inductive reasoning, or what is familiarly called the scientific method.” [Here is an excerpt of the 1974 edition of Leedy's book]

Only two ways! All of humankind! That is the kind of remark which, if taken at face value or uncritically, appears like a great revelation. But the truth is that the two ways that Leedy has in mind are only the classical philosopher’s; and we will see momentarily that the Deduction-Induction twins just may have a distant relative – that is, Abduction – representing a third way.

Rottenberg (1994: 207)[4] credits Aristotle with being “the first” to give “formal expression” through his treatises (some 2,500 years ago) to the process by which humans reason – that is, the “reasoning process.” Aristotle, she notes , suggested that there were two types of reasoning – inductive and deductive. We use these two types of reasoning as the alternative ways to discover Truth. In other words, we use them to observe the world, select impressions, make inferences and generalize (see Rottenberg, 1994: 207). But Aristotle was aware that neither of these two approaches was error-free. At certain stages in the “reasoning process,” we are prone to error.

Deductive (Aristotelian) Logic:  

Here are some definitions to clarify the deductive method:

Deduction = “The principle [procedure] of reasoning from general principles to particular instances” (Theodorson and Theodorson, p. 104). [In deductive reasoning, “specific hypotheses or predictions are derived from broader theoretical principles” T & T, ibid). [This happens a lot, as is to be expected, in literature review; especially the review of theoretical literature]

Deductive Reasoning = Reasoning by deducing, or inference from general to particular; a priori reasoning (see Oxford Dictionary, 1964). [A priori = self-evident, unfettered by the rule of empirical proof]

Deductive Learning: “commences with the rule or principle (theory) which is subsequently applied by the learner” (Cole, 1997: 266); or “where the individual works forward from some hypothesis or other, tests it in practice and obtains results – new experiences” (Cole, 1995: 143).

Deductive Argument = “A deductive argument proceeds from a general statement that the writer assumes to be true to a conclusion that is more specific” (Rottenberg, 1994: 219). [Rottenberg (1994: 212) believes, moreover, that unlike induction which “attempts to arrive at the truth, deduction guarantees sound relationships between statements...called premises.”]

Deduction (Ghosh, 1985: 47) = “the process of drawing generalisation, through a process of reasoning on the basis of certain assumptions which are self-evident or based on observation. In deduction, we deduce generalisations from universal to particular...The main task of deductive logic is to clarify the nature of [the] relation between premises and conclusions in valid arguments.”

The deductive approach is said to have been dominant up to the time of Renaissance. Leedy (1980: 41), for example, notes that “Up to the time of the Renaissance, insight into most problems was sought by means of deductive logic...It relied upon logical reasoning and began with a major premise. This was a statement, similar to an axiom, which seemed to be a self-evident and universally truth: Man is mortal; God is good; the earth is flat.” But the danger is that this major premise may be a “pre-conceived idea” or “preconceived conclusion” (Leedy, 1980: 42) – a ‘dogmatic’ statement – which does not turn out to be true (or grounded) in fact, in which case the conclusions arrived at will be invalid (see Leedy, 1980: 41).

In its simplest form, a deductive argument [a syllogism] has three parts: A major premise, a minor premise and a conclusion. Where each premise is true, then “the conclusion must be true. Unlike the conclusions from induction, which are only probable, the conclusions from deduction are certain” (Rottenberg, 1994: 212). [SYLLOGISM = “a formula of argument consisting of three propositions”: the major premise, the minor premise and the conclusion (see Rottenberg, 1994:213, quoting a dictionary)]

Remember: such a deductive conclusion is deemed to be “certain.” It is considered to be “valid” – that is, to derive from a valid “reasoning process” or a valid form of argument. It is “logically consistent because it follows necessarily from the premises. No other conclusion is possible” (Rottenberg, 1994: 212). However, a form of argument may be valid but the “argument itself” may not satisfy [and will not be indisputable] “if the premises are not true” (Rottenberg, 1994: 212). Thus, “The deductive argument is only as strong as its premises” (Rottenberg, 1994: 213).

Ghosh (1985) had earlier made a similar argument, as follows: The logical validity (“implication”) of an argument “does not depend upon the material [i.e. factual] truth of the premises. The premises may be materially false, but yet the reasoning may be correct” (Ghosh, 1985: 47). [For example (adapted from Ghosh (1985: 48): Democracies have no kings or queens. Britain is a democracy. Therefore, Britain has no king or queen]. However, the truthfulness/validity of a conclusion depends on the “material truth of the premises.”

Inductive Reasoning:

Here, once again, are useful definitions:

Induction = “a process of reasoning whereby we arrive at universal generalisations from particular facts. Induction gives rise to empirical generalisations, and is opposite to deduction. Induction involves a passage from observed to unobserved. Induction involves two processes – observation and generalisation” (Ghosh, 1985: 48).

Induction = “the form of reasoning in which we come to conclusions about the whole on the basis of observations of particular instances” (Rottenberg, 1994: 208). Rottenberg (1994: 211) adds that “An inductive argument proceeds by examining particulars and arriving at a generalization that represents a probable truth.” Moreover, in inductive reasoning, “the reliability of your conclusion depends on the quantity and quality of your observations (Rottenberg, 1994: 208).

Induction = “the process of reasoning from individual instances to general principles” (Theodorson and Theodorson, Dictionary of Sociology, p.199). That is to say, induction means generalization from specific facts; from individual occurrences to general cases. Thus, the experimental method is inductive, since general conclusions derive from individual observations (p. 199). Indeed, T & T observe that “Most sociological studies are” inductive in orientation (p. 200). [But remember that sociological methodology insists that, for such generalizations to hold (to be valid), the sample of individual cases to which empirical attention is paid during both data collection and analysis must be representative (that is, as a totality, a microcosm) of the ‘general cases’ (that is, the population or ‘parameters’ to which the sample refers). One seeks to ensure this at the sampling stage]

Inductive Learning:“the process in which the learner experiences an event or stimulus and draws a conclusion from it, for example some rule or guiding principle” (Cole, 1997: 266); or “where the individual experiences an event, attempts to puzzle it out, and then draws conclusions about it, perhaps in terms of a guiding rule or principle” (Cole, 1995: 142-3).


Abductive Reasoning/Method:

In his Collected Papers, Charles Peirce (1934/1935: 106), who coined the term, defines abduction as "the process of forming an explanatory hypothesis" and adds, importantly, that "every single item of scientific theory which stands established today has been due to abduction." Much has since been written about abduction, which is nowadays commonly characterized as "inference to the best explanation."

For three useful readings on abduction, and its wide application-potential as well as nearness to induction, click on these links:
1. Douven, Igor, "Abduction", The Stanford Encyclopedia of Philosophy (Spring 2011 Edition)
2. Thagard. Paul and Cameron Shelley (1997) "Abductive Reasoning: Logic, visual thinking and coherence"
3. Anonymous Author Wikipedia, A characterization of abduction that I find to be close to Anonymous Author's is to be found in the 'preliminary version' of Hector J. Levesque's article titled "A Knowledge-level account of abduction".

Douven’s (2011) definition of abduction is hard to pin down in a quick quote, but the illustrations in his “General Idea” do clarify the abductive process for those just coming upon the concept. 

Thagard and Shelley (1997), in their densely-argued and well-referenced article, define abductive reasoning as "reasoning in which explanatory hypotheses are formed and evaluated." At the heart of abductive reasoning, they argue, "is the goal of assembling a set of hypotheses (causes) that provide good explanations of the data (effects)." Viewing abductive reasoning that way, they see its applicability in many "important kinds of intellectual tasks, including medical diagnosis, fault diagnosis, scientific discovery, legal reasoning, and natural language understanding." 

According to an Anonymous Author in the Wikipedia link above, abduction is “a form of logical inference that goes from observation to a hypothesis that accounts for the reliable data (observation) and seeks to explain relevant evidence.” On the other hand, abductive reasoning is seen to have the quality of a logical fallacy – the fallacy of attributing an effect (which may be there for all to see) to one’s preferred cause, which not everyone may accept as true, or which may be countervailed by more robust explanations for the “effect” observed by all. Thus, abduction:

“allows inferring a as an explanation of b. Because of this inference, abduction allows the precondition a to be abduced from the consequence b. Deductive reasoning and abductive reasoning thus differ in the direction in which a rule like ‘a entails b’ is used for inference. As such, abduction is formally equivalent to the logical fallacy of affirming the consequent (or Post hoc ergo propter hoc) because of multiple possible explanations for b” (Anonymous Author, Wikipedia)

According to Business Dictionary, abduction is “The type of reasoning whereby one seeks to explain relevant evidence by beginning with some commonly well known facts that are already accepted and then working towards an explanation.” 

In sum, abductive reasoning is the kind of logic which tends to yield a backward-looking conclusion (about the cause of an effect) that is – more often than not, perhaps – not necessarily true or valid [Instead of backward-looking, Shanahan (I have just noted) uses the term "backwards projection (explanation)"]. But abduction often makes valid inferences as well, just as weather forecasting aka predictive deduction does. In weather forecasting, next fortnight’s actual weather may be predicted (explained?) – a prediction/ explanation which turns out to be more or less accurate – on the basis of a juxtaposition of elements of the looming (“over the horizon”) weather on past patterns or trends recorded in a ‘big’ database (which is typically not accessible or explained to the general public), and inferring (or finding) a match[5]. Q: When you predict the weather, are you engaged in explanation or simply deduction?

There is evidently both good and bad abduction, as Charles Sanders Peirce suggested. Good abduction is uncertainty-reducing, for sure. Bad abduction may be the source of much anguish, as we have consistently seen in the Kenyan and US political arenas in both 2012 and 2013. The challenge is how to consistently avoid bad abduction in lay and scientific conversations.


4.Positivistic Foundations of Method:

The Positivistic era had its roots in the Copernican Revolution of the 16th century, deep inside the metaphysical stage, which in principle preceded it. Like the Copernican Revolution itself, Positivism took some 2 to 3 centuries to take hold; that is, up to the early 19th century.

Before the 19th century, the process by which humans discovered knowledge/truth was, generally, dominated by religious and cultural dogma – and by DEDUCTIVE reasoning. The dogma tended to invalidate the DEDUCTION, but not so, conceivably, among the Bushmen (Kung) of the Kalahari. In the 19th century, positivism emerged as a key aspect of social philosophy and social theory (Giddens, 1976)[6].

Let us return briefly to what we were saying about inductive reason, in order to underscore in this section its connections with positivism and science.

Inductive Reasoning “begins, not with a preconceived conclusion – a major premise – but
with an observation” (Leedy, 1980: 42). He (1980: 42) adds that “Renaissance man began
seeking truth by looking steadfastly at the world around him. He started asking questions of
Nature. And Nature responded in the form of observable fact.”

In other words, as Leedy (1980: 42) suggests, the intellectual appeal of inductive reasoning during the Renaissance period was fuelled by “an interest in humanism” and by a pragmatic or empiricist orientation toward “this world and ... its phenomena” – both of which were characteristic of that period. Thus:
:
“Renaissance man soon found that when facts are assembled and studied dispassionately, they frequently suggest hitherto undiscovered truth. Thus, was the scientific method born; and the words mean literally ‘the method that searches after knowledge’ (scienta = L. knowledge, from scire, L. to know)” (Leedy, 1980: 42)

This scientific method (and methodology) underwent considerable development in the 16th century as a result of the work of a number of scholars/thinkers, including: Leonardo, Copernicus, Galileo, Vesalius and Vittorino da Feltre (Leedy, 1980: 42).

Back to positivism:

By positivism Giddens understands:

1.         The view that all knowledge derives from, or pertains to, a reality which we can apprehend or confirm [ontologically] only through the senses [of touch, sight, hearing, smell and taste].
2.         The view that the methods and logic of science, as exemplified by "classical physics", are equally applicable to the study of human society (Giddens, 1976: 130).

Giddens further notes that:
"In the writings of Comte [1798-1857] and Marx [1818-1883] alike, the science of social life was to complete the freeing of the human spirit from religious dogmas and the customary, unexamined beliefs men had about themselves" (Giddens, 1976: 130).

Thus, Western science differs from most types of religious and magical practices [or even lay epistemologies] in the following three ways, among others [But one may pose a 'counter-question': Is Western science a homogeneous worldview characteristic of the entire West? If so, what are its key characteristics? If not, in what specific ways do given non-western cosmologies differ from leading or specific versions or traditions of Western science?]:
1.         Science treats observed phenomena in 'nature' as the manifestations ('outcome') of impersonal forces -- not the "personalized gods, spirits or demons" characteristic of "most" non-western religious and magical systems (Giddens, 1976: 139).
2.         Science institutionalizes the process by which theory is formulated and scientific observations conducted; and, moreover, makes a "public display" of the whole process. That is to say, scientific work is, ideally, legitimated through "free debate and critical testing" (Giddens, 1976: 139). This legitimating characteristic of science (at the core of which is critical, public scrutiny) is lacking in most religious doctrines, argues Giddens (1976: 139).
3.         Science does not involve, as religion and magic often do, “...forms of activity that are alien to western science: including worship in regularized ceremonial, propitiation and sacrifice” (Giddens, 1976: 139)[7].

But Giddens recounts how, following the erosion (in the early 20th century) of faith in scientific knowledge “as exemplar of all knowledge,” there emerged two contesting philosophical camps[8] in the 1920s and 1930s: [1] the logical positivists, who championed the cause of natural science (“scientific rationalism,” scientific knowledge); and [2] the phenomenological and linguistic philosophers who saw natural science's “claims to knowledge [as] secondary to, and dependent upon, ontological premises of the natural attitude” (Giddens' words, emphasis MY's) [Natural attitude = "authority of common sense", ibid].  Anthropologists were at the forefront of this change, touting the merits of the natural attitude. A natural attitude toward knowledge, it was argued, was the most important way to uncover the truth.

It can be argued, then, that the tilt (or re-tilt) toward the natural attitude, which is the bedrock of Qualitative Research Methodology, was the result of a SWOT analysis of the scientific attitude (or “scientific rationalism”) – with its quantitative orientation.

Around the same time as this theoretical debate was going on, moreover, the practice of qualitative research was in vogue in the United States (see, for example, W.I. Thomas and Florian Znaniecki The Polish Peasant in America and Europe). [For a vey recent (2008) example of qualitative analysis, see Malcolm Gladwell’s account (in Outliers) of an Italian community in early 20th century Pennsylvania, USA: It is an account of how the community’s health profile was documented (see “The Roseto Mystery”, pp. 3.12). 

So, one Rationality or Many Rationalities?
[The ‘rationality of science’ (or the ‘scientific attitude’) versus the ‘rationality of
commonsense’ (or the ‘natural attitude’)]

The ‘Rationality of Science’ (or of the ‘Scientific Attitude’): One Rationality
This is the perspective adopted in Weber's study of rational action, which suggests that there is but one standard of rationality. It uses specific means-ends criteria – such as the operational definition of concepts, the representativeness of cases/samples, universality and generalizability of claims – to explain social phenomena. And it explains motivated action using the observer's criteria, and, contrary to the lay actor’s approach to observed reality, entails “the suspension of the belief that things are as they appear” (Giddens, 1976: 35).

The ‘Rationality of Commonsense’ (the ‘Natural Attitude’): Many Rationalities
According to Garfinkel, and Giddens, there are many common-sense rationalities which pertain to the daily rhythm of lay life – as opposed to the demands of social science (Giddens, 1976: 35-36). Reality is socially constructed, as Berger and Luckman suggested.

The concept and practice of ethnomethodology are undergirded by the view that, to paraphrase Giddens (1976: 36), the way that people organize and live their everyday lives is identical to the way that they understand and explain their life experiences; that is, identical to the procedures that they use to explain to themselves and even to others what they see and experience.

Consequently, the differences between Sociology and Natural Science can be highlighted as follows, as suggested by Giddens (1976: 146):
1.         Sociology uses the inductive conclusion with more tact, greater laterality of vision and thinking, as well as the acknowledgment of precursors, unlike natural science which relies entirely on narrowly defined reason
2.         Unlike natural science, Sociology “stands in a subject-subject relation to its ‘field of study,’not a subject-object relation”
3.         Sociology, adopting ethnomethodology’s stance, strives to understand and interpret a world that is already “pre-interpreted… by active subjects … [whose pre-interpretations feature materially in the] …the actual constitution or production of that world”]
4.         Social theory construction, like the general apprehension of social reality “involves a double hermeneutic”[9] – which is not seen in other disciplines (Giddens, 1976: 146).
5.         Sociological generalizations are by definition more nuanced and at great variance with the laws of natural science.




[4] Annette T. Rottenberg (1994) Elements of Argument: A Text and Reader. Fourth Edition. Boston: Bedford Books of St. Martin’s Press.
[5] This sounds like a tautology!
[6] Anthony Giddens. 1976. New Rules of Sociological Method: A Positive Critique of Interpretative Sociologies. London: Hutchinson.
[7] For a detailed discussion of the differences between science and religion/magic see Horton, Robert and Ruth Finnegan. 1973. Modes of Thought. London: Faber.
[8] The two camps have been likewise labeled by Giddens as: The Rationality of Science v. the Rationality of Common Sense
[9] Double Hermeneutic = Double Interpretation. In Double Interpretation, the Observer interprets the words/behavior/actions of the Observed/Respondent, who likewise interprets the Observer’s “seduction” [MY] – and so on.