Measurability: A Key Standard of Scientific Research

Nine key standards, or principles, of scientific research can be gleaned from texts on research method. As I see them, these are:

1. Universality

2. Replicability

3. Control
4. Measurability
5. Validity
6. Reliability (Pending)

7. Objectivity (Pending)

8. Ethics 

9. Representativeness.

I say something about each of the nine standards in separate posts, as shown in the links above. What I want to briefly talk about here is the measurability standard. 

Measurability 

Measurability means susceptibility to measurement. It means, in other words, that what is to be measured must embody quantities and/or qualities -- or have features, dimensions or characteristics -- that can be subjected to appropriate or 'true' measurement. Concerning the defining characteristics of a measurement, Bell (1999) notes that:

"A measurement tells us about a property of something. It might tell us how heavy an object is, or how hot, or how long it is. A measurement gives a number to that property. Measurements are always made using an instrument of some kind. Rulers, stopwatches, weighing scales, and thermometers are all measuring instruments. The result of a measurement is normally in two parts: a number and a unit of measurement, e.g. ‘How long is it?... 2 metres." 

The measurability rule is anchored on the above conceptions, and so requires that the variables around which the researcher intends to collect data should be measurable, or susceptible to acceptable ‘measurement’ (Leedy, 1980: 46)[1]. This is easier done in the natural sciences than in the social sciences; in quantitative studies than in qualitative. Still, one must, in the social sciences too, endeavor to quantify, measure and evaluate. Indeed, the guiding principle of the measurability rule -- its corollary, in other words -- is this: "What can be measured must be measured." Thus, not measuring what can be measured is not an option allowed anyone.

In quantitative research, the measurability standard features prominently in all methodological procedures that revolve around, or build up to, hypothesis testing. The testability of a hypothesis is indeed a function of the measurability of the variables that constitute it. Such measurability is, in turn, dependent on the indicators chosen to represent them; but one is not entirely free to choose just any indicator(s). They must be such as most peers or reviewers or supervisors or sponsors can be persuaded to accept as valid and reliable. Still, one has some leeway in choosing the indicators one wishes to use, so long as the reviewers, supervisors or funding entities are prepared to see or accept one's findings “in the light" of the chosen indicators.

Viswanathan (2010: 285-210) has interrogated in depth the challenges encountered in "Measure development procedures," particularly in connection with quantitatively-oriented social science research. Among these challenges, which he believes are not sufficiently appreciated, is the underpinning assumption, which he faults, that "a construct can be [readily and invariably] isolated and examined." Here is his line of thinking:

"By measuring or manipulating individual constructs, relationships between constructs are studied and substantive hypotheses about these relationships are tested. The very notion that numbers can be assigned to attributes of people, objects, or events is presumed on being able to study attributes or constructs separate from other constructs. After all, measurement relates to rules for assigning numbers to attributes of people, objects and events. [But] this is not the case for many phenomena. A complex network of constructs may influence a phenomenon and may not be separable into individual constructs for purposes of measurement." 

Measurability procedures implemented in qualitative -- that is, 'non-numerical' -- research attempt, not always successfully, to overcome or circumvent such intractables as Viswanathan has pointed out. These procedures include, for example, “comparative judgement (arranging factors in a hierarchy of importance)” or  “scaling (correct versus incorrect responses to a given set of questions)” (Leedy, 1980: 46). Other ‘non-numerical’ strategies that have been used, according to Worsley (1992: 113)[2] include: “careful classification[3], organizing and combining of field-notes [which] requires patient checking and cross-checking and the application of systematic techniques.” But he acknowledges that “numerical methods are sometimes applied to field-data too” (Worsley, 1992: 113). Indicators are often used as proxies/surrogates for more fuzzy or abstract concepts, such as status (Worsley, 1992: 96). Such indicators are inevitably numerical. 

One may also, in connection with the nominal level of measurement (the simplest level), and even with the ordinal level of measurement, calculate frequencies/percentages, and cross-classify in qualitative research. Scaling, moreover, is not just about correct v. incorrect. Incorporating comparative judgement, one may also use scales of: high v. low, do v. don’t, very high to very low, very good to very bad, strongly agree to strongly disagree.

To be fair to everyone, no one is quite attempting stubbornly to square the circle in this measurability conversation. Still, the challenges of measuring require, and will surely be tempered with, constant and open-minded vigilance -- as well as an abundance of capacity to deal with critiques and contrary views, including (so be it) one's own self-criticism. Bell (1999: 1), like others before her, subsumes this conversation under the theme "uncertainty of measurement". Thus:

"Uncertainty of measurement is the doubt that exists about the result of any measurement. You might think that well-made rulers, clocks and thermometers should be trustworthy, and give the right answers. But for every measurement - even the most careful - there is always a margin of doubt. In everyday speech, this might be expressed as ‘give or take’ ... e.g. a stick might be two metres long ‘give or take a centimetre’."

---

^[1] Sarantakos talks of the need for precision in measurement (pp. 18-26)

^[2] Peter Worsley, ed. 1992. The New Introducing Sociology. Revised Third Edition. London: Penguin Books.

^[3] Another word for classification, given by Worsley (1992: 97) is categorization; and both, he argues, are an often overlooked “form of measurement” – even if they are “the simplest level” (that is, the nominal level) of measurement. There are in total four levels of measurement, the others being, in an ascending order of complexity: Ordinal (or rank-order), interval and ‘ratio’ scale (Worsley, 1992: 97-98).



REFERENCES

Bell, Stephanie (1999) Measurement Good Practice Guide: A Beginner's manual to Uncertainty of Measurement. (No. 11, Issue 2). Taddington: National Physics Laboratory

Leedy, Paul D. (1980) Practical Research: Planning and Design. Second Edition. New York:  Macmillan Publishing Co.,
Sarantakos, Sartirios (1994) Social Research. London: The Macmillan Press

Viswanathan, Mathu (2010) "Understanding the Intangibles of Measurement in the Social Sciences," pp. 285-312, in Geoffrey Walford, Eric Tucker and Mathu Viswanathan, Eds. (2010) The SAGE Handbook of Measurement. Los Angeles: SAGE 

Worsley, Peter, ed (1992) The New Introducing Sociology. Revised Third Edition. London: Penguin Books

Representativeness: A Key Standard of scientific Research

Nine key standards, or principles, of scientific research can be gleaned from texts on research method. As I see them, these are:

1. Universality

2. Replicability

3. Control
4. Measurability
5. Validity
6. Reliability (Pending)

7. Objectivity (Pending)

8. Ethics 

9. Representativeness.

I say something about each of the nine standards in other posts. What I want to briefly talk about here is representativeness.

Representativeness[1] [ = Generalizability]: The representativeness standard is applied differently in quantitative and qualitative research. In quantitative research, representativeness refers to the degree to which the sample selected for study can legitimately be said to reflect or embody the broad characteristics of the population in which it is embedded. More pragmatically, a sample is representative if it embodies the specific characteristics of the population that are of interest to a given researcher or study.  Thus, if we were to beam or project a transparency of the sample's outline or contours or segments (or proportions) onto a screen 'pre-loaded' with the population's outline or contours or segments (or proportions), we should expect the two images to perfectly match when the respective scales match. The fundamental question to be 'frontally' interrogated or incontrovertibly answered in any specific quantitative research, as Weber was amply aware, is where and how the material so pre-loaded is, was or can/should be obtained in the first place. 

Broadly speaking, quantitative research begins with the assumption that the contours of a population, in terms of size and structure, are accurately and adequately documented in relevant census or other survey data related to the population; and that the researcher has ample access to the data. The quantitative researcher is enjoined to select a sample the size and components of which are statistically and structurally proportionate to the population that they are supposed to represent. However, for many vexing problems which researchers seek to tackle, and research questions which they may want to answer, such ready-made population maps are in reality non-existent.

Qualitative sociological or anthropological research, likewise, is premised on a pre-existing and deep pre-understanding of the 'broad nature' of the society or population in question, based on prior studies and/or a variety of other sources. However, the reality of field research does not often enough bear out the premise. Consequently, pre-understanding must itself be constructed as an 'immediate' prelude to -- indeed, even as a necessary part of -- the anticipated study, Such pre-understanding then becomes the basis for constructing what Weber, with great insight, termed ideal types. 

Concerning ideal types (idealtypus, in German), Weber (1978: 19-22) argues as follows:

"Sociology seeks to formulate type concepts and generalized uniformities of empirical process. … As in the case of every generalizing science, the abstract character of the concepts of sociology is responsible for the fact that, compared with actual historical reality, they are relatively lacking in fullness of concrete content. To compensate for this disadvantage, sociological analysis can offer a greater precision of concepts. … In all cases, rational or irrational, sociological analysis both abstracts from reality and at the same time helps us to understand it, in that it shows with what degree of approximation a concrete historical phenomenon can be subsumed under one or more of these concepts. For example, the same historical phenomenon may be in one aspect feudal, in another patrimonial, in another bureaucratic, and in still another charismatic. In order to give a precise meaning to these terms, it is necessary for the sociologist to formulate pure ideal types of the corresponding forms of action which in each case involve the highest possible degree of logical integration … But precisely because this is true, it is probably seldom if ever [the case] that a real phenomenon can be found which corresponds exactly to one of these ideally constructed pure types. The case is similar to a physical reaction which has been calculated on the assumption of an absolute vacuum. … When reference is made to 'typical' cases, the term should always be understood, unless otherwise stated, as meaning ideal types, which may in turn be rational or irrational …"

Thus, in qualitative research, as Sarantakos has noted, “sampling...is not based on probability theory, and the size of the sample is usually too small to reflect the attributes of the population concerned. Instead, the sampling procedures used are related to theoretical sampling and are geared towards essential and typical units. Selection [that is, the quest for representativeness] does not stress random procedures but rather theoretical sampling, that is, [it is geared] towards the theoretically important units” (Sarantakos, 1994: 27).

Though it is not usually known or talked about -- or, more importantly, fully appreciated in sociological, or qualitative research, circles -- the fact is that Max Weber's notion of ideal types long ago preemptively freed sociology's qualitative researchers from what one may call the conceptual 'yoke of the quantitative sample'. Incidentally, sociologists' counterparts in anthropology have never felt as constrained by this version of the tyranny of numbers. 

As Priyadarshini (n.d.) has tried to say, moreover:

"Just as an ideal model is constructed by the natural scientists as an instrument and means for knowing nature, so the social scientist creates [ideal types] as a tool for systematizing and comprehending individual facts, against which the investigator can measure reality. It [serves that purpose] because of its separation from empirical reality and difference from it."

Importantly, in this regard, Sarantakos (1994: 27) has further, and pointedly, clarified that while “qualitative researchers consider generalisations to be important, they relate them to the typical cases they study, not to random theory and principles. Generalisation becomes obvious out of the intersubjective clarification of validity. The essential argument here is that generalisation is based on the typical case studied, which is thought to be representative of a species...; what qualitative research claims is that such findings can be interpreted beyond the cases studied and are examples of an ‘exemplar generalisation’, or ‘analytic generalisation’...”

REFERENCES:

Priyadarshini, S. (n.d.) "Weber's 'Ideal Types': Definition, Meaning, Purpose and Use"

Sarantakos, Sartirios (1994) Social Research. London: The Macmillan Press

Weber, Max (1978) Economy and Society: An Outline of Interpretative Sociology (Guenther Ross and Claus Wittich, editors and translators). Berkeley: University of California Press.

---

[1] In quantitative studies this means having such characteristics/capacity/quality as do capture or reflect all the key aspects of the population (or that segment of the population that is of interest).

Thick Description ~ CSO 302

"... : if you want to understand what a science is, you should look in the first instance not at its theories or its findings, and certainly not at what its apologists say about it; you should look at what the practitioners of it do.

In anthropology, or anyway social anthropology, what the practioners (sic) do is ethnography. And it is in understanding what ethnography is, or more exactly what doing ethnography is, that a start can be made toward grasping what anthropological analysis amounts to as a form of knowledge. This, it must immediately be said, is not a matter of methods. From one point of view, that of the textbook, doing ethnography is establishing rapport, selecting informants, transcribing texts, taking genealogies, mapping fields, keeping a diary, and so on. But it is not these things, techniques and received procedures, that define the enterprise.

What defines it is the kind of intellectual effort it is: an elaborate venture in, to borrow a notion from Gilbert Ryle, "thick description." ~ Clifford Geertz (1973: 5-6) [Quote from his book, The Interpretation of Cultures. New York: Basic Books]

Gilbert Ryle's concept of thick description, as popularized in the disciplines of anthropology and sociology by Clifford Geertz (see, also, this article), refers to the successive addition of layer upon layer of detail to a phenomenon or process or event being described -- building up from 'literal' (or simple or lay) to 'thin' (or journalistic) to thick (or 'deep' or scholarly) description -- such that, true to the second law of dialectics, the quantum (quantity) of descriptions progressively transforms into a qualitative synthesis of the 'totality' of aspects of the phenomenon in question. It is like adding pixels and mega-pixels, one measured 'spread' after another, to a picture until one achieves the highest possible level of clarity, or picture quality.

Thick description is widely accepted as a major qualitative data analysis method or technique. However, I have preferred to use the concept synthesis in the above definition, as it more satisfactorily captures what for me is the core purpose of thick description. There is debate, however, concerning the relative merits of analysis and synthesis in pushing the boundaries of knowledge.

Analysis, an anchor concept in both quantitative and qualitative research, which is used routinely and unquestioningly by nearly all scientists (natural and social), does not seem to be quite apt here. B.N Ghosh (1985: 23), for example, has suggested that “analysis plays a more important role than synthesis” during “the initial stages of a science”; but “as the science becomes more and more progressive, synthesis plays a more and more important role. But in every science both the methods are used simultaneously.” He (1985: 24) adds that:

“A science aims at not only arriving at the truth but also at expounding the truth. A synthetic method is necessarily expository because it puts together the elements of a phenomenon in a systematic manner. In synthesis, exposition is possible because the entire elements are completely known and, as such, they can easily be arranged in an orderly way. The discovery of truth, however, is made possible by the method of analysis.”

I think that the aim and effect of thick description is to expound in 'full' the details, the 'totality', of the object of description. However, the end-product of thick description cannot be touted as a "complete" exposition of that which is (to be or has been) described; and I suspect that thick description was never intended for such a purpose.

NOTE: For more on thick description read:
1.This article by Joseph Ponterotto
2. Fei-Wen Liu's article

PS: Quote at the top added on September 11, 2015