Showing posts with label Extrapolation. Show all posts
Showing posts with label Extrapolation. Show all posts

Saturday, October 31, 2015

Representativeness: A Key Standard of scientific Research

Nine key standards, or principles, of scientific research can be gleaned from texts on research method. As I see them, these are:

3. Control
4. Measurability
5. Validity
6. Reliability (Pending)
7. Objectivity (Pending)
8. Ethics 
9. Representativeness.

I say something about each of the nine standards in other posts. What I want to briefly talk about here is representativeness.


Representativeness[1] [ = Generalizability]: The representativeness standard is applied differently in quantitative and qualitative research. In quantitative research, representativeness refers to the degree to which the sample selected for study can legitimately be said to reflect or embody the broad characteristics of the population in which it is embedded. More pragmatically, a sample is representative if it embodies the specific characteristics of the population that are of interest to a given researcher or study.  Thus, if we were to beam or project a transparency of the sample's outline or contours or segments (or proportions) onto a screen 'pre-loaded' with the population's outline or contours or segments (or proportions), we should expect the two images to perfectly match when the respective scales match. The fundamental question to be 'frontally' interrogated or incontrovertibly answered in any specific quantitative research, as Weber was amply aware, is where and how the material so pre-loaded is, was or can/should be obtained in the first place. 

Broadly speaking, quantitative research begins with the assumption that the contours of a population, in terms of size and structure, are accurately and adequately documented in relevant census or other survey data related to the population; and that the researcher has ample access to the data. The quantitative researcher is enjoined to select a sample the size and components of which are statistically and structurally proportionate to the population that they are supposed to represent. However, for many vexing problems which researchers seek to tackle, and research questions which they may want to answer, such ready-made population maps are in reality non-existent.

Qualitative sociological or anthropological research, likewise, is premised on a pre-existing and deep pre-understanding of the 'broad nature' of the society or population in question, based on prior studies and/or a variety of other sources. However, the reality of field research does not often enough bear out the premise. Consequently, pre-understanding must itself be constructed as an 'immediate' prelude to -- indeed, even as a necessary part of -- the anticipated study, Such pre-understanding then becomes the basis for constructing what Weber, with great insight, termed ideal types. 

Concerning ideal types (idealtypus, in German), Weber (1978: 19-22) argues as follows:
"Sociology seeks to formulate type concepts and generalized uniformities of empirical process. … As in the case of every generalizing science, the abstract character of the concepts of sociology is responsible for the fact that, compared with actual historical reality, they are relatively lacking in fullness of concrete content. To compensate for this disadvantage, sociological analysis can offer a greater precision of concepts. … In all cases, rational or irrational, sociological analysis both abstracts from reality and at the same time helps us to understand it, in that it shows with what degree of approximation a concrete historical phenomenon can be subsumed under one or more of these concepts. For example, the same historical phenomenon may be in one aspect feudal, in another patrimonial, in another bureaucratic, and in still another charismatic. In order to give a precise meaning to these terms, it is necessary for the sociologist to formulate pure ideal types of the corresponding forms of action which in each case involve the highest possible degree of logical integration … But precisely because this is true, it is probably seldom if ever [the case] that a real phenomenon can be found which corresponds exactly to one of these ideally constructed pure types. The case is similar to a physical reaction which has been calculated on the assumption of an absolute vacuum. … When reference is made to 'typical' cases, the term should always be understood, unless otherwise stated, as meaning ideal types, which may in turn be rational or irrational …"
Thus, in qualitative research, as Sarantakos has noted, “sampling...is not based on probability theory, and the size of the sample is usually too small to reflect the attributes of the population concerned. Instead, the sampling procedures used are related to theoretical sampling and are geared towards essential and typical units. Selection [that is, the quest for representativeness] does not stress random procedures but rather theoretical sampling, that is, [it is geared] towards the theoretically important units” (Sarantakos, 1994: 27).

Though it is not usually known or talked about -- or, more importantly, fully appreciated in sociological, or qualitative research, circles -- the fact is that Max Weber's notion of ideal types long ago preemptively freed sociology's qualitative researchers from what one may call the conceptual 'yoke of the quantitative sample'. Incidentally, sociologists' counterparts in anthropology have never felt as constrained by this version of the tyranny of numbers. 

As Priyadarshini (n.d.) has tried to say, moreover:
"Just as an ideal model is constructed by the natural scientists as an instrument and means for knowing nature, so the social scientist creates [ideal types] as a tool for systematizing and comprehending individual facts, against which the investigator can measure reality. It [serves that purpose] because of its separation from empirical reality and difference from it."
Importantly, in this regard, Sarantakos (1994: 27) has further, and pointedly, clarified that while “qualitative researchers consider generalisations to be important, they relate them to the typical cases they study, not to random theory and principles. Generalisation becomes obvious out of the intersubjective clarification of validity. The essential argument here is that generalisation is based on the typical case studied, which is thought to be representative of a species...; what qualitative research claims is that such findings can be interpreted beyond the cases studied and are examples of an ‘exemplar generalisation’, or ‘analytic generalisation’...”


REFERENCES:

Priyadarshini, S. (n.d.) "Weber's 'Ideal Types': Definition, Meaning, Purpose and Use"


Sarantakos, Sartirios (1994) Social Research. London: The Macmillan Press

Weber, Max (1978) Economy and Society: An Outline of Interpretative Sociology (Guenther Ross and Claus Wittich, editors and translators). Berkeley: University of California Press.




[1] In quantitative studies this means having such characteristics/capacity/quality as do capture or reflect all the key aspects of the population (or that segment of the population that is of interest).

Tuesday, March 06, 2012

Scope and Limitations in Scientific Research



Introduction

Much confusion surrounds the use of the terms “scope” and “limitations” in scientific research. In any scientific study that we may undertake, it is important that this confusion be cleared before the actual research begins, than after fieldwork – or, worse still, when the findings, having been otherwise processed and analyzed with self-assurance, are being reported to an informed and skeptical audience.

Scope

Scope refers to the portion/proportion of the totality [the content] suggested by the research topic and the problem statement that the researcher intends to cover, or will actually cover. It incorporates the giving of the “reason why”; that is, why the scope is as we are called upon to see it – in terms of Time, Space and/or Number. “Number” here could mean the quantity or size of a variety of factors, such as: sample, the population in which the sample is embedded, social stratum, cohort, age-group, gender group, occupational group, or some other category of “subject.”

The researcher’s task under “Scope” is to specify how much and/or what aspects of the problem or relevant subject-matter is to be tackled, and how large the study area and population will be. In other words, what the researcher states under “Scope” is exactly how much research he/she is going to do [or has to do] to realize the research objectives. This helps to clarify both the sampling frame and the range of hypotheses that one can reasonably expect to test, or the research/guiding questions that one may address. Under “scope”, therefore, the researcher briefly specifies the “area” to be covered, and/or the boundaries  [the Breadth, Length and Depth – in terms of Time, Space and Number] within which the study will be confined.

Limitations

In extrapolating/generalizing research findings, one must know the boundaries of the possible, or of the acceptable claim – even if the claim be in all other ways evidence-based. This is what we call “limitations”, and it is important to know them in advance, or ahead, of data analysis – indeed, before fieldwork. All extrapolations must have boundaries beyond which they cease to be empirically and logically valid. Under “Limitations” the researcher is also supposed to indicate the extent to which the intended scope may not be fully realized.

The implicit question under “Limitations” is this: given your scope, to what extent will your findings be generalizable; that is, to what extent beyond your actual area of study, or your sample? Thus, for a concrete example, if your study covers only one slum area in Nairobi, will this allow you to say anything meaningful or significant about Nairobi as a whole – or even about all slum areas of Nairobi? What about Kenya as a whole? What about other African cities, or other slums in other cities around the world? But Max Weber talked with some animation about ideal types -- ideal-typical cases whose defining characteristics can be inferred into all other yet-to-be-fully-studied cases which observers have a priori reason to believe that they “idealize” and, in turn, idealize them. How far we can legitimately take this is yet to be decided, and I will not engage in the debate right now.

Be that as it may, under “Limitations” the researcher points out the extent to which he/she will be able to generalize or extrapolate the findings, given the scope as already specified. With “Limitations”, therefore, the researcher acknowledges what he/she will not be able to do or cover, and briefly states why -- on logistical, theoretical and/or methodological grounds.

Limitations indicate the boundaries beyond which the findings do not legitimately apply -- or have no significance -- and should not be “stretched,” given the scope. Note that logistical and related problems likely to be encountered, or actually encountered, in the field can further narrow/restrict the boundaries of the possible. In principle, however, the researcher should not, in terms of “Limitations”, concern him/herself with detailing the financial or logistical or other “mundane” constraints likely to encumber the study. [To say all this in other words: By “Limitations of study” we mean to say that, given the scope, study findings will not be extrapolated or generalized beyond a certain boundary (usually implicated, or indeed made self-evident, in the scope). That is, in view of the scope, one is expected to briefly show the contexts/situations in which one’s evidence cannot be used, or is (is not) applicable; contexts/situations to which the evidence/conclusions/findings can (and cannot) be legitimately/profitably applied].

There is a very important exception to all of the foregoing. In a sample survey, extrapolations/ generalizations can and do (are indeed expected to) go beyond the stated scope of the sample to envelope the population (and only that population) of which the sample is statistically representative. Here, the limitations mirror not the boundaries of the sample but of the population of which it is a representative part. Herein lies a critical difference between what we can do with case-study data as opposed to sample-survey data.


Conclusion

Essentially, limitations and generalizability (or extrapolation) derive, follow from, scope. Given the scope – thai is, the sampling parameters of the study – the generalizability of the findings have certain limitations which the researcher is obliged or called upon to clearly state. The findings can be extrapolated to the relevant population (and only that population) only to the extent that the sample accurately represents it. In terms of ideal types, the yet-to-be-surmounted challenge is how to clearly establish, or have broad peer agreement on, the grounds upon which findings (or characteristics) related to the case studied can be, if at all they can, "projected" on to -- that is, generalized to -- the class of cases to which it ideal typically belongs.