Tuesday, March 06, 2012

Scope and Limitations in Scientific Research



Introduction

Much confusion surrounds the use of the terms “scope” and “limitations” in scientific research. In any scientific study that we may undertake, it is important that this confusion be cleared before the actual research begins, than after fieldwork – or, worse still, when the findings, having been otherwise processed and analyzed with self-assurance, are being reported to an informed and skeptical audience.

Scope

Scope refers to the portion/proportion of the totality [the content] suggested by the research topic and the problem statement that the researcher intends to cover, or will actually cover. It incorporates the giving of the “reason why”; that is, why the scope is as we are called upon to see it – in terms of Time, Space and/or Number. “Number” here could mean the quantity or size of a variety of factors, such as: sample, the population in which the sample is embedded, social stratum, cohort, age-group, gender group, occupational group, or some other category of “subject.”

The researcher’s task under “Scope” is to specify how much and/or what aspects of the problem or relevant subject-matter is to be tackled, and how large the study area and population will be. In other words, what the researcher states under “Scope” is exactly how much research he/she is going to do [or has to do] to realize the research objectives. This helps to clarify both the sampling frame and the range of hypotheses that one can reasonably expect to test, or the research/guiding questions that one may address. Under “scope”, therefore, the researcher briefly specifies the “area” to be covered, and/or the boundaries  [the Breadth, Length and Depth – in terms of Time, Space and Number] within which the study will be confined.

Limitations

In extrapolating/generalizing research findings, one must know the boundaries of the possible, or of the acceptable claim – even if the claim be in all other ways evidence-based. This is what we call “limitations”, and it is important to know them in advance, or ahead, of data analysis – indeed, before fieldwork. All extrapolations must have boundaries beyond which they cease to be empirically and logically valid. Under “Limitations” the researcher is also supposed to indicate the extent to which the intended scope may not be fully realized.

The implicit question under “Limitations” is this: given your scope, to what extent will your findings be generalizable; that is, to what extent beyond your actual area of study, or your sample? Thus, for a concrete example, if your study covers only one slum area in Nairobi, will this allow you to say anything meaningful or significant about Nairobi as a whole – or even about all slum areas of Nairobi? What about Kenya as a whole? What about other African cities, or other slums in other cities around the world? But Max Weber talked with some animation about ideal types -- ideal-typical cases whose defining characteristics can be inferred into all other yet-to-be-fully-studied cases which observers have a priori reason to believe that they “idealize” and, in turn, idealize them. How far we can legitimately take this is yet to be decided, and I will not engage in the debate right now.

Be that as it may, under “Limitations” the researcher points out the extent to which he/she will be able to generalize or extrapolate the findings, given the scope as already specified. With “Limitations”, therefore, the researcher acknowledges what he/she will not be able to do or cover, and briefly states why -- on logistical, theoretical and/or methodological grounds.

Limitations indicate the boundaries beyond which the findings do not legitimately apply -- or have no significance -- and should not be “stretched,” given the scope. Note that logistical and related problems likely to be encountered, or actually encountered, in the field can further narrow/restrict the boundaries of the possible. In principle, however, the researcher should not, in terms of “Limitations”, concern him/herself with detailing the financial or logistical or other “mundane” constraints likely to encumber the study. [To say all this in other words: By “Limitations of study” we mean to say that, given the scope, study findings will not be extrapolated or generalized beyond a certain boundary (usually implicated, or indeed made self-evident, in the scope). That is, in view of the scope, one is expected to briefly show the contexts/situations in which one’s evidence cannot be used, or is (is not) applicable; contexts/situations to which the evidence/conclusions/findings can (and cannot) be legitimately/profitably applied].

There is a very important exception to all of the foregoing. In a sample survey, extrapolations/ generalizations can and do (are indeed expected to) go beyond the stated scope of the sample to envelope the population (and only that population) of which the sample is statistically representative. Here, the limitations mirror not the boundaries of the sample but of the population of which it is a representative part. Herein lies a critical difference between what we can do with case-study data as opposed to sample-survey data.


Conclusion

Essentially, limitations and generalizability (or extrapolation) derive, follow from, scope. Given the scope – thai is, the sampling parameters of the study – the generalizability of the findings have certain limitations which the researcher is obliged or called upon to clearly state. The findings can be extrapolated to the relevant population (and only that population) only to the extent that the sample accurately represents it. In terms of ideal types, the yet-to-be-surmounted challenge is how to clearly establish, or have broad peer agreement on, the grounds upon which findings (or characteristics) related to the case studied can be, if at all they can, "projected" on to -- that is, generalized to -- the class of cases to which it ideal typically belongs.