The data evaluation process in qualitative and quantitative research is complex, laborious and time consuming procedure. In social research there are two main approaches to analysis and evaluation of data: manual and computer based. In the former case qualitative research evaluation is treated as ‘intuitive, idiosyncratic and creative’ (Stroh 2000:226). Due to the immersive nature of the participant observation and closeness to a subject a researcher is inclined to see things from the member’s perspective. Thus Cohen and Mannion (1994:52) suggest evaluating materials by means of two stages: ‘external’ and ‘internal criticism’. External criticism is concerned with establishing the ‘authenticity’ (Scott 1990:37) or genuiness of material. It is aimed at the document itself rather than the statements it contains and endeavors to analyse forms of the data rather than the interpretation or meaning. That is way it sets out to discover frauds, inventions or distortions. A set of questions proposed by Platt (1981) can be employed to test observation material on its authenticity:

Does the document make sense or does it contain glaring errors?

Are there different versions of the original document available?

Is there consistency of literary style, handwriting or typeface?

Has the document been transcribed by many copyists?

Does the version available derive from a reliable source?

Internal criticism deals with the accuracy of the data presentation and an evaluator has to establish ‘credibility’, ‘representativeness’ and ‘meaning’ (Scott 1990:53) of the document.

Credibility refers to the question of whether the task is ‘free from error and distortion’ (Macdonald 2001:204). The later may occur when the comments and discussion were made long time after actual observation, or when the account has been made through different hands and the author was not present at the lesson. The task is considered to be representative if all the aspects of the task have been taken place in an accurate way. But missing of some categories might occur, then the question of what is missing, how much and why should be considered.

Representativeness can be affected by the interest or bias of the author to please the reader, or being under pressure, from fear or vanity the writer can distort or omit some facts.

The meaning of a document should be established at two levels: ‘the surface or literal meaning, and the deeper meaning arrived at … interpretative understanding or structural analysis’ (Macdonald 2001:205). The first type embodies the form of the text whereas the second one analyses the content of the message from the point of view of ‘tendencies, sequences, patterns, and orders’ (Ericson, Bareaneck, and Chan 1991:55). Arguably textual analysis should draw to discourse analysis and concentrates only on language features regardless of social setting. Whereas Scott (1990:64) claims that a text is deprived from its real meaning in isolation from the social context. So ‘texts must be studied as socially situated products’ (Scott 1990:65).

Computer-based evaluation

Computer application in qualitative research analysis arguably brings some organisation and system into unstructured material and various paper forms, but definitely is helpful in storing and managing a large amount of materials in ethnography and statistics in quantitative data collection. Sophisticated software packages have been generated for the last years, for example, the Ethnograph (Seidel), QSR NUP∙IST (Richards and Richards), Hyper-RESEARCH (Biber, Kinder) ATLAS/ti (Muhr), SPSS. Computer programmes are of great help for a researcher and can assist in simple functions such as text processing and speed search as in more complicated ones: coding or indexing words and further retrieving them, building theories, making descriptive statistics and inferential one. But Gayle (2000) admonishes that a researcher should remember that computers do not produce results as such, they ‘merely take some of the laborious data management tasks away from the researcher’ Gayle (2000:415).

Chapter 3