Introduction

 

Research, according to Hatch and
Lazaraton (1991), means trying to provide answers to important questions or
problems in an organized, systematic way that generally leads to discovery or
interpretation of facts. Research is therefore believed to contribute profoundly
to knowledge in particular fields. In order to do successful research which
yield valid and reliable results, methodologies need to be carefully chosen and
should reflect the researcher’s overall orientation to their research. In this
essay, I will give a commentary on two most popular data collection methods –
questionnaires and observation. I will begin with a brief definition of two
methods, explaining when and why each is used in certain context and drawing on
my own personal experience as a researcher. Their strengths and weaknesses will
then be discussed and put in comparison when necessary. Lastly, I will identify
any potential problems when using these two methods.

We Will Write a Custom Essay Specifically
For You For Only $13.90/page!


order now

 

1.
When to use and why to use?

1.1
Questionnaire

Defined by Brown (2001:6) as written
instruments that give participants structured, predefined questions or
statements “to which they are to react either by writing out their
answers or selecting them among existing answers”, questionnaire is possibly
the most commonly used method in surveys (Dörnyei, 2007). According to Young
(2016), questionnaires are mostly used to report participants’ background, behaviours
and demographic information. Wallace (1998) adds that questionnaires can be
used to elicit various kinds of data, such as personal perceptions, experiences,
or attitudes. In a classroom context, Wallace argues that questionnaires might
be favourable when the researcher wants to touch upon “knowledge, opinions,
ideas, and experiences of learners, fellow teachers or parents” (1998:124). This
method tends to produce typically quantitative data.

 

 

1.2
Observation

Rather than collecting self-report
data like questionnaires, which primarily bases on what respondents tell the
researcher (Dörnyei, 2007), observation is a direct method that “draws on the
direct evidence of the eye to witness events first hand” (Denscombe: 2007:192).
This calls for a more active role of researchers who need to go in quest of
information and observe what actually happens. Observation is particularly
appropriate for classroom research, in which many aspects of teaching,
learning, or interaction could be usefully investigated. According to Hopkins (1993),
observation plays a vital role in teachers’ professional development. This is
because this method may attest how fixed values, beliefs or opinions have been
applied in actual setting; or how teaching approaches are put into practice with
their real effectiveness (Heighham & Croker, 2009). In agreement with this
viewpoint, Anderson, Herr and Nihlen (1994) claims that observation can be a
useful tool to “help demystify what is actually going on as opposed to what one
might hope or assume is happening” (p.129).

It should also be noted that
observation might be used in conjunction with other methods to add more
evidence and increase the validity of findings (Heighham and Croker, 2009).
Followings are two Bachelor’ theses in my university.

a.      
Ways
teachers deal with students’ oral errors at the production stage and their
effects – a case study in APAX English center.  

b.      The
application of task-based approach to English Speaking activities for first
year fast track students by the teachers of English in FELTE, ULIS, VNU

Rather than being used exclusively on
its own, observation was employed along with interviews and/or questionnaires
in these research. This method offered preliminary information about the respondents’
external behaviours and was followed up by surveys which investigate their
inner beliefs, gaining more insights into the issues.

 

2.
Strengths and Weaknesses

2.1
Questionnaires

2.1.1
Strengths:

Many researchers reach a
consensus on the wide coverage of questionnaires, which gives it an edge over
other methods (Denscombe, 2007; McDonough & McDonough, 1997). It helps
gathering considerable amount of information, from a large number of
respondents and by different means of collecting data in a cost-effective way. Additionally,
Dornyei (2007) claims that questionnaires are a highly versatile tool of survey
which allows the use of both closed-ended and open-ended questions, covering
various topics. Secondly, the value of data is at its best thanks to the use of
“pre-coded answers” from participants, which “fit into a range of options
offered by the researcher” (Denscombe, 2007:159). These data can later be used
to identify any major differences among different research’s findings, to
confirm, reject or form new hypothesis (Kelley et al, 2003). Lastly, Denscombe
emphasizes that not only having advantage for the researchers, questionnaires
are also respondent-friendly since they only need to choose from existing
options which are “spelt out for them” (2007:159) without much time or effort
invested in thinking of the answers. 

 

2.1.2
Weaknesses

There are some main criticisms
regarding the validity of questionnaires. 
First, Dornyei (2007:115) argues that questionnaires could only provide
“thin description of the target phenomena”, which may lead to “superficial
data” and limits in-depth investigation. In accordance with Dornyei’s perspective,
Young (2016) shows that questionnaires only present the issue on the surface
level; for example, the ranking system in a questionnaire may reflect more or
less of different categories but neither indicates nor explains the extent of
how much more or less, in which different researchers may have different
interpretations. Another weakness of questionnaire, pointed out by Wallace
(1998:124) is that this “introspective” method only provides what the
respondents choose to tell. Consequently, checking the reliability and
truthfulness of responses remains a constant challenge.

Furthermore, heavily
relying on self-report, questionnaires may face the presence of response bias
such as the social desirability bias (or prestige bias) which shows the
tendency of people to choose acceptable or desirable answers which do not tell
“what they actually feel or believe” (Dornyei, 2003:12). Another bias detected
by Paulhaus (1991) is acquiescence bias, indicating respondents who are more
inclined to positive connotation and reluctant to provide strong negative
responses. Lastly, contributing to response bias, the halo effect concerns the
human tendency of overgeneralization, showing how the overall impression of
something or somebody may influence respondent’s opinions or attitudes when it
comes to specific details (Oppenheim, 2000). These bias, mainly coming from the
respondents, may heavily affect the validity of the questionnaire. Denscombe
(2007) also points out another potential for the presence of bias which creeps
in the pre-coded questions. He argues that questionnaires may reflect the
researcher’s mindset rather than respondents’ thinking since pre-coded
questions “shape the nature of the responses”, resulting in answers being directed
away from the respondents’ “perceptions of matters to fit in with a line of
thinking established by the researcher”
(p.160).

 

2.2 Observation

2.2.1 Strengths

Similar to questionnaires, observation
also offer a large amount of information regarding participants’ behaviour in a
specific setting (Mackey & Gass, 2005). However, if questionnaires tend to miss the insights that could be
provided by the participants, the main strength of observational data is that
it can “allow the researcher to gain deeper and more multi-layered
understanding of the participants and their context” under investigation
(Mackey & Gass, 2005:176). Moreover, Dornyei (2007) believes that allowing
researchers to immerse themselves in the setting could present more objective
data than “second-hand self-report” one (p.185). Interestingly, he claims that
two approaches of collecting data, namely the unstructured and highly
structured approach, might sometimes be combined to reach the best effect. The
former, often associated with participant observation and believed to generate
qualitative data, tries to find out “not just what happens but why it happens”
(Dunkerton, 1981:145). However, since this depends primarily on the subjective
view of the researcher, the reliability of data may be doubted. On the other
hand, a highly structured approach, which usually refers to systematic
observation, decides the focus of research beforehand through the use of
observation scheme. Though the focus is on overt behaviours, not on why they
happen, the data associated with “selective perception of observers” may
generate objective observations, reducing observer’s bias, for example,
emotions or personal background. (Denscombe, 2007:199). In line with Dornyei’s (2007),
Dunkin and Biddle (1974) proposes a combination of both approaches, which was
effectively applied in Case a (part 1.2). For the first time, an unstructured
approach was adopted; I acted as a teaching assistant being involved in the
classroom’s activity. For the second attempt, I then constructed a
data-collection scheme on the premise of the collected information and acted as
a complete observer. This proved to be an effective blend as the chance to miss
interesting events is likely to decrease, with a more focus on what is of
importance of the research.  

 

2.2.2
Weaknesses

Mackey and Gass (2005) assumes that observation
does not reveal motivation behind participants’ particular actions or behaviours,
in contrast with questionnaires which score high in terms of clear indication
of motivational sources. Another weakness of observation, identified by
Denscombe (2007:193), relates to the issue of observer’s “psychology of memory
and perception”. To be more specific, he mentions three psychological factors
that may greatly influence observation’s direction: “selective recall” relates
to the limited capacity of memory, deterring researcher from remembering all
situations that are being observed; “selective perception” explains how some
specific information is retained why some is rejected; and “accentuated
perception” which shows how physical and mental condition and past experiences
may affect the observation process. These might be legitimate reasons why
findings are interpreted differently among researchers, which may somehow
decrease the research’s reliability.

3.
Problems/issues that need to be addressed

3.1
Questionnaires

Designing
well-constructed questionnaires is a process that requires careful attention
since researchers barely have opportunities to make adjustments once the
questionnaire has been distributed to the participants. There exists problems
during the designing process and after the questionnaires are collected to be analysed.

As for the designing
process, Wallace (1998) and Nunan (1992) emphasizes that the ambiguity of the
questions may cause misunderstanding and misinterpretation for participants. Sharing
the same viewpoint, Young (2016) claims that items wording in the questions can
assume “an unexpected importance”, in which just minor differences can trigger
“different levels of agreement or disagreement” (p.103), particularly in the
case when non-factual data such as attitudinal data is assessed. Another problem
lies in the use of Likert scale. Worcester and Burns (1975) found that a
balanced four-point Likert scale without midpoint pushed respondents,
especially those who slightly lean towards either agreement or disagreement to
the positive end of scale. However, McDonough (1997) assumes that the existence
of the neutral response might be hard to interpret. Likewise, the no-opinion
option should also be used with caution. On the one hand, Schuman and Pressure
(1981) advocates the use of this option since it may reduce respondents’
pressure who have no relevant knowledge to answer the question. On the other
hand, opponents of no-opinion option argue that it may not enhance data quality
and is likely to “preclude measurement of some meaningful opinions” (Krosnick
et al, 2002:371). The researcher then needs to be careful when deciding whether
or not to offer this type of option in the questionnaire.

When questionnaires are
collected for analysis, the issue of the comparability of the translated
version should also be taken into consideration. This back-translation process,
according to McDonough (1997) must ensure to have closest meaning to the
original version to guarantee reliability.

 

3.2
Observation

There are two major
concerns while undertaking this kind of method. 

The first issue relates to the transcription
process. Different from data of questionnaires which can be relatively quickly
processed, by either the researcher or the use of any data analysis software, presenting
observational data remains a challenge. Bailey (2008) has listed several
difficulties such as how to choose contextual data that needs to be interpreted
or how data should be presented, since it is not an easy task to “represent the
full complexity of human interaction” on a written transcript (p.130).

Secondly, as natural setting is
highlighted as a condition for observation to take place (Denscombe, 2007), it
is advisable to avoid disrupting the naturalness of the setting to see things
as they normally occur. Denscombe (2007) notices the observer effect – the
presence of the observer in the context- may make participants “aware of being studied- conscious of it- and then react in a way that is
not usual”, or “embarrassed, or disguise normal practice” (p.66). Regarding
this effect as “observer paradox”, Richards (2003:108) suggests that the use of
note-taking should not be made apparent to the participant. He also affirms
that the relationship between the observer and the participant should also be
given more thoughts, considering how it might affect individual and group’s
performances, and ultimately the research’s findings.

 

Conclusion

In this essay, I have discussed several strengths,
weaknesses and relevant issues of two research methodologies: questionnaires
and observation, each of which is a very common research tool that is
specifically designed for certain purposes. In many cases one data collection
method might be complemented by the use of other methods for triangulation – a
powerful technique to add depth to the data analysis as well as improve the
validity and reliability of the research. Both aforementioned methods demand
great attention and subtlety in many aspects of designing process. Being aware
of the potential problems researchers might face when adopting these methods is
then essential to raise the standards of good research practice.  

x

Hi!
I'm Erica!

Would you like to get a custom essay? How about receiving a customized one?

Check it out