Experimental Design

1. Naturalistic observation is a research tool in which a subject is observed in its natural habitat without any manipulation by the observer.
2. Participant observation is one type of data collection method. The aim is to gain a close and intimate familiarity with a given group of individuals through an intensive involvement in their cultural environment, usually over an extended period of time.
3. A case study involves an up-close, in-depth, and detailed examination of a subject, as well as its related contextual conditions.
4. An existence proof is a demonstration that a given psychological phenomenon can occur.
5. In the design of experiments, the experimental group is the group of participants that receives the manipulation.
6. In an experiment, the control group is the group of participates who receive either no manipulation or a standard manipulation.
7. Random assignment is an experimental technique for assigning human participants or animal subjects to different groups in an experiment by a chance procedure.
8. A blind is an experiment in which information about the test is kept from the participant until after the test.
9. Double-blind describes an especially stringent way of conducting an experiment which attempts to eliminate subjective, unrecognized biases carried by both an experiment's subjects (usually human) and its conductors.
10. In the design of an experiment, the independent variable represents the input or cause. This is the variable that the experimenter manipulates.
11. In the design of an experiment, the dependent variable represents the output or effect. It is the variable that an experimenter measures to see whether the manipulation has an effect.


A simple confounding case: in this model, given Z, there is no association between X and Y. However, not observing Z will create spurious association between X and Y. Z is called a confounding factor.

12. A confound is an extraneous variable in a statistical model that correlates with both the dependent variable and the independent variable. In experimental design, it is any difference between the experimental and control groups other than the independent variable.
13. With positive correlation, the greater value of one variable mainly corresponds with a greater value of the other variable.
14. With negative correlation, a greater value of one variable mainly corresponds with a lower value of the other variable.
15. A scatter plot is a type of mathematical diagram using Cartesian coordinates to display values as a set of points for typically two variables for a set of data.
    
    

    Waiting time between eruptions and the duration of the eruption for the Old Faithful Geyser in Yellowstone National Park, Wyoming, USA. This chart suggests there are generally two 'types' of eruptions: short-wait-short-duration, and long-wait-long-duration.

    
    
16. Illusory correlation is the phenomenon of perceiving a relationship between variables (typically people, events, or behaviors) even when no such relationship exists.


The placebo effect can be produced by inert tablets, by sham surgery, and by false information, such as when electrical stimulation is turned 'off' in those with Parkinson's disease implanted brain electrodes.

17. Sometimes patients given a simulated or otherwise medically ineffectual treatment will have a perceived or actual improvement in a medical condition, a phenomenon commonly called the placebo effect.
18. In a balanced placebo design, half of the participants are told they are receiving the active drug and half are told they are receiving the placebo - but only half of the participants in each group actually receive the drug or placebo as told, permitting independent and combined assessment of drug and placebo effects.
19. A nocebo is an inert substance or form of therapy that creates harmful effects in a patient.
20. External validity is the validity of generalized (causal) inferences in scientific research, usually based on experiments. It is the extent to which the results of a study can be generalized to other situations and to other people.
21. Internal validity is a property of scientific studies which reflects the extent to which a causal conclusion based on a study is warranted. Such warrant is constituted by the extent to which a study minimizes systematic error or bias.
22. Face validity is the extent to which a test is subjectively viewed as covering the concept it purports to measure. It refers to the transparency or relevance of a test as it appears to test participants.
23. Content validity refers to the extent to which a measure represents all facets of a given social construct.
24. Construct validity is the degree to which a test measures what it claims, or purports, to be measuring.
25. Criterion validity is the extent to which a measure is related to an outcome.
26. Incremental validity describes how much a new psychometric assessment increases the predictive ability beyond that provided by an existing method of assessment, whether the new test adds much information that might be obtained with simpler, already existing methods.
    
    

    
    
27. In the psychometrics, reliability is the overall consistency of a measure, whether the measure produces similar results under consistent conditions.
28. Test-retest reliability is the variation in measurements taken by a single person or instrument on the same item, under the same conditions, and in a short period of time.
29. In statistics, inter-rater reliability is the degree of agreement among raters. It gives a score of how much homogeneity, or consensus, there is in the ratings given by judges.
30. The fallacy of positive instances is the act of pointing to individual cases or data that seem to confirm a particular position, while ignoring a significant portion of related cases or data that may contradict that position.
31. The idea behind the practice of meta-analysis is that there is a common truth behind all conceptually similar studies. The aim is to use approaches from statistics to derive a pooled estimate.
32. Publication bias is a type of bias with regard to what academic research is likely to be published among what is available to be published. This type of bias can affect the outcome of literature reviews of claims about support for a hypothesis.




A funnel plot expected without the file drawer problem. The largest studies converge at the tip while smaller studies show more or less symmetrical scatter at the base.



A funnel plot expected with the file drawer problem. The largest studies still cluster around the tip, but the bias against publishing negative studies has caused the smaller studies as a whole to have an unjustifiably favorable result to the hypothesis.



33. The Pygmalion effect, or Rosenthal effect, is the phenomenon whereby higher expectations lead to an increase in performance.
34. The experimenter expectancy effect occurs when researchers' hypotheses causes them to unintentionally bias the outcome of a study.
35. The Hawthorne effect (also referred to as the observer effect) is a type of reactivity in which individuals modify or improve an aspect of their behavior in response to their awareness of being observed.
36. Demand characteristics refers to an experimental artifact where participants form an interpretation of the purpose of an experiment and subconsciously change their behavior to fit that interpretation.
37. Reactivity is a phenomenon that occurs when individuals alter their performance or behavior due to the awareness that they are being observed. It is a significant threat to a research study's internal validity
38. A type of study with validity problems, a self-report study is a type of survey, questionnaire, or poll in which respondents read the question and select a response by themselves without researcher interference.
39. A field of applied statistics, survey methodology studies the sampling of individual units from a population and the associated data collection techniques, such as questionnaire construction and methods for improving the accuracy of responses.
40. Halo effect is a cognitive bias in which an observer's overall impression of a person, company, brand, or product influences the observer's feelings and thoughts about that entity's character or properties.
41. The leniency effect is the tendency of raters to provide ratings that are overly generous.
42. The error of central tendency is the unwillingness of raters to provide extreme (very high or very low) ratings.