brycewang-stanford/educational-research-methods

Educational Research Methods

A comprehensive skill for conducting rigorous educational research using both quantitative and qualitative methodologies. Covers study design, data collection instruments, analysis techniques, and reporting standards specific to education scholarship.

Study Design Frameworks

Quantitative Designs

Educational quantitative research typically follows one of these designs:

| Design | Purpose | Example | |--------|---------|---------| | Randomized controlled trial (RCT) | Causal inference | Random assignment to instruction methods | | Quasi-experimental | Causal inference without randomization | Pre-post comparison with matched control | | Correlational | Relationship exploration | Survey linking self-efficacy to GPA | | Longitudinal panel | Change over time | Tracking cohort achievement K-12 | | Cross-sectional survey | Snapshot description | National teacher satisfaction survey |

Qualitative Designs

Common qualitative traditions in education:

• Ethnography: Extended immersion in a classroom or school culture to produce thick description
• Case study: In-depth examination of a bounded system (a program, a school, a student)
• Grounded theory: Iterative coding to build theory from interview and observation data
• Phenomenology: Exploring the lived experience of participants (e.g., first-generation college students)
• Action research: Practitioners systematically studying their own practice to improve it

Mixed Methods

Sequential and concurrent mixed-methods designs are increasingly common in education research:

Sequential Explanatory:
  Phase 1: Quantitative survey (n=500) --> identify patterns
  Phase 2: Qualitative interviews (n=20) --> explain patterns

Concurrent Triangulation:
  QUAN data collection + QUAL data collection (simultaneous)
  --> merge and compare findings at interpretation stage

Embedded Design:
  Primary: RCT measuring learning outcomes
  Secondary: Classroom observations embedded within treatment arm

Data Collection Instruments

Survey and Questionnaire Design

import pandas as pd
from scipy import stats

# Reliability analysis for a Likert-scale instrument
def cronbach_alpha(df: pd.DataFrame) -> float:
    """
    Compute Cronbach's alpha for internal consistency reliability.
    df: DataFrame where each column is an item, each row a respondent.
    Acceptable threshold: alpha >= 0.70 for research purposes.
    """
    n_items = df.shape[1]
    item_vars = df.var(axis=0, ddof=1)
    total_var = df.sum(axis=1).var(ddof=1)
    alpha = (n_items / (n_items - 1)) * (1 - item_vars.sum() / total_var)
    return round(alpha, 4)

# Example usage with a 6-item motivation scale
data = pd.DataFrame({
    'item1': [4, 5, 3, 4, 5, 3, 4, 5],
    'item2': [3, 4, 3, 4, 5, 2, 4, 4],
    'item3': [4, 5, 4, 5, 4, 3, 5, 5],
    'item4': [3, 4, 2, 3, 5, 2, 3, 4],
    'item5': [4, 5, 3, 4, 5, 3, 4, 5],
    'item6': [3, 4, 3, 4, 4, 3, 4, 4],
})

alpha = cronbach_alpha(data)
print(f"Cronbach's alpha: {alpha}")
# alpha >= 0.70 indicates acceptable internal consistency

Observation Protocols

Structured classroom observation instruments:

• CLASS (Classroom Assessment Scoring System): Measures teacher-student interactions across emotional support, classroom organization, and instructional support
• RTOP (Reformed Teaching Observation Protocol): Evaluates inquiry-based instruction in STEM
• Flanders Interaction Analysis: Codes teacher talk, student talk, and silence in timed intervals

Interview Protocols

Semi-structured interview best practices for educational research:

1. Begin with rapport-building questions before moving to core topics
2. Use open-ended prompts: "Tell me about..." rather than yes/no questions
3. Prepare follow-up probes for each core question
4. Pilot the protocol with 2-3 participants and revise
5. Plan for 45-60 minute sessions to allow depth without fatigue

Analysis Techniques

Quantitative Analysis for Education Data

import statsmodels.api as sm
from statsmodels.formula.api import mixedlm

# Hierarchical Linear Model (HLM) -- essential for nested
# education data (students within classrooms within schools)
# Example: predicting math achievement from student SES
# and classroom teaching quality

model = mixedlm(
    "math_score ~ student_ses + teaching_quality",
    data=df,
    groups=df["school_id"],
    re_formula="~teaching_quality"
)
result = model.fit()
print(result.summary())

# Effect size calculation (Cohen's d)
def cohens_d(group1, group2):
    n1, n2 = len(group1), len(group2)
    var1, var2 = group1.var(), group2.var()
    pooled_std = ((( n1 - 1) * var1 + (n2 - 1) * var2) / (n1 + n2 - 2)) ** 0.5
    return (group1.mean() - group2.mean()) / pooled_std

Qualitative Coding

Thematic analysis workflow (Braun and Clarke, 2006):

1. Familiarization: Read transcripts multiple times, take initial notes
2. Initial coding: Generate codes systematically across the dataset
3. Theme search: Collate codes into candidate themes
4. Theme review: Check themes against coded extracts and full dataset
5. Theme definition: Refine names and write analytic narrative
6. Report: Select vivid, compelling quotes that capture each theme

Tools: NVivo, ATLAS.ti, MAXQDA, or open-source Taguette for coding.

Reporting Standards

APA and AERA Guidelines

Educational research follows the APA Publication Manual (7th edition) and the AERA Standards for Reporting on Empirical Social Science Research:

• Report effect sizes alongside p-values for all statistical tests
• Describe the sample demographics in detail (age, gender, race/ethnicity, SES)
• Discuss both statistical significance and practical significance
• For qualitative work, describe researcher positionality and reflexivity
• Include limitations section addressing threats to validity

Key Journals

• American Educational Research Journal (AERJ)
• Educational Researcher
• Journal of Educational Psychology
• Review of Educational Research
• Teaching and Teacher Education
• International Journal of Educational Research

Ethical Considerations

Educational research involving human subjects (especially minors) requires Institutional Review Board (IRB) approval. Key considerations include informed consent from parents/guardians, assent from minors, data de-identification, and equitable participant selection. The Belmont Report principles (respect for persons, beneficence, justice) guide all education research ethics.