Research ArticleEDUCATION RESEARCH

Assessing faculty professional development in STEM higher education: Sustainability of outcomes

See allHide authors and affiliations

Science Advances  18 Mar 2016:
Vol. 2, no. 3, e1501422
DOI: 10.1126/sciadv.1501422
  • Fig. 1 Teaching practice at the end of a semester.

    The frequency (mean ± SE) with which FIRST and non-FIRST faculty (n = 11 pairs) used various types of interactive activities during a semester (paired t test or Wilcoxon signed-rank test, *P < 0.05, **P < 0.01), as reported on the Teaching Practice Survey. “Discussion”—student discussions in pairs or small groups to answer a question; “Apply principles”—classroom interactions that required students to apply principles presented in class to a novel question; “Use data”—activities in which students use data to answer questions while working in small groups; “Clickers”—clicker questions that test conceptual understanding; “Evaluate their thinking”—individual writing activities that require students to evaluate their own thinking (1 = never; 2 = several times per semester; 3 = once per week; 4 = once per class; 5 = more than once per class).

  • Fig. 2 Comparison of RTOP scores per category.

    Distribution of RTOP scores for teaching videos of FIRST and non-FIRST faculty.

  • Fig. 3 Distribution of Bloom category levels for goals and assessments.

    Percentage (mean ± SE) of course goals as stated in the course syllabi (top) and assessment points per course (bottom) that were categorized into each Bloom category.

  • Table 1 Characteristics of matched pairs of a FIRST faculty participant and a non-FIRST faculty colleague.

    Knowledge of and experience with active learning, perception of departmental commitment, and challenges to implementing active learning were calculated as summed Likert responses to the Teaching Background Survey (Appendix S1). Large course sizes are those with >75 students per course. Data are presented as mean ± SE.

    GroupFemale
    participants (%)
    Teaching
    experience (years)
    Active-learning
    knowledge
    Active-learning
    experience
    Departmental
    commitment
    Active-learning
    challenge
    Large course
    size (%)
    FIRST562.1 ± 2.335.2 ± 2.334.8 ± 1.821.3 ± 4.851.5 ± 1.218
    Non-
    FIRST
    504.1 ± 4.335.3 ± 2.734.7 ± 2.523.2 ± 3.248.7 ± 2.418
  • Table 2 Regression analysis results for the effects of faculty participation in the FIRST project (treatment) on RTOP scores from expert reviews of faculty teaching.

    Model r2 = 0.65. Challenges to active-learning implementation were calculated as summed Likert responses to the Teaching Background Survey (Appendix S1). P values in the right-hand column are from two-tailed t tests.

    CoefficientParameter estimateSEPStandardized estimate
    Intercept61.1914.920.00040
    Treatment15.952.74<0.00010.73
    Gender2.063.160.520.09
    Class enrollment−0.050.030.13−0.22
    Challenges to active learning−0.430.300.16−0.18
  • Table 3 Comparison of FIRST participants who were selected for this paired study with nonselected FIRST participants upon completion of the FIRST program.

    Sample size is given in parentheses. Teaching experience is expressed as years before participation in FIRST. Knowledge of and experience with active learning and teaching confidence were calculated as summed Likert responses to survey questions identical to those in the Teaching Background Survey (Appendix S1). The RTOP score refers to the average RTOP score for videos of classroom teaching. Data are presented as mean ± SE.

    GroupFemale participants
    (%)
    Teaching experience
    (years)
    Active-learning
    knowledge
    Active-learning
    experience
    Teaching
    confidence
    RTOP score
    Paired study55 (n = 20)0.44 ± 0.1 (17)42.2 ± 2.9 (18)40.6 ± 2.4 (18)3.0 ± 0.2 (18)49.6 ± 1.4
    (20)
    Nonparticipants66 (n = 180)0.73 ± 0.1 (141)43.4 ± 0.9 (108)37.5 ± 0.8 (108)3.1 ± 0.1 (108)46.0 ± 0.7
    (150)
  • Table 4 Characteristics of instruments used in the paired study.

    Sample size refers to the number of faculty participants who submitted complete data for an instrument.

    InstrumentAcronymPurposeWhen usedItems and
    subscales
    Response
    type
    Sample size
    Teaching Background
    Survey (Appendix S1)
    BKParticipants’ confidence, knowledge of and
    experience with teaching and pedagogy
    Beginning of
    the study
    79 itemsOrdinal, writ
    ten
    18 pairs
    Approaches to Teaching
    Inventory (19, 20)
    ATIParticipants’ perceptions of teaching strategies used
    in a course
    Beginning of
    the course
    22 items,
    2 subscales
    Likert20 pairs
    Self-Efficacy
    Survey (26)
    SEParticipants’ confidence in their teaching abilityBeginning of
    the course
    4 itemsLikert20 pairs
    Teaching Practice
    Survey (Appendix S2)
    TPSParticipants’ perceived use of different classroom
    teaching practices and approaches to assessment
    End of the
    course
    30 itemsLikert11 pairs
    Experience of Teaching
    Questionnaire (40)
    ETQParticipants’ perceptions of environmental factors
    that are likely to influence teaching practices
    End of the
    course
    32 items,
    5 subscales
    Likert20 pairs
    Learning and Studying
    Questionnaire (25)
    LSQStudents’ perceptions of their reasons for taking the
    course and approaches to learning and studying
    Beginning of
    the course
    56 items,
    3 sections,
    5 subscales
    Likert13 pairs
    Experiences of
    Teaching and Learning
    Questionnaire (25)
    ETLQStudents’ perceptions of course demands and
    learning achieved, the teaching-learning
    environment, and their approaches to learning
    and studying
    End of the
    course
    77 items,
    4 sections
    Likert11 paired
    courses
    Reformed Teaching Ob
    servation
    Protocol (21)
    RTOPExpert ratings of the extent to which learner-centered
    teaching practices are used in a class
    End of the
    study
    25 items,
    5 subscales
    Ordinal19 FIRST facul
    ty, 17 non-FIRST
    faculty
  • Supplementary Materials

    This PDF file includes:

    • Appendix S1. Paired Teaching Background Survey.
    • Appendix S2. Teaching Practice Survey.

    Download PDF

    Files in this Data Supplement:

Navigate This Article