Research ArticleSOCIAL SCIENCES

Challenges to capture the big five personality traits in non-WEIRD populations

See allHide authors and affiliations

Science Advances  10 Jul 2019:
Vol. 5, no. 7, eaaw5226
DOI: 10.1126/sciadv.aaw5226
  • Fig. 1 Prediction of income with the Big Five PTs and cognitive ability.

    The top panel presents the relationship between income and the different PTs. We estimated the coefficients and their 95th confidence intervals running the following regression separately for each country: yi=α0+β0cogi+PT=1PT=5βPTPTi+ϵi, where yi is the income of person i (transformed into the rank of income and scaled from 0 to 100). For cogi, the full literacy test was used when available, and the partial literacy test was used otherwise (see the Supplementary Materials). The last line pools all observations from all countries, controlling for the best measure of cognitive ability available in each country (country-fixed effects are not needed since relative rankings were calculated by country). All regressors are standardized, hence coefficients can be interpreted as the effect of 1 SD in the regressor on the percentile of the rank of income. The bottom panel is based on similar regressions, but replacing the five PTs by one index averaging the five values (for each observation). It only includes the nine countries with full literacy test included in the STEP database. The last line pools all observations from the nine countries. The coefficient of cognitive ability (literacy) is significantly higher than the coefficient of the Big Five index (at the 90% level) in four out of nine countries and in the pooled regression.

  • Fig. 2 Cronbach’s alpha as a function of the number of items by type of data.

    The estimates for the survey data are based on surveys with at least six items per PT, while the estimates for the internet data are based on data from the 14 STEP countries, using all 44 items of the BFI. The U.S. data also use all items of the BFI. For each dataset and for each number of items n, we calculate Cronbach’s alpha for every possible combination of n items before averaging it across all combinations and then averaging it across datasets (by type of data collection).

  • Fig. 3 Relationship between psychometric indicators and cognitive ability.

    In each figure, the level of observation is the largest possible geographical division in the country (regions, provinces, or district). We apply a weight that is the inverse of the number of geographical divisions to give the same weight to each country. The calculations of congruence coefficient, Cronbach’s alpha, absolute value of acquiescence bias, and enumerator bias are described in Materials and Methods. Enumerator bias measures the share of the variation in responses (by PT) that can be explained by systematic biases due to which enumerator administrated each survey. Cognitive ability is measured by the full literacy test, also described in the Supplementary Materials. The nine countries in the regression are the nine countries with full literacy test included in the STEP surveys.

  • Table 1 Congruence, factor structures obtained by PCA, and comparison with theoretical scale.

    Congruence coefficient (first column) is a proxy for the similarity of the factor structure, obtained from the correlation between factor loadings between two samples (in this case, the sample of the corresponding line is compared to factor loadings of the U.S. data). A detailed description of the calculation of the congruence coefficient is provided in Materials and Methods. In the rest of the table, each column represents one item (the same across datasets), sorted by PTs, and an “R” in its name means that it is a reverse-coded item (see Supplementary Materials section 1.1 for the phrasing of each item). The number that appears in each cell indicates for which factor the item has the highest factor loading in the PCA (with Procrustes rotation on U.S. data). Cell entries are in red bold font when the factor with highest loading differs from the one that the item aims to measure. For other surveys, only identifiers are provided to preserve anonymity. All data were corrected for acquiescence bias before the analysis.

    U.S. data
    Congruence
    coefficient
    OpennessConscientiousnessExtraversionAgreeablenessEmot. stability
    O1O2O3C1C2RC3E1E2RE3A1A2A3ES1RES2ES3R
    U.S. data1111222333444555
    STEP survey data
    Congruence
    coefficient
    OpennessConscientiousnessExtraversionAgreeablenessEmot. stability
    O1O2O3C1C2RC3E1E2RE3A1A2A3ES1RES2ES3R
    Ghana0.68411124333111555
    Kenya0.71214221333444555
    Sri Lanka0.71211121331444555
    Yunnan0.75114222333144555
    Laos0.70113421134444555
    Vietnam0.69111222433431555
    Philippines0.59311424331324455
    Bolivia0.84111222333444555
    Colombia0.72324212333444555
    Macedonia0.67114111333444525
    Serbia0.79111224333444555
    Ukraine0.81211242333411555
    Georgia0.77311122313441555
    Armenia0.82111122333441555
    Average0.73
    Other survey data
    Congruence
    coefficient
    OpennessConscientiousnessExtraversionAgreeablenessEmot. stability
    O1O2O3C1C2RC3E1E2RE3A1A2A3ES1RES2ES3R
    D10.78211222332454555
    D20.69235212333444515
    D30.92111222333444555
    D40.60344324334114515
    D50.67211124331444555
    D60.68211223235144555
    D70.66111244233454552
    D80.71211525333254542
    D90.73112242333144555
    Average0.71
    Internet data
    Congruence
    coefficient
    OpennessConscientiousnessExtraversionAgreeablenessEmot. stability
    O1O2O3C1C2RC3E1E2RE3A1A2A3ES1RES2ES3R
    Ghana0.97111222333444555
    Kenya0.86111222333444555
    Sri Lanka0.64312222333212555
    China0.98111222333444555
    Laos0.81115222333445555
    Vietnam0.98111222333444555
    Philippines0.85111242333444555
    Bolivia0.94111222333444555
    Colombia0.97111222333444555
    Macedonia0.76111222343444555
    Serbia0.98111222333444555
    Ukraine0.98111222333444555
    Georgia0.90111222333444555
    Armenia0.97111222333444555
    Average0.90
  • Table 2 Psychometric indicators by database.

    Datasets, psychometric measures, and different datasets and subsamples are described in detail in the main text and Supplementary Materials. All calculations were done after correcting for acquiescence bias. Within correlation, between correlation, Cronbach’s alpha, and congruence coefficient are first calculated by dataset before calculating a nonweighted average across all datasets. See table S6 for calculations for each dataset separately. In the case of within correlation, Cronbach’s alpha, and congruence coefficient, for each dataset, we first calculate it by PT and then average it across PT (before averaging across datasets).

    No. of
    items
    No. of
    datasets
    No. of
    observations
    Within
    correlation
    Between
    correlation
    Cronbach’s
    alpha (avg.
    of 5 PTs)
    Congruence
    coeff. (avg.
    of 5 PTs)
    Restricting each dataset to the 15 items available in step surveys
    STEP surveys151440,5840.260.090.490.73
    Other survey data15914,0510.170.080.350.71
    All survey data (that have the 15 STEP items)152354,6350.220.090.440.73
    Internet data1514198,3560.320.090.570.90
    U.S. data1516420.450.100.70-
    Using all items available in each dataset
    All survey data10 to 442994,7510.230.100.510.72
    Survey data with 44 items4466,0170.170.110.620.67
    Internet data4414198,3560.300.090.770.89
    U.S. data4416420.400.090.85-
    Subsample of respondents
    STEP surveys (tertiary education)15149,7470.260.090.490.73
    Internet data (number of observations ≤ STEP)151429,5280.320.090.570.87
    Internet data (age 26 to 48)151450,6220.330.100.570.89

Supplementary Materials

  • Supplementary material for this article is available at http://advances.sciencemag.org/cgi/content/full/5/7/eaaw5226/DC1

    Additional data description

    Fig. S1. Prediction of income with imperfect proxy of cognitive ability and the Big Five PTs.

    Table S1. Descriptive statistics of STEP data.

    Table S2. Descriptive statistics of other survey data and reference data.

    Table S3. Descriptive statistics of internet data.

    Table S4. Psychometric indicators by database, using data without correcting for acquiescence bias.

    Table S5. Cronbach’s alpha by PT, using STEP survey data, without versus with acquiescence bias correction.

    Table S6. Psychometric indicators by dataset.

    Table S7. Cronbach’s alpha by PT and database.

    Table S8. Average item-by-item correlation coefficients in different databases.

    Table S9. Psychometric indicators for Colombia, comparing randomly assigned face-to-face versus self-administrated surveys.

  • Supplementary Materials

    This PDF file includes:

    • Additional data description
    • Fig. S1. Prediction of income with imperfect proxy of cognitive ability and the Big Five PTs.
    • Table S1. Descriptive statistics of STEP data.
    • Table S2. Descriptive statistics of other survey data and reference data.
    • Table S3. Descriptive statistics of internet data.
    • Table S4. Psychometric indicators by database, using data without correcting for acquiescence bias.
    • Table S5. Cronbach’s alpha by PT, using STEP survey data, without versus with acquiescence bias correction.
    • Table S6. Psychometric indicators by dataset.
    • Table S7. Cronbach’s alpha by PT and database.
    • Table S8. Average item-by-item correlation coefficients in different databases.
    • Table S9. Psychometric indicators for Colombia, comparing randomly assigned face-to-face versus self-administrated surveys.

    Download PDF

    Files in this Data Supplement:

Stay Connected to Science Advances

Navigate This Article