Research ArticleRESEARCH METHODS

Document co-citation analysis to enhance transdisciplinary research

See allHide authors and affiliations

Science Advances  03 Jan 2018:
Vol. 4, no. 1, e1701130
DOI: 10.1126/sciadv.1701130
  • Fig. 1 DCA.

    DCA illustration of the conversion of citation data (A) to a co-citation network (B) and the resulting node (C) and edge (D) metrics before and after trimming (E).

  • Fig. 2 Systems thinking document co-citation network.

    Results trimmed at the following co-citation frequency levels: (A) ≥3, (B) ≥5, (C) ≥7, and (D) ≥9. A key is provided in the lower right panel (E). Nodes represent co-cited documents with top co-cited documents among a community labeled by author(s) and year published. Node shape and color represent assigned community determined by smart local moving (SLM) detection for each network. Edges represent co-citations between documents with frequencies represented by width and color tone. Communities in the ≥3 network of fewer than three documents were not included in the visual because these four small communities were complete and isolated. Visualization was made with organic layout in Cytoscape (34).

  • Fig. 3 Subjects of co-citation communities.

    The top three WorldCat subject labels are shown for each of the main communities. The color, shape, and bolded number correspond to the co-citation communities in the ≥3 systems thinking network in Fig. 2A. Numbers of co-cited documents within each community that have a topic label are reported.

  • Fig. 4 Comprehensive co-citation network.

    A co-citation network generated from comprehensive search criteria with edges trimmed to frequencies of three or more co-citations. Documents matching those in the ≥3 systems thinking network of Fig. 2A are colored blue. Top co-cited documents from Table 1 are labeled.

  • Table 1 Highly co-cited documents.

    Top three co-cited documents among seven assigned communities for the ≥3 network in Fig. 2A. Communities containing fewer than three documents are omitted.

    CommunityReference to co-cited documentTimes citedDegree
    0P. M. Senge, The Fifth Discipline: The Art and Practice of the Learning Organization (Doubleday and Company, 1990).6290
    J. W. Forrester, Industrial Dynamics (Massachusetts Institute of Technology, 1961).2946
    J. D. Sterman, Business Dynamics: Systems Thinking and Modeling for a Complex World (Irwin/McGraw-Hill, 2000).2817
    1P. Checkland, Systems Thinking, Systems Practice (Wiley, 1981).59111
    R. L. Ackoff, Creating the Corporate Future: Plan or be Planned for (Wiley, 1981).1847
    P. Checkland, J. Scholes, Soft Systems Methodology in Action (Wiley, 1990).2542
    2W. Ulrich, Critical Heuristics of Social Planning: A New Approach to Practical Philosophy (P. Haupt, 1983).2384
    C. W. Churchman, The Systems Approach (Delacorte Press, 1968).1865
    C. W. Churchman, The Design of Inquiring Systems: Basic Concepts of Systems and Organization (Basic Books, 1971).1549
    3O. Ben Zvi Assaraf, N. Orion, Development of system thinking skills in the context of earth system education. J. Res. Sci. Teach. 42 (5), 518–560 (2005).923
    M. J. Jacobson, U. Wilensky, Complex systems in education: Scientific and educational importance and implications for the learning sciences. J. Learn. Sci. 15 (1), 11–34 (2006).821
    M. Frank, Engineering systems thinking and systems thinking. J. Syst. Eng. 3 (3), 163–168 (2000).821
    4M. C. Jackson, Systems Methodology for the Management Sciences (Plenum Press, 1991).2483
    R. L. Flood, M. C. Jackson, Creative Problem Solving: Total Systems Intervention (Wiley, 1991a).2469
    R. L. Flood, M. C. Jackson, Critical Systems Thinking: Directed Readings (J. Wiley, 1991b).1450
    5L. von Bertalanffy, General System Theory: Foundations, Development, Applications (George Braziller, 1968).2634
    M. Mulej, R. Espejo, M. C. Jackson, S. Kajzer, J. Mingers, P. Mlakar, N. Mulej, V. Potočan, M. Rebernik, A. Rosicky, B. S. Umpleby, D. Uršič, R. Vallee, Dialektična in druge mehkosistemske teorije: (podlage za celovitost in uspeh managementa) (Ekonomsko-poslovna fakulteta, 2000).410
    M. Davidson, Uncommon Sense: The Life and Thought of Ludwig von Bertalanffy (1901–1972), Father of General Systems Theory (Tarcher, 1976).49
    6S. J. Leischow, A. Best, W. M. Trochim, P. I. Clark, R. S. Gallagher, S. E. Marcus, E. Matthews, Systems thinking to improve the public’s health. Am. J. Prev. Med. 35 (2), S196–S203 (2008).75
    J. B. Homer, G. B. Hirsch, System dynamics modeling for public health: Background and opportunities. Am. J. Public Health 96 (3), 452–458 (2006).45
    W. M. Trochim, D. A. Cabrera, B. Milstein, R. S. Gallagher, S. J. Leischow, Practical challenges of systems thinking and modeling in public health. Am. J. Public Health 96 (3), 538–546 (2006).54
  • Table 2 Validation.

    Validation results of the systems thinking network are displayed for each of the following trim levels: three, five, seven, and nine or more co-citations. The number of documents and the number of co-citations in each network are indicated. Internal consistency is reported as Spearman’s rank correlations of times cited by source documents to degree of co-citation. Community validity was tested using a χ2 test for independence between assigned network communities and subject communities. Stability was measured as the number of co-cited documents in the comprehensive network also found in the systems thinking network, and a Spearman’s rank correlation of the degree of co-citation for documents matched between the two networks.

    Systems thinking network
    trim levels
    ≥3≥5≥7≥9
    Network metrics
    Co-cited documents (number of nodes)246713519
    Co-citations (number of edges)1,29227110544
    Internal consistency
    Spearman’s value (S)1,099,36915,4682151584
    P value<0.001<0.001<0.0010.034
    ρ0.560.740.700.49
    Community validity
    X2494.5585.4045.1617.47
    Degrees of freedom (df)280483012
    P value<0.001<0.0010.0370.13
    Stability to comprehensive network
    Number of matching documents68362418
    S26,6134,4591716510
    P value<0.0010.00950.250.47
    ρ0.450.420.250.47
  • Table 3 Steps adapted from a general process for mapping knowledge domains were implemented to build a co-citation network from bibliographic data.
    StepGeneral process (29)Implementation in this study
    1. Data acquisitionSelect an appropriate data source.Search the Web of Science Core Collection for articles from different research areas whose titles contain “system(s) thinking” to export database entries.
    2. ProcessingSelect a unit of analysis and extract the necessary data from the selected sources.Select cited reference list from each document’s bibliographic entry and use R to merge duplicate citations for co-citation analysis.
    3. AnalysisChoose an appropriate similarity measure and then calculate similarity values.Calculate co-citation network using Science of Science (Sci2) and apply multiple thresholds to reveal different co-citation levels.
    4. VisualizationCreate a data layout using a clustering or ordination algorithm.Perform SLM community detection to group co-cited documents and use Cytoscape to visualize the network, communities, and co-cited documents.

Supplementary Materials

  • Supplementary material for this article is available at http://advances.sciencemag.org/cgi/content/full/4/1/e1701130/DC1

    fig. S1. Scatterplot of times cited and degree of co-citation for ≥3 systems thinking network.

    fig. S2. Network representation of co-cited documents organized as subject communities.

    table S1. Tabulation of documents from identified co-cited communities (≥3 network) to identified subject communities (fig. S2).

    table S2. An annotated bibliography of six useful resources for understanding DCA and other types of bibliographic networks.

  • Supplementary Materials

    This PDF file includes:

    • fig. S1. Scatterplot of times cited and degree of co-citation for ≥3 systems thinking network.
    • fig. S2. Network representation of co-cited documents organized as subject communities.
    • table S1. Tabulation of documents from identified co-cited communities (≥3 network) to identified subject communities (fig. S2).
    • table S2. An annotated bibliography of six useful resources for understanding DCA and other types of bibliographic networks.

    Download PDF

    Files in this Data Supplement:

Navigate This Article