3
views
0
recommends
+1 Recommend
1 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Comparing new tools of artificial intelligence to the authentic intelligence of our global health students

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Introduction

          The transformative feature of Artificial Intelligence (AI) is the massive capacity for interpreting and transforming unstructured data into a coherent and meaningful context. In general, the potential that AI will alter traditional approaches to student research and its evaluation appears to be significant. With regard to research in global health, it is important for students and research experts to assess strengths and limitations of GenAI within this space. Thus, the goal of our research was to evaluate the information literacy of GenAI compared to expectations that graduate students meet in writing research papers.

          Methods

          After completing the course, Fundamentals of Global Health (INTH 401) at Case Western Reserve University (CWRU), Graduate students who successfully completed their required research paper were recruited to compare their original papers with a paper they generated by ChatGPT-4o using the original assignment prompt. Students also completed a Google Forms survey to evaluate different sections of the AI-generated paper (e.g., Adherence to Introduction guidelines, Presentation of three perspectives, Conclusion) and their original papers and their overall satisfaction with the AI work. The original student to ChatGPT-4o comparison also enabled evaluation of narrative elements and references.

          Results

          Of the 54 students who completed the required research paper, 28 (51.8%) agreed to collaborate in the comparison project. A summary of the survey responses suggested that students evaluated the AI-generated paper as inferior or similar to their own paper (overall satisfaction average = 2.39 (1.61–3.17); Likert scale: 1 to 5 with lower scores indicating inferiority). Evaluating the average individual student responses for 5 Likert item queries showed that 17 scores were < 2.9; 7 scores were between 3.0 to 3.9; 4 scores were ≥ 4.0, consistent with inferiority of the AI-generated paper. Evaluation of reference selection by ChatGPT-4o ( n = 729 total references) showed that 54% ( n = 396) were authentic, 46% ( n = 333) did not exist. Of the authentic references, 26.5% (105/396) were relevant to the paper narrative; 14.4% of the 729 total references.

          Discussion

          Our findings reveal strengths and limitations on the potential of AI tools to assist in understanding the complexities of global health topics. Strengths mentioned by students included the ability of ChatGPT-4o to produce content very quickly and to suggest topics that they had not considered in the 3-perspective sections of their papers. Consistently presenting up-to-date facts and references, as well as further examining or summarizing the complexities of global health topics, appears to be a current limitation of ChatGPT-4o. Because ChatGPT-4o generated references from highly credible biomedical research journals that did not exist, our findings conclude that ChatGPT-4o failed an important component in using information effectively. Moreover, misrepresenting trusted sources of public health information is highly concerning, particularly given recent experiences from the COVID-19 pandemic and more recently in reporting on the impact of, and response to natural disasters. This is a significant limitation of GenAI’s ability to meet information literacy standards expected of graduate students.

          Supplementary Information

          The online version contains supplementary material available at 10.1186/s13040-024-00408-7.

          Related collections

          Most cited references30

          • Record: found
          • Abstract: not found
          • Article: not found

          Towards a common definition of global health

            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found
            Is Open Access

            “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy

              Bookmark
              • Record: found
              • Abstract: found
              • Article: found
              Is Open Access

              Global health competencies and approaches in medical education: a literature review

              Background Physicians today are increasingly faced with healthcare challenges that require an understanding of global health trends and practices, yet little is known about what constitutes appropriate global health training. Methods A literature review was undertaken to identify competencies and educational approaches for teaching global health in medical schools. Results Using a pre-defined search strategy, 32 articles were identified; 11 articles describing 15 global health competencies for undergraduate medical training were found. The most frequently mentioned competencies included an understanding of: the global burden of disease, travel medicine, healthcare disparities between countries, immigrant health, primary care within diverse cultural settings and skills to better interface with different populations, cultures and healthcare systems. However, no consensus on global health competencies for medical students was apparent. Didactics and experiential learning were the most common educational methods used, mentioned in 12 and 13 articles respectively. Of the 11 articles discussing competencies, 8 linked competencies directly to educational approaches. Conclusions This review highlights the imperative to document global health educational competencies and approaches used in medical schools and the need to facilitate greater consensus amongst medical educators on appropriate global health training for future physicians.
                Bookmark

                Author and article information

                Contributors
                paz@case.edu
                Journal
                BioData Min
                BioData Min
                BioData Mining
                BioMed Central (London )
                1756-0381
                18 December 2024
                18 December 2024
                2024
                : 17
                : 58
                Affiliations
                [1 ]Master of Public Health Program, Department of Population and Quantitative Health Sciences, Case Western Reserve University School of Medicine, ( https://ror.org/051fd9666) Cleveland, OH USA
                [2 ]School of Medicine, Case Western Reserve University, ( https://ror.org/051fd9666) Cleveland, OH USA
                [3 ]Bioethics Department, Case Western Reserve University, ( https://ror.org/051fd9666) Cleveland, OH USA
                [4 ]Nutrition Department, Case Western Reserve University, ( https://ror.org/051fd9666) Cleveland, OH USA
                [5 ]Anthropology Department, Case Western Reserve University, ( https://ror.org/051fd9666) Cleveland, OH USA
                [6 ]Pathology Department, Case Western Reserve University, ( https://ror.org/051fd9666) Cleveland, OH USA
                [7 ]Division of Infectious Diseases and HIV Medicine, ( https://ror.org/051fd9666) Cleveland, OH USA
                [8 ]Center for Global Health and Diseases, Cleveland, OH USA
                [9 ]Department of Public Health, ( https://ror.org/04thj7y95) Cleveland, OH USA
                Article
                408
                10.1186/s13040-024-00408-7
                11656723
                39696442
                94ef2213-1810-490e-887b-3fb429f509cd
                © The Author(s) 2024

                Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

                History
                : 19 September 2024
                : 19 November 2024
                Categories
                Research
                Custom metadata
                © BioMed Central Ltd., part of Springer Nature 2024

                Bioinformatics & Computational biology
                Bioinformatics & Computational biology

                Comments

                Comment on this article

                scite_
                0
                0
                0
                0
                Smart Citations
                0
                0
                0
                0
                Citing PublicationsSupportingMentioningContrasting
                View Citations

                See how this article has been cited at scite.ai

                scite shows how a scientific paper has been cited by providing the context of the citation, a classification describing whether it supports, mentions, or contrasts the cited claim, and a label indicating in which section the citation was made.

                Similar content84

                Most referenced authors311