Can ChatGPT be the Plastic Surgeon's New Digital Assistant? A Bibliometric Analysis and Scoping Review of ChatGPT in Plastic Surgery Literature

We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.

To develop a short instrument, called DISCERN, which will enable patients and information providers to judge the quality of written information about treatment choices. DISCERN will also facilitate the production of new, high quality, evidence-based consumer health information. An expert panel, representing a range of expertise in consumer health information, generated criteria from a random sample of information for three medical conditions with varying degrees of evidence: myocardial infarction, endometriosis, and chronic fatigue syndrome. A graft instrument, based on this analysis, was tested by the panel on a random sample of new material for the same three conditions. The panel re-drafted the instrument to take account of the results of the test. The DISCERN instrument was finally tested by a national sample of 15 information providers and 13 self help group members on a random sample of leaflets from 19 major national self help organisations. Participants also completed an 8 item questionnaire concerning the face and content validity of the instrument. Chance corrected agreement (weighted kappa) for the overall quality rating was kappa = 0.53 (95% CI kappa = 0.48 to kappa = 0.59) among the expert panel, kappa = 0.40 (95% CI kappa = 0.36 to kappa = 0.43) among information providers, and kappa = 0.23 (95% CI kappa = 0.19 to kappa = 0.27) among self help group members. Higher agreement levels were associated with experience of using the instrument and with professional knowledge of consumer health information. Levels of agreement varied across individual items on the instrument, reflecting the need for subjectivity in rating certain criteria. The trends in levels of agreement were similar among all groups. The final instrument consisted of 15 questions plus an overall quality rating. Responses to the questionnaire after the final testing revealed the instrument to have good face and content validity and to be generally applicable. DISCERN is a reliable and valid instrument for judging the quality of written consumer health information. While some subjectivity is required for rating certain criteria, the findings demonstrate that the instrument can be applied by experienced users and providers of health information to discriminate between publications of high and low quality. The instrument will also be of benefit to patients, though its use will be improved by training.

The Internet is a widely used information resource for patients with inflammatory bowel disease, but there is variation in the quality of Web sites that have patient information regarding Crohn's disease and ulcerative colitis. The purpose of the current study is to systematically evaluate the quality of these Web sites. The top 50 Web sites appearing in Google using the terms "Crohn's disease" or "ulcerative colitis" were included in the study. Web sites were evaluated using a (a) Quality Evaluation Instrument (QEI) that awarded Web sites points (0-107) for specific information on various aspects of inflammatory bowel disease, (b) a five-point Global Quality Score (GQS), (c) two reading grade level scores, and (d) a six-point integrity score. Thirty-four Web sites met the inclusion criteria, 16 Web sites were excluded because they were portals or non-IBD oriented. The median QEI score was 57 with five Web sites scoring higher than 75 points. The median Global Quality Score was 2.0 with five Web sites achieving scores of 4 or 5. The average reading grade level score was 11.2. The median integrity score was 3.0. There is marked variation in the quality of the Web sites containing information on Crohn's disease and ulcerative colitis. Many Web sites suffered from poor quality but there were five high-scoring Web sites.