Situating governance and regulatory concerns for generative artificial intelligence and large language models in medical education

0
Situating governance and regulatory concerns for generative artificial intelligence and large language models in medical education
  • IBM. What is Generative AI? [cited 2024 10 Sep 2024]; Available from: (2024).

  • IBM. What are Large Language Models (LLMs)? [cited 2024 10 Sep 2024]; Available from: (2024).

  • Masters, K. Artificial intelligence in medical education. Med. Teach. 41, 976–980 (2019).

    Article 
    PubMed 

    Google Scholar 

  • Hanycz, S. A. & Antiperovitch, P. A practical review of generative AI in cardiac electrophysiology medical education. J. Electrocardiol. 90, 153903 (2025).

    Article 
    PubMed 

    Google Scholar 

  • Lu, M.Y. et al. A Multimodal generative AI Copilot for human pathology. Nature (2024).

  • Wang, X. et al. Foundation model for predicting prognosis and adjuvant therapy benefit from digital pathology in GI cancers. J. Clin. Oncol. p. JCO2401501 (2025).

  • Eynon, R. & Young, E. Methodology, legend, and rhetoric: the constructions of AI by academia, industry, and policy groups for lifelong learning. Sci., Technol., Hum. Values 46, 166–191 (2020).

    Article 

    Google Scholar 

  • Bell, G., Burgess, J., Thomas, J. & Sadiq, S. Rapid Response Information Report: Generative AI—Language Models (LLMs) and Multimodal Foundation Models (MFMs), (Australian Council of Learned Academies, 2023).

  • Masters, K. Ethical use of artificial intelligence in health professions education: AMEE Guide No. 158. Med Teach. 45, 574–584 (2023).

    Article 
    PubMed 

    Google Scholar 

  • Tolsgaard, M. G. et al. The fundamentals of Artificial Intelligence in medical education research: AMEE Guide No. 156. Med. Teach. 45, 565–573 (2023).

    Article 
    PubMed 

    Google Scholar 

  • Reznick, R. Harris, K., Horsley, T. & Mohsen, H. Task force report on artificial intelligence and emerging digital technologies, Royal College of Physicians and Surgeons of Canada (2020).

  • Gordon, M. et al. A scoping review of artificial intelligence in medical education: BEME Guide No. 84. Med Teach. 46, 446–470 (2024).

    Article 
    PubMed 

    Google Scholar 

  • Lee, J., Wu, A. S., Li, D. & Kulasegaram, K. M. Artificial intelligence in undergraduate medical education: a scoping review. Acad. Med. 96, S62–S70 (2021).

    Article 
    PubMed 

    Google Scholar 

  • Alam, F., Lim, M. A. & Zulkipli, I. N. Integrating AI in medical education: embracing ethical usage and critical understanding. Front. Med.10, 1279707 (2023).

    Article 

    Google Scholar 

  • AAIN Generative AI Working Group, AIN Generative Artificial Intelligence Guidelines (Austraian Academic Integrity Network, 2023).

  • Valiant, L. The Importance of Being Educable: A New Theory of Human Uniqueness (Princeton University Press, 2024).

  • Foltynek, T. et al. ENAI recommendations on the ethical use of artificial intelligence in education. Int. J. Educ. Integr. 19 (2023).

  • Preiksaitis, C. & Rose, C. Opportunities, challenges, and future directions of generative artificial intelligence in medical education: scoping review. JMIR Med Educ. 9, e48785 (2023).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Dong, H., Lio, J., Sherer, R. & Jiang, I. Some learning theories for medical educators. Med. Sci. Educ. 31, 1157–1172 (2021).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Kolb, D. A. Experiential Learning: Experience as the Source of Learning and Development (Prentice-Hall, 1984).

  • Ericsson, K. A. & Staszewski, J. J. skilled memory and expertise: mechanisms of exceptional performance. In: Klahr, D., Kotovsky, K. Editors Complex Information Processing, 235–267 (Lawrence Erlbaum Associates, 1989).

  • Wang, S. et al. Artificial intelligence in education: a systematic literature review. Expert Syst. Appl. 252 (2024).

  • Macnamara, B. N. et al. Does using artificial intelligence assistance accelerate skill decay and hinder skill development without performers’ awareness? Cogn. Res. Princ. Implic. 9, 46 (2024).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Tran, M., Balasooriya, C. & Semmler, C., Rhee, J. Generative artificial intelligence: the ‘more knowledgeable other’ in a social constructivist framework of medical education. npj DIgit. Med. Under Review (2025).

  • Safranek, C. W., Sidamon-Eristoff, A. E., Gilson, A. & Chartash, D. The role of large language models in medical education: applications and implications. JMIR Med. Educ. 9, e50945 (2023).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Linardatos, P., Papasteranopoulos, V. & Kotsiantis, S. Explainable AI: a review of machine learning interpretability methods. Entropy 23 (2020).

  • Bearman, M. & Ajjawi, R. Learning to work with the black box: Pedagogy for a world with artificial intelligence. Br. J. Educ. Technol. 54, 1160–1173 (2023).

    Article 

    Google Scholar 

  • Clusmann, J. et al. The future landscape of large language models in medicine. Commun. Med. 3, 141 (2023).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • van der Niet, A. G. & Bleakley, A. Where medical education meets artificial intelligence: ‘Does technology care? Med Educ. 55, 30–36 (2021).

    Article 
    PubMed 

    Google Scholar 

  • Reddy, S. Generative AI in healthcare: an implementation science informed translational path on application, integration and governance. Implement Sci. 19, 27 (2024).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Mykhailov, D. Philosophical dimension of today’s educational technologies: framing ethical landscape of the smart education domain. NaUKMA Res. Pap. Philo. Relig. Stud. 68–75 (2023).

  • Moritz, S., Romeike, B., Stosch, C. & Tolks, D. Generative AI (gAI) in medical education: Chat-GPT and co. GMS J. Med. Educ. 40, Doc54 (2023).

    PubMed 
    PubMed Central 

    Google Scholar 

  • Brin, D. et al. Comparing ChatGPT and GPT-4 performance in USMLE soft skill assessments. Sci. Rep. 13, 16492 (2023).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Alfertshofer, M. et al. Sailing the seven seas: a multinational comparison of ChatGPT’s performance on medical licensing examinations. Ann. Biomed. Eng. 52, 1542–1545 (2024).

    Article 
    PubMed 

    Google Scholar 

  • Lucas, H. C., Upperman, J. S. & Robinson, J. R. A systematic review of large language models and their implications in medical education. Med. Educ. (2024).

  • Zack, T. et al. Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study. Lancet Digit. Health 6, e12–e22 (2024).

    Article 
    CAS 
    PubMed 

    Google Scholar 

  • Li, Z. et al. Large language models and medical education: a paradigm shift in educator roles. Smart Learning Environ. 11(2024).

  • Chan, K. S. & Zary, N. Applications and challenges of implementing artificial intelligence in medical education: integrative review. JMIR Med. Educ. 5, e13930 (2019).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Tolsgaard, M. G., Boscardin, C. K., Park, Y. S., Cuddy, M. M. & Sebok-Syer, S. S. The role of data science and machine learning in Health Professions Education: practical applications, theoretical contributions, and epistemic beliefs. Adv. Health Sci. Educ. Theory Pract. 25, 1057–1086 (2020).

    Article 
    PubMed 

    Google Scholar 

  • Balasooriya, C. et al. Learning, teaching and assessment in health professional education and scholarship in the next 50 years. FoHPE 25 (2024).

  • Abd-Alrazaq, A. et al. Large language models in medical education: opportunities, challenges, and future directions. JMIR Med. Educ. 9, e48291 (2023).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Lomis, K. et al. Artificial Intelligence for Health Professions Educators. NAM Perspect. 2021 (2021).

  • Nagi, F. et al. Applications of artificial intelligence (AI) in medical education: a scoping review. Stud. Health Technol. Inf. 305, 648–651 (2023).

    Google Scholar 

  • Boscardin, C. K., Gin, B., Golde, P. B. & Hauer, K. E. ChatGPT and Generative artificial intelligence for medical education: potential impact and opportunity. Acad. Med 99, 22–27 (2024).

    Article 
    PubMed 

    Google Scholar 

  • Artsi, Y. et al. Large language models for generating medical examinations: systematic review. BMC Med. Educ. 24, 354 (2024).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Laupichler, M. C., Rother, J. F., Grunwald Kadow, I. C., Ahmadi, S. & Raupach, T. Large language models in medical education: comparing ChatGPT- to human-generated exam questions. Acad. Med 99, 508–512 (2024).

    Article 
    PubMed 

    Google Scholar 

  • Scott, K. & Hart, J. Digital technologies in health: implications for health professional education. FoHPE. 25 (2024).

  • Pearce, J. & Chiavaroli, N. Rethinking assessment in response to generative artificial intelligence. Med, Educ. 57, 889–891 (2023).

    PubMed 

    Google Scholar 

  • Fawns, T. & Schuwirth, L. Rethinking the value proposition of assessment at a time of rapid development in generative artificial intelligence. Med. Educ. 58, 14–16 (2024).

    Article 
    PubMed 

    Google Scholar 

  • Rampton, V., Mittelman, M. & Goldhahn, J. Implications of artificial intelligence for medical education. Lancet Digit. Health 2, e111–e112 (2020).

    Article 
    PubMed 

    Google Scholar 

  • Jackson, P. et al. Artificial intelligence in medical education – perception among medical students. BMC Med. Educ. 24, 804 (2024).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Australian Academy of Technological Sciences and Engineering (ATSE) and A.I.f.M.L. (AIML), Responsible AI: Your questions answered. 2023: Canberra, Adelaide.

  • Moy, S. et al. Patient perspectives on the use of artificial intelligence in health care: a scoping review. J. Patient Cent. Res Rev. 11, 51–62 (2024).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Mikkelsen, J. G., Sorensen, N. L., Merrild, C. H., Jensen, M. B. & Thomsen, J. L. Patient perspectives on data sharing regarding implementing and using artificial intelligence in general practice – a qualitative study. BMC Health Serv. Res 23, 335 (2023).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Khullar, D. et al. Perspectives of patients about artificial intelligence in health care. JAMA Netw. Open 5, e2210309 (2022).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Kudina, O. & de Boer, B. Co-designing diagnosis: Towards a responsible integration of Machine Learning decision-support systems in medical diagnostics. J. Eval. Clin. Pract. 27, 529–536 (2021).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Mykhailov, D. A moral analysis of intelligent decision-support systems in diagnostics through the lens of Luciano Floridi’s information ethics. Hum. Aff. 31, 149–164 (2021).

    Article 

    Google Scholar 

  • Juravle, G., Boudouraki, A., Terziyska, M. & Rezlescu, C. Trust in artificial intelligence for medical diagnoses. Prog. Brain Res. 253, 263–282 (2020).

    Article 
    PubMed 

    Google Scholar 

  • Candelon, F., di Carlo, R, C., De Bondt, M. & Evgenious, T. AI regulation is coming. Harvard Bus. Rev. (2021).

  • Longoni, C., Bonezzi, A. & Morewedge, C. K. Resistance to medical artificial intelligence. J. Consum. Res. 46, 629–650 (2019).

    Article 

    Google Scholar 

  • Sauerbrei, A., Kerasidou, A., Lucivero, F. & Hallowell, N. The impact of artificial intelligence on the person-centred, doctor-patient relationship: some problems and solutions. BMC Med. Inf. Decis. Mak. 23, 73 (2023).

    Article 

    Google Scholar 

  • de Boer, B. & Kudina, O. What is morally at stake when using algorithms to make medical diagnoses? Expanding the discussion beyond risks and harms. Theor. Med. Bioeth. 45, 245–266 (2021).

    Article 

    Google Scholar 

  • Kingsford, P. A. & Ambrose, J. A. Artificial intelligence and the doctor-patient relationship. Am. J. Med 137, 381–382 (2024).

    Article 
    PubMed 

    Google Scholar 

  • Lorenzini, G., Arbelaez Ossa, L., Shaw, D. M. & Elger, B. S. Artificial intelligence and the doctor-patient relationship expanding the paradigm of shared decision making. Bioethics 37, 424–429 (2023).

    Article 
    PubMed 

    Google Scholar 

  • Mittelstadt, B. The Impact of Artificial Intelligence on the Doctor-Patient Relationship (Council of Europe 2021).

  • Mittermaier, M., Raza, M. & Kvedar, J. C. Collaborative strategies for deploying AI-based physician decision support systems: challenges and deployment approaches. NPJ Digit. Med. 6, 137 (2023).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • U.S. Department of Health and Human Services. Artificial Intelligence (AI) at HHS. [cited 10 Sep 2024]; Available from: (2024).

  • Jonnagaddala, J. & Wong, Z. S. Privacy preserving strategies for electronic health records in the era of large language models. NPJ Digit. Med. 8, 34 (2025).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Productivity Commission, Australian Government. Making the most of the AI opportunity: The challenges of regulating AI: Canberra (2024).

  • Nikolic, S. et al. ChatGPT, Copilot, Gemini, SciSpace and Wolfram versus higher education assessments: an updated multi-institutional study of the academic integrity impacts of Generative Artificial Intelligence (GenAI) on assessment, teaching and learning in engineering. Austr. J. Eng. Educ. 29, 1–28 (2024).

  • Schuwirth, L. The need for national licensing examinations. Med. Educ. 41, 1022–1023 (2007).

    Article 
    PubMed 

    Google Scholar 

  • Schuwirth, L. W. & Van der Vleuten, C. P. Programmatic assessment: from assessment of learning to assessment for learning. Med. Teach. 33, 478–485 (2011).

    Article 
    PubMed 

    Google Scholar 

  • Bhanji, F. et al. Competence by design: the role of high-stakes examinations in a competence based medical education system. Perspect. Med. Educ. 13, 68–74 (2024).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Shumailov, I. et al. AI models collapse when trained on recursively generated data. Nature 631, 755–759 (2024).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar 

  • De Angelis, L. et al. ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health. Front. Public Health 11, 1166120 (2023).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Xu, X., Chen, Y. & Miao, J. Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review. J. Educ. Eval. Health Prof. 21, 6 (2024).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Ngo, B., Nguyen, D. & vanSonnenberg, E. The cases for and against artificial intelligence in the medical school curriculum. Radio. Artif. Intell. 4, e220074 (2022).

    Article 

    Google Scholar 

  • Franco D’Souza, R., Mathew, M., Mishra, V. & Surapaneni, K. M. Twelve tips for addressing ethical concerns in the implementation of artificial intelligence in medical education. Med. Educ. Online 29, 2330250 (2024).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Fleisher, L. A. & Economou-Zavlanos, N. J. Artificial Intelligence can be regulated using current patient safety procedures and infrastructure in hospitals. JAMA Health Forum 5, e241369 (2024).

    Article 
    PubMed 

    Google Scholar 

  • Chouffani El Fassi, S. et al. Not all AI health tools with regulatory authorization are clinically validated. Nat. Med. 30, 2718–2720 (2024).

  • Australian Human Rights Commission. Australia Needs AI Regulation [cited 3 Dec 2024]; Available from: (2023).

  • Mesko, B. & Topol, E. J. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. NPJ Digit. Med. 6, 120 (2023).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • GIbson, D., Kovanovic, V., Ifenthaler, D., Dexter, S. & Feng, S. Learning theories for artificial intelligence promoting learning processes. Br. J. Educ. Technol. 54, 1125–1146 (2023).

    Article 

    Google Scholar 

  • Yu, H. & Guo, Y. Generative artificial intelligence empowers educational reform: current status, issues, and prospects. Front. Educ. 8 (2023).

  • Gniel, H. AI: A Regulatory Perspective (Australian Government Tertiary Education Quality and Standards Agency, 2023).

  • The Royal Australian College of General Practitioners. Artificial intelligence in primary care [cited 2024 20/09/2024]; Available from: (2024).

  • Knopp, M. I. et al. AI-enabled medical education: threads of change, promising futures, and risky realities across four potential future worlds. JMIR Med Educ. 9, e50373 (2023).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Australian Health Practitioner Regulation Agency. Meeting your professional obligations when using Artificial Intelligence in healthcare [cited 3 Dec 2024]; Available from: (2024).

  • Australian Government Digital Transformation Agency, Policy for the Responsible Use of AI in Government, Commonwealth of Australia (Digital Transformation Agency) (2024).

  • The White House. Fact Sheet: Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence [cited 13 Dec 2024]; Available from: (2023).

  • Council of Europe, Council of Europe Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law, Council of Europe Treaty Series No. 225 (2024).

  • Department for Science, I.T., The Bletchley Declaration by Countries Attending the AI Safety Summit, 1–2 November 2023, Department for Science, Innovation & Technology (2023).

  • United Nations AI Advisory Body, Governing AI for Humanity (2024).

  • Australian Government Department of Industry, S.a.R., Safe and responsible AI in Australia consultation: Australian Government’s interim response. Commonwealth of Australia (2024).

  • Wellner, G. A postphenomenological guide to AI regulation. J. Hum.-Technol. Relat. 2, 1–18 (2024).

    Google Scholar 

  • The White House. Blueprint for an AI Bill of Rights: Making Automated Systems Work for the American People [cited 20 Sep 2024]; Available from: (2024).

  • Government of the United Kingdom, National AI Strategy, HM Government (2021).

  • Digital Policy Office: The Government of the Hong Kong Special Administrative Region of the People’s Republic of China, Ethical Artificial Intelligence Framework (2024).

  • Ministry of Education, C., Sports, Science and Technology – Japan, White Paper on Science, Technology, and Innovation: How AI will transform Science, Technology and Innovation. (Ministry of Education, Culture, Sports, Science and Technology, 2024).

  • Australian Government Department of Industry, S.a.R., Voluntary AI Safety Standard. (Commonwealth of Australia, 2024).

  • link

    Leave a Reply

    Your email address will not be published. Required fields are marked *