"Large Language Models are neural network language models with remarkable conversational skills. They generate human-like responses and engage in interactive conversations. However, they often generate highly convincing statements that are verifiably wrong or provide inappropriate responses. Today there is no way to be certain about the quality, evidence level, or consistency of clinical information or supporting evidence for any response. These chatbots are unsafe tools when it comes to medical advice and it is necessary to develop new frameworks that ensure patient safety," said Prof. Stephen Gilbert, Professor for Medical Device Regulatory Science at Else Kröner Fresenius Center for Digital Health at TU Dresden.
Challenges in the regulatory approval of large language models
Most people research their symptoms online before seeking medical advice. Search engines play a role in decision-making process. The forthcoming integration of LLM-chatbots into search engines may increase users' confidence in the answers given by a chatbot that mimics conversation. It has been demonstrated that LLMs can provide profoundly dangerous information when prompted with medical questions.
LLM's underlying approach has no model of medical "ground truth," which is inherently dangerous. Chat interfaced LLMs have already provided harmful medical responses and have already been used unethically in 'experiments' on patients without consent. Almost every medical LLM use case requires regulatory control in the EU and US. In the US their lack of explainability disqualifies them from being 'non devices'. LLMs with explainability, low bias, predictability, correctness, and verifiable outputs do not currently exist and they are not exempted from current (or future) governance approaches.
Read more at Science Daily
No comments:
Post a Comment