Tue Sep 17 05:58:00 UTC 2024: ## AI Medical Platform Fails the Test: OpenEvidence Criticized for Incompetent Handling of SCAMs

**Portland, OR** – Dr. Mark Crislip, a renowned infectious disease specialist and author of the popular blog “Edgy Doc,” has launched a scathing critique of OpenEvidence, a self-proclaimed “leading AI-powered medical information platform.” Crislip, known for his outspoken views on science-based medicine (SBM), argues that the AI platform demonstrates significant limitations in its understanding of unconventional and scientifically unsupported therapies, commonly referred to as SCAMs.

In a detailed analysis published on his blog, Crislip tested OpenEvidence’s capabilities by querying the platform on various SCAMs, including homeopathy, chiropractic, and acupuncture. He found that OpenEvidence’s responses were demonstrably flawed, failing to accurately represent the scientific consensus and often resorting to misleading or unsubstantiated claims.

For instance, when asked about the efficacy of homeopathy, OpenEvidence cited studies with methodological limitations and failed to recognize the fundamental implausibility of water being effective in treating various diseases. Similarly, the platform’s response to chiropractic, while acknowledging its potential benefit for musculoskeletal pain, included references to studies involving osteopathic manipulation, potentially blurring the lines between chiropractic and other manipulative therapies.

In its assessment of acupuncture, OpenEvidence provided a seemingly positive evaluation, citing a range of studies claiming its effectiveness for various conditions. However, Crislip argues that these positive findings can be largely attributed to poor study design, biases, and inadequate scientific rigor.

“OpenEvidence’s responses to my queries about SCAMs were deeply concerning,” states Dr. Crislip. “The platform failed to grasp the basic principles of SBM and seemed to accept even the most outlandish claims without critical analysis.” He further argues that the platform’s tendency to rely heavily on systematic reviews and meta-analyses, while neglecting the importance of primary research, further highlights its shortcomings in understanding the nuances of complex medical topics.

While acknowledging OpenEvidence’s potential for summarizing core treatment approaches for common infectious diseases, Dr. Crislip concludes that the platform’s lack of critical thinking and inadequate understanding of SBM render it an unreliable source of medical information, particularly when it comes to unconventional therapies. He urges OpenEvidence to address its shortcomings and integrate SBM principles into its AI model to better serve the medical community.

“The SCAM blind spot of evidence-based medicine is alive and well at OpenEvidence,” concludes Dr. Crislip. “They need to add SBM programming to their AI model. I wonder if that blind spot is amenable to remediation? I hope they have the concept of a plan to incorporate Science-Based Medicine into their AI.”

This critique highlights the growing concern surrounding the application of AI in healthcare. While AI offers potential benefits in medical research and diagnostics, it is crucial to ensure that AI-powered platforms are developed and utilized responsibly, with a strong emphasis on critical thinking and scientific accuracy.

Read More