The Department of Defense and Google have joined forces to create an AI-powered microscope, the Augmented Reality Microscope (ARM), which has been developed to assist health care professionals in diagnosing cancer. The cost of the microscope to health care systems is estimated at $90,000 - $100,000. Professionals feel that this product will be of great help to medical personnel in smaller labs as they confront staffing shortages and a rise in patient numbers.
At the VA hospital in Seattle, Dr. Nadeem Zafar was using his microscope to settle a debate with a colleague about a prostate cancer case. As a pathologist, Zafar had to consider bodily tissues and fluids to diagnose conditions. However, the microscope he was using was no ordinary device – it was an artificial intelligence-powered microscope developed by Google and the Department of Defense. The team ran the case through the microscope and it showed that Zafar was right, to the point that his colleague agreed. This is what makes the AI-powered Augmented Reality Microscope (ARM) technology unique, as it can act as an ‘arbitrator’. It is still in the early stages of development, but early research shows it could be a beneficial tool for pathologists without easy access to a second opinion.
There are currently 13 ARMs in existence, one of which is being utilized by Mitre, a non-profit organization devoted to working with government agencies in solving technology-related challenges. Researchers there are using the ARM to detect vulnerabilities that could create issues in clinical settings. From the outside, the ARM appears like a microscope that may be seen in a high school biology classroom, with its beige color and its large eyepiece and tray for examining glass slides. It is, however, connected to a computer tower hosting AI models. When a glass slide is loaded under the microscope, the AI models can display cancerous areas by way of a glowing green line which is visible through the eyepiece and on a separate monitor. The AI is also able to suggest the seriousness of the cancer, and a black and white heat map will appear on the monitor to demonstrate the limits of the cancer in pixelated form. In August, CNBC witnessed the ARM and its functions demonstrated by the researchers at the Mitre facility.
Patrick Minot, a senior autonomous systems engineer at Mitre, said that using the AI directly in the microscope's field of view avoids disrupting the pathologists' typical workflow. This convenience was a deliberate design decision. In recent years, pathologists have encountered staffing deficits, much like many other facets of healthcare. At the same time, the increasing age of the general population has boosted the number of cases they must attend to. These factors can have potentially serious outcomes if the specialists are overloaded and miss something. As a way to enhance efficiency, numerous organizations have been attempting to digitize the pathologists' responsibilities. Nevertheless, digital pathology can be difficult to implement due to the large data storage capacity required by a single slide, which might lead to considerable overhead expenses for small health systems. The ARM is not intended to take the place of digital pathology systems, but Minot said it could assist health establishments to avoid the need for them. Pathologists can also take pictures of slides with the ARM's software, incurring much lower storage costs in the process. The ARM usually costs healthcare organizations between $90,000 and $100,000, according to Minot. Additionally, the ARM ensures that the traditional microscope is still a central component of the pathologists' practice. Minot has found himself the subject of many warnings not to interfere with their microscopes, which he jokingly mentioned.
Dr. Niels Olson, the chief medical officer of the Defense Innovation Unit (DIU) is well aware of the struggles pathologists face. Established by the Department of Defense in 2015, the DIU enables the military to adopt the most modern technology that businesses are creating. Under this division, Olson had a contract with many companies to bypass lengthy bureaucratic issues. Olson, who was also a U.S. Navy veteran, was sent to Guam, a U.S. island in Micronesia in 2018, where he served as a laboratory medical director and blood bank director for the Naval Hospital, the only pathologist in the facility. This forced him to make difficult decisions and diagnoses alone. "Not only was it my job to determine if a cancer was present," Olson told CNBC in an interview, "but I also needed to confirm that it was not cancer. Without someone else to help me, that could be a really nerve-racking experience." He then went on to explain that if he had the Augmented Reality Microscope (ARM) in Guam at the time, it would have been a great resource as a second opinion. Although the ARM is designed to assist pathologists, Olson explained that it won't take the place of doctors. He also added that the technology would be especially beneficial to smaller, remote labs, and medical residents in training.
Back in 2016, while Olson was a resident at the Naval Medical Center in San Diego, he reached out to one of his connections at Google to inform him of his ideas for the ARM. After going to the Google office building and watching the ARM identify cancer on slides he presented, everyone in the room was ecstatic. Subsequently, after a DIU product manager saw Olson's research, the two wrote an article together in 2019 that discussed how the DoD and Silicon Valley could join forces and use the federal government's colossal healthcare database within the civilian healthcare system. Jenoptik was then picked to create the hardware and Google to develop the software for the ARM. Google Cloud's global director of healthcare strategy and solutions, Aashima Gupta, made it clear that none of their personnel or infrastructure have access to the encrypted data stored and used to train the ARM's four algorithms for breast, cervical, prostate, and mitosis cancer diagnosis.
After conducting initial research to see if the ARM would be effective, the DIU published a paper in the Journal of Pathology Informatics in fall 2022. David Jin, the lead author of the paper and deputy director for AI assessment at the Department of Defense's Chief Digital and Artificial Intelligence Office, said that the results from the study of the AI detecting breast cancer metastasis in lymph nodes were promising but there were some caveats. He told CNBC in an interview that there was still "huge" amounts of testing to be done before it can be used in patient care. The author, Olson, who returned to the DIU in 2020, noted that independent tests had yet to be conducted on the other three models (prostate cancer, mitosis, and cervical cancer). Research with the ARM is on-going and the DIU is seeking feedback from organizations like Mitre and health systems like Veterans Affairs. The DIU has begun to contemplate how to expand the use of the technology and collaborate with regulators. Agreements with Google and Jenoptik have been reached that will let the ARM be used by the military and commercially. The DIU is hoping to make the ARM available to all government users through the GSA site this fall. Zafar of VA Puget Sound said the ARM will help pathologists, but most of all, the public will benefit from its accuracy, speed, and cost-effectiveness. "AI is here and will keep advancing," he said. "We should not be afraid of these technologies, but focus on using them to meet our medical and health care needs."
Comments