In the world of science, AI has been like a fascinating maze. It is so attractive because it promises to find hidden patterns in data and change things, especially in medical tests. But as COVID-19 made the world look for new solutions, problems with AI started showing up.
A case in point is the use of AI in analyzing chest X-rays to diagnose COVID-19. At first, everyone was really excited about this. They said AI could do better than people at spotting infections. But when scientists in Kansas took a good look, they found a big problem. AI systems were not necessarily learning to discern clinical features but rather exploiting consistent background differences in the dataset.
This discovery is just the start of a bigger problem. It shows there is a crisis in making things happen the same way again. The problem, it seems, lies in the unchecked and often ill-informed application of AI, leading to a flood of papers making claims that cannot be replicated or worse, are incorrect or impractical.
Computer scientist Sayash Kapoor from Princeton University aptly describes the situation as a “widespread issue impacting many communities beginning to adopt machine-learning methods.” A sentiment echoed by Aeronautical engineer Lorena Barba, who believes that scientific machine learning in the physical sciences is facing pervasive problems.
The main issue is that AI is like clay. If we are not careful, scientists can shape and mold it until it gives the results they want. Not being careful, along with problems like data leaks and unfair data, shows that AI in science is not in a good place right now.
To deal with these problems, people are trying to set rules for how scientists should tell everyone about their AI work. Kapoor and friends made a checklist. It is like a to-do list to make sure scientists share everything about their AI work, like how good the data is, how they built it and if there is a risk of data leaking. However, challenges persist, especially in computational sciences where providing sufficient details for full reproducibility remains a daunting task.
