Fractal patterns reveal hidden limits to AI predictability
Researchers at the University of Sydney have identified a fundamental mathematical reason why artificial intelligence systems can learn different behaviors even when trained under almost identical conditions, a discovery that could help improve the reliability and trustworthiness of AI technologies. Об этом пишет сайт Phys.org.
The study by Professor Pulin Gong from the School of Physics and his Ph.D. student Andrew Ly has been published in Nature Machine Intelligence. As AI systems become increasingly embedded in health care, business, research and everyday life, ensuring they behave consistently and predictably has become a major challenge. The new research suggests that some forms of AI unpredictability are not due to engineering flaws or inconsistencies in computer hardware, but arise from the underlying mathematics of how neural networks learn.
"Researchers have often attributed differences in AI behavior to external sources of randomness, such as the order in which training data is presented or tiny numerical variations introduced by hardware," Gong said. "Our findings point to something deeper. Even when these factors are removed, extremely small differences in a model's starting conditions can still lead to different outcomes."
Neural networks, the technology behind today's most advanced AI systems, learn by adjusting millions or even billions of internal parameters during training. The Sydney team investigated why networks sometimes arrive at different solutions despite using the same data, training procedure and deterministic computing environment. Ly said, "Imagine two AI systems trained to detect cancer using the same medical images and the same training procedure. Tiny differences in their initial parameters could lead them to learn different behaviors."
The researchers trained approximately 500,000 neural networks, analyzing systems used for tasks including image recognition and text processing. Evidence consistent with a "riddled basins" fractal structure was found across a wide range of network architectures, datasets and training methods. In one larger image-recognition network, making the starting settings 10 billion times more precise produced no measurable improvement in reproducibility, and altering the starting settings by a single "bit" sent training to a different internal configuration in close to half of cases.
The work also revealed a surprising trade-off between performance and reproducibility. The settings enabling networks to learn most effectively also made their outcomes harder to predict and reproduce. Reducing the "learning rate" made the results reproducible, but the networks learned less effectively. Rather than viewing unpredictability as a problem that can be completely engineered away, the researchers argue it may represent a fundamental feature of how many neural networks operate.
