According to DeepLearning.AI, researchers have introduced a new method to estimate exactly how many bits of information a language model memorizes from its training data. The team conducted rigorous experiments using hundreds of GPT-2–style models trained on both synthetic datasets and subsets of FineWeb. By comparing the negative log likelihood of trained models to that of stronger baseline models, the researchers were able to measure model memorization with greater accuracy. This advancement offers AI industry professionals practical tools to assess and mitigate data leakage and overfitting risks, supporting safer deployment in enterprise environments (source: DeepLearning.AI, August 28, 2025).