Artificial Intelligence (AI) systems become deployed into practical and safety-critical systems that interact with humans due to their great achievements in performance. However, careless deployment raises concerns due to untrusting behaviors of AI systems. In this talk, I will discuss my efforts on learning and quantifying the uncertainty of predictions from AI models under distribution shift to enhance the trustworthiness of AI systems. In particular, I focus on a prediction set, which quantifies the uncertainty of predictions via a set of possible label predictions, where the set size represents the uncertainty. This prediction set comes with the correctness guarantee even under distribution shift from covariate shift to local distribution shift by Byzantine adversaries in redundant systems. The practical value of the correct prediction sets are demonstrated in computer vision and blockchain applications. In closing, I will discuss directions to enhance the trustworthiness of systems with large language models, cyber-physical systems, Rust binary analysis, and the composition of multiple AI and non-AI components via uncertainty learning.

Sangdon Park is a postdoctoral researcher at the Georgia Institute of Technology, mentored by Taesoo Kim. He earned his Ph.D. in Computer and Information Science from the University of Pennsylvania in 2021, where he was advised by Insup Lee and Osbert Bastani. His research interest focuses on designing safe and secure AI systems by understanding from theory to implementation and applications, mainly in computer security, computer vision, robotics, cyberphysical systems, and natural language processing.