Dragomir emphasizes the importance of building robust systems in machine learning by focusing on inductive bias and proper structure for better generalization. He highlights the need to detect rare examples during training, which can enhance model performance and trustworthiness. Understanding when to trust these models is crucial for safe and effective deployment.