Training neural networks on edge devices presents unique challenges, particularly in optimizing hyperparameters without draining resources or compromising privacy. By drawing inspiration from marginal likelihood approaches, a new algorithm allows for partitioning both data and models, enabling each subnetwork to validate its generalization capabilities while minimizing the risk of privacy budget depletion. This innovative method enhances efficiency in federated learning scenarios.