The discussion highlights the hardware requirements for on-premises model training, emphasizing the use of standard machines with 32 cores and sufficient RAM. It’s crucial to understand that model serving is CPU-bound, meaning the number of models served cannot exceed the available cores. Additionally, while RAM needs are primarily dictated by the total size of the models, having ample cores is essential for fast inference and response times.
