Randy discusses the architectural necessity of segmenting applications into functional pools at eBay, leading to a stateless design. This approach allows for horizontal scalability and flexible upgrades without session state on the server, compelling developers to create applications that can seamlessly handle user interactions across multiple services. The result is a disciplined architecture that enhances operational efficiency and application design.