AI for search at Etsy

Neuroevolution

Neuroevolution combines evolutionary algorithms with neural networks to enhance search capabilities. Andrew Stanton explains that neuroevolution is particularly useful in scenarios where computing a gradient is challenging, such as in reinforcement learning. It allows for model optimization without assuming a specific objective function, making it competitive in certain domains 1.

Neural evolution is more of a way of learning models based on these kind of beliefs of populations, of answers that can kind of compete with each other.

--- Andrew Stanton

This approach has gained traction due to its ability to evolve both network structures and weights, offering a viable alternative to traditional methods like stochastic gradient descent 2.

   

Challenges

Implementing evolutionary algorithms in real-world applications presents unique challenges, especially in diverse data environments like Etsy's marketplace. Andrew Stanton highlights the difficulty of balancing multiple objectives, such as ensuring both new and established sellers succeed, which requires sophisticated optimization strategies 3.

You have this black box function, which have all these different factors, which have these trade offs between them.

--- Andrew Stanton

To address these challenges, techniques like evolutionary strategies are employed, which can be combined with traditional methods to improve efficiency and maintain search effectiveness 1.