Reproducing Deep Learning Models

Reproduction Steps

Rajiv Shah embarked on the task of reproducing a deep learning model designed to predict earthquakes, highlighting the challenges and initial steps involved. He appreciated the open access to data and code, which facilitated his reproduction efforts on a standard laptop, demonstrating the model's accessibility 1. Shah emphasized the importance of adhering to best practices in machine learning, such as proper data partitioning to prevent leakage, which are crucial for successful model building 2.

One of the great things about this paper was they made the data freely available, as well as the code that they used for building the models.

--- Rajiv Shah

The discussion also touched on the necessity for the machine learning community to establish guidelines to help newcomers avoid common pitfalls 2.

   

Collaboration

Feedback and collaboration played a significant role in Shah's reproduction process. While he did not receive feedback from the original authors, he engaged with other researchers from the earthquake community who identified similar issues with the model 1. This collaboration led to the development of a larger article aimed at addressing these concerns within the academic community.

I have not heard back from the original authors, but part of one of the reasons why I actually finally published the blog and published the notebooks to make it reproducible so anybody could run the models on the data themselves was I was contacted by some other researchers that actually come from the earthquake community.

--- Rajiv Shah

Shah's experience underscores the value of open science and the potential for interdisciplinary collaboration to enhance research quality and impact 1.