Web Scraping Insights
The discussion highlights the limitations of APIs in providing comprehensive data for research purposes. Diarmuid emphasizes that valuable information often resides on websites, particularly for academics studying organizations. He shares his own experience of needing detailed administrative data about nonprofits, which is frequently not included in standard data downloads. This underscores the importance of web scraping as a tool for accessing richer, unstructured information.In this clip
From this podcast
Software Engineering Radio - the podcast for professional software developers
Episode 503: Diarmuid McDonnell on Web Scraping
Related Questions