Can survey data improve predictions?

Survey data can indeed improve predictions, primarily by incorporating large quantities and diverse types of auxiliary data.

1. Volume and Diversity of Data: Increasing the volume of both resume data and survey data enhances the quality of predictions. Surveys, although smaller in scale, are crucial, and their integration with extensive auxiliary data, such as resumes, scales the predictive accuracy significantly 1.

2. Transfer Learning: This technique leverages large-scale datasets to improve predictions on smaller survey datasets. For instance, using resume data for pretraining models has resulted in better predictions on subsequent survey data 2.

3. Combining Data Sources: Surveys, when combined with other forms of data (e.g., digital footprints, economic indicators), enhance the predictive power. For example, in econometric studies, predictive models utilizing this combination have been effective in estimating wage gaps and unemployment risks 3.

   Improving Predictive Models

   Keyon Vafa

   Kyle Polich

   Keyon Vafa discusses the factors that can drive the accuracy of predictive models, including the importance of data volume and the impact of survey data. He highlights the potential for better predictions by increasing the volume of resume data and the smaller survey data set. Additionally, he explores the possibility of reaching a point where further improvements require changes to the model or the use of other data sources.

   

   Data Skeptic

   CAREER Prediction

   Explore similar conversationsView full episode summary

   1

   2

   3

However, it's also noted that there are inherent limitations and potential biases. It's crucial to audit and validate the data and model predictions to mitigate systemic biases 3.