When & Where to Stop Predictive Modeling?
Share on LinkedIn Share on Facebook Share on Twitter When a data scientist develops a Predictive Model, he doesn’t know where to stop, when to stop and which model alternative to select? Here is what I think should be a framework to follow. The AIRS (accuracy, Implementability, reliability and stability) framework will help to take a scientific decision. Let us describe how it will work. Accuracy: When one develops a predictive model, he always decides what should be the accuracy value to stop the model development iteration. Though this target was decided and accepted based on a predefined measure that was agreed with the business or with the customer beforehand, the data scientist has a major role to play during model development/iteration process. The predefined measure can be qualitative and/or quantitative. This measure can be evaluated during model development (in-sample) or after model in model production (out-sample). For example, if you are developing a fore