The primary objective of this paper is to develop a family of GARCH models, combining them with popular Recurrent Neural Network (RNN) models, which can capture the high nonlinear relationships between past return innovations and conditional variance, which is overlooked by standard GARCH models. The next objective is to apply Markov Switching GARCH model and see the differences in the predictive accuracy (according to AIC and BIG criteria and in two viewpoints: MSB and MAD) between the standard GARCH models, GJRGARCH model, RNN-GARCH models and Markov Switching GARCH model by comparing their out-of-sample forecasts. The dataset consists of a series of daily returns obtained from the National Stock Exchange (NSE) for the Indian Equity Markets. The results indicated that the proposed RNN-GARCH model and RNN-Markov Switching models are accurate and quick prediction methods.