A Multi-Level Ensemble Framework for Adaptive Deep Learning in High-Dimensional Data Environments with Heterogeneous Feature Distributions

Abstract

The explosive arrival of high-dimensional data in a wide variety of fields, including bioinformatics, finance, and image processing, offers serious problems to classical deep learning models, particularly when the feature distributions are not homogeneous. In this paper, a new multi-level ensemble-based adaptive deep learning strategy is proposed for effectively processing high-dimensional data with heterogeneous feature distributions. The reason the proposed model works is that it incorporates feature space partitioning ideas, adaptive deep learning models, and ensemble aggregation to enhance robustness, improve interpretability, and improve predictive performance. The model uses feature heterogeneity to segregate the input space, thus utilizing different deep learning models with different levels of subspace properties. This is followed by a dynamic aggregation mechanism of the ensemble that adapts to changing data distributions to maintain high accuracy and generalizability. Experiments on benchmark data, comprising gene expression data and remote sensing data, confirm that the proposed method is significantly more accurate, computationally and memory efficient and resistant to overfitting compared to the baseline models. The work presents a scalable framework to address the challenges brought about by high-dimensional, heterogeneous data, which is a manifestation of future, more reliable and flexible AI systems being brought to practical use.

Country : USA

1 Lalmohan Behera2 Venkataram Poosapati

  1. Senior IEEE Member and IETE Member
  2. Independent Data Engineer

IRJIET, Volume 6, Issue 2, February 2022 pp. 94-102

doi.org/10.47001/IRJIET/2022.602016

References

  1. Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. science, 313(5786), 504-507.
  2. Guyon, I., & Elisseeff, A. (2003). An introduction to variable and feature selection. Journal of machine learning research, 3(Mar), 1157-1182.
  3. Jolliffe, I. (2011). Principal component analysis. In International encyclopedia of statistical science (pp. 1094-1096). Springer, Berlin, Heidelberg.
  4. Jacobs, R. A., Jordan, M. I., Nowlan, S. J., & Hinton, G. E. (1991). Adaptive mixtures of local experts. Neural computation, 3(1), 79-87.
  5. Pan, S. J., & Yang, Q. (2009). A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10), 1345-1359.
  6. Dietterich, T. G. (2000, June). Ensemble methods in machine learning. In International workshop on multiple classifier systems (pp. 1-15). Berlin, Heidelberg: Springer Berlin Heidelberg.
  7. Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8), 1798-1828.
  8. Chen, T., &Guestrin, C. (2016, August). Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining (pp. 785-794).
  9. Kouw, W. M., & Loog, M. (2019). A review of domain adaptation without target labels. IEEE transactions on pattern analysis and machine intelligence, 43(3), 766-785.
  10. Breiman, L. (1996). Bagging predictors. Machine learning, 24(2), 123-140.
  11. Freund, Y., &Schapire, R. E. (1997). A decision-theoretic generalization of online learning and an application to boosting. Journal of computer and system sciences, 55(1), 119-139.
  12. Caruana, R. (1997). Multitask learning. Machine learning, 28(1), 41-75.
  13. Zhang, Y., & Yang, Q. (2021). A survey on multi-task learning. IEEE transactions on knowledge and data engineering, 34(12), 5586-5609.
  14. Zheng, C., Wang, C., & Jia, N. (2019). An ensemble model for multi-level speech emotion recognition. Applied Sciences, 10(1), 205.
  15. Johnstone, I. M., & Titterington, D. M. (2009). Statistical challenges of high-dimensional data. Philosophical transactions of the Royal Society A: Mathematical, physical and engineering sciences, 367(1906), 4237-4253.
  16. Liu, R., & Gillies, D. F. (2016). Overfitting in linear feature extraction for classification of high-dimensional image data. Pattern Recognition, 53, 73-86.
  17. Nakamura-Zimmerer, T., Gong, Q., & Kang, W. (2021). Adaptive deep learning for high-dimensional Hamilton--Jacobi--Bellman equations. SIAM Journal on Scientific Computing, 43(2), A1221-A1247.
  18. Li, M., & Wang, Z. (2020). Deep learning for high-dimensional reliability analysis. Mechanical Systems and Signal Processing, 139, 106399.
  19. Gao, L., Song, J., Liu, X., Shao, J., Liu, J., & Shao, J. (2017). Learning in high-dimensional multimedia data: the state of the art. Multimedia Systems, 23(3), 303-313.
  20. Fang, S. H., & Wang, C. H. (2015). A novel fused positioning feature for handling the heterogeneous hardware problem. IEEE Transactions on Communications, 63(7), 2713-2723.
  21. Seni, G., & Elder, J. (2010). Ensemble methods in data mining: improving accuracy through combining predictions. Morgan & Claypool Publishers.