Enhancing Nonparametric Tests: Insights for Computational Intelligence and Data Mining

Shahid Yousufi Aqmal; Fermle Erdely S

doi:10.69725/raida.v1i3.168

Authors

Shahid Yousufi A a. School of Computer Science and Electronic Engineering, University of Essex, Wivenhoe Park, Colchester, United Kingdom
Fermle Erdely S b. Interactive Coventry Limited, Coventry University Technology Park, Puma Way, Coventry, CV1, United Kingdom

DOI:

https://doi.org/10.69725/raida.v1i3.168

Keywords:

Nonparametric tests, Computational intelligence, Data mining, Post hoc analysis, Statistical reliability

Abstract

Objective: With the aim of improving monitoring reliability and interpretability of CI and DM experimental statistical tests, we evaluate the performance of cutting-edge nonparametric tests and post hoc procedures.

Methods: A Friedman Aligned Ranks test, Quade test, and multiple post hoc corrections Bonferroni-Dunn and Holm were used to comparative analyze data. These approaches were employed to algorithm performance metrics with varied datasets to evaluate their capability to detect meaningful differences and control Type I errors.
Results: Advanced nonparametric methods consistently outperformed traditional parametric tests, offering robust results in heterogeneous datasets. The Quade test was the most powerful and stable, and the post hoc procedures greatly increased the power of the pairwise comparisons.
Novelty: We evaluate advanced nonparametric methods in CI and DM experiments: the Friedman Aligned Ranks test, the Quade test, and post hoc procedures (Bonferroni-Dunn and Holm). These methods represent a departure from traditional parametric tests that depend on assumptions of normality and homogeneity of variance, allowing for more flexible and robust approaches to analyses of complex, heterogeneous datasets. By comparing the strength and efficacy of these methods, the research also delivers common guidelines for their use; as well as demonstrating their utility in realistic situations characterized by non-standard and dispersed data.
Implications for Research: The findings have far-reaching theoretical and pragmatic implications for scholars in CI and DM. On a theoretical level, this work undermines the common bias towards parametric techniques, providing an increasingly robust framework for comparative analysis in experimental research. This work improves understanding of the adaptation of statistical tests to fit the complexities of real-world data by highlighting the advantages of advanced nonparametric methods, specifically the Quade test and post hoc corrections. Practical implications The results give owners of data summaries actionable recommendations, which will assist researchers in the selection of statistical methods that are tuned to the nature of their datasets, resulting in improved reliability and interpretability of future evaluations of algorithms. Thus, this endeavor will promote more powerful and statistically appropriate methods in CI and DM studies, leading to more confident and valid claims surrounding algorithmic performance.

Downloads

Download data is not yet available.

Author Biographies

Shahid Yousufi A, a. School of Computer Science and Electronic Engineering, University of Essex, Wivenhoe Park, Colchester, United Kingdom

Dr. Shahid Yousufi Aqmal
School of Computer Science and Electronic Engineering, University of Essex, Wivenhoe Park, Colchester, United Kingdom
Fermle Erdely S, b. Interactive Coventry Limited, Coventry University Technology Park, Puma Way, Coventry, CV1, United Kingdom

Dr Fermle Erdely Suckling
Reseacher Lecture
Interactive Coventry Limited, Coventry University Technology Park, Puma Way, Coventry, CV1, United Kingdom

References

Albahri, A. S., Duhaim, A. M., Fadhel, M. A., Alnoor, A., Baqer, N. S., Alzubaidi, L., Albahri, O. S., Alamoodi, A. H., Bai, J., Salhi, A., Santamaría, J., Ouyang, C., Gupta, A., Gu, Y., & Deveci, M. (2023). A systematic review of trustworthy and explainable artificial intelligence in healthcare: Assessment of quality, bias risk, and data fusion. Information Fusion, 96, 156–191. https://doi.org/https://doi.org/10.1016/j.inffus.2023.03.008 DOI: https://doi.org/10.1016/j.inffus.2023.03.008

Albahri, O. S., Zaidan, A. A., Albahri, A. S., Zaidan, B. B., Abdulkareem, K. H., Al-qaysi, Z. T., Alamoodi, A. H., Aleesa, A. M., Chyad, M. A., Alesa, R. M., Lim, C. K., Lakulu, M. M., Ibrahim, A. B., & Rashid, N. A. (2020). Systematic review of artificial intelligence techniques in the detection and classification of COVID-19 medical images in terms of evaluation and benchmarking: Taxonomy analysis, challenges, future solutions and methodological aspects. Journal of Infection and Public Health, 13(10), 1381–1396. https://doi.org/https://doi.org/10.1016/j.jiph.2020.06.028 DOI: https://doi.org/10.1016/j.jiph.2020.06.028

Ali, R., Lee, S., & Chung, T. C. (2017). Accurate multi-criteria decision making methodology for recommending machine learning algorithm. Expert Systems with Applications, 71, 257–278. https://doi.org/https://doi.org/10.1016/j.eswa.2016.11.034 DOI: https://doi.org/10.1016/j.eswa.2016.11.034

Berger, F., Joest, D., Barbers, E., Quade, K., Wu, Z., Sauer, D. U., & Dechent, P. (2024). Benchmarking battery management system algorithms - Requirements, scenarios and validation for automotive applications. ETransportation, 22, 100355. https://doi.org/https://doi.org/10.1016/j.etran.2024.100355 DOI: https://doi.org/10.1016/j.etran.2024.100355

Bernárdez, B., Durán, A., Parejo, J. A., & Ruiz–Cortés, A. (2018). An experimental replication on the effect of the practice of mindfulness in conceptual modeling performance. Journal of Systems and Software, 136, 153–172. https://doi.org/https://doi.org/10.1016/j.jss.2016.06.104 DOI: https://doi.org/10.1016/j.jss.2016.06.104

Bi, Z., Zhang, C. W. J., Wu, C., & Li, L. (2022). New digital triad (DT-II) concept for lifecycle information integration of sustainable manufacturing systems. Journal of Industrial Information Integration, 26, 100316. https://doi.org/https://doi.org/10.1016/j.jii.2021.100316 DOI: https://doi.org/10.1016/j.jii.2021.100316

Campelo, F., & Wanner, E. F. (2020). Sample size calculations for the experimental comparison of multiple algorithms on multiple problem instances. Journal of Heuristics, 26(6), 851–883. https://doi.org/10.1007/s10732-020-09454-w DOI: https://doi.org/10.1007/s10732-020-09454-w

Carrasco, J., García, S., Rueda, M. M., Das, S., & Herrera, F. (2020). Recent trends in the use of statistical tests for comparing swarm and evolutionary computing algorithms: Practical guidelines and a critical review. Swarm and Evolutionary Computation, 54, 100665. https://doi.org/https://doi.org/10.1016/j.swevo.2020.100665 DOI: https://doi.org/10.1016/j.swevo.2020.100665

Chen, C., Napolitano, R., Hu, Y., Kar, B., & Yao, B. (2024). Addressing machine learning bias to foster energy justice. Energy Research & Social Science, 116, 103653. https://doi.org/https://doi.org/10.1016/j.erss.2024.103653 DOI: https://doi.org/10.1016/j.erss.2024.103653

Chevalier, M., Davis, B. A. S., Heiri, O., Seppä, H., Chase, B. M., Gajewski, K., Lacourse, T., Telford, R. J., Finsinger, W., Guiot, J., Kühl, N., Maezumi, S. Y., Tipton, J. R., Carter, V. A., Brussel, T., Phelps, L. N., Dawson, A., Zanon, M., Vallé, F., … Kupriyanov, D. (2020). Pollen-based climate reconstruction techniques for late Quaternary studies. Earth-Science Reviews, 210, 103384. https://doi.org/https://doi.org/10.1016/j.earscirev.2020.103384 DOI: https://doi.org/10.1016/j.earscirev.2020.103384

Deaton, A., & Cartwright, N. (2018). Understanding and misunderstanding randomized controlled trials. Social Science & Medicine, 210, 2–21. https://doi.org/https://doi.org/10.1016/j.socscimed.2017.12.005 DOI: https://doi.org/10.1016/j.socscimed.2017.12.005

Diez-Olivan, A., Del Ser, J., Galar, D., & Sierra, B. (2019). Data fusion and machine learning for industrial prognosis: Trends and perspectives towards Industry 4.0. Information Fusion, 50, 92–111. https://doi.org/https://doi.org/10.1016/j.inffus.2018.10.005 DOI: https://doi.org/10.1016/j.inffus.2018.10.005

Ezugwu, A. E., Ikotun, A. M., Oyelade, O. O., Abualigah, L., Agushaka, J. O., Eke, C. I., & Akinyelu, A. A. (2022). A comprehensive survey of clustering algorithms: State-of-the-art machine learning applications, taxonomy, challenges, and future research prospects. Engineering Applications of Artificial Intelligence, 110, 104743. https://doi.org/https://doi.org/10.1016/j.engappai.2022.104743 DOI: https://doi.org/10.1016/j.engappai.2022.104743

Fu, Y., Yang, G., Pu, R., Li, Z., Li, H., Xu, X., Song, X., Yang, X., & Zhao, C. (2021). An overview of crop nitrogen status assessment using hyperspectral remote sensing: Current status and perspectives. European Journal of Agronomy, 124, 126241. https://doi.org/https://doi.org/10.1016/j.eja.2021.126241 DOI: https://doi.org/10.1016/j.eja.2021.126241

Garcia-Perez, A., Cegarra-Navarro, J. G., Sallos, M. P., Martinez-Caro, E., & Chinnaswamy, A. (2023). Resilience in healthcare systems: Cyber security and digital transformation. Technovation, 121, 102583. https://doi.org/https://doi.org/10.1016/j.technovation.2022.102583 DOI: https://doi.org/10.1016/j.technovation.2022.102583

Hernández-Maldonado, V. M., Erdely, A., Díaz-Viera, M., & Rios, L. (2024). Fast procedure to compute empirical and Bernstein copulas. Applied Mathematics and Computation, 477, 128827. https://doi.org/https://doi.org/10.1016/j.amc.2024.128827 DOI: https://doi.org/10.1016/j.amc.2024.128827

Jimenez-Mesa, C., Ramirez, J., Suckling, J., Vöglein, J., Levin, J., & Gorriz, J. M. (2023). A non-parametric statistical inference framework for Deep Learning in current neuroimaging. Information Fusion, 91, 598–611. https://doi.org/https://doi.org/10.1016/j.inffus.2022.11.007 DOI: https://doi.org/10.1016/j.inffus.2022.11.007

LaTorre, A., Molina, D., Osaba, E., Poyatos, J., Del Ser, J., & Herrera, F. (2021). A prescription of methodological guidelines for comparing bio-inspired optimization algorithms. Swarm and Evolutionary Computation, 67, 100973. https://doi.org/https://doi.org/10.1016/j.swevo.2021.100973 DOI: https://doi.org/10.1016/j.swevo.2021.100973

Li, L.-M., Lu, K.-D., Zeng, G.-Q., Wu, L., & Chen, M.-R. (2016). A novel real-coded population-based extremal optimization algorithm with polynomial mutation: A non-parametric statistical study on continuous optimization problems. Neurocomputing, 174, 577–587. https://doi.org/https://doi.org/10.1016/j.neucom.2015.09.075 DOI: https://doi.org/10.1016/j.neucom.2015.09.075

Madureira, L., Popovič, A., & Castelli, M. (2021). Competitive intelligence: A unified view and modular definition. Technological Forecasting and Social Change, 173, 121086. https://doi.org/https://doi.org/10.1016/j.techfore.2021.121086 DOI: https://doi.org/10.1016/j.techfore.2021.121086

Niankara, I. (2024). Evaluating the influence of digital strategy on the interplay between quality certification and sales performance using data science and machine learning algorithms. Journal of Open Innovation: Technology, Market, and Complexity, 10(3), 100354. https://doi.org/https://doi.org/10.1016/j.joitmc.2024.100354 DOI: https://doi.org/10.1016/j.joitmc.2024.100354

Niso, G., Krol, L. R., Combrisson, E., Dubarry, A. S., Elliott, M. A., François, C., Héjja-Brichard, Y., Herbst, S. K., Jerbi, K., Kovic, V., Lehongre, K., Luck, S. J., Mercier, M., Mosher, J. C., Pavlov, Y. G., Puce, A., Schettino, A., Schön, D., Sinnott-Armstrong, W., … Chaumon, M. (2022). Good scientific practice in EEG and MEG research: Progress and perspectives. NeuroImage, 257, 119056. https://doi.org/https://doi.org/10.1016/j.neuroimage.2022.119056 DOI: https://doi.org/10.1016/j.neuroimage.2022.119056

Olikh, O. (2024). A test of meta-heuristic algorithms for parameter extraction of next-generation solar cells with S-shaped current–voltage curves. Materials Science and Engineering: B, 307, 117506. https://doi.org/https://doi.org/10.1016/j.mseb.2024.117506 DOI: https://doi.org/10.1016/j.mseb.2024.117506

Osaba, E., Villar-Rodriguez, E., Del Ser, J., Nebro, A. J., Molina, D., LaTorre, A., Suganthan, P. N., Coello Coello, C. A., & Herrera, F. (2021). A Tutorial On the design, experimentation and application of metaheuristic algorithms to real-World optimization problems. Swarm and Evolutionary Computation, 64, 100888. https://doi.org/https://doi.org/10.1016/j.swevo.2021.100888 DOI: https://doi.org/10.1016/j.swevo.2021.100888

Parmezan, A. R. S., Souza, V. M. A., & Batista, G. E. A. P. A. (2019). Evaluation of statistical and machine learning models for time series prediction: Identifying the state-of-the-art and the best conditions for the use of each model. Information Sciences, 484, 302–337. https://doi.org/https://doi.org/10.1016/j.ins.2019.01.076 DOI: https://doi.org/10.1016/j.ins.2019.01.076

Pelletier, F., Masson, C., & Tahan, A. (2016). Wind turbine power curve modelling using artificial neural network. Renewable Energy, 89, 207–214. https://doi.org/https://doi.org/10.1016/j.renene.2015.11.065 DOI: https://doi.org/10.1016/j.renene.2015.11.065

Rasheed, K., Qayyum, A., Ghaly, M., Al-Fuqaha, A., Razi, A., & Qadir, J. (2022). Explainable, trustworthy, and ethical machine learning for healthcare: A survey. Computers in Biology and Medicine, 149, 106043. https://doi.org/https://doi.org/10.1016/j.compbiomed.2022.106043 DOI: https://doi.org/10.1016/j.compbiomed.2022.106043

Razavi, S., Jakeman, A., Saltelli, A., Prieur, C., Iooss, B., Borgonovo, E., Plischke, E., Lo Piano, S., Iwanaga, T., Becker, W., Tarantola, S., Guillaume, J. H. A., Jakeman, J., Gupta, H., Melillo, N., Rabitti, G., Chabridon, V., Duan, Q., Sun, X., … Maier, H. R. (2021). The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support. Environmental Modelling & Software, 137, 104954. https://doi.org/https://doi.org/10.1016/j.envsoft.2020.104954 DOI: https://doi.org/10.1016/j.envsoft.2020.104954

Roth, J., Sant’Anna, P. H. C., Bilinski, A., & Poe, J. (2023). What’s trending in difference-in-differences? A synthesis of the recent econometrics literature. Journal of Econometrics, 235(2), 2218–2244. https://doi.org/https://doi.org/10.1016/j.jeconom.2023.03.008 DOI: https://doi.org/10.1016/j.jeconom.2023.03.008

Saad, A. M., Dulaimi, M., & Zulu, S. L. (2023). Broader use of the Modern Methods of Construction (MMC) in the UK public sector: A Business Model Canvas (BMC) perspective. Journal of Open Innovation: Technology, Market, and Complexity, 9(2), 100035. https://doi.org/https://doi.org/10.1016/j.joitmc.2023.100035 DOI: https://doi.org/10.1016/j.joitmc.2023.100035

Sanchis-Segura, C., & Wilcox, R. R. (2024). From means to meaning in the study of sex/gender differences and similarities. Frontiers in Neuroendocrinology, 73, 101133. https://doi.org/https://doi.org/10.1016/j.yfrne.2024.101133 DOI: https://doi.org/10.1016/j.yfrne.2024.101133

Segundo-Marcos, R., Carrillo, A. M., Fernández, V. L., & Daza González, M. T. (2023). Age-related changes in creative thinking during late childhood: The contribution of cooperative learning. Thinking Skills and Creativity, 49, 101331. https://doi.org/https://doi.org/10.1016/j.tsc.2023.101331 DOI: https://doi.org/10.1016/j.tsc.2023.101331

Shmueli, G., Ray, S., Velasquez Estrada, J. M., & Chatla, S. B. (2016). The elephant in the room: Predictive performance of PLS models. Journal of Business Research, 69(10), 4552–4564. https://doi.org/https://doi.org/10.1016/j.jbusres.2016.03.049 DOI: https://doi.org/10.1016/j.jbusres.2016.03.049

Sivarajah, U., Kamal, M. M., Irani, Z., & Weerakkody, V. (2017). Critical analysis of Big Data challenges and analytical methods. Journal of Business Research, 70, 263–286. https://doi.org/https://doi.org/10.1016/j.jbusres.2016.08.001 DOI: https://doi.org/10.1016/j.jbusres.2016.08.001

Veček, N., Črepinšek, M., & Mernik, M. (2017). On the influence of the number of algorithms, problems, and independent runs in the comparison of evolutionary algorithms. Applied Soft Computing, 54, 23–45. https://doi.org/https://doi.org/10.1016/j.asoc.2017.01.011 DOI: https://doi.org/10.1016/j.asoc.2017.01.011

Yamasaki, M., Freire, R. Z., Seman, L. O., Stefenon, S. F., Mariani, V. C., & dos Santos Coelho, L. (2024). Optimized hybrid ensemble learning approaches applied to very short-term load forecasting. International Journal of Electrical Power & Energy Systems, 155, 109579. https://doi.org/https://doi.org/10.1016/j.ijepes.2023.109579 DOI: https://doi.org/10.1016/j.ijepes.2023.109579

Yu, Z., Guindani, M., Grieco, S. F., Chen, L., Holmes, T. C., & Xu, X. (2022). Beyond t test and ANOVA: applications of mixed-effects models for more rigorous statistical analysis in neuroscience research. Neuron, 110(1), 21–35. https://doi.org/10.1016/j.neuron.2021.10.030 DOI: https://doi.org/10.1016/j.neuron.2021.10.030

Zhang, J., Wang, Y., Zhao, Y., & Cai, X. (2018). Applications of inferential statistical methods in library and information science. Data and Information Management, 2(2), 103–120. https://doi.org/https://doi.org/10.2478/dim-2018-0007 DOI: https://doi.org/10.2478/dim-2018-0007

Zhou, T., Song, Z., & Sundmacher, K. (2019). Big Data Creates New Opportunities for Materials Research: A Review on Methods and Applications of Machine Learning for Materials Design. Engineering, 5(6), 1017–1026. https://doi.org/https://doi.org/10.1016/j.eng.2019.02.011 DOI: https://doi.org/10.1016/j.eng.2019.02.011