Human and Information Interaction

Determining user interface accessibility evaluation indicators in virtual education systems using the BWM method

Document Type : Original Article

Authors

Yaghoub Norouzi 1
Nayereh JafariFar 2
Zahra Bighlari 3

1 Department of Knowledge and Information Science. Department of Knowledge and Information Science, University of Qom, Iran

2 MSc in Knowledge and Information Science; Department of Knowledge and Information Science; Qom University of Medical Sciences, Qom, Iran.

3 MSc in Knowledge and Information Science; Department of Knowledge and Information Science; the University of Qom, Hazrat-e Ma’soumeh University (HMU), Qom. Iran

Abstract
purpose: The article aims to identify and prioritize indicators for evaluating the accessibility capabilities of the user interface in virtual education systems.

Methodology: In step 1, to identify the evaluation indicators from the indicators listed in ISO 9241, a localized checklist was prepared by the virtual education systems inside the country. Then, using the Fuzzy Delphi method and the opinions of experts from the higher education system of Iran in the field of accessibility of education and e-learning systems, the identified indicators were modified and finalized. In the next step, these indicators were prioritized using the BWM method from the point of view of experts, and their importance was determined.

Findings: The final framework of indicators for evaluating the accessibility capabilities of the interaction environment (user interface) of virtual education systems was compiled and prioritized in four general indicators (general guidelines and requirements, inputs, outputs, support services, help, and online documentation) and 24 sub-indices. For this purpose, LINGO software was used. Based on the findings of the research, the component "Compatibility with accessible technologies" won the first rank among all sub-indices. The component "online documentation" was ranked second and "setting accessibility levels" was ranked third. The last rank (rank 24) was awarded to "Camera".

Conclusion: There is no consensus on the standard framework for evaluating the accessibility of virtual education systems. The review of the conducted research showed that there is a research gap in the field of not comprehensively identifying and presenting a comprehensive and coherent picture to evaluate the accessibility of the interaction environment in virtual education systems and it was concluded that to improve the use of virtual education systems, identifying and prioritizing the factors It is necessary to evaluate the accessibility of virtual education systems. The innovation of this article is to provide a comprehensive framework for identifying and prioritizing the accessibility evaluation indicators of the interactive environment in virtual education systems localized for the country. The internet speed in Iran is not high and turning on the camera during virtual classes due to the high volume of the internet it consumes causes disconnection and communication between them. The statistical community of the research was aware of this fact, therefore, according to the existing conditions, they assigned the least weight to this index.

Keywords

accessibility
Fuzzy Delphi
User interface
best-worst method (BWM)
virtual education system
interactive environment
Abran, A., Khelifi, A., Suryn, W., & Seffah, A. (2003). Usability meanings and interpretations in ISO standards. Software quality journal, 11(4), 325-338. [DOI:10.1023/A:1025869312943]
Abran, A., Khelifi, A., Suryn, W., & Seffah, A. (2003). Usability meanings and interpretations in ISO standards. Software quality journal, 11(4), 325-338.
Amado-Salvatierra, H. R., Hernández, R., & Hilera, J. R. (2012). Implementation of accessibility standards in the process of course design in virtual learning environments. Procedia Computer Science, 14, 363-370. [DOI:10.1016/j.procs.2012.10.042]
Askarinejad, M. (2021). Providing a Structural Model for the Use of E-Learning System with Empha-sis on the Mediation of Behavioral
Amado-Salvatierra, H. R., Hernández, R., & Hilera, J. R. (2012). Implementation of accessibility standards in the process of course design in virtual learning environments. Procedia Computer Science, 14, 363-370.
Azeta, A. A., Ayo, C. K., Atayero, A. A., & Ikhu-Omoregbe, N. A. (2010). Application of voiceXML in e-learning systems. In Cases on Successful E-Learning Practices in the Developed and Developing World: Methods for the Global Information Economy (pp. 92-108). IGI Global. [DOI:10.4018/978-1-60566-942-7.ch007]
Askarinejad, M. (2021). Providing a Structural Model for the Use of E-Learning System with Empha-sis on the Mediation of Behavioral
Azeta, A. A., Ayo, C. K., Atayero, A. A., & Ikhu-Omoregbe, N. A. (2010). Application of voiceXML in e-learning systems. In Cases on Successful E-Learning Practices in the Developed and Developing World: Methods for the Global Information Economy (pp. 92-108). IGI Global.
Bouzon, M., Govindan, K., Rodriguez, C. M. T., & Campos, L. M. (2016). Identification and analysis of reverse logistics barriers using method and AHP. Resources, conservation and recycling, 108, 182-197. [DOI:10.1016/j.resconrec.2015.05.021]
Chang, V., Walters, R. J., & Wills, G. (2013). The development that leads to the Cloud Computing Business Framework. International Journal of Information Management, 33(3), 524-538. [DOI:10.1016/j.ijinfomgt.2013.01.005]
Chua, B. B., & Dyson, L. E. (2004, December). Applying the ISO 9126 model to the evaluation of an e-learning system. In Proc. of ASCILITE (Vol. 5, No. 8, pp. 184-190).
Bouzon, M., Govindan, K., Rodriguez, C. M. T., & Campos, L. M. (2016). Identification and analysis of reverse logistics barriers using method and AHP. Resources, conservation and recycling, 108, 182-197.
Dawood, K. A., Zaidan, A. A., Sharif, K. Y., Ghani, A. A., Zulzalil, H., & Zaidan, B. B. (2021). Novel multi-perspective usability evaluation framework for selection of open source software based on BWM and group VIKOR techniques. International Journal of Information Technology & Decision Making, 1-91. [DOI:10.1142/S0219622021500139]
Chang, V., Walters, R. J., & Wills, G. (2013). The development that leads to the Cloud Computing Business Framework. International Journal of Information Management, 33(3), 524-538.
Forman, E. H., & Selly, M. A. (2001). Decision by objectives: how to convince others that you are right. World Scientific.
Chua, B. B., & Dyson, L. E. (2004, December). Applying the ISO 9126 model to the evaluation of an e-learning system. In Proc. of ASCILITE (Vol. 5, No. 8, pp. 184-190).
Granić, A. (2008). Experience with usability evaluation of e-learning systems. Universal Access in the Information Society, 7(4), 209-221. [DOI:10.1007/s10209-008-0118-z]
Dawood, K. A., Zaidan, A. A., Sharif, K. Y., Ghani, A. A., Zulzalil, H., & Zaidan, B. B. (2021). Novel multi-perspective usability evaluation framework for selection of open source software based on BWM and group VIKOR techniques. International Journal of Information Technology & Decision Making, 1-91.
Hsu, Y. L., Lee, C. H., & Kreng, V. B. (2010). The application of Method and Fuzzy AHP in lubricant regenerative technology selection. Expert Systems with Applications, 37(1), 419-425. [DOI:10.1016/j.eswa.2009.05.068]
Ishikawa, A., Amagasa, M., Shiga, T., Tomizawa, G., Tatsuta, R., & Mieno, H. (1993). The max-min Delphi method and method via fuzzy integration. Fuzzy sets and systems, 55(3), 241-253. [DOI:10.1016/0165-0114(93)90251-C]
Kannan, D., de Sousa Jabbour, A. B. L., & Jabbour, C. J. C. (2014). Selecting green suppliers based on GSCM practices: Using fuzzy TOPSIS applied to a Brazilian electronics company. European Journal of operational research, 233(2), 432-447. [DOI:10.1016/j.ejor.2013.07.023]
Forman, E. H., & Selly, M. A. (2001). Decision by objectives: how to convince others that you are right. World Scientific.
Kiget, N. K., Wanyembi, G., & Peters, A. I. (2014). Evaluating usability of e-learning systems in universities. International Journal of Advanced Computer Science and Applications, 5(8). [DOI:10.14569/IJACSA.2014.050815]
Lotfi Zadeh, A. (1995). Fuzzy sets, Information and Control, 8, 338-353. [DOI:10.1016/S0019-9958(65)90241-X]
Granić, A. (2008). Experience with usability evaluation of e-learning systems. Universal Access in the Information Society, 7(4), 209-221.
Medina-Flores, R., & Morales-Gamboa, R. (2015). Usability evaluation by experts of a learning management system. IEEE Revista Iberoamericana de Tecnologias del Aprendizaje, 10(4), 197-203. [DOI:10.1109/RITA.2015.2486298]
Hsu, Y. L., Lee, C. H., & Kreng, V. B. (2010). The application of Method and Fuzzy AHP in lubricant regenerative technology selection. Expert Systems with Applications, 37(1), 419-425.
Muhammad, A. H., Siddique, A., Youssef, A. E., Saleem, K., Shahzad, B., Akram, A., & Al-Thnian, A. B. S. (2020). A hierarchical model to evaluate the quality of web-based e-learning systems. Sustainability, 12(10), 40-71. [DOI:10.3390/su12104071]
Ishikawa, A., Amagasa, M., Shiga, T., Tomizawa, G., Tatsuta, R., & Mieno, H. (1993). The max-min Delphi method and method via fuzzy integration. Fuzzy sets and systems, 55(3), 241-253.
Nurhudatiana, A., Hiu, A. N., & Ce, W. (2018, September). Should I use laptop or smartphone? a usability study on an online learning application. In 2018 International Conference on Information Management and Technology (ICIMTech) (pp. 565-570). IEEE. [DOI:10.1109/ICIMTech.2018.8528134]
Pourrajabi Talemi, S. (2018), Rcognizing and Prioritizing the Effective Factors in the Success of Advertisements Using BWM, Master's Thesis, Rahbord Shomal Institue. [In Persian].
Kannan, D., de Sousa Jabbour, A. B. L., & Jabbour, C. J. C. (2014). Selecting green suppliers based on GSCM practices: Using fuzzy TOPSIS applied to a Brazilian electronics company. European Journal of operational research, 233(2), 432-447.
Rezaei, J. (2015). Best-worst multi-criteria decision-making method. Omega, 53, 49-57. [DOI:10.1016/j.omega.2014.11.009]
Rezaei, J. (2016). Best-worst multi-criteria decision-making method: Some properties and a linear model. Omega, 64, 126-130. [DOI:10.1016/j.omega.2015.12.001]
Kiget, N. K., Wanyembi, G., & Peters, A. I. (2014). Evaluating usability of e-learning systems in universities. International Journal of Advanced Computer Science and Applications, 5(8).
Sadeghi, S., Rasouli, N., & Jandaghi, G. (2016). Identifying and prioritizing contributing factors in supply chain competitiveness by using PLS-BWM techniques (case study: Payam shoes company). World Scientific News, 2(49), 117-143.
Lotfi Zadeh, A. (1995). Fuzzy sets, Information and Control, 8, 338-353.
van Roekel, W. S. (2017). Improving international logistics performance measurement. Master of Science in Systems Engineering, Policy Analysis and Management, Netherlands: Delft University of Technology.
Medina-Flores, R., & Morales-Gamboa, R. (2015). Usability evaluation by experts of a learning management system. IEEE Revista Iberoamericana de Tecnologias del Aprendizaje, 10(4), 197-203.
Muhammad, A. H., Siddique, A., Youssef, A. E., Saleem, K., Shahzad, B., Akram, A., & Al-Thnian, A. B. S. (2020). A hierarchical model to evaluate the quality of web-based e-learning systems. Sustainability, 12(10), 40-71.
Nurhudatiana, A., Hiu, A. N., & Ce, W. (2018, September). Should I use laptop or smartphone? a usability study on an online learning application. In 2018 International Conference on Information Management and Technology (ICIMTech) (pp. 565-570). IEEE.
Rezaei, J. (2015). Best-worst multi-criteria decision-making method. Omega, 53, 49-57.
Rezaei, J. (2016). Best-worst multi-criteria decision-making method: Some properties and a linear model. Omega, 64, 126-130.
Sadeghi, S., Rasouli, N., & Jandaghi, G. (2016). Identifying and prioritizing contributing factors in supply chain competitiveness by using PLS-BWM techniques (case study: Payam shoes company). World Scientific News, 2(49), 117-143.
van Roekel, W. S. (2017). Improving international logistics performance measurement. Master of Science in Systems Engineering, Policy Analysis
and Management, Netherlands: Delft University of Technology.
Human and Information Interaction
Volume 10, Issue 1
Spring 2023
Pages 1-23