In the first case, researchers are provided with an overview on 1) the main studies’ features and 2) the trends in reporting quality assessments. In our opinion, characterizing current literature on these issues is relevant for both researchers and practitioners. So far, there is still an uncertainty about how and to what extent studies ensure that the instruments used are adequate to measure digital competencies in higher education. However, the evaluation process itself and, more specifically, the quality of the employed instrument are two important topics that have not been extensively addressed in the past. This literature has been characterized on several occasions from different perspectives: ranging from concept use to organizational infrastructures, strategic leadership, and teaching practices. As recent reviews showed, the topic of digital competence in higher education comprises of a large and fertile body of studies. Regardless of the structure of the instrument, its quality is given by the fulfillment degree of two psychometric properties: reliability and validity. As a consequence, it largely depends on the quality of the employed instrument.

The act of evaluating digital competencies is, from a theoretical perspective, a measurement task. The latter is particularly important not only for diagnosing the educational community, but also to verify the extent to which an intervention program has been effective. Higher education institutions play an essential role in this context, that is, by integrating different strategies to provide these digital competencies to their educational community and building coherent evaluation processes and instruments. For this reason, it is essential that professionals have sufficient ICT skills and competencies, adapting to the demands of the modern working world. In a world governed by Information and Communication Technologies (ICTs), most of the essential processes of modern society are automatized in one way or another. In addition, we estimate that the probability of finding studies with acceptable or good reporting practices increases over time. We found that studies in which several areas of digital competence were measured were more likely to report quality assessments. In general, the studies had a moderate to high depth of evidence on the assessments performed. 36% of the studies did not report any quality assessment, while less than 50% covered both groups of reliability and validity assessments at the same time.

The studies were mostly published in low tier journals. Ad hoc, self-reported questionnaires measuring various digital competence areas were the most commonly used method for data collection.

The results indicate that most of the studies focused on medium to large samples of European university students, who attended social science programs. 73 primary studies were selected from the published literature in the last 10 years in order to 1) characterize the literature, 2) evaluate the reporting practice of quality assessments, and 3) analyze which variables explain such reporting practices. In this study, we report on a Systematic Mapping Study (SMS) on how the quality of the quantitative instruments used to measure digital competencies in higher education is assured.

Citizens 2.0.17 traits