Introduction and theoretical framework
Despite its frequent use in various contexts, the concept of quality is perceived in different meanings, which usually result from the context in which quality is mentioned. If it is problematic to define a general definition of quality, then it seems no less difficult to define quality in the context of education. Rýdl (2002) states that applying a uniform quality assessment model in education is extremely difficult, even while maintaining the generality of such a model. Similarly, Janík (2013) and Veteška (2016) argue that it is difficult to achieve a uniform understanding of the concept of quality in education, because different actors define quality differently, with regard to their pursued interests. Nevertheless, it is important to define and systematically evaluate quality in education (Schindlerová et al., 2015).
Quality in school can be understood absolutely or relatively. In the first case, the quality of educational processes means the desired level of functioning or production of these processes, which can be prescribed by certain requirements, and can therefore be objectively measured and evaluated (Průcha, 1996). The second option is the tools of excellence based on the transformational concept of quality, which rather than the system and procedures is related to the constant improvement and transformation of the school (Sallis, 2002). In this concept, the main goal of the school is the satisfaction of its customers. However, student satisfaction is not a homogenous and sufficiently researched construct. Findings (Wiers-Jenssen et al., 2002; Richardson, 2005) show that it is a concept strongly influenced by a number of contextual factors, regardless of the quality of teaching. As an argument against equating student satisfaction with the quality of teaching, Richardson (2005) also used a reference to the research results of psychologists dealing with the intellectual development of students (Perry, 1970; Baxter Magolda, 1992), whose findings confirm that intellectual development is associated with a certain discomfort.
Aldridge (2016), Corradino and Fogarty (2016) and others, on the other hand, perceive student satisfaction as one of the indicators of quality and draw attention to its influence on student engagement, educational results and access to education. Monitoring the satisfaction of graduates was often not a priority for secondary schools. Tracking school pupils after graduation was not always easy. Schools often had to rely on the fact that graduates would contact them if they wanted to participate in the survey. However, with the development of the Internet, social networks and communication technologies in general, the tracking of graduates becomes easier and, thanks to the possibilities of electronic questionnaires, also fast, efficient and user-friendly.
At present, gathering feedback from graduates is commonly used as a quality management tool, and in the field of vocational education it has been legislatively recommended since 2017 (Council Recommendation C 423/01, 2017). In the professional press, this issue has not yet been sufficiently processed, the National Institute for Education (now the National Pedagogical Institute) was engaged in monitoring, or tracing the educational path of graduates through comparative analyses, for the purpose of ascertaining the employment of graduates on the labor market and monitoring the unemployment rate according to the majors studied, as well as monitoring the unemployment rate from the point of view of regions in the Czech Republic. (Chamoutová et al., 2020)
It is possible to determine the satisfaction of pupils with their school in real time or in the form of a longitudinal survey, in which the satisfaction of pupils is examined after a certain period of time or retrospectively. It can be assumed that in the case of a time gap, the opinions of pupils - graduates on the quality of the school, their satisfaction with the school, the quality of teaching and the usability of knowledge will differ from the opinions that were formulated in real time.
Methodology
The aim of the investigation was to verify the construction of a questionnaire for a n/a sample of one secondary vocational school, which was created with the aim of finding out the opinions of school graduates on the quality of teaching and their satisfaction with the school they attended through feedback. The partial goal was to find out whether the graduates of the school where the preliminary research was carried out work in the field they studied, whether their studies at secondary school brought them any benefits for future employment, whether they continue their studies at university, or what education they have already achieved and what they would recommend to improve in retrospect.
Part of the investigation was also a comparison of the opinions of graduates with secondary data, which the monitored school has been obtaining since 2010 in the form of an annual survey of pupils on the satisfaction and quality of teaching. The currently obtained data were compared with the summary data obtained in the year in which the school conducted its own satisfaction survey. The compared questions were completely identical in both questionnaires. In the course of the investigation, the assumptions determining the level of satisfaction of the graduates with the school and the quality of teaching in retrospect and the effect of time since graduation on the evaluation of satisfaction and quality of teaching were verified.
Research sample
The survey was conducted on a sample of 226 graduates of a public secondary vocational school established by the capital city of Prague. From the school years 2010/2011 to 2014/2015, only 26 graduates took part in the survey, so these data were excluded from the survey and the data of 200 respondents from 2015–2020 were used. The school, where the survey was conducted, educates pupils in five education programs of category "M", i.e. in fields of education providing secondary education with matriculation exam. The capacity of the school is 900 pupils, approximately 200 graduates leave the school every year. The Czech School Inspectorate evaluates the school as an "example of good practice", while its strengths are "quality teaching", "school climate", "students' results in matriculation exams", "material conditions for education", "equipment with ICT technology", "high interest of applicants for admission to education", "mutual respect and an open form of communication" and "school presentation and marketing".
2,036 graduates from the years 2010–2020, with whom the school had contact in its databases, were contacted by e-mail, 186 e-mails were returned as undeliverable, 1,624 graduates did not respond to repeated calls. All respondents were involved in the survey, none of the questionnaires were discarded. The low number of respondents is one of the limits of the survey and is reflected in the discussion. The structure of the respondents is shown in Table 1.
Table 1 Number of graduates involved in the survey
Investigation methods
Data collection was carried out from November 2 to 20, 2020. For data collection, a questionnaire was created containing 29 items divided into two parts, five questions were open. (Note. You can find it in the appendix) In the first part, respondents answered questions about their current job position, further studies, level of education and the amount of acquired knowledge that they used during their studies at university or in practice. In the second part, the questionnaire contained questions about graduates' satisfaction with the school they attended, opinions on the quality of teaching and teachers. These questions were identical to the questions that the school has been asking students since 2015 in the form of regular evaluation. The questionnaire was not anonymous, the graduates were identified by name, completed educational program and year of graduation. Closed questions were analyzed using descriptive statistics in the Excel program, averages and differences in answers were calculated. Open questions were evaluated qualitatively, due to their number, it was not necessary to code the questions. Due to the covid-19 pandemic, the questionnaire was distributed electronically, via the Microsoft Forms platform. The questionnaire could also be completed via mobile phones. Respondents answered on a five-point Likert scale.
To compare data over time, we used secondary data that the school obtains through a questionnaire that has been used to determine the quality of the school and student satisfaction since 2015. These data were obtained from a representative sample of respondents (the average return was 75%), and the respondents in that year were students in the first to fourth year of secondary school.
Results
From table 2, which presents the calculations of average answers, it is clear that in all monitored items the answers are above the middle of the scale (2.5 and more). Graduates from the 2015/2016 school year express the greatest satisfaction with the school and the quality of teaching (4.74, respectively 4.59). On the contrary, recent graduates from 2019/2020 express the relatively least satisfaction with the school (4.19), while graduates from 2017/2018 rate the quality of teaching the worst (4.05).
Graduates rated the applicability of knowledge for studying at universities the best in the 2018/2019 school year (4.3), the worst in 2017/2018 (3.51). The graduates rated the usability of the acquired knowledge for practical life the best in 2016/2017 (3.11) and the worst in 2018/2019 (2.65). When comparing the two observed criteria, graduates more positively assess the usefulness of the acquired education for studying at universities than for practical life.
Table 2 Calculations of average values in individual items
Table 3 shows the calculations of "r" correlations, which were performed on the graduate data using the Pearson correlation coefficient, and which showed a strong linear relationship between the quality of teaching and graduate satisfaction with the school (0.950). Pupils perceive a relatively weak relationship between the quality of teaching and the benefit of acquired knowledge for their further university studies (0.328). The linear dependence between students' satisfaction and the usefulness of their high school studies for university studies is almost zero (0.085). It is interesting to compare the dependence between the usability of the acquired education for practice and the satisfaction of pupils and the quality of teaching. Here, there is even a negative dependence (-0.537, respectively -0.560).
Table 3 Correlations of individual monitored items
Answers to open questions also brought beneficial findings. Among the most frequent answers to the question of what was most beneficial about studying in high school, the graduates answered - teaching foreign languages, professional and foreign experience, approach of teachers, study of professional subjects, emphasis on independence, quality of teaching or preparation for universities itself. Graduates would welcome more lectures on professional topics, seminars or a better connection between teaching and practice. All assess the education as high quality, they are satisfied with it, and 92% of those interviewed would choose the same secondary school. In the field of further education, 90% of school graduates continued full-time studies at universities, 10% of respondents chose private schools. Among the preferred universities were Charles University, Czech University of Agriculture, Czech Technical University, University of Economics or University of Chemical Technology. Among the fields of study, they preferred economics and management, law, humanitarian studies, and media and creative fields.
Interesting results were obtained by comparing the individual answers with the secondary data that the school obtains through a standardized questionnaire since 2015 (table 4). In all monitored items, pupils retroactively evaluate their satisfaction with the school and the quality of the school better than when they attended school.
Table 4 Comparison of opinions over time
This fact is shown in table 5, which proves that the longer the distance the graduates have from the school, the higher their satisfaction with the school and the quality of teaching. Students and graduates from 2019/2020 have the smallest difference in opinions about satisfaction (0.1) and quality of teaching (0.09). On the contrary, the biggest differences are between pupils and graduates from 2015/2016 (0.89 satisfaction and 0.55 teaching quality), while the shorter the time since graduation, the more the differences in opinions continuously decrease.
Table 5 Differences in opinions over time
Discussion and conclusions
The set goals of the investigation were met. The created questionnaire proved itself in the pilot and the necessary data was obtained through it. Despite the limits typical of retrospective studies, the survey conducted on a sample of 200 graduates of the Prague secondary vocational school produced interesting results. Although the results cannot be interpreted causally, it turned out that graduates, as well as students who evaluated their satisfaction with the school and the quality of teaching in a given year, rate it above the average on a five-point Likert scale. In this regard, previously published surveys (Liška & Veteška, 2021) confirm that Czech secondary vocational school students are not dissatisfied at school and that preconceived assumptions have been confirmed. In addition to the number of answers that the school management provided useful feedback related to the quality of teaching, the effectiveness of education, the identification of the hidden curriculum, the development of the school and the application of graduates, based on the calculations of the Pearson correlation coefficient, it can be assumed that it is not very important for the graduates what benefit their high school studies have for their further studies or practical life.
When comparing the retrospective assessment with the assessment in the given year, the effect of time was evident. The longer the time since graduation, the more positively the graduates evaluate the school. It is too bold to speculate about the reasons for this finding (e.g. displacement of negative experiences, idealization of the past based on negative experiences in the present, situational factors, etc.), however, this information may be important for the school management, as it allows them to perceive the quality in the school more comprehensively. The conducted investigation also has its other limits. An indisputable limitation is the selection of the sample. The survey was conducted on a limited number of graduates of one secondary vocational school. It can be assumed that with a different number of respondents and sample selection, different conclusions could be reached. In subsequent research, it would therefore be appropriate to work with a larger number of respondents from more schools and conduct the survey repeatedly, so that the results of the survey could provide a more comprehensive picture of the quality and effectiveness of education.
A number of secondary schools are looking for tools through which they can monitor and verify the quality of their education. One of these tools is the determination of graduates' satisfaction with their school and the quality of its teaching. Through the tracking of graduates, the school can find out a number of other facts, e.g. the quality of school educational programs, professional internships, foreign and extracurricular activities, as well as the usefulness of the acquired education for further graduate studies or practical life. Information about the integration of graduates in the labor market, career paths and further study continuation may also be interesting. Although obtaining feedback from graduates tends to be more focused on the tertiary level of education, it also brings positive effects at a lower level of education.
Author: PhDr. Mgr. Roman Liška, Ph.D., MBA Department of Andragogy and Educational Management, Faculty of Education, Charles University in Prague. This output was created within the Cooperatio program (Charles University, 2022-2026), scientific field of Education and Pedagogy (GEED).
Literature
Aldridge, J.M., Fraser, B.J., Fozdar, F., Ala'i, K., Dantas, J., & Afari, E. (2016). Students' perceptions of school climate as determinants of well-being, resilience and identity. Improving Schools 19 (1): 5–26
Baxter Magolda, MB (1992). Students' epistemologies and academic experiences: Implications for pedagogy. The Review of Higher Education, 15(3), 265-287.
Corradino, C. & Forgarty, K. (2016). Positive emotions and academic achievement. Applied psychology opus. New York University. Retrieved from http://steinhardt.nyu.edu/appsych/opus/issues/2016/spring/
Chamoutová, D., Novotná, H., Líbal, M., Trhlíková, J., & Vojtěch, J. (2020). Employment of school graduates on the labor market (1st ed.). National Pedagogical Institute of the Czech Republic.
Recommendation of the Council of November 20, 2017 on monitoring the application of graduates (2017). Official Journal of the European Union.
Janík, T. (2013). Quality (in) education: a content-oriented approach to research and improvement of teaching. Masaryk University.
Liška, R., & Veteška, J. (2021). Comparison of the satisfaction of pupils of schools using quality management systems with pupils of ordinary schools. e-Pedagogium, 21(3), 20–41
Perry, W.G. (1970). Forms of intellectual and ethical development in the college years: A scheme. New York: Holt, Rinehart & Winston
Prucha, J. (1996). Pedagogical evaluation: evaluation of educational programs, processes and results. Brno: Masaryk University.
Richardson, J.T.E. (2005). Instruments for obtaining student feedback: A review of the literature. Assessment & Evaluation in Higher Education, 30(4), 387-415.
Rydl, K. (2002). To the concept of quality in education and its evaluation. E-Pedagogium, 2(1), 104–123.
Sallis, E. (2002). Total quality management in education. Sterling, VA: Stylus Pub.
Schindler, L., Puls-Elvidge, S., Welzant, H., & Crawford, L. (2015). Definitions of Quality in Higher Education: A Synthesis of the Literature, 5(3), 3–13. https://files.eric.ed.gov/fulltext/EJ1132898.pdf.
Veteška, J. (2016). Overview of andragogy: an introduction to the study of adult education and learning. Portal.
Wiers-Jenssen, J., Stensaker, B. & Grøgaard, J.B. (2002). Student satisfaction: towards an empirical deconstruction of the concept. Quality in Higher Education, 8(2), 183–195. DOI: 10.1080/ 1353832022000004377.