NVAO's radical change in Flanders: assessment based on an institution's individuality23/02/2021

NVAO fundamentally assumes that Flemish colleges of higher education and universities offer quality education and can also guarantee this quality themselves. The accreditation organisation now extends this mindset of trust to all assessment procedures. This means, among other things, that in the new Flemish quality assurance system, NVAO will no longer impose a defined framework with fixed criteria and templates for the assessment of new programmes and for the initial accreditation of existing programmes. Here, a NVAO committee of experts still comes along, but it starts from the individuality of the higher education institutions and their system of quality assurance (the 'own direction') and from an open dialogue. 

NVAO's innovative Appreciative Approach means more freedom but also more responsibility for the institutions. Both sides have to get used to this radically different way of thinking and working. Meanwhile, the first existing programmes in Flanders have been assessed under the new system. The graduate programme in shop management of VIVES is the first programme to receive a new style report, tailored to their own needs. Time to sound out Michel Maricau, Jan Labbe and Anja Vanroose of VIVES to find out how they experienced this different approach.

Each from their own specific background, they prepared and monitored the assessment together. With these three around the virtual table, NVAO has a complementary insight into the impact of the new system on the programme and the institution. This impact was already felt during the preparations. The previous framework from NVAO offered the contradictory advantage of being clear and unquestionable; it 'just had to be that way'. Now it was up to VIVES to fill in and shape everything itself. This brought many opportunities, but also great responsibility.  

Michel Maricau: "This is a fundamental difference to the previous system. Before, NVAO gave you a framework and everyone had to follow it, full stop. With the freedom we now have, we can no longer hide behind 'NVAO's instructions'. Now it really has to come from us. With this freedom we did not just do things haphazardly, we followed our own framework. The preparation for the NVAO assessment has confirmed internally that this own framework works fine. The colleagues have experienced for themselves that the instruments we develop are not 'compulsory numbers' but pay off. They help us focus on the added value of the programmes, both internally and externally." 

Jan Labbe adds: "Thanks to the freedom we were given by NVAO and the fact that we could use our own framework, we immediately got freedom internally as well. Because everyone within VIVES knows the cycle and knows their place in it. That creates trust. Because we could use that cycle, we were given a lot of room by our managers to prepare. We really started from the training vision. This was already informally available within the programme, but we seized the opportunity of the assessment to make it explicit. Together with the lecturers and the focus groups, we concisely summarised the 'soul' of the programme in four principles. We used these principles as anchor points throughout the preparations and in the report. This made it possible for everyone to identify with them. 

Another advantage of being able to work within one's own framework is no longer having to produce 'paper' solely for NVAO. Whereas this mountain of paper was still very large, especially in the period 2005-2013, it is now as good as gone. Anja Vanroose: "You can use the data and formats you have, without having to squeeze them into an external framework. Because shop management is a small programme, we mainly work with focus groups to collect data, for example. This also allows us to build up a good relationship with the field, which is much less the case when uniform surveys are sent out. We simply shared the results of those focus groups digitally with NVAO."  

What still has to be prepared specifically for NVAO is the self-evaluation report. But the fact that institutions can start from their own direction there as well, makes the process a lot easier. Michel Maricau: "Everything concerning evaluation, for example, belongs in our own framework under Educational Process. It used to be in the NVAO framework under GKW3 Realised Final Level. Each arrangement can be justified, but the big advantage now is that within an institution you have the same way of working that refers to our didactic concept, our educational vision." 

It is also a good test to get your own approach right, as it turns out. Jan Labbe: "We have anyway a good internal interaction between content, education and quality assurance. Because we were able to stay within our own framework, that interplay worked extra well during the preparation. We have also grown in it throughout the process. The perspectives of development and accountability are two different things. You don't always see that yourself anymore because you are in the middle of developing. At first, for example, we had a text with both descriptive and more evaluative elements. At a certain point, we decided to give a description first and then a critical reflection per chapter. It's nice that you can make those choices yourself now." 

The fact that VIVES was allowed to help shape the assessment itself was also new. Michel Maricau: "We were able to indicate ourselves which focus points we wanted to discuss and how we saw the actual course of the assessment. We prepared this in close consultation with the NVAO process coordinator. Actually, our core team did not consist of three, but of four people. This cooperation also offers you the certainty that the process you are going through is formally in order. When, for example, we had to change our proposal for a two-day site visit because of Corona, we were able to check it with the process coordinator straight away."  

This ability to shape the assessment itself proves indeed to be an excellent instrument to achieve an open dialogue. Michel Maricau is pleased with this positive effect: "It is a relatively new thing within our own direction to ask each programme what their focal points are, about which they want input from our committees. This turns out to be very efficient in order to have a real conversation. In this way, an audit (i.e. assessment by our own panel of experts, within the framework of their own direction) adds information, value and appreciation to a programme. A classic question that many of our programmes struggle with is whether they should be more specialised or more generalist. Well, from now on, they can submit that to our committees. The feedback they receive is not the truth, but it does offer a different perspective. And multiple perspectives are the basic principle of quality assurance. 

But would this positive effect also hold up when assessed by an external NVAO committee? In the past, this sometimes led to window dressing, Michel Maricau points out: "Certainly in the previous period (2005-2013), programmes sometimes used window dressing. They just had to get the accreditation. With the new NVAO approach, I have never had that feeling. Of course you want to show the best of yourself as a programme. That is logical and it is necessary. But this does not prevent you from thinking thoroughly and critically, on the contrary. It is precisely by being given the freedom by NVAO and by taking the responsibility as an institution and a programme that one should not be afraid of the assessment. 

In practice, NVAO's Appreciative Approach proved indeed to be appreciative and to lead to a dialogue instead of a one-sided accountability. Jan Labbe: "The first conversation was more about gathering information, the second was really an open conversation about our focal points and thinking together about future challenges. Anja Vanroose adds: "Because the sector is represented in the committee, you can go deeper into specific challenges. We notice, for example, that the intake of students is still too limited. This is a bottleneck profession that is not immediately seen as attractive. You can't solve something like that on your own as a programme; it is nice to be able to raise it in the NVAO committee and get input." Enquiries revealed the same positive feeling among the lecturers. "During the preparation they had the feeling that they had to defend themselves, but after the interview that feeling was completely gone. They felt that the committee was very involved with what was going on within the study programme." 

The talks ultimately resulted in a positive report, not only in terms of accreditation, but also in terms of the added value of the programme. Michel Maricau: "We clearly recognise the programme in the report. The committee substantiates its advice where necessary and does not omit to formulate recommendations. You can see our focus points there. The committee confirms that we are asking the right questions and also provides extensive input. The recommendations are therefore not so much about pressing problems that must be tackled immediately; it is more about helping to map out a direction for the coming years." 

Jan Labbe confirms: "The assessment in the new system does not put an unnecessary focus on grading. It is simply sufficient or not. This means that you are not blinded by the number of 'stars'. The report very quickly goes to the content and it is precisely with this qualitative feedback that you can move forward as a programme. To me, that is the essence of working in self-governance: expertise is not an illusion of perfection. That used to be the case: there was a bar and you had to try to keep to it as best you could. Nowadays, expertise means making responsible choices and really going for it. And that is a completely different approach. In fact, you simply expose your process instead of your product. Now, as an education, you are doing well when you clearly know why you are doing something, even if it doesn't result in a 'perfect' product." 

Finally, Michel Maricau, who as an 'old hand' has already experienced three different quality assurance systems in Flanders, is happy to give a few tips from VIVES to get even more added value out of the assessment by an NVAO committee: "We already mentioned it, but it is essential to expose the 'soul' of the programme as soon as possible. This ensures that you yourself also work in an appreciative manner and start from what the programme stands for. In addition, it is important that the support services guide the programmes in their critical reflection. All too often we assume that this is automatic, but it quickly becomes snowed under in day-to-day operations. It can therefore really help training programmes to progress if you keep on asking questions, if you get them to think systematically about how they can put into practice what they are trying to achieve.