6. How to ensure that training has been effective
- How do I plan the training so that it helps participants to retain the material?
- How do I get feedback on the training and whether it has helped participants in their panel work?
We know that much of the knowledge and skills learned on a training event is lost very quickly unless it is put into use straightaway on a day-to-day basis. We noted in section 4.1. Knowing your participants that, for many of our participants, there may be a significant gap between training and carrying out panel work. If training is to be effective it has to be retained long enough to be taken into the workplace or real-life situation – in our case, into the panel situation. We can maximise the chances of this in various ways:
- Make sure the design of training matches the needs that you and your organisation have identified. In this way you should be able to ensure that there will be opportunities for participants to use the knowledge and skills acquired on the training event, even though these might be some time in the future
- Make sure that anyone from your organisation who will be working with your panel members also knows what the training has included, and is prepared to coach panel members when they carry out panel work to help them to put skills and knowledge into practice. It is very useful if members of your organisation can attend at least one training event
- Make sure that the training includes practical examples which reflect the actual work of the panel member. These can be descriptions of the work, case studies, or practice activities such as mock meetings. Introducing activities into the training which encourage ‘doing’, rather than just ‘listening’ or ‘speaking’, also encourages retention of the material
- If you have the time during the training event try to encourage participants to discover the principles involved in panel work for themselves. We tend to remember the knowledge that we have discovered for ourselves better than things we are simply told. You could use case-studies for problem-based learning activities, or give small groups open-ended questions or problems to discuss. Remember to share the solutions that individuals or groups discover with the whole group after the exercise
- Give participants an aide-mémoire of the process that they will be using as a panel member. This could take the form of a guide to procedure (just bullet-points or headlines) which you could refer to during the training. It can then be used by participants during their panel work to ‘jog their memory’ of the main points that they learned on training
- When you design hand-outs or other training materials for the participants think about how the participant will use them after the training and try to ensure that they will still be comprehensible and useful even after the training has finished. If the training is changed after you deliver it for the first time you will need to remember to update those who have already had the training. If it is possible, it might be a good idea to put the training materials on a part of your organisation’s website dedicated to panel members. Then you will be able to update training materials easily.
- Encourage action planning on the part of participants. It was noted above (in section 3.4.2. Reflection, summary of the day, action planning) that it is a good idea to allow time in the training event for participants to reflect on their learning and put together some action points to be started when they return home. These action points will give a trigger for the participant to keep thinking about the training event, and to carry on with their learning after the training until they have a chance to carry out some panel work. It is also useful if the new panel member is given the opportunity to reflect after the first panel work, and see whether there are any new action points to consider (see section 6.3 Continued engagement with participants)
- Encourage follow-up between participants, and if you have the time, with yourself. This could be through email, online, get-togethers, support groups, etc. This will help to build up a community of panel members who can support each other (see section 6.3 Continued engagement with participants).
- If you can, use past participants as mentors. Once a panel member has been trained and has carried out some panel activity he/she may be prepared to mentor new panel members. The mentoring relationship may have many forms so you will want to put in writing what the mentor is expected to do. It may simply involve being available should the new panel member wish to email with a question. Or it could include meeting and discussing the new panel member’s work.
It is no good encouraging your participants to retain what they have learned on the programme if what you have delivered is not relevant to the learning outcomes, or has not been delivered in a way that helps them to learn. So you need to find out how effective the training has been and, if it has not worked as well as you wanted, to change some aspects of it. This is the process of evaluation and it is a very important part of the overall training cycle (see the diagram in Annex 7: The training cycle).
There are different options for carrying out evaluation and gathering feedback.
- Spot checks during the training: you can find out how participants are doing on the training event by asking questions now and again. This method is probably most helpful during a longer training event where participants have time to settle down and start learning, and where the learning outcomes are build upon each other sequentially. You could ask general questions like ‘What have you learned so far?’ or more detailed ones about the content of the programme, like ‘Can you explain the three main sections of the Qualifications Framework?’. Participants could write down their answers quickly so that you could gather them in for analysis. Simply observing the participants will also give you an idea of whether they are understanding what is going on and are finding the training material intelligible.
- Give out a ‘Reactionnaire’ straight after the training has finished. As the name implies this kind of evaluation instrument gathers the participants’ reaction to the training. It can be short or detailed, and ask a few questions or many. You could use this kind of questionnaire to ask participants whether the aims of the training have been met. To do that you would need to make sure that you had set the aims of the training, and then decide what questions you would ask the participants to try to establish whether your aims have been met. An example is given in Annex 8: Example of a training ‘Reactionnaire’.
- Evaluation of key learning outcomes: this is a more sophisticated kind of evaluation questionnaire. It can ask all the questions in a Reactionnaire, but it concentrates on asking the participants whether they think they have been able to meet the learning outcomes of the training. This kind of evaluation demands (a) that you have set learning outcomes; and (b) that the participants know what they are.
- Group discussion: if you have some time after the training event has ended and some of your participants can stay around for an hour you could run a group discussion on how the training went. You could ask the same questions as in the Reactionnaire or learning outcome questionnaire, but having the group of people in the room means that you can drill down into their responses and get more information about why aspects of the training worked well, and why some did not. Or you could focus on the aspects of the training that you personally were less confident about. Remember though that the opinions expressed by the small group may not be completely representative of the whole group of participants.
- Delayed questionnaire: it can be very useful to give participants a questionnaire some time after the training (and ideally after they have carried out some panel work) to ask whether the training was appropriate for their panel task and whether they have been able to put the training to use. This can be in addition to a Reactionnaire and/or Evaluation of learning outcomes, depending on how much time and money you can spend on evaluation. It might make more sense to invest in detailed evaluation at the start of a training cycle, so that you can be sure that the training is meeting its aims, and then perhaps reduce evaluation, or use different evaluation instruments, later on.
- Pre-training questionnaire: the delayed questionnaire could be combined with a pre-training questionnaire which asks the participants for their perception of how well prepared they are for panel work. They can then be asked the same questions after training and/or after their first panel work to see whether they feel more confident and better prepared for their work. If they do not, then something is not working and you need to think again about the learning outcomes of the training.
These kinds of questionnaires are designed to help with your own developmental needs and with the development of the training event. They will provide useful information for you but may not be sophisticated enough for other purposes, such as demonstrating a business case, or applying for funding. If you need sophisticated or statistical information you may need to consult a professional who can devise evaluation methods for you. If you are tempted to think that evaluation is a waste of time, remember that it will actually save you time and effort in the future. If you have chosen your learning outcomes carefully, and delivered training effectively, this means that you will produce panel members who know the process, adhere to that process, behave professionally and produce reports on time. If this is not happening the training is a waste of time and money, and your life will be harder not easier. So you need to find out whether your training is working, and if it is not, why not.
Continued engagement with participants
In the last section we noted that giving participants the opportunity to keep in contact after the training was one of the ways you could increase retention of the training material and encourage participants to put the training into action when they carry out panel work. In this way participants can continue to ask questions about panel work and increase their knowledge of the processes that they will be working in. They can also swap ideas for dealing with difficult situations. They can mentor or buddy one another to develop good practice in panel activities. In this way they will build up a community of panel members who will help to improve not only their own performances but also increase the consistency with which panel activities are carried out.
The most obvious way that you can facilitate this activity is providing an online facility, like a ‚chat room’, or by enabling participants to keep in email contact with each other. Unless you have made this a condition of the training, you should make sure that all participants are happy about having personal details such as email addresses circulated to everyone else. You should also make it clear what the rules for communication are, and whether anyone will moderate the chat room or emails.
If you have the resources, offering refresher training, annual meetings, or workshops are very good ways of keeping your community of panel members together and will give you opportunities to update them on any changes to the panel processes. Of course, providing these activities is expensive and panel members may not have the time to attend, so you might choose to do this updating using a website. Another alternative is to set aside a small amount of time before a panel activity (that is, a review, accreditation event, etc.) to update and refresh panel members’ knowledge before they start a new piece of panel work.
You can also encourage new panel members to reflect on their first panel event and draw up any action points that they feel they should address. If you have the resources, it would be good for a member of your agency to go through the action points with the new panel member to ensure that they can be dealt with successfully – perhaps with a mentor, or through using information on your website.
If a member of your agency is present with the panel team in its panel work, then you might be able to give a new panel member immediate feedback on their first panel activity. If you do this, then ideally you should have some criteria against which to give feedback. Some examples might be: was written work completed before the deadline? was the panel member punctual at the panel event? did the panel member contribute to panel discussions constructively and knowledgeably? did the panel member behave in a professional way with meetings with the institution? Work out what you want your panel member to do, and how you want them to behave. Then draw up your feedback criteria to match.
In order to be most effective training should not end when the last participant leaves the training room. If you can encourage panel members to stay in contact with you and with each other you will help to produce a community of panel members who will not only be interested in keeping their knowledge up to date, but also help to improve panel processes, and provide a source of support for you and for each other.
- 1. Introduction
- 2. Overall aim and general area
- 3. The contents, session aims and learning outcomes
- 4. Taking the needs of the participants into account
- 5. Effective delivery of the training event
- 6. How to ensure that training has been effective
King, G., 2012. Handbook for training panel members for quality assurance procedures, ECA Occassional Paper. The Hague.