3. The contents, session aims and learning outcomes

From ECApedia
Jump to: navigation, search

We now need to list the detailed information for each of the areas identified: the actual review process; the ethos and expectations of quality assurance agencies; the knowledge, skills and attitudes to effectively and professionally carry out a review.


Contents of the training

Once you have decided on the broad areas that the training must cover, you will need to think in more detail about what those areas will include. Above we identified the following broad areas for a training event: “introduce panel members to the actual review process; explain the ethos and expectations of the agency; give them the knowledge and skills and attitudes to carry out a review effectively and professionally”. We now need to list the detailed information for each of these areas. For example, if we concentrate on “introduce panel members to the actual review process”, we might decide that the kind of information that participants will need to know includes: where to find information about the review process, how to get support from the agency in understanding the review process; an introduction to the review handbook; an introduction to the National Qualifications Framework for awards; who is involved in the review process; the main stages of the review process”, and so on.

Repeating this process for all the broad areas that you identified will give a complete list of the materials which your training event needs to include. This is the material which will make up the individual sessions of your training event and give it its overall structure. But before we think about that, we need to make sure that we know (a) what the aims of each session are; and (b) what we expect our participants to have achieved in each part of the training.

Session aims

It might be helpful to remember the following:
     *   a Session Aim is a statement of what you (the trainer) intend to accomplish
     *   a Learning Outcome is a statement of what the participant is expected to achieve.


By setting the aims for each part of the training you will make it clear to participants what you are trying to do and the material you intend to cover. This will help to make the participants’ expectations of the training realistic and it will enable them to give you more accurate feedback on the training afterwards. It will help the participants to know exactly what will or will not be covered in the training event. It is important that participants know the aims before they decide to come to training so that they can decide whether it is appropriate for them or not. This will help to reduce resentment or disaffection on the training event from participants who expected something quite different!

Learning outcomes

Learning outcomes are simply the things that you want participants to have achieved by a certain point in time, probably by the end of the training event (unless you are carrying out any kind of follow-up activity). Achievements can include knowledge, skills or attitudes.

Drawing up learning outcomes is important because it will help you to know that the training material you intend to present is relevant to your overall aim for the training. It will help you to know that you are delivering material which will enable your participants to acquire the specific skills, attitudes or knowledge that they need to carry out the work that you will ask them to do. Having learning outcomes will also help you to test whether participants have achieved what you need them to achieve, so they are also a way of evaluating the training (see section 6.2. Evaluation). Learning outcomes also make it easier for participants to assess for themselves what they have learned.

When you draw up the learning outcomes, be specific. Learning outcomes should include a verb because you will expect your participants to have done something. It is important to choose the verbs in the learning outcome carefully. For example if you are presenting participants with information about the review process you may expect them to read the information, recall it, interpret it, synthesise it, evaluate it, and so on. Think carefully about the exact wording that is needed so that participants know what you expect of them. Some examples of verbs that you might consider are given in the table in Annex 3: Verbs for learning outcomes.

One way of checking whether the learning outcome will be useful for the participant’s learning and assessment of that learning is to use the familiar SMART questions: is the learning outcome Specific, Measurable, Attainable, Relevant and Time-bound? But do not worry if you cannot always meet these requirements. For example, it might be difficult to always find learning outcomes which give measurable results since some of the skills that you might wish participants to acquire will be attitudes or ‘soft’ skills: communication, empathy, gaining trust, and so on. Similarly, having a deadline for when a learning outcome should be achieved can also be problematic, since in many cases you will not know for sure that a panel member has achieved the learning outcome until they have carried out a review – and you may not be able to get feedback about that. So if some of the learning outcomes do not look too SMART it is not a serious problem! But it is worth using the questions as a check, to make sure that you have thought seriously about the wording of each learning outcome.

When you are writing learning outcomes bear the following points in mind:
-   Make the learning outcome brief and to the point; if the learning outcome begins to take up several sentences it probably means that you have amalgamated several different outcomes and you may find it difficult to know whether each one has been covered in the training
-   Set a target date; this will usually be by the end of the training, but bear in mind the point above, that participants may only be able to demonstrate that they have attained the learning outcome in the actual review
-   If particular resources will be needed to attain the learning outcome make sure it is clear who will provide them and where they can be found
-   Make the learning outcome realistic – what can you and the participants do as a result of one training event?
-   Make it challenging enough to be worth setting, not something that you already expect the participants can do
-   Specify conditions under which the learning outcome must be achieved, e.g. alone or with a buddy or in a team; using a laptop or flip chart, etc.
-   Select learning outcomes which are consistent with overall goals and values (yours, those of the participants and those of your agency/organisation); if the ethos of your agency demands politeness and collegiality do not expect participants to learn aggressive questioning behaviour!


So an example of a learning outcome might be:

“By the end of the training event each panel member will be able to use four different kinds of questioning technique in a review meeting and will have practised these techniques in a mock meeting with other participants”.

This is a reasonable learning outcome because it describes the precise knowledge to be acquired and the need for the participant to have used that knowledge in practice. However, in order to assess whether the trainer had addressed all of the material relevant to the learning outcome, and that the learning outcome was achieved by the participants, it would be even better to divide this into two learning outcomes.

For example,

By the end of the training event each panel member will
     (a)   know how to use four different kinds of questioning technique in a review meeting and
     (b)   have practised these techniques in a mock meeting with other participants.


Structure of the training – participant engagement

You have now got an outline of the material which you need to cover in your training event, divided into sessions, each with a session aim and participant learning outcomes. You will now need to think about how to structure this set of material – what kind of a programme for the training event will best ensure that participants have a good chance of achieving the learning outcomes.

As you consider the programme it is necessary to think about not only the presentation of the technical material (knowledge, skills, attitudes) as discussed above, but also other general sessions which will help the participants to engage with the training fully. To do this properly you need to have an understanding of who your participants are and we shall return to this (in section 4. Taking the needs of the participants into account). But before that we shall look at what kinds of general sessions might be useful.

Introduction to the training

First, it is very important to set the scene for your training – this will help to set the kind of ethos that you want to generate in your training event. Starting off well and making sure that your participants know that you are committed to the training, and committed to enabling them to learn, will make the rest of the training go well. So do not leave this section out, even if you do not have much time. It is a particularly important section if you do not know the participants but will need to work with them in future. It helps to build up rapport and commitment in your participants and this will encourage a professional attitude in them as they go about the work of being a panel member.

The kind of elements that you might want to cover include:


Introduce yourself and, if there is time, let participants introduce themselves in a plenary session. It is important to introduce yourself so that you can give a short explanation of why you are qualified to give the training – this will increase the credibility of the event. If participants are able to have the opportunity to say just a little about themselves this may help them to identify better with other participants, and be prepared to engage more fully. If it is a large group and time is short, divide the participants into small groups and let the members of each group introduce each other. If you have the luxury of a longer training event there are many ways that you can use the opening session to help participants get to know one another and begin to feel comfortable working together. Such activities are sometimes called ‘ice-breakers’. You should ensure that any activities that you use are suitable for the audience that you are training, and are relevant to the kind of skills and information you are delivering. Many websites and publications provide examples of such activities. Some are included in Annex 2: References .

Reminder of the aims of the training event

It is worth making sure that the participants know what you are aiming to do in the training event. This will help to make their expectations realistic and lessen the likelihood that participants complain that you did not give them the relevant information. You should have specified the aims before inviting participants to the training so that they know what they are getting!

Learning on the event

Tell the participants what you will do or provide to help them learn effectively on the event. You could encourage them to fill out a learning log or list of issues and questions that they want to think about; you may give them other kinds of handouts, such as the PowerPoint slides you are using. Emphasize that you yourself are a learning resource and so are other participants. The point of setting ground rules for learning (see next point) is to enable everyone to learn as effectively as they can during the event, and to enjoy it too!

Ground rules and confidentiality

If you have time allow the participants to suggest the ground rules for the event, e.g. expecting everyone to be punctual, no use of mobile phones; openness to others’ opinions; respect for other participants; and so on. If you are short on time, give a list of the ground rules that you would like to use, and ask for any objections to them. You can modify them quickly to accommodate your participants. It is particularly important to agree a confidentiality agreement. During the training participants might swap confidential information about their organizations. The group of participants needs to agree that what is discussed in the training room is confidential, or to agree not to share sensitive information.

Recap of the expected learning outcomes of the training event

Again, this will help to remind participants what the event is all about and to remove any false expectations about the material that you will cover. It will also give participants a set of benchmarks against which they can evaluate their learning on the event.

Outline of the training event and house-keeping announcements

You should provide participants with a programme with the running order for the event. Apart from start and finish times you may wish to leave out other start times for individual sessions so that you have a chance to modify the event as you go along. You might want to spend longer on a session that participants are finding difficult, for example, and cut down somewhere else. Also let participants know about basic house-keeping details: arrangement for emergency evacuation or fire drills; where the cloak rooms are; whether refreshments will be available all the time or just at set times. Try to reduce as much uncertainty as possible so that the participants feel at home and welcome. This is the start of good communication.

Reflection, summary of the day, action planning

If you have time, build opportunities into the programme for participants to think about the material you have delivered and to note any questions they have or issues they want to pursue. If participants keep a list of these matters then you can return to them at the end of the event. You can build in a question and answer session at the end of the event, or at certain points during the event.

Participants can use the list of issues and ideas that they build up for an action planning session at the end of the event. It is always useful to encourage participants to have a few specific actions that they can carry out when they go back to their workplace. This will help to transfer the learning that they have done during the event into the real world (see section 6. How to ensure that training has been effective).

Structure of the training – presenting the information

To recap: so far you have got an outline of the material which you need to cover in your training event, probably divided into several sessions, each with a session aim and participant learning outcomes. You have begun to think about how you will structure this material in your training event by thinking about the general sessions (Introductions, action planning, and so on) that you might want to include. Now you need to think about the technical material that you will deliver – information about knowledge, skills or aptitudes required for review panel work. You need to think about how to structure this set of material to ensure that participants have a good chance of achieving the learning outcomes. For example, you might decide to put all information about skills in one session, or you might consider that it will take more than one session. You might want to introduce the skills in the order that the reviewer will use them. You might want to separate off some skills because not all reviewers will use them – or because some might have them already. You might also consider that some learning outcomes should be achieved before the training event starts, perhaps by online or distance communication.

So although at this stage you have a good idea of the material to be covered, and the learning outcomes to be achieved, you still need to ask questions about how you structure the training material. Remember that your aim in doing this is to try to ensure that participants have the best possible chance of achieving the learning outcomes. After all, that is your overall goal. This is the result which will make your job as a QA agency staff member easier to do, and increase the job satisfaction of the panel members that you are training.

To structure the training material you need to be able to answer three simple but important questions:

It might be helpful to remember the following:
     i     When and in what order will the material be delivered?
     ii    What resources are needed to deliver the material?
     iii   How will the material be delivered?


We shall look at some possible answers to the first two questions; how training material can be delivered effectively is the subject of section 5. Effective delivery of the training event.

Order and timing

One of the first things you need to consider in planning a training is how much of it needs to be carried out face-to-face and how much could be delivered by other means such as online or by sending out training documents by email. Electronic facilities are obviously very useful for sending out documentation or exercises that you want participants to see before any face-to-face event takes place. You could combine sending out reading materials with a quiz or test to ensure that participants have read it – although this does not guarantee that they have understood it! You could also ask participants to carry out exercises which they must bring along to the face-to-face training, and which could be part of an exercise at that stage. If your organisation has a website where documentation about the review/accreditation process and any national frameworks or codes of practice can be found, then it is possible to ask participants to look at this before the training.

If your participants are very busy and time is short it might be tempting to deliver all of the training online. This is very understandable, and if the materials are well written and combined with interactive online scenarios or tests it can be very valuable. But remember that in this case your participants will not get the chance to practise skills with other participants, and you will not have the chance to observe them, or to get to know them. If these possibilities are important to you then you will need to build in a face-to-face stage for the training.

We have already started to think about time constraints. If our participants are senior members of organisations then their time will be very precious. Some participants may have to take leave from their organisation to attend training and in that case finance may be an issue. Your own time is also very valuable – you may be delivering this training in addition to all your other tasks. For all these reasons you must make the most of the time that you have available, and deliver the training in the most cost-effective way for all those who will be present. This is true whether the training is three hours long or three days long. Participants must always feel that their time has been well spent.

You will also need to take into account any travelling time that you and participants will need to reach the training venue. Even if your face-to-face training takes only three hours, that might still mean a whole day travelling or even an overnight stay for some participants. Bear this in mind when you think about what time your training will start and what time it will end.

Also think carefully about who your participants are. (There is more information about this in the next section.) Do the participants know each other already? If so, you can probably spend less time on introductions and getting to know each other. Do they have non-academic needs to think about, e.g. child care to organize? If all participants are local, but some have child care responsibilities, then you might want to start at a time that allows for those responsibilities. Knowing your participants is the start of good communication with them.

You also need to plan when you will have breaks in the programme, how long they will be and whether refreshments will be served. If your training event includes lunch think carefully about what you will do directly after lunch. This is sometimes called the “graveyard session”, for good reason: people may be sleepy and unresponsive. You might want to plan something more active or interactive straight after lunch. A plain PowerPoint presentation is probably best avoided!

Once you have decided on the length of event, number and length of sessions, and breaks, you need to check that the material is in a sequence which is intelligible and makes sense to participants. Usually it is preferable in the kind of training that you will deliver to panel members to build up the material sequentially – content later in the programme will build on what you delivered earlier. This is not the only way, however. You could instead have some standalone sessions which could be delivered in any order. In that case it is important for you to explain how they relate to one another, and to reflect on this as a plenary group when all the sessions have been delivered.

In some kinds of training it can also be useful to give participants slightly different tasks to do, perhaps in small groups. This is less likely to be the case in trainings of panel members where it is necessary for each panel member to be familiar with all the material, so that they can carry out their panel review duties consistently. You might want to consider separate parallel sessions for different kinds of panel members, such as panel secretaries, or student panel members. However, in the interests of consistency for the review process, these separate sessions should only be a minor part of the overall training event.


Probably the major resource which you will need to consider is time – the time you spend developing and delivering the training, and the time your participants spend preparing for it and attending it (if you have a face-to-face event). Time is money because there will be a financial cost associated with you spending your time providing training, and for your participants to do the training. But there will also be opportunity costs: if you and your participants are doing training, you cannot be doing anything else. Because of this, as was noted above, you need to ensure that the training is worthwhile for everyone involved: that the learning outcomes set for the training are appropriate for what your agency needs to achieve, and that the participants can achieve the learning outcomes.

But there are other resources which your training event may need which will also affect the costs of the event. You will need to consider these so that you can budget accurately for the training event, and so that your agency knows how much the event will cost. The checklist in Annex 4: Example of an aide-mémoire will help you to think about what you might need for the training.

In particular it is worth thinking very carefully about the venue of the training event and the layout of the training room. You may not have very much choice about these aspects of training, but you should still think about what is the ideal arrangement for the training, and find out in advance whether this is available. If it is not, then you will be in a position to think about how you could change things before you start the training.

For training of panel members it is likely that you will want to carry out some interactive exercises. You may consider that the participants should have the opportunity to carry out exercises like a mock meeting, or analysing a self-evaluative document or case study. You may simply want to allow them time to discuss issues and problems about the review activity. Whenever training is going to be interactive you should try to arrange the training room as flexibly as you can. For example, using an IT training room with fixed desks and monitors will not be ideal. Similarly a lecture theatre with tiers of seating will not enable you to get the most out of training. The ideal is to have the room arranged with a number of small tables so that participants can sit in small groups and have a space for small group work. Light chairs that can be moved around the room easily are ideal so that participants can work in different areas with different people. A space for a table where resource or stationery items can be laid out is helpful, as is a separate table for refreshments, if these will be provided. If you are using PowerPoint or the internet you will also need a computer, projector, screen and internet connection – and someone who can show you how to use them! The diagram in Annex 5: Organisation of the training room shows an ideal arrangement. If this cannot be provided for you, at least try to make sure that you have a room that is big enough for participants to move their chairs into small groups – and chairs that are light enough to be able to do that!

An important constraint on you the trainer is how many people you feel capable of training at one time. Panel member training is a special type of training because it is necessary to build up a relationship with the participants so that when they leave training they will have accepted your agency’s values for their work. This will help them to carry out the review process with professionalism and commitment, even if you are not with them. Training can be a very important way of building this relationship between you and the panel members, and between the panel members themselves. This is one reason why training delivered entirely online may not be ideal for training panel members. If you need to use training to start building a community of reviewers then you need to be able to meet participants and to give each participant some individual attention – however brief – during the training. You also need to give them the opportunity to approach you with their problems and ideas. For this reason a general guideline would be to have no more than 30 participants to one trainer. However, if you need to give feedback to each participant individually, or if there are a lot of interactive exercises to be facilitated, one trainer to 15 participants might be more realistic. You might want to consider working with a colleague if you need to train a large number of participants at the same time. This will give the participants the added bonus of some variety in training delivery!

Handbook chapters


King, G., 2012. Handbook for training panel members for quality assurance procedures, ECA Occassional Paper. The Hague.