Difference between revisions of "External quality assurance approaches in Europe"

From ECApedia
Jump to: navigation, search
m
Line 4: Line 4:
 
#A short description of the European [[Quality assurance|QA]] landscape
 
#A short description of the European [[Quality assurance|QA]] landscape
 
#An overview of ten dimensions of [[EQA]] systems which can be used for characterising EQA systems  
 
#An overview of ten dimensions of [[EQA]] systems which can be used for characterising EQA systems  
#A small assignment consisting of reading 4 attached short profiles of EQA systems/agencies and filling in and returning the attached matrix for the own EQA system.
+
#A small assignment consisting of reading 4 short profiles of EQA systems/agencies and filling in the matrix for the own EQA system.
 +
 
 
As background materials the experts may read about the [[Principles for the Selection of Experts|ECA principles for the selection of experts]] and the [[Recommendations for mutual recognition of institutional evaluations|ECA recommendations for institutional evaluations]].
 
As background materials the experts may read about the [[Principles for the Selection of Experts|ECA principles for the selection of experts]] and the [[Recommendations for mutual recognition of institutional evaluations|ECA recommendations for institutional evaluations]].
  

Revision as of 09:38, 24 August 2012

External quality assurance approaches in Europe: a short overview

This text consists of three brief parts:

  1. A short description of the European QA landscape
  2. An overview of ten dimensions of EQA systems which can be used for characterising EQA systems
  3. A small assignment consisting of reading 4 short profiles of EQA systems/agencies and filling in the matrix for the own EQA system.

As background materials the experts may read about the ECA principles for the selection of experts and the ECA recommendations for institutional evaluations.

The European quality assurance landscape

Higher education and therefore quality assurance within higher education remain the responsibility of the national governments and the institutions in each country. Nevertheless, there are two developments that have contributed to the emergence of a European QA landscape: QA initiatives by the European Commission and the inclusion of a QA action line in the Bologna process. In addition, the work carried out by the European Consortium for Accreditation in higher education (ECA) has put the topic of mutual recognition on the agenda by connecting the European work on quality and qualifications, and intensifying the cooperation on QA.

European Commission

QA initiatives by the European Commission started in the first half of the 1990s primarily because the Erasmus exchange programme showed the need to enable the recognition of credits study abroad based on assurance that the quality of those courses abroad was equivalent to those at home. The European Commission was interested in stimulating European mobility by introducing a European dimension to the then only handful of countries which had a national QA system in place. In 1991 the European Council of Ministers took the decision to start European pilots projects for evaluating quality in higher education which were funded by the Commission and carried out in 17 countries in 1994-1995. These pilot projects created an informal network which was formalised by the creation of the European Network for Quality Assurance (ENQA) , with funding by the Commission, after the adoption of a Recommendation of the European Parliament and Council in 1998. When the Bologna process concentrated on QA the Commission continued to use its influence and resources to steer important QA developments, most notably the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG) and the European Quality Assurance Register for Higher Education (EQAR). The Commission also built on its legal powers for the regulated professions and used the power of its purse to foster the development of quality labels in disciplines such as engineering, chemistry, music, etc. Professional associations and accreditation agencies in these disciplines were encouraged to organise themselves on a European level and establish European quality labels and accreditation mechanisms for programmes in these disciplines. This development was supported by the second Recommendation of the European Parliament and Council in 2006 which called for Member States to enable institutions to seek accreditation from registered agencies outside their own country. The Recommendation also invited the Commission to continue its support for cooperation on QA and to present triennial progress reports regarding QA of which the first one was published in 2009.

In 1999 the Bologna Declaration mentioned “European co-operation in quality assurance with a view to developing comparable criteria and methodologies” as one of the six objectives for achieving the European Higher Education Area in the next decade.

The second Ministerial Communiqué (Prague, 2001) emphasised “the necessity of close European cooperation and mutual trust in and acceptance of national quality assurance systems; “to design scenarios for mutual acceptance of evaluation and accreditation/certification mechanisms”; and it made a call “to collaborate in establishing a common framework of reference and to disseminate best practice”.

The third Ministerial Communiqué (Berlin, 2003) established the principle that the primary responsibility for QA rests with each institution. It introduced an ambitious agenda by specifying four elements which national QA systems should include by 2005. It also called on ENQA and the European associations of institutions (EUA, EURASHE) and students (ESIB, now called ESU) “to develop an agreed set of standards, procedures and guidelines on quality assurance, to explore ways of ensuring an adequate peer review system for quality assurance and/or accreditation agencies or bodies”.

In Bergen (2005) the Ministers took up the first part by adopting the developed Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG). These ESG were influenced by the codes of good practices that were adopted previously by ECA in 2004 and the International Network for Quality Assurance Agencies in Higher Education (INQAAHE) in 2003.

In Bergen the ENQA proposal for peer reviews of QA agencies on the basis of the ESG was also endorsed . ENQA in cooperation with EUA, EURASHE and ESIB was asked to develop a European register of quality assurance agencies based on national review.

In London (2007) the Ministers adopted the E4 proposal for a European register and specified that it should be “voluntary, self-financing, independent and transparent”. The development of this register had also been supported in the 2006 Recommendation of the European Parliament and Council. In March 2008 the European Quality Assurance Register for Higher Education (EQAR) was founded by the E4.

In the Ministerial Communiqué of Leuven/Louvain-la-Neuve (2009) the Ministers asked the E4 to ensure that “the European Quality Assurance Register is evaluated externally, taking into account the views of the stakeholders”. This external evaluation took place in 2011. The review panel recommended a reflection on EQAR’s strategic role, including the relationship with ENQA. Both organisations require that QA agencies substantially comply to the ESG in order to become or stay full member (ENQA) or be included in EQAR. However, the criteria and interpretation used are not completely identical and the decisions are made by different bodies (the ENQA Board and Register Committee). In addition, not all ENQA full members apply for EQAR. As of April 2012 there are 44 ENQA full members and 28 agencies listed on EQAR.

The development of qualifications frameworks is another outcome of the Bologna process which, although not in the QA action line, has influenced QA significantly. The adoption by the Ministers in 2005 of an overarching Qualifications Framework for the European Higher Education Area (EHEA) which specifies three cycles (Bachelor and short cycle programmes, Master, Doctorate) with generic descriptors for these cycles and corresponding amounts of ECTS is an important landmark. It has made it possible for each country to develop a national qualifications framework which is in line with the EHEA framework and therefore comparable to the frameworks of other countries .

In recent years the Communiqués underline the importance of developing learning outcomes. Both qualifications frameworks and learning outcomes are now part of most EQA systems, i.e. it is assessed or evaluated whether institutions develop programmes that are in line with the European requirements for qualifications frameworks and learning outcomes.

European Consortium for Accreditation in higher education (ECA)

ECA was founded in 2003 by a dozen accreditation organisations that wanted to cooperate specifically on accreditation and get to faster and deeper cooperation than could be accomplished within the much larger and heterogeneous ENQA network . The Consortium has as its main aim the mutual recognition of accreditation and QA decisions which was encouraged explicitly in the Communiqués of Bergen and London and in the 2006 Recommendation of the European Parliament and Council. This has resulted in a number of mutual recognition agreements and in a multilateral agreement regarding the mutual recognition of accreditation of joint programmes (MULTRA). An important reason for mutual recognition of accreditation and QA decisions is that it can facilitate the recognition of accredited qualifications, and thereby enhance the European mobility strived for in the Bologna Process. ECA has therefore cooperated with recognition authorities (i.e. ENIC-NARICs) by setting up a joint declaration on mutual recognition (2005) , and more recently by encouraging good practices on the recognition of degrees awarded by joint programmes. It is believed that mutual recognition is particularly valuable in the field of joint programmes as the institutions involved have to undergo multiple national accreditation or QA procedures, and recognition issues are often all but clear. Perhaps of equivalent importance is that the strive for mutual recognition has led to an intensified cooperation on QA, resulting in practical cooperations (including joint programmes) and in an agreed set of principles. These principles include the selection of experts (2005), the accreditation of joint programmes (2007), and the use of learning outcomes in accreditation procedures (2009). In 2009 ECA members also agreed on recommendations for institutional evaluations (accreditations, audits or assessmenta). As a shift from programme accreditation to institutional approaches took place in many countries it was seen as necessary to state what these institutional evaluations should entail to make mutual recognition possible. The recommendations focus on the panel, site visit, and publication requirements for standards, criteria and outcomes. The 2009 progress report of the European Commission mentioned ECA, its mutual recognition agreements and publications of accreditation decisions on the European website Qrossroads as good practices.

The ESG are certainly among the most important outcomes of the Bologna process. The ESG consist of 3 parts comprising internal QA of higher education institutions, the process of external QA, and external QA agencies. Although the level of implementation of the ESG still varies nationally, the cyclical external reviews of agencies which is mandatory for both ENQA full membership and inclusion in EQAR, means that compliance to the ESG is checked . In 2011 the E4 have done a study on the implementation of the ESG which concluded that the ESG have impacted on the development of QA processes on the national, institutional and agency level. The E4 recommend the Ministers meeting in Bucharest in April 2012 to mandate them to carry out a careful revision of the ESG to improve their clarity, applicability and usefulness whilst maintaining the current principles.

Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG)

European standards and guidelines for internal quality assurance within higher education institutions

Policy and procedures for quality assurance

Institutions should have a policy and associated procedures for the assurance of the quality and standards of their programmes and awards. They should also commit themselves explicitly to the development of a culture which recognises the importance of quality, and quality assurance, in their work. To achieve this, institutions should develop and implement a strategy for the continuous enhancement of quality. The strategy, policy and procedures should have a formal status and be publicly available. They should also include a role for students and other stakeholders.

Approval, monitoring and periodic review of programmes and awards:

Institutions should have formal mechanisms for the approval, periodic review and monitoring of their programmes and awards.

Assessment of students

Students should be assessed using published criteria, regulations and procedures which are applied consistently.

Quality assurance of teaching staff

Institutions should have ways of satisfying themselves that staff involved with the teaching of students are qualified and competent to do so. They should be available to those undertaking external reviews, and commented upon in reports.

Learning resources and student support

Institutions should ensure that the resources available for the support of student learning are adequate and appropriate for each programme offered.

Information systems

Institutions should ensure that they collect, analyse and use relevant information for the effective management of their programmes of study and other activities.

Public information

Institutions should regularly publish up to date, impartial and objective information, both quantitative and qualitative, about the programmes and awards they are offering.

European standards for the external quality assurance of higher education

Use of internal quality assurance procedures

External quality assurance procedures should take into account the effectiveness of the internal quality assurance processes described in Part 1 of the European Standards and Guidelines.

Development of external quality assurance processes

The aims and objectives of quality assurance processes should be determined before the processes themselves are developed, by all those responsible (including higher education institutions) and should be published with a description of the procedures to be used.

Criteria for decisions

Any formal decisions made as a result of an external quality assurance activity should be based on explicit published criteria that are applied consistently.

Processes fit for purpose

All external quality assurance processes should be designed specifically to ensure their fitness to achieve the aims and objectives set for them.

Reporting

Reports should be published and should be written in a style, which is clear and readily accessible to its intended readership. Any decisions, commendations or recommendations contained in reports should be easy for a reader to find.

Follow-up procedures

Quality assurance processes which contain recommendations for action or which require a subsequent action plan, should have a predetermined follow-up procedure which is implemented consistently.

Periodic reviews

External quality assurance of institutions and/or programmes should be undertaken on a cyclical basis. The length of the cycle and the review procedures to be used should be clearly defined and published in advance.

System-wide analyses

Quality assurance agencies should produce from time to time summary reports describing and analysing the general findings of their reviews, evaluations, assessments etc.

European standards for external quality assurance agencies

Use of external quality assurance procedures for higher education

The external quality assurance of agencies should take into account the presence and effectiveness of the external quality assurance processes described in Part 2 of the European Standards and Guidelines.

Official status

Agencies should be formally recognised by competent public authorities in the European Higher Education Area as agencies with responsibilities for external quality assurance and should have an established legal basis. They should comply with any requirements of the legislative jurisdictions within which they operate.

Activities

Agencies should undertake external quality assurance activities (at institutional or programme level) on a regular basis.

Resources

Agencies should have adequate and proportional resources, both human and financial, to enable them to organise and run their external quality assurance process(es) in an effective and efficient manner, with appropriate provision for the development of their processes and procedures.

Mission statement

Agencies should have clear and explicit goals and objectives for their work, contained in a publicly available statement.

Independence

Agencies should be independent to the extent both that they have autonomous responsibility for their operations and that the conclusions and recommendations made in their reports cannot be influenced by third parties such as higher education institutions, ministries or other stakeholders.

External quality assurance criteria and processes used by the agencies

The processes, criteria and procedures used by agencies should be pre-defined and publicly available. These processes will normally be expected to include:

  1. a self-assessment or equivalent procedure by the subject of the quality assurance process;
  2. an external assessment by a group of experts, including, as appropriate, (a) student member(s), and site visits as decided by the agency;
  3. publication of a report, including any decisions, recommendations or other formal outcomes;
  4. a follow-up procedure to review actions taken by the subject of the quality assurance process in the light of any recommendations contained in the report.

Accountability procedures

Agencies should have in place procedures for their own accountability.

Characteristics of national external quality assurance systems

Professional or disciplinary evaluations and accreditations have been known for many decades in some disciplines (e.g. engineering, medicine, accountancy, architecture, social work) in some countries (most notably the UK but also e.g. engineering in France). However, external quality assurance on a national scale (for all disciplines) is a relatively new phenomenon in Europe. The origins of systematic external quality assurance on a national level can be traced back to the period from the mid 1980s till the beginnings of the 1990s when a small number of countries (France, the Netherlands, UK, Denmark) introduced it in their higher education systems. Later on in the 1990s EQA took off when more European countries introduced national EQA systems. After the Bologna Declaration, specifically following the Berlin Communiqué, national EQA systems became the norm. Today there are not only national EQA systems but also a European infrastructure to which the national QA systems have to relate. In spite of this European dimension there still is a wide diversity among national EQA systems, in aims, methodologies, standards, consequences and other characteristics. These differences reflect the diverse origins, history, policies and national legislation of the higher education systems. Nevertheless, there are also many commonalities in EQA due to the European developments, networking, cooperation and policy-borrowing.

National EQA systems can be analysed by looking at ten dimensions which together characterise each EQA system in a unique manner. These “fingerprints” of EQA systems can be placed against the background of the ESG. This way both the national and the European level are addressed. The Table below shows the ten dimensions of EQA systems and the ESG standard to which each dimension relates.

Ten dimensions of EQA

Dimensions ESG reference

1. Aims Each EQA system has at least one (and mostly more) of the following aims:

  • Transparency of educational provision and processes

Self-evaluation and site visit reports contribute to transparency. Making excellence visible (which is becoming more popular in EQA systems) is also a way of being transparent.

  • Improvement of quality of educational provision and processes

Improvement is often cited as the most important aim of EQA. It is also the most difficult one to measure; whether improvement of education is a consequence of internal or external QA is difficult to determine as there are many interfering other factors.

  • Accountability to stakeholders (government, society)

EQA can be a way to make HEIs accountable and to show whether public money for providing education is spent well. Accountability is often seen as at odds with improvement but in practice nearly all EQA systems have both aims, although accreditation tends to be more accountability-oriented than evaluations without public reports and clear outcomes.

The outcomes of EQA may lead to the validation or recognition of the qualifications obtained and provide trust to society that academic or professional standards are maintained. Several EQA systems may address this aim: e.g. programme accreditation as a prerequisite for recognition of degrees; an institutional audit which confirms that (self-accrediting) universities have sound validation procedures for qualifications in place and that academic standards are upheld; professional accreditation which confirms that a programme complies to the professional standards in this field.

2.2

2. Agency (agencies) coordinating the EQA

  • Status of the agency
  • Mission statement
  • Independence of the agency
  • Internal QA of agency


3.2
3.5
3.6
3.8

3. Type, level, range and cycle of EQA

  • Names used for EQA (terminology is often confusing; e.g. accreditation, evaluation, audit can mean different things in different systems, and the use of terms such as “systems accreditation” (“Systemakkreditiering”) is confined to specific countries)
  • Institutions
    • new and/or existing?
    • all or selection (e.g. universities of applied sciences or universities)?
    • focusing on education, research, services?
  • Programmes
    • new and/or existing?
    • all or selection (some subjects)?
    • 1st, 2nd, 3rd cycle?
  • Cyclical/periodic EQA (regular intervals)?

3.3









2.7


4. EQA methods and experts

  • Self-assessment
    • Does the self-assessment report contain descriptions, judgements, and recommendations?
    • Which kind of performance indicators or statistical information or other data are included?
    • Is there an evaluation of strengths and weaknesses?
  • External assessment
    • Does a site visit take place (in some systems there are no site visits for new programmes)?
    • What is the representation of the HEI at the site visit?
  • External experts
    • Is there a check on independence of experts?
    • Are there selection criteria for experts?
    • What is the composition of the panel: academics, professional field, student, QA , educational and international expertise?

3.7

5. Standards and criteria

  • Descriptive/checklists or prescriptive/formal criteria?
  • Minimum standards/basic quality/ thresholds or “above threshold”?
  • Internal QA system
  • Public information
  • Learning outcomes
    • intended and achieved
    • alignment with qualifications frameworks and subject benchmarks
    • international reference points
  • Assessment of students
  • Teaching staff
  • Learning resources and student support
  • Specific criteria?
    • Entry requirements
    • Duration requirements (for whole programme or part of it)
    • Curriculum/courses requirements
    • Quantitative requirements (e.g. student/staff ratio, graduation rates, proportion of PhDs or professors, etc.)



1.1, 1.2, 1.6
1.7



1.3
1.4
1.5





6. Reports

  • Written by whom?
  • Analysis/judgements substantiated?
  • Commendations and/or recommendations included?
  • Public?
  • Easy accessible for its intended readership?

3.7
2.5

7. Decision-making

  • Criteria and rules for making decisions
  • Validity of decisions (e.g. accreditation periods)
  • Appeals procedures


2.3

8. Consequences and follow-up

  • Directly or indirectly linked to funding?
  • Recognition of degrees?
  • Consequences for reputation/rankings?
  • Follow-up procedures?
  • System-wide analyses?
  • Intended (actions following internal and/or external assessment) and unintended (e.g. game-playing, resistance) consequences


2.6
2.8

9. Costs and benefits of EQA

  • Costs
    • Resources of agency
    • Costs of applications for reviews
    • Costs of internal QA of HEIs
    • Opportunity costs (e.g. time spent on QA is time not spent on teaching)
  • Benefits
  • Improvement of education
  • Quality culture/continuous improvement (internal QA)
  • Compliance (to standards/criteria)

3.4

10. Stakeholders perspectives

  • Expectations of stakeholders regarding EQA (cf. 1. Aims)
  • Involvement of stakeholders (government, employers, alumni, society, students, academics, HEIs)
    • in internal QA of HEIs
    • in external QA (reviews)
    • in follow-up to EQA
  • Satisfaction of stakeholders with EQA
2.4