Benefit: A favourable
or helpful situation or condition resulting from programme implementation.
Data: Material gathered during the course of an evaluation that serves as
the basis for information, discussion, and inference.
Data Collection Methods: The way facts and information about a programme
and its performance are amassed.
Data Types
Baseline Data: The situation at the outset
of a programme/project or at a specific juncture against which results and
accomplishments can be measured and assessed.
Existing Data: Application to evaluation
of data already being collected, whether by the Agency or elsewhere.
Performance
Data/Information: Data or
information collected to ÒmeasureÓ (verify) progress in achieving the intended
outcomes (as measured by the Performance Indicators) in order to help account
for and manage programme performance.
Primary Data: Data collected by an
evaluation team specifically for the purposes of the evaluation.
Secondary Data: Data collected and recorded
by others (usually at an earlier juncture) for purposes other than the current
evaluation.
Evaluation: The process of conducting a formal, in-depth examination through
objective and systematic analysis of the manner and extent to which a programme
element achieves intended results (outputs, outcomes and objectives) in terms
of its efficiency, effectiveness, impact and relevance (and, for TC,
sustainability).
Evaluation Approach/Design: The conceptual framework for conducting an
evaluation and methods used to collect information, analyze data, and arrive at
conclusions.
Evaluation Instrument or Tool:
The method to collect information or data that answers the evaluation
questions. This includes an array of items including performance measurement,
survey questionnaires, interviews, etc.
Evaluation Results: The
combination of findings, conclusions and recommendations derived from a
specific evaluation.
Evaluation Criteria: The ÒstandardsÓ by which programmes are
appraised and judged.
Effectiveness: The extent to
which a programme element achieves its objectives.
Efficiency: The productivity of the
implementation process in terms of how economically inputs are converted into
outputs.
Relevance: The degree to which a programme
element objectives and results are pertinent, valid and appropriate in meeting
Member State needs and priorities.
Impact: The longer-term effect or
consequence, direct or indirect, on the identified need which, when combined
with other efforts, results from Agency involvement.
Sustainability: The ability to maintain or
enhance a project after the withdrawal of external support.
Evaluation Models
Simple model is quick or rapid response
evaluation covering a minimum of indicators and can be done entirely
internally.
Standard model uses more complex
indicators and data collection methods, but is till done internally, and takes
more time.
Extended model involves collecting data
from external stakeholders, uses more complex indicators and analysis and
accordingly takes more time.
Evaluation Results: The combination of findings, conclusions and
recommendations derived from the evaluation process and culminating in a
written report.
Evaluation Timing
Ex-post: An examination of a
programme/project carried out some time after its completion in order to assess
its relevance, effectiveness, sustainability and impact.
Interim/Formative: Explanation of a
programme/project at a specific juncture during its implementation (often
mid-term) to assess the probability of achieving its objectives.
Terminal: An examination of progress made
towards the attainment of programme/project objectives.
Evaluation Types
Formative: An evaluation conducted while
the programme/project is still in progress, yet far enough advanced for the
evaluation results to have a formative effect upon it.
Impact: An examination of the effects and
benefits brought about by a programme/project.
Mid-term: An evaluation carried out during
implementation of a programme/project, primarily focused on the administrative
aspects thereof.
Participatory: An approach to an
evaluation which involves programme participants, including external
beneficiaries of the programme element, in the collective examination and
appraisal of the relevant programme area.
Process: An examination of the ways and
means of programme/project planning and implementation in order to identify
potential for systemic improvement.
Self-evaluation: An internal evaluation
conducted by the persons responsible for or involved with programme element,
i.e. those responsible for carrying out the programme or project. It may be a component part of and input
to a larger evaluation approach involving an ÒindependentÓ programme
evaluation.
Summative: An evaluation conducted after
the programme/project is completed, thereby serving to summarize its results
and accomplishments.
External
Factors: Those factors Ð
events, circumstances or decisions Ð beyond the control of programme
co-ordinators but which may affect (positively or negatively) the
implementation and performance of the programme and, therefore, its ability to
achieve its objectives.
Follow-up Action: The actions taken or decisions made in order to
utilize information or experience from specific monitoring or evaluation
exercises.
Measure:
To verify or observe the
presence or absence of an intended (planned) state, such as an outcome.
Observe:
To watch something
happening or a situation and to record what is seen.
Programme ÒLogicÓ
Need/Problem
(of Member State(s)): The requirement or problem existing in Member States addressed by the AgencyÕs
programme, which points to the rationale of why the Agency (rather than some other organization or
Member States themselves) is uniquely capable of helping or is mandated to
help.
Impact: The longer-term effect or change, direct
or indirect, on the identified need/ problem (as stated in the Overall
Objective) which, when combined with other efforts, results from Agency
involvement.
Objective:
A situation that should exist at the end of a specific programme/project period
as a result of the Agency activities that contribute to meeting the identified
needs of Member States.
Outcome:
A specific identifiable change brought about by Agency activities within a
specific period.
Performance
Indicator: A feature,
characteristic, or yardstick used to ÒmeasureÓ or observe (indicate) the
progress of a programme element towards achieving its expected results leading
to its objectives. They relate
exclusively to outcomes.
Output:
The products or services to be delivered within the context of a
programme/project.
Activity:
Processes that transform inputs into outputs.
Input/Resource:
Organizational, financial, human and material resources required to carry out
the programme element activities and to produce the outputs within a given time-frame.
Performance: The actual achievement of programme results,
including expected outcomes. It extends
the definition of ÒsuccessÓ from the accomplishment of activities and
successful delivery of desired outputs to the inducement of expected outcomes. Performance is successful if the
outcome is observed or verified.
Performance Assessment (Measurement): The process of assessing (measuring) outcomes
through the use of established performance indicators and parameters affecting
them.
Programme: This term has
multiple uses: (1) A generic term used to
include all elements of the AgencyÕs programme structure, including
Major Programmes, Programmes, Subprogrammes and Projects/Support Services. (2) A grouping of related subprogrammes
directed at the attainment of one or more objectives that contribute to the
broader objectives of the major programme.
Programme
Element: A
structural component that comprises a formally designated ÒprogrammeÓ Ð e.g., Major Programme, Programme,
Subprogramme, Project/Support Activity.
Results: What was achieved as a
culmination of outputs, outcomes and impact of a programme element (e.g.,
subprogramme or project).
Results-based Approach: A process that is
directed at achieving desired results. It is a process in which (a) programme
formulation revolves around a set of predefined objectives and expected
results; (b) expected results justify the resource requirements which are
derived from and linked to outputs required to achieve such results; and (c)
actual performance in achieving results is measured by objective performance
indicators.
Self-evaluation:
An internal process primarily for
the use of managers to improve their performance results. By definition, it is conducted by
persons responsible for or involved with project design and/or implementation
(i.e., those responsible for carrying out the project or programme element).
Stakeholder Participants: Persons designated by other organizational
entities (usually Departments) which have an interest in the outcome of the
evaluation to serve as their ÒrepresentativeÓ on matters related to a specific
PROGRAMME evaluation.
Transparency: Provision of sufficient information in
an easily understandable manner to higher levels of management and Member
States so they can make decisions about the programme.
Variable: A characteristic that can take on different
values.
Verify:
Examine information to confirm its accuracy or correctness.