Utility Standards

The utility standards guarantee that an evaluation is oriented to the information needs of the intended users of the evaluation. (Widmer, 2003)

U1 Stakeholder Indentification
U2 Evaluator Credibility
U3 Information Scope and Selection
U4 Values Identification
U5 Report Clarity
U6 Report Timeliness and Dissemination
U7 Evaluation Impact

See the Program Evaluations Metaevaluation Checklist for more information about checkpoints for each standard and how to score them.

Close

Feasibility Standards

The feasibility standards ensure tht an evaluation is conducted in a realistic, well-considered, diplomatic and cost-conscious manner (Widmer, 2003)

F1 Practical Procedures
F2 Political Viability
F3 Cost Effectiveness

See the Program Evaluations Metaevaluation Checklist for more information about checkpoints for each standard and how to score them.

Close

Propriety Standards

The propriety standards ensure that an evaluation is carried out in a legal and ethical manner and that the welfare of the stakeholders is given due attention. (Widmer, 2003)

P1 Service Orientation
P2 Formal Written Agreements
P3 Rights of Human Subjects
P4 Human Interactions
P5 Complete and Fair Assessment
P6 Disclosure of Findings
P7 Conflict of Interest
P8 Fiscal Responsibility

See the Program Evaluations Metaevaluation Checklist for more information about checkpoints for each standard and how to score them.

Close

Accuracy Standards

The accuracy standards ensure that an evaluation produces and disseminates valid and useable information. (Widmer, 2003)

A1 Program Documentation
A2 Context Analysis
A3 Described Purposes and Procedures
A4 Defensible Information Sources
A5 Valid Information
A6 Reliable Information
A7 Systematic Information
A8 Analysis of Quantitative Information
A9 Analysis of Qualitative Information
A10 Justified Conclusions
A11 Impartial Reporting
A12 Metaevaluation

See the Program Evaluations Metaevaluation Checklist for more information about checkpoints for each standard and how to score them.

Close

What standards exist to guide metaevaluations of program evaluations?

The two primary instruments for judging program evaluations are The Program Evaluations Standards (Joint Committee on Standards for Educational Evaluation, 1994) and The AEA Guiding Principles (American Evaluation Association, 1995).

The Program Evaluation Standards and The Guiding Principles are both endorsed by The American Journal of Evaluation (AJE). They have been found to be compatible and complementary. Although they both have limitations, they can be used in concert to provide evaluations that are sound and fair.

The Joint Committee Standards

The Joint Committee Standards present detailed criteria for determing an evaluation's utility, feasibility, propriety, and accuracy. The standards were developed to be used primarily in the evaluation of education and training programs and educational personnel.

A checklist to guide metaevaluations was developed by Dr. Stufflebeam based on The Joint Committee Standards. It includes 30 standards, each with 10 checkpoints. Judgments about each standard can be scored on each checkpoint using a scale of 0-10 where 0 = poor and 10 = excellent.

Example

The Guiding Principles for Evaluators

The Guiding Principles were developed for evaluators to guide the evaluation of a broader range of evaluations. While not as detailed as The Evaluation Standards, they require that evaluations be "systematic and data based, conducted by evaluators with the requisite competence, and honest; embody respect for all participating and affected persons; and take into account the diversity of interests and values that may be related to the general and public welfare." (Stufflebeam, 2001)

A checklist was developed by Dr. Stufflebeam based on the AEA Guiding Principles for Evaluators to help evaluators systematically apply the principles to evaluation studies. Judgments about each principle are reached by examining adherence to stated checkpoints using a scale ranging from poor (0) to excellent (100).