text:
Program evaluation
From Wikipedia, the free encyclopedia
Jump to: navigation, search
Program evaluation is a systematic method for collecting, analyzing, and using information to answer questions about projects, policies and programs,[1] particularly about their effectiveness and efficiency. In both the public and private sectors, stakeholders often want to know whether the programs they are funding, implementing, voting for, receiving or objecting to are producing the intended effect. While program evaluation first focuses around this definition, important considerations often include how much the program costs per participant, how the program could be improved, whether the program is worthwhile, whether there are better alternatives, if there are unintended outcomes, and whether the program goals are appropriate and useful.[2] Evaluators help to answer these questions, but the best way to answer the questions is for the evaluation to be a joint project between evaluators and stakeholders.[3]
The process of evaluation is considered to be a relatively recent phenomenon. However, planned social evaluation has been documented as dating as far back as 2200 BC.[4] Evaluation became particularly relevant in the U.S. in the 1960s during the period of the Great Society social programs associated with the Kennedy and Johnson administrations.[5][6] Extraordinary sums were invested in social programs, but the impacts of these investments were largely unknown.
Program evaluations can involve both quantitative and qualitative methods of social research. People who do program evaluation come from many different backgrounds, such as sociology, psychology, economics, social work, and public policy. Some graduate schools also have specific training programs for program evaluation.
Contents
[hide]
1 Doing an evaluation
1.1 Assessing needs
1.2 Assessing program theory
1.3 Assessing implementation
1.4 Assessing the impact (effectiveness)
1.4.1 Program Outcomes
1.4.2 Measuring Program Outcomes
1.5 Assessing efficiency
2 Determining causation
3 Reliability, validity and sensitivity in program evaluation
3.1 Reliability
3.2 Validity
3.3 Sensitivity
4 Evaluating Collective Impact
5 Planning a program evaluation
6 Methodological constraints and challenges
6.1 The shoestring approach
6.2 Budget constraints
6.3 Time constraints
6.4 Data constraints
6.5 Five-tiered approach
6.6 Methodological challenges presented by language and culture
7 Utilization results
7.1 Persuasive utilization
7.2 Direct (instrumental) utilization
7.3 Conceptual utilization
7.4 Variables affecting utilization
7.5 Guidelines for maximizing utilization
8 Internal versus external program evaluators
8.1 Internal evaluators
8.2 External evaluators
9 Three paradigms
9.1 Positivist
9.2 Interpretive
9.3 Critical-emancipatory
10 Empowerment evaluation
10.1 Establishing a mission
10.2 Taking stock
10.3 Planning for the future
11 Transformative Paradigm
11.1 Paradigms
11.1.1 Axiology (Values and Value Judgements)
11.1.2 Ontology (Reality)
11.1.3 Epistemology (Knowledge)
11.1.4 Methodology (Systematic Inquiry)
11.2 Lenses
11.2.1 Critical Race Theory
11.2.2 Feminist Theory
11.2.3 Queer/LGBTQ Theory
12 Government requirements
13 CIPP Model of evaluation
13.1 History of the CIPP model
13.2 CIPP model
13.3 Four aspects of CIPP evaluation
13.4 Using CIPP in the different stages of the evaluation
14 See also
15 References
16 Further reading
17 External links
Doing an evaluation[edit]
Program evaluation may be conducted at several stages during a program's lifetime. Each of these stages raises different questions to be answered by the evaluator, and correspondingly different evaluation approaches are needed. Rossi, Lipsey and Freeman (2004) suggest the following kinds of assessment, which may be appropriate at these different stages:
Assessment of the need for the program
Assessment of program design and logic/theory
Assessment of how the program is being implemented (i.e., is it being implemented according to plan? Are the program's processes maximizing possible outcomes?)
Assessment of the program's outcome or impact (i.e., what it has actually achieved)
Assessment of the program's cost and efficiency
Assessing needs[edit]
A needs assessment examines the population that the program intends to target, to see whether the need as conceptualized in the program actually exists in the population; whether it is, in fact, a problem; and if so, how it might best be dealt with. This includes identifying and diagnosing the actual problem the program is trying to address, who or what is affected by the problem, how widespread the problem is, and what are the measurable effects that are caused by the problem. For example, for a housing program aimed at mitigating homelessness, a program evaluator may want to find out how many people are homeless in a given geographic area and what their demographics are. Rossi, Lipsey and Freeman (2004) caution against undertaking an intervention without properly assessing the need for one, because this might result in a great deal of wasted funds if the need did not exist or was misconceived.
Needs assessment involves the processes or methods used by evaluators to describe and diagnose social needs[7] This is essential for evaluators because they need to identify whether programs are effective and they cannot do this unless they have identified what the problem/need is. Programs that do not do a needs assessment can have the illusion that they have eradicated the problem/need when in fact there was no need in the first place. Needs assessment involves research and regular consultation with community stakeholders and with the people that will benefit from the project before the program can be developed and implemented. Hence it should be a bottom-up approach. In this way potential problems can be realized early because the process would have involved the community in identifying the need and thereby allowed the opportunity to identify potential barriers.
The important task of a program evaluator is thus to: First, construct a precise definition of what the problem is.[7] Evaluators need to first identify the problem/need. This is most effectively done by collaboratively including all possible stakeholders, i.e., the community impacted by the potential problem, the agents/actors working to address and resolve the problem, funders, etc. Including buy-in early on in the process reduces potential for push-back, miscommunication, and incomplete information later on.
Second, assess the extent of the problem.[7] Having clearly identified what the problem is, evaluators need to then assess the extent of the problem. They need to answer the ‘where’ and ‘how big’ questions. Evaluators need to work out where the problem is located and how big it is. Pointing out that a problem exists is much easier than having to specify where it is located and how rife it is. Rossi, Lipsey & Freeman (2004) gave an example that: a person identifying some battered children may be enough evidence to persuade one that child abuse exists. But indicating how many children it affects and where it is located geographically and socially would require knowledge about abused children, the characteristics of perpetrators and the impact of the problem throughout the political authority in question.
This can be difficult considering that child abuse is not a public behavior, also keeping in mind that estimates of the rates on private behavior are usually not possible because of factors like unreported cases. In this case evaluators would have to use data from several sources and apply different approaches in order to estimate incidence rates. There are two more questions that need to be answered:[8] Evaluators need to also answer the ’how’ and ‘what’ questions[8] The ‘how’ question requires that evaluators determine how the need will be addressed. Having identified the need and having familiarized oneself with the community evaluators should conduct a performance analysis to identify whether the proposed plan in the program will actually be able to eliminate the need. The ‘what’ question requires that evaluators conduct a task analysis to find out what the best way to perform would be. For example, whether the job performance standards are set by an organization or whether some governmental rules need to be considered when undertaking the task.[8]
Third, define and identify the target of interventions and accurately describe the nature of the service needs of that population[7] It is important to know what/who the target population is/are – it might be individuals, groups, communities, etc. There are three units of the population: population at risk, population in need and population in demand[7]
Population at risk: are people with a significant probability of developing the risk e.g. the population at risk for birth control programs are women of child bearing age.
Population in need: are people with the condition that the program seeks to address; e.g. the population in need for a program that aims to provide ARV’s to HIV positive people are people that are HIV positive.
Population in demand: that part of the population in need that agrees to be having the need and are willing to take part in what the program has to offer e.g. not all HIV positive people will be willing to take ARV’s.
Being able to specify what/who the target is will assist in establishing appropriate boundaries, so that interventions can correctly address the target population and be feasible to apply<[7]
There are four steps in conducting a needs assessment:[9]
Perform a ‘gap’ analyses
Evaluators need to compare current situation to the desired or necessary situation. The difference or the gap between the two situations will help identify the need, purpose and aims of the program.
Identify priorities and importance
In the first step above, evaluators would have identified a number of interventions that could potentially address the need e.g. training and development, organization development etc. These must now be examined in view of their significance to the program’s goals and constraints. This must be done by considering the following factors: cost effectiveness (consider the budget of the program, assess cost/benefit ratio), executive pressure (whether top management expects a solution) and population (whether many key people are involved).
Identify causes of performance problems and/or opportunities
When the needs have been prioritized the next step is to identify specific problem areas within the need to be addressed. And to also assess the skills of the people that will be carrying out the interventions.
Identify possible solutions and growth opportunities
Compare the consequences of the interventions if it was to be implemented or not.
Needs analysis is hence a very crucial step in evaluating programs because the effectiveness of a program cannot be assessed unless we know what the problem was in the first place.
Assessing program theory[edit]
The program theory, also called a logic model or impact pathway,[10] is an assumption, implicit in the way the program is designed, about how the program's actions are supposed to achieve the outcomes it intends. This 'logic model' is often not stated explicitly by people who run programs, it is simply assumed, and so an evaluator will need to draw out from the program staff how exactly the program is supposed to achieve its aims and assess whether this logic is plausible. For example, in an HIV prevention program, it may be assumed that educating people about HIV/AIDS transmission, risk and safe sex practices will result in safer sex being practiced. However, research in South Africa increasingly shows that in spite of increased education and knowledge, people still often do not practice safe sex.[11] Therefore, the logic of a program which relies on education as a means to get people to use condoms may be faulty. This is why it is important to read research that has been done in the area. Explicating this logic can also reveal unintended or unforeseen consequences of a program, both positive and negative. The program theory drives the hypotheses to test for impact evaluation. Developing a logic model can also build common understanding amongst program staff and stakeholders about what the program is actually supposed to do and how it is supposed to do it, which is often lacking (see Participatory impact pathways analysis). Of course, it is also possible that during the process of trying to elicit the logic model behind a program the evaluators may discover that such a model is either incompletely developed, internally contradictory, or (in worst cases) essentially nonexisistent. This decidedly limits the effectiveness of the evaluation, although it does not necessarily reduce or eliminate the program.[12]
Creating a logic model is a wonderful way to help visualize important aspects of programs, especially when preparing for an evaluation. An evaluator should create a logic model with input from many different stake holders. Logic Models have 5 major components: Resources or Inputs, Activities, Outputs, Short-term outcomes, and Long-term outcomes [13] Creating a logic model helps articulate the problem, the resources and capacity that are currently being used to address the problem, and the measurable outcomes from the program. Looking at the different components of a program in relation to the overall short-term and long-term goals allows for illumination of potential misalignments. Creating an actual logic model is particularly important because it helps clarify for all stakeholders: the definition of the problem, the overarching goals, and the capacity and outputs of the program.[13]
Rossi, Lipsey & Freeman (2004) suggest four approaches and procedures that can be used to assess the program theory.[7] These approaches are discussed below.
Assessment in relation to social needs [7]
This entails assessing the program theory by relating it to the needs of the target population the program is intended to serve. If the program theory fails to address the needs of the target population it will be rendered ineffective even when if it is well implemented.[7]
Assessment of logic and plausibility[7]
This form of assessment involves asking a panel of expert reviewers to critically review the logic and plausibility of the assumptions and expectations inherent in the program's design.[7] The review process is unstructured and open ended so as to address certain issues on the program design. Rutman (1980), Smith (1989), and Wholly (1994) suggested the questions listed below to assist with the review process.[7]
Are the program goals and objectives well defined?
Are the program goals and objectives feasible?
Is the change process presumed in the program theory feasible?
Are the procedures for identifying members of the target population, delivering service to them, and sustaining that service through completion well defined and suffiient?
Are the constituent components, activities, and functions of the program well defined and sufficient?
Are the resources allocated to the program and its various activities adequate?
Assessment through comparison with research and practice [7]
This form of assessment requires gaining information from research literature and existing practices to assess various components of the program theory. The evaluator can assess whether the program theory is congruent with research evidence and practical experiences of programs with similar concepts.[7]
Assessment via preliminary observation [7]
This approach involves incorporating firsthand observations into the assessment process as it provides a reality check on the concordance between the program theory and the program itself.[7] The observations can focus on the attainability of the outcomes, circumstances of the target population, and the plausibility of the program activities and the supporting resources.[7]
These different forms of assessment of program theory can be conducted to ensure that the program theory is sound.
Assessing implementation[edit]
Process analysis looks beyond the theory of what the program is supposed to do and instead evaluates how the program is being implemented. This evaluation determines whether the components identified as critical to the success of the program are being implemented. The evaluation determines whether target populations are being reached, people are receiving the intended services, staff are adequately qualified. Process evaluation is an ongoing process in which repeated measures may be used to evaluate whether the program is being implemented effectively. This problem is particularly critical because many innovations, particularly in areas like education and public policy, consist of fairly complex chains of action. Many of which these elements rely on the prior correct implementation of other elements, and will fail if the prior implementation was not done correctly. This was conclusively demonstrated by Gene V. Glass and many others during the 1980s. Since incorrect or ineffective implementation will produce the same kind of neutral or negative results that would be produced by correct implementation of a poor innovation, it is essential that evaluation research assess the implementation process itself.[14] Otherwise, a good innovative idea may be mistakenly characterized as ineffective, where in fact it simply had never been implemented as designed.
Assessing the impact (effectiveness)[edit]
The impact evaluation determines the causal effects of the program. This involves trying to measure if the program has achieved its intended outcomes, i.e. program outcomes.
Program Outcomes[edit]
An outcome is the state of the target population or the social conditions that a program is expected to have changed.[7] Program outcomes are the observed characteristics of the target population or social conditions, not of the program. Thus the concept of an outcome does not necessarily mean that the program targets have actually changed or that the program has caused them to change in any way.[7]
There are two kinds of outcomes, namely outcome level and outcome change, also associated with program effect.[7]
Outcome level refers to the status of an outcome at some point in time.
Outcome change refers to the difference between outcome levels at different points in time.
Program effect refers to that portion of an outcome change that can be attributed uniquely to a program as opposed to the influence of some other factor.
Measuring Program Outcomes[edit]
Outcome measurement is a matter of representing the circumstances defined as the outcome by means of observable indicators that vary systematically with changes or differences in those circumstances.[7] Outcome measurement is a systematic way to assess the extent to which a program has achieved its intended outcomes.[15] According to Mouton (2009) measuring the impact of a program means demonstrating or estimating the accumulated differentiated proximate and emergent effect, some of which might be unintended and therefore unforeseen.[16]
Outcome measurement serves to help you understand whether the program is effective or not. It further helps you to clarify your understanding of your program. But the most important reason for undertaking the effort is to understand the impacts of your work on the people you serve.[16] With the information you collect, you can determine which activities to continue and build upon, and which you need to change in order to improve the effectiveness of the program.
This can involve using sophisticated statistical techniques in order to measure the effect of the program and to find causal relationship between the program and the various outcomes. More information about impact evaluation is found under the heading 'Determining Causation'.
Assessing efficiency[edit]
Finally, cost-benefit or cost-efficiency analysis assesses the efficiency of a program. Evaluators outline the benefits and cost of the program for comparison. An efficient program has a lower cost-benefit ratio. There are two types of efficiency, namely, static and dynamic. While static efficiency concerns achieving the objectives with least costs, dynamic efficiency concerns continuous improvement. [17]
Determining causation[edit]
Perhaps the most difficult part of evaluation is determining whether the program itself is causing the changes that are observed in the population it was aimed at. Events or processes outside of the program may be the real cause of the observed outcome (or the real prevention of the anticipated outcome).
Causation is difficult to determine. One main reason for this is self selection bias.[18] People select themselves to participate in a program. For example, in a job training program, some people decide to participate and others do not. Those who do participate may differ from those who do not in important ways. They may be more determined to find a job or have better support resources. These characteristics may actually be causing the observed outcome of increased employment, not the job training program.
Evaluations conducted with random assignment are able to make stronger inferences about causation. Randomly assigning people to participate or to not participate in the program, reduces or eliminates self-selection bias. Thus, the group of people who participate would likely be more comparable to the group who did not participate.
However, since most programs cannot use random assignment, causation cannot be determined. Impact analysis can still provide useful information. For example, the outcomes of the program can be described. Thus the evaluation can describe that people who participated in the program were more likely to experience a given outcome than people who did not participate.
If the program is fairly large, and there are enough data, statistical analysis can be used to make a reasonable case for the program by showing, for example, that other causes are unlikely.
Reliability, validity and sensitivity in program evaluation[edit]
It is important to ensure that the instruments (for example, tests, questionnaires, etc.) used in program evaluation are as reliable, valid and sensitive as possible. According to Rossi et al. (2004, p. 222),[7] 'a measure that is poorly chosen or poorly conceived can completely undermine the worth of an impact assessment by producing misleading estimates. Only if outcome measures are valid, reliable and appropriately sensitive can impact assessments be regarded as credible'.
Reliability[edit]
The reliability of a measurement instrument is the 'extent to which the measure produces the same results when used repeatedly to measure the same thing' (Rossi et al., 2004, p. 218).[7] The more reliable a measure is, the greater its statistical power and the more credible its findings. If a measuring instrument is unreliable, it may dilute and obscure the real effects of a program, and the program will 'appear to be less effective than it actually is' (Rossi et al., 2004, p. 219).[7] Hence, it is important to ensure the evaluation is as reliable as possible.
Validity[edit]
The validity of a measurement instrument is 'the extent to which it measures what it is intended to measure' (Rossi et al., 2004, p. 219).[7] This concept can be difficult to accurately measure: in general use in evaluations, an instrument may be deemed valid if accepted as valid by the stakeholders (stakeholders may include, for example, funders, program administrators, et cetera).
Sensitivity[edit]
The principal purpose of the evaluation process is to measure whether the program has an effect on the social problem it seeks to redress; hence, the measurement instrument must be sensitive enough to discern these potential changes (Rossi et al., 2004).[7] A measurement instrument may be insensitive if it contains items measuring outcomes which the program couldn't possibly effect, or if the instrument was originally developed for applications to individuals (for example standardized psychological measures) rather than to a group setting (Rossi et al., 2004).[7] These factors may result in 'noise' which may obscure any effect the program may have had.
Only measures which adequately achieve the benchmarks of reliability, validity and sensitivity can be said to be credible evaluations. It is the duty of evaluators to produce credible evaluations, as their findings may have far reaching effects. A discreditable evaluation which is unable to show that a program is achieving its purpose when it is in fact creating positive change may cause the program to lose its funding undeservedly.[improper synthesis?]
Steps to Program Evaluation Framework
According to the Center for Disease Control (CDC) there are six steps to a complete program evaluation. The steps described are: engage stakeholder, describe the program, focus the evaluation design, gather credible evidence, justify conclusions, and ensure use and share lessons learned.[19] These steps can happen in a cycle framework to represent the continuing process of evaluation.
Evaluating Collective Impact[edit]
Though program evaluation processes mentioned here are appropriate for most programs, highly complex non-linear initiatives, such as those using the collective impact (CI) model, require a dynamic approach to evaluation. Collective impact is "the commitment of a group of important actors from different sectors to a common agenda for solving a specific social problem"[20] and typically involves three stages, each with a different recommended evaluation approach:
Early phase: CI participants are exploring possible strategies and developing plans for action. Characterized by uncertainty.
Recommended evaluation approach: Developmental evaluation to help CI partners understand the context of the initiative and its development:[21] "Developmental evaluation involves real time feedback about what is emerging in complex dynamic systems as innovators seek to bring about systems change."[22]
Middle phase: CI partners implement agreed upon strategies. Some outcomes become easier to anticipate.
Recommended evaluation approach: Formative evaluation to refine and improve upon the progress, as well as continued developmental evaluation to explore new elements as they emerge. Formative evaluation involves "careful monitoring of processes in order to respond to emergent properties and any unexpected outcomes."[23]
Later phase: Activities achieve stability and are no longer in formation. Experience informs knowledge about which activities may be effective.
Recommended evaluation approach: Summative evaluation "uses both quantitative and qualitative methods in order to get a better understanding of what [the] project has achieved, and how or why this has occurred."[24]
Planning a program evaluation[edit]
Planning a program evaluation can be broken up into four parts: focusing the evaluation, collecting the information, using the information, and managing the evaluation.[25] Program evaluation involves reflecting on questions about evaluation purpose, what questions are necessary to ask, and what will be done with information gathered. Critical questions for consideration include:
What am I going to evaluate?
What is the purpose of this evaluation?
Who will use this evaluation? How will they use it?
What questions is this evaluation seeking to answer?
What information do I need to answer the questions?
When is the evaluation needed? What resources do I need?
How will I collect the data I need?
How will data be analyzed?
What is my implementation timeline?
Methodological constraints and challenges[edit]
The shoestring approach[edit]
The "shoestring evaluation approach" is designed to assist evaluators operating under limited budget, limited access or availability of data and limited turnaround time, to conduct effective evaluations that are methodologically rigorous(Bamberger, Rugh, Church & Fort, 2004).[26] This approach has responded to the continued greater need for evaluation processes that are more rapid and economical under difficult circumstances of budget, time constraints and limited availability of data. However, it is not always possible to design an evaluation to achieve the highest standards available. Many programs do not build an evaluation procedure into their design or budget. Hence, many evaluation processes do not begin until the program is already underway, which can result in time, budget or data constraints for the evaluators, which in turn can affect the reliability, validity or sensitivity of the evaluation. > The shoestring approach helps to ensure that the maximum possible methodological rigor is achieved under these constraints.
Budget constraints[edit]
Frequently, programs are faced with budget constraints because most original projects do not include a budget to conduct an evaluation (Bamberger et al., 2004). Therefore, this automatically results in evaluations being allocated smaller budgets that are inadequate for a rigorous evaluation. Due to the budget constraints it might be difficult to effectively apply the most appropriate methodological instruments. These constraints may consequently affect the time available in which to do the evaluation (Bamberger et al., 2004).[26] Budget constraints may be addressed by simplifying the evaluation design, revising the sample size, exploring economical data collection methods (such as using volunteers to collect data, shortening surveys, or using focus groups and key informants) or looking for reliable secondary data (Bamberger et al., 2004).[26]
Time constraints[edit]
The most time constraint that can be faced by an evaluator is when the evaluator is summoned to conduct an evaluation when a project is already underway if they are given limited time to do the evaluation compared to the life of the study, or if they are not given enough time for adequate planning. Time constraints are particularly problematic when the evaluator is not familiar with the area or country in which the program is situated (Bamberger et al., 2004).[26] Time constraints can be addressed by the methods listed under budget constraints as above, and also by careful planning to ensure effective data collection and analysis within the limited time space.
Data constraints[edit]
If the evaluation is initiated late in the program, there may be no baseline data on the conditions of the target group before the intervention began (Bamberger et al., 2004).[26] Another possible cause of data constraints is if the data have been collected by program staff and contain systematic reporting biases or poor record keeping standards and is subsequently of little use (Bamberger et al., 2004).[26] Another source of data constraints may result if the target group are difficult to reach to collect data from - for example homeless people, drug addicts, migrant workers, et cetera (Bamberger et al., 2004).[26] Data constraints can be addressed by reconstructing baseline data from secondary data or through the use of multiple methods. Multiple methods, such as the combination of qualitative and quantitative data can increase validity through triangulation and save time and money. Additionally, these constraints may be dealt with through careful planning and consultation with program stakeholders. By clearly identifying and understanding client needs ahead of the evaluation, costs and time of the evaluative process can be streamlined and reduced, while still maintaining credibility.
All in all, time, monetary and data constraints can have negative implications on the validity, reliability and transferability of the evaluation. The shoestring approach has been created to assist evaluators to correct the limitations identified above by identifying ways to reduce costs and time, reconstruct baseline data and to ensure maximum quality under existing constraints (Bamberger et al., 2004).
Five-tiered approach[edit]
The five-tiered approach to evaluation further develops the strategies that the shoestring approach to evaluation is based upon.[27] It was originally developed by Jacobs (1988) as an alternative way to evaluate community-based programs and as such was applied to a statewide child and family program in Massachusetts, U.S.A.[28] The five-tiered approach is offered as a conceptual framework for matching evaluations more precisely to the characteristics of the programs themselves, and to the particular resources and constraints inherent in each evaluation context.[27] In other words, the five-tiered approach seeks to tailor the evaluation to the specific needs of each evaluation context.
The earlier tiers (1-3) generate descriptive and process-oriented information while the later tiers (4-5) determine both the short-term and the long-term effects of the program.[29] The five levels are organized as follows:
Tier 1: needs assessment (sometimes referred to as pre-implementation)[30]
Tier 2: monitoring and accountability
Tier 3: quality review and program clarification (sometimes referred to as understanding and refining)[31]
Tier 4: achieving outcomes
Tier 5: establishing impact
For each tier, purpose(s) are identified, along with corresponding tasks that enable the identified purpose of the tier to be achieved.[31] For example, the purpose of the first tier, Needs assessment, would be to document a need for a program in a community. The task for that tier would be to assess the community's needs and assets by working with all relevant stakeholders.[31]
While the tiers are structured for consecutive use, meaning that information gathered in the earlier tiers is required for tasks on higher tiers, it acknowledges the fluid nature of evaluation.[29] Therefore, it is possible to move from later tiers back to preceding ones, or even to work in two tiers at the same time.[31] It is important for program evaluators to note, however, that a program must be evaluated at the appropriate level.[30]
The five-tiered approach is said to be useful for family support programs which emphasise community and participant empowerment. This is because it encourages a participatory approach involving all stakeholders and it is through this process of reflection that empowerment is achieved.[28]
Methodological challenges presented by language and culture[edit]
The purpose of this section is to draw attention to some of the methodological challenges and dilemmas evaluators are potentially faced with when conducting a program evaluation in a developing country. In many developing countries the major sponsors of evaluation are donor agencies from the developed world, and these agencies require regular evaluation reports in order to maintain accountability and control of resources, as well as generate evidence for the program’s success or failure.[32] However, there are many hurdles and challenges which evaluators face when attempting to implement an evaluation program which attempts to make use of techniques and systems which are not developed within the context to which they are applied.[33] Some of the issues include differences in culture, attitudes, language and political process.[33][34]
Culture is defined by Ebbutt (1998, p. 416) as a "constellation of both written and unwritten expectations, values, norms, rules, laws, artifacts, rituals and behaviors that permeate a society and influence how people behave socially".[34] Culture can influence many facets of the evaluation process, including data collection, evaluation program implementation and the analysis and understanding of the results of the evaluation.[34] In particular, instruments which are traditionally used to collect data such as questionnaires and semi-structured interviews need to be sensitive to differences in culture, if they were originally developed in a different cultural context.[35] The understanding and meaning of constructs which the evaluator is attempting to measure may not be shared between the evaluator and the sample population and thus the transference of concepts is an important notion, as this will influence the quality of the data collection carried out by evaluators as well as the analysis and results generated by the data.[35]
Language also plays an important part in the evaluation process, as language is tied closely to culture.[35] Language can be a major barrier to communicating concepts which the evaluator is trying to access, and translation is often required.[34] There are a multitude of problems with translation, including the loss of meaning as well as the exaggeration or enhancement of meaning by translators.[34] For example, terms which are contextually specific may not translate into another language with the same weight or meaning. In particular, data collection instruments need to take meaning into account as the subject matter may not be considered sensitive in a particular context might prove to be sensitive in the context in which the evaluation is taking place.[35] Thus, evaluators need to take into account two important concepts when administering data collection tools: lexical equivalence and conceptual equivalence.[35] Lexical equivalence asks the question: how does one phrase a question in two languages using the same words? This is a difficult task to accomplish, and uses of techniques such as back-translation may aid the evaluator but may not result in perfect transference of meaning.[35] This leads to the next point, conceptual equivalence. It is not a common occurrence for concepts to transfer unambiguously from one culture to another.[35] Data collection instruments which have not undergone adequate testing and piloting may therefore render results which are not useful as the concepts which are measured by the instrument may have taken on a different meaning and thus rendered the instrument unreliable and invalid.[35]
Thus, it can be seen that evaluators need to take into account the methodological challenges created by differences in culture and language when attempting to conduct a program evaluation in a developing country.
Utilization results[edit]
There are three conventional uses of evaluation results: persuasive utilization, direct (instrumental) utilization, and conceptual utilization.
Persuasive utilization[edit]
Persuasive utilization is the enlistment of evaluation results in an effort to persuade an audience to either support an agenda or to oppose it. Unless the 'persuader' is the same person that ran the evaluation, this form of utilization is not of much interest to evaluators as they often cannot foresee possible future efforts of persuasion.[7]
Direct (instrumental) utilization[edit]
Evaluators often tailor their evaluations to produce results that can have a direct influence in the improvement of the structure, or on the process, of a program. For example, the evaluation of a novel educational intervention may produce results that indicate no improvement in students' marks. This may be due to the intervention not having a sound theoretical background, or it may be that the intervention is not conducted as originally intended. The results of the evaluation would hopefully cause to the creators of the intervention to go back to the drawing board to re-create the core structure of the intervention, or even change the implementation processes.[7]
Conceptual utilization[edit]
But even if evaluation results do not have a direct influence in the re-shaping of a program, they may still be used to make people aware of the issues the program is trying to address. Going back to the example of an evaluation of a novel educational intervention, the results can also be used to inform educators and students about the different barriers that may influence students' learning difficulties. A number of studies on these barriers may then be initiated by this new information.[7]
Variables affecting utilization[edit]
There are five conditions that seem to affect the utility of evaluation results, namely relevance, communication between the evaluators and the users of the results, information processing by the users, the plausibility of the results, as well as the level of involvement or advocacy of the users.[7]
Guidelines for maximizing utilization[edit]
Quoted directly from Rossi et al. (2004, p. 416).:[7]
Evaluators must understand the cognitive styles of decisionmakers
Evaluation results must be timely and available when needed
Evaluations must respect stakeholders' program commitments
Utilization and dissemination plans should be part of the evaluation design
Evaluations should include an assessment of utilization
Internal versus external program evaluators[edit]
The choice of the evaluator chosen to evaluate the program may be regarded as equally important as the process of the evaluation. Evaluators may be internal (persons associated with the program to be executed) or external (Persons not associated with any part of the execution/implementation of the program). (Division for oversight services,2004). The following provides a brief summary of the advantages and disadvantages of internal and external evaluators adapted from the Division of oversight services (2004), for a more comprehensive list of advantages and disadvantages of internal and external evaluators, see (Division of oversight services, 2004).
Internal evaluators[edit]
Advantages
May have better overall knowledge of the program and possess informal knowledge of the program
Less threatening as already familiar with staff
Less costly
Disadvantages
May be less objective
May be more preoccupied with other activities of the program and not give the evaluation complete attention
May not be adequately trained as an evaluator.
External evaluators[edit]
Advantages
More objective of the process, offers new perspectives, different angles to observe and critique the process
May be able to dedicate greater amount of time and attention to the evaluation
May have greater expertise and evaluation brain
Disadvantages
May be more costly and require more time for the contract, monitoring, negotiations etc.
May be unfamiliar with program staff and create anxiety about being evaluated
May be unfamiliar with organization policies, certain constraints affecting the program.
Three paradigms[edit]
Positivist[edit]
Potter (2006)[36] identifies and describes three broad paradigms within program evaluation . The first, and probably most common, is the positivist approach, in which evaluation can only occur where there are "objective", observable and measurable aspects of a program, requiring predominantly quantitative evidence. The positivist approach includes evaluation dimensions such as needs assessment, assessment of program theory, assessment of program process, impact assessment and efficiency assessment (Rossi, Lipsey and Freeman, 2004).[37] A detailed example of the positivist approach is a study conducted by the Public Policy Institute of California report titled "Evaluating Academic Programs in California's Community Colleges", in which the evaluators examine measurable activities (i.e. enrollment data) and conduct quantitive assessments like factor analysis.[38]
Interpretive[edit]
The second paradigm identified by Potter (2006) is that of interpretive approaches, where it is argued that it is essential that the evaluator develops an understanding of the perspective, experiences and expectations of all stakeholders. This would lead to a better understanding of the various meanings and needs held by stakeholders, which is crucial before one is able to make judgments about the merit or value of a program. The evaluator’s contact with the program is often over an extended period of time and, although there is no standardized method, observation, interviews and focus groups are commonly used. A report commissioned by the World Bank details 8 approaches in which qualitative and quantitative methods can be integrated and perhaps yield insights not achievable through only one method.[39]
Critical-emancipatory[edit]
Potter (2006) also identifies critical-emancipatory approaches to program evaluation, which are largely based on action research for the purposes of social transformation. This type of approach is much more ideological and often includes a greater degree of social activism on the part of the evaluator. This approach would be appropriate for qualitative and participative evaluations. Because of its critical focus on societal power structures and its emphasis on participation and empowerment, Potter argues this type of evaluation can be particularly useful in developing countries.
Despite the paradigm which is used in any program evaluation, whether it be positivist, interpretive or critical-emancipatory, it is essential to acknowledge that evaluation takes place in specific socio-political contexts. Evaluation does not exist in a vacuum and all evaluations, whether they are aware of it or not, are influenced by socio-political factors. It is important to recognize the evaluations and the findings which result from this kind of evaluation process can be used in favour or against particular ideological, social and political agendas (Weiss, 1999).[40] This is especially true in an age when resources are limited and there is competition between organizations for certain projects to be prioritised over others (Louw, 1999).[41]
Empowerment evaluation[edit]
Main article: Empowerment evaluation
Empowerment evaluation makes use of evaluation concepts, techniques, and findings to foster improvement and self-determination of a particular program aimed at a specific target population/program participants.[42] Empowerment evaluation is value oriented towards getting program participants involved in bringing about change in the programs they are targeted for. One of the main focuses in empowerment evaluation is to incorporate the program participants in the conducting of the evaluation process. This process is then often followed by some sort of critical reflection of the program. In such cases, an external/outsider evaluator serves as a consultant/coach/facilitator to the program participants and seeks to understand the program from the perspective of the participants. Once a clear understanding of the participants perspective has been gained appropriate steps and strategies can be devised (with the valuable input of the participants) and implemented in order to reach desired outcomes.
According to Fetterman (2002)[42] empowerment evaluation has three steps;
Establishing a mission
Taking stock
Planning for the future
Establishing a mission[edit]
The first step involves evaluators asking the program participants and staff members (of the program) to define the mission of the program. Evaluators may opt to carry this step out by bringing such parties together and asking them to generate and discuss the mission of the program. The logic behind this approach is to show each party that there may be divergent views of what the program mission actually is.
Taking stock[edit]
Taking stock as the second step consists of two important tasks. The first task is concerned with program participants and program staff generating a list of current key activities that are crucial to the functioning of the program. The second task is concerned with rating the identified key activities, also known as prioritization. For example, each party member may be asked to rate each key activity on a scale from 1 to 10, where 10 is the most important and 1 the least important. The role of the evaluator during this task is to facilitate interactive discussion amongst members in an attempt to establish some baseline of shared meaning and understanding pertaining to the key activities.In addition, relevant documentation (such as financial reports and curriculum information) may be brought into the discussion when considering some of the key activities.
Planning for the future[edit]
After prioritizing the key activities the next step is to plan for the future. Here the evaluator asks program participants and program staff how they would like to improve the program in relation to the key activities listed. The objective is to create a thread of coherence whereby the mission generated (step 1) guides the stock take (step 2) which forms the basis for the plans for the future (step 3). Thus, in planning for the future specific goals are aligned with relevant key activities. In addition to this it is also important for program participants and program staff to identify possible forms of evidence (measurable indicators) which can be used to monitor progress towards specific goals. Goals must be related to the program's activities, talents, resources and scope of capability- in short the goals formulated must be realistic.
These three steps of empowerment evaluation produce the potential for a program to run more effectively and more in touch with the needs of the target population. Empowerment evaluation as a process which is facilitated by a skilled evaluator equips as well as empowers participants by providing them with a 'new' way of critically thinking and reflecting on programs. Furthermore, it empowers program participants and staff to recognize their own capacity to bring about program change through collective action.[43]
Transformative Paradigm[edit]
The transformative paradigm is integral in incorporating social justice in evaluation. Donna Mertens, primary researcher in this field, states that the transformative paradigm, "focuses primarily on viewpoints of marginalized groups and interrogating systemic power structures through mixed methods to further social justice and human rights".[44] The transformative paradigm arose after marginalized groups, who have historically been pushed to the side in evaluation, began to collaborate with scholars to advocate for social justice and human rights in evaluation. The transformative paradigm introduces many different paradigms and lenses to the evaluation process, leading it to continually call into question the evaluation process.
Both the American Evaluation Association and National Association of Social Workers call attention to the ethical duty to possess cultural competence when conducting evaluations. Cultural competence in evaluation can be broadly defined as a systemic, response inquiry that is actively cognizant, understanding, and appreciative of the cultural context in which the evaluation takes place; that frames and articulates epistemology of the evaluation endeavor; that employs culturally and contextually appropriate methodology; and that uses stakeholder-generated, interpretive means to arrive at the results and further use of the findings.[45] Many health and evaluation leaders are careful to point out that cultural competence cannot be determined by a simple checklist, but rather it is an attribute that develops over time. The root of cultural competency in evaluation is a genuine respect for communities being studied and openness to seek depth in understanding different cultural contexts, practices and paradigms of thinking. This includes being creative and flexible to capture different cultural contexts, and heightened awareness of power differentials that exist in an evaluation context. Important skills include: ability to build rapport across difference, gain the trust of the community members, and self-reflect and recognize one’s own biases.[46]
Paradigms[edit]
The paradigms axiology, ontology, epistemology, and methodology are reflective of social justice practice in evaluation. These examples focus on addressing inequalities and injustices in society by promoting inclusion and equality in human rights.
Axiology (Values and Value Judgements)[edit]
The transformative paradigm’s axiological assumption rests on four primary principles:[44]
The importance of being culturally respectful
The promotion of social justice
The furtherance of human rights
Addressing inequities
Ontology (Reality)[edit]
Differences in perspectives on what is real are determined by diverse values and life experiences. In turn these values and life experiences are often associated with differences in access to privilege, based on such characteristics as disability, gender, sexual identity, religion, race/ethnicity, national origins, political party, income level, are, language, and immigration or refugee status.[44]
Epistemology (Knowledge)[edit]
Knowledge is constructed within the context of power and privilege with consequences attached to which version of knowledge is given privilege.[44] "Knowledge is socially and historically located within a complex cultural context".[47]
Methodology (Systematic Inquiry)[edit]
Methodological decisions are aimed at determining the approach that will best facilitate use of the process and findings to enhance social justice; identify the systemic forces that support the status quo and those that will allow change to happen; and acknowledge the need for a critical and reflexive relationship between the evaluator and the stakeholders.[44]
Lenses[edit]
While operating through social justice, it is imperative to be able to view the world through the lens of those who experience injustices. Critical Race Theory, Feminist Theory, and Queer/LGBTQ Theory are frameworks for how we think should think about providing justice for marginalized groups. These lenses create opportunity to make each theory priority in addressing inequality.
Critical Race Theory[edit]
Critical Race Theory(CRT)is an extension of critical theory that is focused in inequities based on race and ethnicity. Daniel Solorzano describes the role of CRT as providing a framework to investigate and make visible those systemic aspects of society that allow the discriminatory and oppressive status quo of racism to continue.[48]
Feminist Theory[edit]
The essence of feminist theories is to "expose the individual and institutional practices that have denied access to women and other oppressed groups and have ignored or devalued women" [49]
Queer/LGBTQ Theory[edit]
Queer/LGBTQ theorists question the heterosexist bias that pervades society in terms of power over and discrimination toward sexual orientation minorities. Because of the sensitivity of issues surrounding LGBTQ status, evaluators need to be aware of safe ways to protect such individuals’ identities and ensure that discriminatory practices are brought to light in order to bring about a more just society.[44]
Government requirements[edit]
Given the Federal budget deficit, the Obama Administration moved to apply an "evidence-based approach" to government spending, including rigorous methods of program evaluation. The President's 2011 Budget earmarked funding for 19 government program evaluations for agencies such as the Department of Education and the United States Agency for International Development (USAID). An inter-agency group delivers the goal of increasing transparency and accountability by creating effective evaluation networks and drawing on best practices.[50] A six-step framework for conducting evaluation of public health programs, published by the Centers for Disease Control and Prevention (CDC), initially increased the emphasis on program evaluation of government programs in the US. The framework is as follows:
Engage stakeholders
Describe the program.
Focus the evaluation.
Gather credible evidence.
Justify conclusions.
Ensure use and share lessons learned.
CIPP Model of evaluation[edit]
Main article: CIPP evaluation model
History of the CIPP model[edit]
The CIPP model of evaluation was developed by Daniel Stufflebeam and colleagues in the 1960s.CIPP is an acronym for Context, Input, Process and Product. CIPP is an evaluation model that requires the evaluation of context, input, process and product in judging a programme’s value. CIPP is a decision-focused approach to evaluation and emphasises the systematic provision of information for programme management and operation.[51]
CIPP model[edit]
The CIPP framework was developed as a means of linking evaluation with programme decision-making. It aims to provide an analytic and rational basis for programme decision-making, based on a cycle of planning, structuring, implementing and reviewing and revising decisions, each examined through a different aspect of evaluation –context, input, process and product evaluation.[51]
The CIPP model is an attempt to make evaluation directly relevant to the needs of decision-makers during the phases and activities of a programme.[51] Stufflebeam’s context, input, process, and product (CIPP) evaluation model is recommended as a framework to systematically guide the conception, design, implementation, and assessment of service-learning projects, and provide feedback and judgment of the project’s effectiveness for continuous improvement.[51]
Four aspects of CIPP evaluation[edit]
These aspects are context, inputs, process, and product. These four aspects of CIPP evaluation assist a decision-maker to answer four basic questions:
What should we do?
This involves collecting and analysing needs assessment data to determine goals, priorities and objectives. For example, a context evaluation of a literacy program might involve an analysis of the existing objectives of the literacy programme, literacy achievement test scores, staff concerns (general and particular), literacy policies and plans and community concerns, perceptions or attitudes and needs.[51]
How should we do it?
This involves the steps and resources needed to meet the new goals and objectives and might include identifying successful external programs and materials as well as gathering information.[51]
Are we doing it as planned?
This provides decision-makers with information about how well the programme is being implemented. By continuously monitoring the program, decision-makers learn such things as how well it is following the plans and guidelines, conflicts arising, staff support and morale, strengths and weaknesses of materials, delivery and budgeting problems.[51]
Did the programme work?
By measuring the actual outcomes and comparing them to the anticipated outcomes, decision-makers are better able to decide if the program should be continued, modified, or dropped altogether. This is the essence of product evaluation.[51]
Using CIPP in the different stages of the evaluation[edit]
The CIPP model is unique as an evaluation guide as it allows evaluators to evaluate the program at different stages, namely: before the program commences by helping evaluators to assess the need and at the end of the program to assess whether or not the program had an effect.
CIPP model allows you to ask formative questions at the beginning of the program, then later gives you a guide of how to evaluate the programs impact by allowing you to ask summative questions on all aspects of the program.
Context: What needs to be done? Vs. Were important needs addressed?
Input: How should it be done? Vs. Was a defensible design employed?
Process: Is it being done? Vs. Was the design well executed?
Product: Is it succeeding? Vs. Did the effort succeed?
See also[edit]
Impact assessment
Impact evaluation
Participatory Impact Pathways Analysis
Policy analysis
Policy studies
References[edit]
Jump up ^ Administration for Children and Families (2010) The Program Manager's Guide to Evaluation. Chapter 2: What is program evaluation?.
Jump up ^ Shackman, Gene. "What Is Program Evaluation: A Beginner's Guide". The Global Social Change Research Project. Retrieved April 8, 2012.
Jump up ^ "Hints for Conducting Strong Evaluations". Program Evaluation. CDC - Office of the Associate Director for Program - Program Evaluation. Retrieved April 8, 2012.
Jump up ^ Shadish, W. R., Cook, T. D., & Leviton, L. C. (1991). Foundations of program evaluation: Theories of practice. Newbury Park, CA: Sage. CS1 maint: Multiple names: authors list (link)
Jump up ^ "U.S. Department of Labor -- Brief History of DOL - Eras of the New Frontier and the Great Society, 1961-1969". dol.gov.
Jump up ^ National Archives, Records of the Office of Management and Budget (1995) 51.8.8 Records of the Office of Program Evaluation
^ Jump up to: a b c d e f g h i j k l m n o p q r s t u v w x y z aa ab ac ad ae af Rossi, P. Lipsey, M. W., & Freeman, H.E. (2004). Evaluation: A systematic approach (7th ed.). Thousand Oaks, CA: Sage.
^ Jump up to: a b c Barbazette, J. (2006). "What is needs assessment?" (PDF).
Jump up ^ Rouda, R. H.; Kusy, M. E. (1995). "Needs assessment: The first step".
Jump up ^ Centers for Disease Control and Prevention. Framework for Program Evaluation in Public Health. MMWR 1999;48(No. RR-11).
Jump up ^ Van der Riet, M. (2009). 'The production of context: using activity theory to understand behaviour change in response to HIV and AIDS.' Unpublished doctoral dissertation. University of KwaZulu-Natal, Pietermaritzburg.
Jump up ^ Eveland, JD. (1986) "Small Business Innovation Research Programs: Solutions Seeking Problems". In D. Gray, T. Solomon, and W. Hetzner (eds.), Technological Innovation: Strategies for a New Partnership. Amsterdam: North-Holland
^ Jump up to: a b McLaughlin, J. A., & Jordan, G. B. (1999). Logic models: a tool for telling your programs performance story. Evaluation and program planning, 22(1), 65-72.
Jump up ^ Eveland, JD (1986) "Diffusion, Technology Transfer, and Implementation: Thinking and Talking about Change". Knowledge. 8(2):303-322.
Jump up ^ The Intermediary Development Series. Measuring Outcomes. Dare Mighty Things, INC.
^ Jump up to: a b Mouton, J (2009). "Assessing the impact of complex social interventions". Journal of Public Administration. 44: 849–865.
Jump up ^ Shahab, Sina; Clinch, J Peter; O’Neill, Eoin (21 July 2017). "Impact-based planning evaluation: Advancing normative criteria for policy analysis". Environment and Planning B: Urban Analytics and City Science: 239980831772044. doi:10.1177/2399808317720446.
Jump up ^ Delbert Charles Miller, Neil J. Salkind (2002) Handbook of Research Design & Social Measurement. Edition: 6, revised. Published by SAGE,.
Jump up ^ Center for Disease Control. "Evaluation Framework". CDC. Retrieved 20 September 2012.
Jump up ^ Kania, John; Kramer, Mark. "Collective Impact". Stanford Social Innovation Review. Stanford Center on Philanthropy and Civil Society at Stanford University. Retrieved 19 September 2014.
Jump up ^ Preskill, Hallie; Parkhurst, Marcie; Juster, Jennifer Splansky. "Guide to Evaluating Collective Impact" (PDF). www.fsg.org. FSG, Inc. Retrieved 2014-09-19.
Jump up ^ Patton, Michael. "Evaluation Approaches and Techniques". The Evaluators' Institute. George Washington University. Retrieved 2014-09-19.
Jump up ^ "Formative Evaluation". Community Sustainability Engagement Evaluation Toolbox. Retrieved 19 September 2014.
Jump up ^ "Summative Evaluation". Community Sustainability Engagement Evaluation Toolbox. Retrieved 19 September 2014.
Jump up ^ Taylor-Powell, Ellen; Steele, Sarah; Douglah, Mohammad. "Planning a Program Evaluation" (PDF). University of Wisconsin Extension. Retrieved September 20, 2014.
^ Jump up to: a b c d e f g Bamberger, M. (2004). "Shoestring Evaluation: Designing Impact Evaluations under Budget, Time and Data Constraints". American Journal of Evaluation. 25: 5–37. doi:10.1177/109821400402500102.
^ Jump up to: a b Jacobs, F. H. (2003). "Child and Family Program Evaluation: Learning to Enjoy Complexity". Applied Developmental Science. 7 (2): 62–75. doi:10.1207/S1532480XADS0702_3.
^ Jump up to: a b Weiss, H.B., & Greene, J.C. (1992). An empowerment partnership for family support and education programs and evaluations. Family Science Review, 5(1,2): 131-148.
^ Jump up to: a b "Five-tiered approach to evaluation". National Parenting Education Network (n.d.). Retrieved September 18, 2012.
^ Jump up to: a b Bailey, S. J.; Deen, M. Y. (2002). "A Framework for introducing program evaluation to extension faculty and staff". Journal of Extension. Retrieved September 18, 2012.
^ Jump up to: a b c d "Five-tiered approach to program evaluation" (PdF). ag.arizona.edu. Retrieved September 2012. Check date values in: |access-date= (help)
Jump up ^ Bamberger, M. (2000). "The Evaluation of International Development Programs: A View from the Front". American Journal of Evaluation. 21 (1): 95–102. doi:10.1177/109821400002100108.
^ Jump up to: a b Smith, T. (1990). "Policy evaluation in third world countries: some issues and problems". Asian Journal of Public Administration. 12: 55–68. doi:10.1080/02598272.1990.10800228.
^ Jump up to: a b c d e Ebbutt, D. (1998). "Evaluation of projects in the developing world: some cultural and methodological issues". International Journal of Educational Development. 18 (5): 415–424. doi:10.1016/S0738-0593(98)00038-8.
^ Jump up to: a b c d e f g h Bulmer, M.; Warwick, D. (1993). Social research in developing countries: surveys and censuses in the Third World. London: Routledge. ISBN 0-471-10352-7.
Jump up ^ Potter, C. (2006). Program Evaluation. In M. Terre Blanche, K. Durrheim & D. Painter (Eds.), Research in practice: Applied methods for the social sciences (2nd ed.) (pp. 410-428). Cape Town: UCT Press.
Jump up ^ Rossi, P., Lipsey, M.W., & Freeman, H.E. (2004). Evaluation: a systematic approach (7th ed.). Thousand Oaks: Sage.
Jump up ^ Gill, Andrew. "Evaluating Academic Programs in California's Community Colleges". PPIC.
Jump up ^ Woolcock, Michael. "Integrating Qualitative and Quantitative Approaches in Program Evaluation" (PDF). Retrieved 21 September 2011.
Jump up ^ Weiss, C.H. (1999). Research-policy linkages: How much influence does social science research have? World Social Science Report, pp. 194-205.
Jump up ^ Louw, J. (1999). Improving practice through evaluation. In D. Donald, A. Dawes & J. Louw (Eds.), Addressing childhood adversity (pp. 60-73). Cape Town: David Philip.
^ Jump up to: a b Fetterman, D.M. (2002). "Empowerment evaluation: Building communities of practice and a culture of learning". American Journal of Community Psychology. 30 (1): 81–102.
Jump up ^ Fetterman, D.M. (2005). Empowerment evaluation principles in action: Assessing levels of commitment. In D.M. Fetterman & A. Vandersman (Eds.), Empowerment evaluation principles in practice(pp.27-41). New York: Guilford Press
^ Jump up to: a b c d e f Mertens, D and Wilson, A. (2012). Program Evaluation Theory and Practice: A Comprehensive Guide. New York, NY: The Guilford Press. pp. 168–180.
Jump up ^ SenGupta, S., Hopson, R., & Thompson-Robinson, M. (2004). Cultural competence in evaluation:an overview. New Directions in Evaluation, 102. pp. 5–19.
Jump up ^ Endo, T., Joh, T., & Yu, H. (2003). Voices from the field: Health evaluation leaders in multicultural evaluation. Oakland, CA: Policy Research Associates. p. 5.
Jump up ^ Mertens, D., Yamashita, M. (2010). Mission statement for the American Evaluation Association's Topical Interest Group: Mixed Methods in Evaluation. Washington DC. p. 48.
Jump up ^ Solorzano, D. (1997). "Images and Words that wound: Critical race theory, racial stereotyping and teacher education.". Teacher Education Quarterly. 24: 5–19.
Jump up ^ Brabeck, M. & Brabeck, K. (2009). Feminist Perspective on Research Ethics. Thousand Oaks, CA: Sage. p. 39.
Jump up ^ "Program Evaluation" (PDF). White House. Retrieved 20 September 2011.
^ Jump up to: a b c d e f g h Robinson, B. (2002).The CIPP has its formation from the earlier stages where there were no paragraphs or any acronyms for any product or stanzas. The CIPP approach to evaluation.Collit project: A background note from Bernadette
Further reading[edit]
Suchman, Edward A. Evaluative Research: Principles and Practice in Public Service & Social Action Programs (1967)
Rivlin, Alice M. Systematic Thinking for Social Action (1971)
Weiss, Carol H. Evaluative Research: Methods of Assessing Program Effectiveness (1972)
Cook, Thomas D. and Campbell, Donald T. Quasi-Experimentation: Design & Analysis for Field Settings (1979)
Boulmetis, John and Dutwin, Phyllis. The ABCs of Evaluation (2005)
Rossi, Peter H., Freeman, Howard A. and Lipsey, Mark W.. Evaluation. A Systematic Approach (1999)
Preskill, Hallie, Parkhurst, Marcie, and Juster, Jennifer Splansky. "Guide to Evaluating Collective Impact" (2014)
Borland, J., & Tseng, Y. P. (2011). A Primer on Doing Evaluation of Social Programs. Parity, 24(7), 8.
Wholey Joseph S, Hatry Harry P, Newcomer Kathryn E, et al. (2010). Handbook of Practical Program Evaluation Third Edition. San Francisco: Jossey-Bass. ISBN 978-0-470-52247-9.
Khaidukov, Danil; Tasalov, Kirill; Schepetina, Ekaterina; Chueva, Ekaterina (2016). Improvement of methodological approaches to enhancing the efficiency of state programs of the Russian Federation // Lomonosov Moscow State University Science Conference «Effective management», Poligraf Service, Moscow, pp. 65–73
The Program Manager's Guide to Evaluation, Second Edition, Office of Planning, Research & Evaluation, Administration for Children and Families.
External links[edit]
Administration for Children and Families The Program Manager's Guide to Evaluation. Discussion of evaluation, includes chapters on Why evaluate, What is evaluation.
BetterEvaluation BetterEvaluation: Sharing information to improve evaluation
American Evaluation Association Includes a link to Evaluation resources such as organizations, links sites, email discussion lists, consultants and more
Canadian Evaluation Society Includes a link to Evaluation information such as services, professional development, resources, organizations, regional chapters
CDC six-step framework. Also available here Centers for Disease Control and Prevention. Framework for Program Evaluation in Public Health. MMWR 1999;48(No. RR-11). Includes a description of logic models in the Steps section.
Handbook of Research Design & Social Measurement. Delbert Charles Miller, Neil J. Salkind (2002) Edition: 6, revised. Published by SAGE.
EvalPartners (EvalYouth) International Evaluation Network and Training tools
The EvaluationWiki - The mission of EvaluationWiki is to make freely available a compendium of up-to-date information and resources to everyone involved in the science and practice of evaluation. The EvaluationWiki is presented by the non-profit Evaluation Resource Institute.
Free Resources for Program Evaluation and Social Research Methods This is a gateway to resources on program evaluation, how to, online guides, manuals, books on methods of evaluation and free software related to evaluation.
Innovation Network A nonprofit organization working to share planning and evaluation tools and know-how. The organization provides online tools, consulting, and training, for nonprofits and funders.
Links to Assessment and Evaluation Resources List of links to resources on several topics, including: centers, community building, education and training in evaluation; Foundations; Indiana government & organizations; Links collected by...; Logic models; performance assessment & electronic portfolios, political & private groups or companies, professional assns, orgs & pubs, Purdue University, United States Government, web searches for publications by author & topic, and Vivisimo topical meta searches.
Maine Legislature's Office of Program Evaluation & Government Accountability An excellent example of a governmental Program Evaluation office with links to several detailed reports which include methodology, evaluations results, recommendations and action plans.
National Legislative Program Evaluation Society Includes links to state offices of program evaluation and/or performance auditing in the US.
Real Evaluation Tools and tips for developing an evaluation plan.
Stephanie Evergreen's Blog Helpful resources for Data Visualization and Reporting
4-H Youth Development Programs This is a teaching website and provides help wishing to evaluate their programs.
Program evaluation and Outcome management - The Urban Institute
Retrieved from "https://en.wikipedia.org/w/index.php?title=Program_evaluation&oldid=804468274"
Categories:
Impact assessment
Educational evaluation methods
Public policy research
Hidden categories:
CS1 maint: Multiple names: authors list
CS1 errors: dates
Articles that may contain original research from February 2012
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Article
Talk
Variants
Views
Read
Edit
View history
More
Search
Navigation
Main page
Contents
Featured content
Current events
Random article
Donate to Wikipedia
Wikipedia store
Interaction
Help
About Wikipedia
Community portal
Recent changes
Contact page
Tools
What links here
Related changes
Upload file
Special pages
Permanent link
Page information
Wikidata item
Cite this page
Print/export
Create a book
Download as PDF
Printable version
Languages
Français
Italiano
עברית
Português
Русский
Edit links
This page was last edited on 9 October 2017, at 05:40.
Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.
Privacy policy
About Wikipedia
Disclaimers
Contact Wikipedia
Developers
Cookie statement
Mobile view

values: mw-page-base noprint

values: mw-head-base noprint

values: firstHeading firstHeading en

values: siteSub noprint

values: jump-to-nav mw-jump

attrs: href class title

values:
/wiki/Program_(management)
mw-redirect
Program (management)

value: p

values:
/wiki/Lyndon_Johnson
mw-redirect
Lyndon Johnson

values:
/wiki/Quantitative_method
mw-redirect
Quantitative method

values:
/wiki/Qualitative_method
mw-redirect
Qualitative method

values: Edit section: Assessing the impact (effectiveness)

values: Edit section: Measuring Program Outcomes

values: Edit section: Reliability, validity and sensitivity in program evaluation

values: The material near this tag may be based upon an improper synthesis of sources. (February 2012)

values: Edit section: Evaluating Collective Impact

values: Edit section: Planning a program evaluation

values: Edit section: Methodological constraints and challenges

values: Edit section: Methodological challenges presented by language and culture

values: Edit section: Direct (instrumental) utilization

values: Edit section: Variables affecting utilization

values: Edit section: Guidelines for maximizing utilization

values: Edit section: Internal versus external program evaluators

values:
/wiki/Cultural_competence
mw-redirect
Cultural competence

values: Edit section: Axiology (Values and Value Judgements)

values: Edit section: Methodology (Systematic Inquiry)

values:
/wiki/Critical_Race_Theory
mw-redirect
Critical Race Theory

values:
/wiki/Feminist_theories
mw-redirect
Feminist theories

values:
/wiki/Programme_management
mw-redirect
Programme management

values:
/wiki/Factor_of_production
mw-redirect
Factor of production

values: Edit section: Four aspects of CIPP evaluation

values: Edit section: Using CIPP in the different stages of the evaluation

values: Category:CS1 maint: Multiple names: authors list

values:
nofollow
external text
//doi.org/10.1207%2FS1532480XADS0702_3

values: -moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;

values:
nofollow
http://www.betterevaluation.org/
external text

values:
nofollow
http://www.eval.org/
external text

values:
nofollow
external text
http://www.eval.org/resources.asp

values:
nofollow
http://www.evaluationcanada.ca/
external text

values:
nofollow
external text
https://www.cdc.gov/eval/framework.htm

values:
nofollow
http://www.evalpartners.org/
external text

values:
nofollow
external text
http://www.evaluationwiki.org

values:
nofollow
http://gsociology.icaap.org/methods/
external text

values:
nofollow
external text
http://www.innonet.org

values:
nofollow
http://www.maine.gov/legis/opega/
external text

values:
nofollow
http://www.ncsl.org/nlpes/
external text

values:
nofollow
http://realevaluation.com/
external text

values:
nofollow
http://stephanieevergreen.com/blog/
external text

values: Category:Articles that may contain original research from February 2012

values: Discussion about edits from this IP address [ctrl-alt-n]

values: A list of edits made from this IP address [ctrl-alt-y]

values: You are encouraged to create an account and log in; however, it is not mandatory

values: You're encouraged to log in; however, it's not mandatory. [ctrl-alt-o]

values:
/wiki/Program_evaluation
View the content page [ctrl-alt-c]
c

values: Discussion about the content page [ctrl-alt-t]

values: p-namespaces navigation vectorTabs p-namespaces-label

values:
p-variants
navigation
vectorMenu emptyPortlet
p-variants-label

values: Past revisions of this page [ctrl-alt-h]

values: p-views navigation vectorTabs p-views-label

values:
p-cactions
navigation
vectorMenu emptyPortlet
p-cactions-label

values:
mw-wiki-logo
/wiki/Main_Page
Visit the main page

values:
/wiki/Wikipedia:File_Upload_Wizard
Upload files [ctrl-alt-u]
u

values: A list of all special pages [ctrl-alt-q]

values: Permanent link to this revision of the page

values: Link to connected data repository item [ctrl-alt-g]

values: Printable version of this page [ctrl-alt-p]

values: Mission d'évaluation des politiques publiques – French

values: Valutazione delle politiche pubbliche – Italian

values: הערכת ביצוע – Hebrew

values: Оценка программ – Russian

values: mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-0 ns-subject page-Program_evaluation rootpage-Program_evaluation vector-nav-directionality skin-vector action-view


text:
Say Yes to the Dress
From Wikipedia, the free encyclopedia
Jump to: navigation, search
For other versions of the show, see Say Yes to the Dress (disambiguation).
Say Yes to the Dress
Genre
Reality
Narrated by
Roger Craig Smith
Country of origin
United States
Original language(s)
English
No. of seasons
15
No. of episodes
270 (list of episodes)
Production
Executive producer(s)
Grace Inge
Camera setup
Multiple
Running time
22 minutes
Production company(s)
Half Yard Productions
Release
Original network
TLC
Picture format
480i (SDTV)
1080i (HDTV)
Original release
October 12, 2007 (2007-10-12) – present
Chronology
Related shows
Say Yes to the Dress: Atlanta
Say Yes to the Dress: Bridesmaids
Say Yes to the Dress: Big Bliss
Say Yes to the Dress: Randy Knows Best
Randy to the Rescue
External links
Website
Production website
Say Yes to the Dress is an American reality television series on TLC which follows events at Kleinfeld Bridal in Manhattan. Kleinfeld Bridal recently expanded into the Canadian market by adding a store in Toronto. The series shows the progress of individual sales associates, managers, and fitters at the store, along with profiling brides as they search for the perfect wedding dress. Common themes include overwhelming advice of friends and family, the ability of the "perfect dress" to help a bride overcome personal difficulty, struggle with weight and body image concerns, and the challenge of staying in budget, especially in the case of dresses by Kleinfeld's exclusive designer, Pnina Tornai (one of the few designers ever mentioned by name in the show). Dresses sold on the show range from $1,300 to $40,000.[1][2][3]
Contents
[hide]
1 Cast
1.1 Mara Urshel
1.2 Ronald Rothstein
1.3 Randy Fenoli
1.4 Dorothy Silver
1.5 Nicole Sacco
1.6 Nitsa Glezelis
1.7 Joan Roberts
1.8 Camille Coffey
1.9 Vera Skenderis
1.10 Audrey Pisani
1.11 Keasha Rigsby
1.12 Debbie Asprea
2 Episodes
3 Spinoffs
4 References
5 External links
Cast[edit]
Mara Urshel[edit]
Mara Urshel is one of the co-owners of Kleinfeld. Earlier in her career in the retail industry, she served as senior vice president and general merchandise manager at Saks Fifth Avenue. She worked there for twenty years. After she left Saks, she was employed by Casual Corner and Geoffrey Beene Company in executive management positions. She then purchased the Kleinfeld Bridal store with Ronald Rothstein and Wayne Rogers on July 9, 1999.[4]
Ronald Rothstein[edit]
Ronald (Ronnie) Rothstein is one of the co-owners of Kleinfeld. He was successful academically; he graduated from Wharton School at the University of Pennsylvania in 1964 and received his law degree from the University of Miami in 1968. He then became a member of the Florida Bar. In 1976, he started his own consumer products company named Oh Dawn, but after eight years, he sold Oh Dawn to an American Stock Exchange company. He did however stay on in an executive capacity for several more years. On July 9, 1999 he purchased the Kleinfeld Bridal store with Mara Urshel and Wayne Rogers.[5]
Randy Fenoli[edit]
Main article: Randy Fenoli
Randy Fenoli is the Fashion Director for Kleinfeld. He was born in Mt. Vernon, Illinois, and grew up with a love of fashion. He began sewing dresses when he was only nine years old. When he got older, he branched out into the areas of make-up artistry, hair styling, and entertainment. He then enrolled with the New York's Fashion Institute of Technology. Later, he was offered a job working for Vivian Dessy Diamond, of the Vivian Diamond company. He was then offered a job at Kleinfeld where he worked until 2012. He is now an independent consultant.
Dorothy Silver[edit]
Dorothy Silver is the Director of Sales and Merchandising for Kleinfeld. She began her career in New York as a floor manager at Bonwit Teller, but she has spent the majority of her twenty-five years in the retail industry at Kleinfeld. She works alongside Nicole Sacco and Joan Roberts.[6]
Nicole Sacco[edit]
Nicole Sacco is the Director of Fittings and Sales at Kleinfeld. She has worked there for thirteen years and has many responsibilities, primarily helping consultants with their clients, whether it's finding a dress, closing the sale, or just making sure the client is happy. She works alongside Dorothy Silver and Joan Roberts.[7][8]
Nitsa Glezelis[edit]
Nitsa Glezelis is the Director of Alterations at Kleinfeld. She was born in Kos, Greece, and has been in the retail industry since she was twelve years old. She has worked at Kleinfeld for eighteen years. She works alongside Vera Skenderis.[9]
Joan Roberts[edit]
Joan Roberts is the Director of Sales/Bridal Manager at Kleinfeld. Before coming to Kleinfeld, she had worked in the retail industry for twenty five years. She works alongside Nicole Sacco and Dorothy Silver.[10]
Camille Coffey[edit]
Camille Coffey is one of the several bridal consultants at Kleinfeld. She was first introduced to Kleinfeld when she was shopping for a dress to wear to her son's wedding. She applied for a job and has been working there ever since.[11]
Vera Skenderis[edit]
Vera Skenderis is the Alterations Manager at Kleinfeld. She was born in Athens, Greece. She has been working in the retail industry for thirty-four years. She works alongside Nitsa Glezelis.[12]
Audrey Pisani[edit]
Audrey Pisani is one of the several bridal consultants at Kleinfeld. She was born in Brooklyn and has worked in the retail industry for twenty years. Fifteen of those years has been spent working in bridal wear.[13]
Keasha Rigsby[edit]
Keasha Rigsby is one of the several bridal consultants at Kleinfeld. She has been working in the retail industry for fifteen years. Six of those years has been spent working in bridal wear. She was first introduced to Kleinfeld when she was shopping for wedding dresses with her cousin. She met one of the co-owners, Ronnie Rothstein, applied for a job, and has been working there ever since. Keasha has not appeared on Say Yes to the Dress since the January 2011 season. She will be appearing on a new show entitled Keasha’s Perfect Dress on the Canadian Slice network.[14] This show will follow Keasha as she opens a new bridal salon.[15]
Debbie Asprea[edit]
Debbie Asprea is one of the several bridal consultants at Kleinfeld. She has been working in the retail industry for eighteen years. She has been with Kleinfeld for fifteen of those eighteen years. She attributes her love of fashion to her father, who was a dress contractor.[16]
Episodes[edit]
Main article: List of Say Yes to the Dress episodes
Season
Episodes
Season Premiere
Season Finale
1 6 October 12, 2007 (2007-10-12) November 16, 2007 (2007-11-16)
2 24 July 15, 2008 (2008-07-15) December 26, 2008 (2008-12-26)
3 18 March 6, 2009 (2009-03-06) July 31, 2009 (2009-07-31)
4 19 September 11, 2009 (2009-09-11) December 18, 2009 (2009-12-18)
5 24 April 23, 2010 (2010-04-23) July 30, 2010 (2010-07-30)
6 18 January 7, 2011 (2011-01-07) April 8, 2011 (2011-04-08)
7 18 October 7, 2011 (2011-10-07) December 30, 2011 (2011-12-30)
8 18 June 15, 2012 (2012-06-15) August 17, 2012 (2012-08-17)
9 19 December 28, 2012 (2012-12-28) March 1, 2013 (2013-03-01)
10 18 August 16, 2013 (2013-08-16) November 1, 2013 (2013-11-01)
11 18 February 21, 2014 (2014-02-21) May 23, 2014 (2014-05-23)
12 17 October 10, 2014 (2014-10-10) December 19, 2014 (2014-12-19)
13 18 March 6, 2015 (2015-03-06) June 5, 2015 (2015-06-05)
14 20 March 4, 2016 (2016-03-04) June 24, 2016 (2016-06-24)
15 16 March 4, 2017 (2017-03-04) July 15, 2017 (2017-07-15)
Spinoffs[edit]
Say Yes to the Dress: Atlanta (July–September 2010; July 2011–) is the first spinoff, which focuses on Bridals by Lori, a bridal shop in Atlanta, Georgia. Other than the shop, staff and production company (NorthSouth Productions produced the Atlanta series), the elements are the same as the original series. The Atlanta series itself also led to two spinoffs:
Say Yes to the Dress: Bridesmaids (July 2011–) is a spinoff focused on bridesmaid dresses and the bridesmaid showroom at Bridals by Lori.[17]
Say Yes to the Dress: Monte's Take (July 2011 – February 2012) is a podcast hosted by Monte Durham, the bridal image consultant at Bridals by Lori; and TLC Interactive Producer Candace Keener. The weekly podcast provides listeners wedding tips and tricks.
Say Yes to the Dress: Big Bliss (September–October 2010; April 2011–) is the second spin-off. The shop, staff, venue and producers are the same as the original series, except that all of Kleinfeld's clientele on this program are brides that are plus-sized.
Say Yes to the Dress: Randy Knows Best (April 2011–August 2013): Randy Fenoli, Kleinfeld's fashion director, uses top ten lists and past episode clips to share his tips on a variety of bridal categories.[18]
Randy to the Rescue (June 2012–August 2013): Randy Fenoli travels across the country where lucky brides-to-be get a special consultation with Randy, where he will give a few pointers on how to improve their bridal style.[19]
Say Yes to the Dress: Australia (2016): An international adaptation of the format on TLC Australia, featuring couture wedding gown designer Adam Dixon. The series is the first local production on the channel.[20]
Say Yes to the Dress: UK (2016) the second international adaptation that premiered on TLC UK
Say Yes to the Dress: Ireland (2017) which premiered on RTÉ 2.
Say Yes to the Dress: Asia (2017) was also created for TLC Asia, shot in Malaysia with Malaysian hosts, featuring 18 Southeast Asian women and their dresses.[21]
Say Yes: Wedding SOS (January 2018–) is a wedding makeover spin-off series focusing on transforming couples who have let themselves go.
References[edit]
Jump up ^ Nadelson, Reggie (February 18, 2012). "Virginal or strapless? How New York says Yes to the dress". BBC Magazine. Retrieved July 26, 2013.
Jump up ^ Coffin J. 2010. Teenagers Portrayed in Television. Journal of Psychology 41
Jump up ^ "'Say Yes to the Dress' Season 13 to Premiere Friday, March 6 on TLC". February 5, 2015. Retrieved November 28, 2016.
Jump up ^ "Mara, Co-Owner of Kleinfeld: Say Yes to the Dress: TLC". Tlc.howstuffworks.com. January 17, 2011. Retrieved July 5, 2012.
Jump up ^ "Ronnie, Co-Owner of Kleinfeld: Say Yes to the Dress: TLC". Tlc.howstuffworks.com. Retrieved July 5, 2012.
Jump up ^ "Behind the Seams: About our Bloggers". Kleinfeldbridal.com. Retrieved July 5, 2012.
Jump up ^ "Say Yes to the Dress | Nicole". Kleinfeldbridal.com. Retrieved July 5, 2012.
Jump up ^ "Q&A with Nicole — Say Yes To The Dress Blog : TLC". Blogs.discovery.com. May 20, 2010. Retrieved July 5, 2012.
Jump up ^ "Say Yes to the Dress | Nitsa". Kleinfeldbridal.com. Retrieved July 5, 2012.
Jump up ^ "Say Yes to the Dress | Joan". Kleinfeldbridal.com. Retrieved July 5, 2012.
Jump up ^ "Say Yes to the Dress | Camile". Kleinfeldbridal.com. Retrieved July 5, 2012.
Jump up ^ "Say Yes to the Dress | Designer Bios". Kleinfeldbridal.com. Retrieved July 5, 2012.
Jump up ^ "Say Yes to the Dress | Audrey". Kleinfeldbridal.com. Retrieved July 5, 2012.
Jump up ^ "Slice commissions "Keasha's Perfect Dress"". Retrieved November 28, 2016.
Jump up ^ "Say Yes to the Dress | Keasha". Kleinfeldbridal.com. Retrieved July 5, 2012.
Jump up ^ "Say Yes to the Dress | Debbie". Kleinfeldbridal.com. Retrieved July 5, 2012.
Jump up ^ "BRIDESMAIDS GET THE SPOTLIGHT ON TLC'S NEWEST INSTALLMENT OF "SAY YES TO THE DRESS"". TLC Press Release. May 13, 2011. Retrieved July 15, 2011.
Jump up ^ "TLC Gets Married to Another Installment of "Say Yes to the Dress"". The Futon Critic. March 7, 2011. Retrieved March 7, 2011.
Jump up ^ TLC: "About Randy To The Rescue"
Jump up ^ Knox, David (30 May 2016). "Auditions: Say Yes To The Dress: Australia". TV Tonight. Retrieved 30 May 2016.
Jump up ^ Say Yes to the Dress: Asia official webpage
External links[edit]
Official website
Kleinfeld Bridal
Bridals by Lori
Pnina Tornai
Say Yes to the Dress: Monte's Take Podcast
[hide]
v
t
e
TLC original programming
7 Little Johnstons
10 Years Younger
48 Hours
90 Day Fiancé
All About Sex
Breaking Amish
Buying Naked
Cake Boss
Counting On
Curvy Brides
Extreme Cheapskates
Four Weddings
Gypsy Sisters
A Haunting
Home Sweet Bus
I Am Jazz
I Found The Gown
Leah Remini: It's All Relative
The Little Couple
Little People, Big World
Long Island Medium
Long Lost Family
Love, Lust or Run
My 600-lb Life
My Big Fat Fabulous Life
My Big Fat Gypsy Wedding
My Big Fat American Gypsy Wedding
My Giant Life
My Strange Addiction
Next Great Baker
Kate Plus 8
Kindred Spirits
OutDaughtered
Outrageous 911
Our Little Family
Paranormal Lockdown
Randy to the Rescue
Say Yes to the Dress
Say Yes to the Dress: Atlanta
Say Yes to the Dress: Bridesmaids
Say Yes to the Dress: Randy Knows Best
Say Yes to the Dress: The Big Day
Secretly Pregnant
Sex Sent Me to the ER
Something Borrowed, Something New
The Willis Family
Too Close to Home
Trading Spaces
Untold Stories of the E.R.
Who Do You Think You Are?
See also Template:Former TLC Programming
Retrieved from "https://en.wikipedia.org/w/index.php?title=Say_Yes_to_the_Dress&oldid=834430640"
Categories:
2000s American television series
2007 American television series debuts
2010s American television series
Television shows set in New York City
TLC (TV network) programs
Wedding television shows
Hidden categories:
Use mdy dates from July 2013
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Article
Talk
Variants
Views
Read
Edit
View history
More
Search
Navigation
Main page
Contents
Featured content
Current events
Random article
Donate to Wikipedia
Wikipedia store
Interaction
Help
About Wikipedia
Community portal
Recent changes
Contact page
Tools
What links here
Related changes
Upload file
Special pages
Permanent link
Page information
Wikidata item
Cite this page
Print/export
Create a book
Download as PDF
Printable version
Languages
Español
Français
Italiano
Português
Edit links
This page was last edited on 5 April 2018, at 18:04.
Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.
Privacy policy
About Wikipedia
Disclaimers
Contact Wikipedia
Developers
Cookie statement
Mobile view
Enable previews

key: html

type: local

value: div

attrs: id class

values: mw-page-base noprint

values: mw-head-base noprint

value: a

value: i

attrs: id class lang

values: firstHeading firstHeading en

values: siteSub noprint

values: jump-to-nav mw-jump

attrs: role class

values:
note
hatnote navigation-not-searchable

value: th

attrs: colspan class style

values: text-align:center;font-size:125%;font-weight:bold;font-style: italic; background: #CCCCFF; padding: 0.25em 1em; line-height: 1.5em;

value: tr

value: img

attrs: alt src width height data-file-width data-file-height

attrs: href class

value: td

attrs: colspan style

attrs: href title

values: text-align:center;background: #CCCCFF; padding: 0.25em 1em; line-height: 1.5em;

values: Wharton School of the University of Pennsylvania

values:
nofollow
http://www.bridalsbylori.com/
external text

values: -moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;

values:
nofollow
http://www.kleinfeldbridal.com/
external text

values:
nofollow
http://www.pninatornai.co.il/
external text

values:
/wiki/My_Big_Fat_Gypsy_Wedding
mw-redirect
My Big Fat Gypsy Wedding

values: Something Borrowed, Something New (TV series)

values: Who Do You Think You Are? (U.S. TV series)

values: nowraplinks hlist collapsible autocollapse navbox-inner

values: Category:2000s American television series

values: Category:2007 American television series debuts

values: Category:2010s American television series

values: Category:Television shows set in New York City

values: Discussion about edits from this IP address [ctrl-alt-n]

values: A list of edits made from this IP address [ctrl-alt-y]

values: You are encouraged to create an account and log in; however, it is not mandatory

values: You're encouraged to log in; however, it's not mandatory. [ctrl-alt-o]

values:
/wiki/Say_Yes_to_the_Dress
View the content page [ctrl-alt-c]
c

values:
/wiki/Main_Page
Visit the main page [ctrl-alt-z]
z

values: Featured content – the best of Wikipedia

values: Find background information on current events

values:
/wiki/Special:Random
Load a random article [ctrl-alt-x]
x

values: portal navigation p-navigation p-navigation-label

values: Guidance on how to use and edit Wikipedia

values: About the project, what you can do, where to find things

values: A list of recent changes in the wiki [ctrl-alt-r]

values: portal navigation p-interaction p-interaction-label

values: List of all English Wikipedia pages containing links to this page [ctrl-alt-j]

values: Recent changes in pages linked from this page [ctrl-alt-k]

values: /static/images/wikimedia-button-1.5x.png 1.5x, /static/images/wikimedia-button-2x.png 2x

values: /static/images/poweredby_mediawiki_132x47.png 1.5x, /static/images/poweredby_mediawiki_176x62.png 2x

values:
suggestions
display: none; font-size: 13px;

values: mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-0 ns-subject page-Say_Yes_to_the_Dress rootpage-Say_Yes_to_the_Dress skin-vector action-view