Hawthorne effect and M&E

The data collection and observations of programs/project beneficiaries for results are fundamental task of M&E professionals. The true or credible reflections from data/observation always been suspicious when reports prepared or presented or announced. For example in WASH programs, one of  the desired result would be behavior change of community in regards of hand washing. The program desired community should wash hand with soaps frequently after the use of wash room, Before and after eating. During the survey/observation, data analysis and finding presentation, the  behavior change of community could be concluded. But this M&E report encircled with suspicious when post program completion phase would not able to realize the change. This often arise the doubt on credibility of M&E as well as on presented report. This situation could be explained from the principle of Hawthorne effect. 

The Hawthorne effect is also called the observer effect. When an experiment or research is being conducted, the tendency for the subjects to alter any part of their behavior because they are aware of the fact that they are being watched, this is the Hawthorne effect.

The phenomenon of Hawthorne effect is mostly used in management to enhance the productivity of the employees. In management this effect advocates, when work of people are observed their productivity enhanced. The principle is the employee will be conscious on their work when  observed and they try to do more. Giving more attention to people brings out the best in them. The effects last till the observation. After abandoning the observation, people tends to go back to original status.

M&E requires the data collection from program/project beneficiaries to monitor and evaluate the program effects. The data from survey or observations are likely to be affected from Hawthorne effect. The beneficiaries should be on conscious mode as they are being watched and supported. There will be tendency of giving only positive response rather than being critical with program/project. Also people anticipate more program/projects would be launched in their area when pleasing program/project authorities. This makes the result  more of speculation. The same should happened to the example above. The project beneficiary either did not want to displease the project/program personnel or they abandon the behavior once community felt they are no longer observed from projects/programs.

The Hawthorne effect is challenge for M&E deliverable. The conscious behavior of  respondents on one hand should result be in exaggerated form. In another, the tendency of people behavior reversion into previous status after detaching them from observation, questions the credibility of results when observation was there.

One of the mitigation from the Hawthorne effect would be finding a way to quantify and eliminate the effect during data collection. This would be difficult task as well as challenging. The findings and conclusions could be also presented mentioning the estimation of The Hawthorne effect. The utilization of triangulation tool could also be another solution to eliminate and estimation of the effect. The indirect observation or proxy questionnaire could be designed.

Thinking from the sustainability aspect for the desired change during the program/project design would be another measure to eliminate the Hawthorne effect.

The consideration of the Hawthorne effect in M&E is practiced less. Further research and development of system to identify and measurement of Hawthorne effect should enhance the credibility of the efforts M&E professionals.

Consideration of local context in development of indicators

Defining the program indicators are essential task of M&E. Without indicators, no any measurement can be performed. If there's no measurement then management of program must be in chaos.

The quality in indicators must be ascertained to have effectiveness and credibility in measurement. The consideration of local context should be taken to enhance the quality of indicators. For example: One of outcome indicator of child nutrition program could be, 

      - Percentages of children increased in meat consumption at least once a week for nutrition enhancement in X area

Thinking straight forward, the above indicator follow the SMART standard. But if further analyzed from the local contextual perspective, it might not be so smart. If X area compromise more than 80% vegetarian population, it is unlikely the outcome measurement would represent true picture of the population. This type of measurement rise on one hand, the credibility question and on another the intention of program itself.

Without understanding the local culture and social dynamics, the developed M&E framework will be in limbo and worthless.

During the construction of indicators as far as possible, the targeted project beneficiary should also be involved or the indicators should be tested and discussed with stakeholders. This mitigate in large extent the hidden untidiness of developed indicators. 

Going back to above example, either the indicator could be omitted if program is in initial phase or could be redefined if program is in implementation or data should be dis-aggregated into vegetarian and non-vegetarian in proper manner if program is in post completion phase.