Power of Survey Proved

Today, election result of Delhi (Indian capital city) legislative Assembly declared. Following my earlier post on "Power of Survey", the result proved the power of survey. The table below presents the comparison on exit poll prediction and actual result.

On analyzing the result with respect to projection, it can be seen only six seats deviated for two party. This is normal. It is hard to find any survey that can precisely present the result. Nor still any body can design the survey that can precisely give the result. Always the survey results are in margin of error.

Here's if you go to earlier post, you can see the table that presents result of exit poll conducted by five media houses. In that table, four media houses has presented the exit poll prediction in range. In this range AAP party actual result touched the upper range while BJP actual result touched the lower range.

So, during the sample size calculation, we usually jot down to two terms: "Level of confidence" and "Margin of Error". These two terms are most important part of any M&E survey. It is also hardest part to explain in layman language to general mass.

Statistically, if you say 95% Level of confidence, it means a range of values that you can be 95% certain contains the population mean.Lets reflect ourselves being layman, can I make any sense on this definition?

Describing margin of error is easy compare to level of confidence.A margin of error tells you how many percentage points your results will differ from the real population value. It is simply understood, if anybody said this survey result has 10% margin of error then that means the presented survey result will vary by plus minus 10% from real population value.

But, the absolute presentation of Margin of error without mentioning the level of confidence will be wrong and misguide the audience. So, most survey results are presented in .... result with 95% confidence level and 10% margin of error.

Again, reflecting from layman perspective, it is hard to understand confidence level. Should any layman understand, surveyor are 95% confidence in their survey. Or should understand I as a user/reader of survey report be confident on 95% level in the results.

Both the understanding are wrong. It should be if same survey carried out 100 times, .... result will be same for 95 times with ...... result shall vary by plus minus 10%.

The Delhi election exit poll results are analogous with "Level of confidence" and "Margin of Error". The survey (exit poll) carried out five times by five media houses with same pattern of results and actual result had touch the predicted upper and lower range.

Thus it can be said "Power of Survey again Proved".

Power of Survey

The Capital of India, Delhi, has observed the election for Delhi Assembly. In the large set of population, Indian media usually conduct and forecast the exit poll of every election. Most of time it used to be correct. 

I remember, in the last national election exit poll conducted by National media had correctly announced BJP will win the election with the majority. Before the election, news media were very much critical with BJP and questions the winning of BJP in that election. But turned upside down after the exit poll.

As of general obvious rule, the result of survey do not used to be exact and pin point.

The following Table presents the forecast of Delhi election by five big media. The results to be announced on coming Tuesday, 11 Feb. 2020. Lets see, this time how much power of survey will be reflected in the exit poll.

Best of luck to AAP. The party formed by a common man and had demonstrated the power of common man.

No Baseline, No Worry

MEL professionals often worried about the baseline. The worries includes baseline value, methods for baseline establishment and much more satisfying the project colleges/stakeholders. Further worries worsen when project executives did not bother to have baseline in the start of project. Should project collect baseline data mandatorily? I say "Yes" if project is in initial phase. Here, collection does not mean, it should be primary data collection. May adopt any method, but should have baseline at least in intended project result indicators. What if it is in mid-time or end phase of project? I say "No Worry".

Baseline is just a benchmark. What is the height of Mount Everest? Everybody knows it is 8848 meter. What is the baseline for the height of Mount Everest? It is a Sea which is marked as zero "0" value for to measure any height of earth surface. If sea was marked as 1000 meter, the height of Mount Everest would be 9848 meter.

Similarly, what is the freezing temperature of water? Is it 0 °C or 32 °F? Both are right. Here you should make attention on the unit of measurement. In the Celsius measurement unit the value is "0" and in Fahrenheit measurement unit the value is "32".

Definitely, the measurement of Human characteristic is complex than properties of physical materials. It can make a sense, utilizing appropriate methods and tools. How should we measure the awareness level of people? Certainly, we should face the lack in universal unit and method to measure the awareness level of people? I think it is not a problem, until you follow, M&E professional ethics and integrity. Just like, freezing fact of water do not differ whether you measure in Celsius or Fahrenheit. The difference of height between sea and peak of Mount Everest remain same whether you give "0" or "1000" value to sea level. 

The following figure is from the Linkedin post by Ann-Murray A. Brown. This is about "Three Things you can do if you did not collect baseline data." This could be mantra to lessen the worry of M&E professionals.

Data analysis explained in one shot !

Data analysis and result presentation are fundamental work of M&E. The literature on data analysis goes to many tons and it should be. We can go deeper as we discussed more. But the essence of data analysis should not be forgotten. Sometimes we lost in jumble of data set. Many times we could not able to direct/advise data analyst what to do with data.

I found following figure in LinkedIn that explained in one shot about the essence of data analysis. I supposed this could be tagged in the desk of M&E professionals.

Ethical issue in M&E and Learning

I have been taking online course in www.philanthropyu.org on "From Data Collection to data use". This is specialized course I ever found in M&E learning.

The course discuss on ethical principle in first module. This is very much mind blowing to me. I think, should it be for M&E professional and practitioner: either individual or organization.

Source: from google search

Ethics is not more than correct moral behavior. The course module one explains "Ethics are a big deal in M&E. If used properly, M&E is a tool for distinguishing truth from fiction and for creating just, positive solutions. If unethical choices are made, however, it can be difficult to know what is true. Poor, unwise, or unjust decisions can be made. Even worse, your data practices may endanger the very people you hope to help."

Ethics is not a rule neither can trial under any law in the court.It is a behavior that is anticipated by society, stakeholders or peers. Most of the time, M&E professionals have to face ethical dilemma: A choice where every option has some ethical issues.

The following are some ethical standard that should better followed:

1) Do no harm: For example-informed consent for data collection and data use. Keeping respondents data anonymous and confidential.

2) Honesty: For example- Ensuring accurate data and data analysis as well as accurate result presentation. Presenting the limitation of result and findings. Disclosing the organizational interest with the outcome.

3) Competence: From vast tools and processes one should use in which she/he is confident and skillful and use competently. The competency can be improved through continuous learning and practice.

Not following the ethical issue will result the questioning towards the credibility of M&E. The donors, stakeholders and beneficiary will loose trust on M&E. Ultimately M&E profession shall be in limbo.