This article and the report it was based on didn’t simply pique my interest because of the topic and how it would impact Liberia given the country is going through a milestone elections this year, it did because I currently run the largest research and evaluation firm in Liberia. We conduct a hefty amount of research for international organizations, so any report instilling this amount of fear and attempting to undermine the peace we have enjoyed would pique our interest.
I went to check out the report and after reading it, I quickly concluded that Catholic Relief Services (CRS) had an agenda and it was not to “advance justice and positive peace” as stated in the report – it was about fear mongering and attempting to undermine our peace. The reasons for my conclusion rest in how this study was designed. From bias in the way the questions were constructed, wrong context analysis on Liberia, apparent lack of rigor in the data collection process, bad sampling design and study methodology, to lack of national representation of the study and more, and any good researcher could tear up this research at first glance so I am very concerned that CRS chose to publish this report knowing the gross limitations and the potential impact it has on Liberia.
Don’t get me wrong. We do want and welcome reports that are critical of our current state and can help us maintain our peace, but this is not one. It lacks the rigor and reeks of an alternate agenda. When I read the report, I quickly reached out to few of my research buddies who conduct polling and research globally to get their perspectives and they concurred with my conclusions and here are the 4 core reasons:
1. Inherent ambiguity and bias in how the survey tools were constructed
The questions are very ambiguous or biased in a way that a typical respondent may answer negatively. For example, questions 20 and 21 of the survey tools read: “Do you believe that Liberia can descend into another violent national-level conflict?” and “What is your assessment of the risk that Liberia can still descend into a violent national-level conflict, such as civil war, despite the peace it currently enjoys?”.
There is a big difference between CAN and WILL in constructing survey questions. “Can” leads the respondents and the fact is “anything CAN happen”. Given the current state of the U.S, if that question is asked today, no one is 100% sure that the US CAN’T descend into a conflict. Secondly, there is no time period for this conflict. Had the question been written, “within the next year” or “within the next 5 years”, we could define impending conflict. But the researchers present a purely hypothetical option that might occur at any time in the future.
2. Inability of the research to design target specific questions could lead to questioning the result of the report
Most of the questions were asked to all respondents, when it would have been more appropriate to ask the question to only certain respondents. For example, Question 9 asks the respondent to compare Liberia’s current situation to the situation before the war. This would make sense for someone who is age 43 or older (approximately age 15 in 1989). Anyone younger than this would not have first-hand memories to be able to judge the pre-war situation.
I was 10 when the war started and I cannot remember any critical “situation before the war”. I remembered that my parents both worked and they worked very hard to send us to the best schools. What I do not remember was all the nuances that my older brothers, sisters and some friends now explain. They remember details that I was just too young to comprehend. CRS should have focused on asking target-respondent specific questions in this study, unfortunately we do not see a lot of that in their report.
3. Wrong context analysis on reconciliation and negative peace in Liberia
Almost every time friends and family who are not Liberian visit Liberia, they are always amazed at stories about former fighters roaming freely. Many can’t seem to understand how Liberians could “forgive” someone who killed their friends and family members and now see the person roaming around freely and even serving in government without reacting. In my mind, I am always wondering if those people who they think have forgiven would react if given the chance because that is in fact the true definition of reconciliation.
Using terms like “reconciled” in the CRS report is too vague to produce an interpretable response. Does “reconciled” mean that former rebels have shaken the hands of their victims and said “sorry”? Does it mean that all stolen property has been returned? Does it mean that your neighbor during the war who stole some of your rice has apologized and returned it? “Reconciled” means different things to different people — and after war, people often feel like they have never been compensated or appreciated for what they suffered.
Related to reconciliation is the concept of negative peace or positive peace. The report states that “66.3% of the respondents described it as “negative peace”—implying that even though Liberians are not openly fighting, they live under circumstances (e.g., laws, rules, regulations, policies, programs, and practices) as well as conditions (such as neglect, poverty, discrimination, etc.) that inflict hardship”. First, Question 10, which CRS mentioned extensively in the report is biased towards producing a negative response. Based on the wording of the question, one could easily say the US is in a situation of negative peace —even before Trump was elected! If you look at all the incidents of police brutality and race issues in the U.S, one could deduce that the U.S is certainly going through a period of negative peace.
I doubt even Johan Galtung, the father of peace studies, would have used the definition of “negative peace” in the context of Liberia knowing the historical context of the country. As I previously stated, one could easily say that the US, Nigeria, Ghana and so many other countries are experiencing negative peace.
4. Lack of rigor in the study approach and design
Bad sampling design
While I doubt that sampling cause any of these results, the issue with the sampling questions CRS’s credibility and level of rigor. In the report, CRS attempts to justify why it did not use the most recent national population census which was conducted in 2008. The report states that CRS “planned to use the most recent population data from the national population census, however the most recent population census was conducted in 2008, which is outdated data for the purposes of sampling. Additionally, the national population census data provided information on population distribution per county, but not down to community level.” These two factors led them to use the national electoral roll used for the 2011 general elections from the National Elections Commission (NEC) because it provided, “the most comprehensive and in-depth proportional distribution of Liberia’s population across the country. It shows population distribution down to the level of electoral districts and further down to the specific polling stations. Also, it provides maps of the electoral districts and GPS locations of the polling stations.”
Even if I bought the justification, why didn’t CRS use the 2014 Voter Registration Update from the very National Elections Commission (NEC) since that is the latest? The statement that the 2011 data from NEC is the “most comprehensive and in-depth proportional distribution of Liberia’s population across the country” is plain false! 2014 is!
Furthermore, Liberia Institute of Statistics & Geo-Information Services (LISGIS) has a document on its website that projects the new population of the country based on the 2008 census results. The projection is done by County, District, Clan and Households and can be found here: https://www.lisgis.net/pg_img/Population%202008,%202014%20by%20County,%20District,%20Clan%20and%20Households,%20Liberia%20260914.pdf. Therefore the claim that the census population distribution was done per county, but not down to community level is also false! Why CRS did not use the new projections leads me to question its motives further.
Lastly, while reading the report, I also discovered that CRS did not provide enough information on how households and respondents within households were selected to participate. The report simply states that “procedures for determining specific households from which interviewees were picked as well as the process for identifying and selecting specific individuals for the interviews were discussed at the data collection training that took place in Gbarnga 10-11 March 2016”. CRS owes it to the public to provide more details regarding this subject.
No data triangulation whatsoever
After separately analyzing both types of data (qualitative and quantitative), it’s often critical to conduct a joint analysis using a triangulation process. However, the report did not mention anything about data triangulation once! Triangulation is a powerful technique that facilitates validation of data through cross verification from two or more sources. In particular, it refers to the application and combination of several research methodologies in the study of the same phenomenon (Erlandson, Harris, Skipper, & Allen, 1993; Lincoln & Guba, 1985; Bogdan & Biklen, 2007; Silverman, 1993).
The triangulation process could have enabled CRS to validate the outcome of the study while offering robust recommendations and strategies to ensure peace in Liberia. The lack of triangulation leads me to think that CRS had a pre-determined mindset regarding the topic.
Lack of clarity on QA process or technical competence of the field researchers
The report states a lot about its methodology, but nothing on quality control and quality assurance of the data collection process. I am unsure if any quality control measures were taken so I have serious questions and reservations about the validity of the data. For example, was the data collection instrument pretested? Was the process piloted? Were field quality assurance procedures put in place to ensure that correct respondents were sampled in each polling unit? How were they sure that the data was not collected under the mango tree?
When I read that training was conducted from March 10 to March 11 of 2016 and data collection and upload started on March 12th, it led me to conclude that the training process was very poor. I have never…and I mean NEVER seen a data collection training for a national study lasting only two days. From the statement in the report, it means that piloting was not done, supervisor training was not done and no time to adjust and retrain the team. If piloting was done, CRS would not have experienced a situation where “several data clerks lost interviews in the process of uploading”. Note that CRS did not say how many were lost, but simply “several data clerks lost interviews in the process of uploading”.
No focus group discussions conducted whatsoever
The research only conducted key informant interviews (KIIs) as part of the qualitative methods. It would have also been best to conduct focus group discussions (FGDs) to gain other insights into the perceptions of people. Relying only on KIIs leads to lack of triangulation within the data collected for the study. Based on our experiences, KIIs are typically conducted with an influencer or elite in the community and their opinions often do not represent the community.
Why would an organization such as Catholic Relief Services that should be focused on peace and reconciliation allow for such a botched-up report to be released to cause fear? While the report provides some seemingly good recommendations, there are many other reports within the development community in Liberia with similar recommendations and they managed to achieve similar outcomes without causing fear and they were pretty rigorous studies.
The only conclusion is that CRS had intentions and agenda beyond advancing justice and positive peace in Liberia. The report was also meant to instill fear and undermine the peace in Liberia.