How do individuals who closely follow events in Iraq observe progress in state capacity and governance, two contested but related concepts? Those looking at different sectors may count Improvised Explosive Device attacks or Megawatts generated, but there are of course many intangible, often qualitative elements that may give this data more meaning.
This article discusses indices and reports that attempt to measure civilian state capacity, or lack of it. Many definitions of capacity also include the ability of the state to control its geographical territory through armed force and state monopoly on violence, but security is not a focus of IEI. Here, we assume that strong capacity of institutions leads to improved governance outcomes and reduced state fragility and therefore, improved security.
Indices of governance (such as the World Bank’s Worldwide Governance Indicators project- WGI) often consult country specialists but invariably contain data from a vast array of sources from non-specialists in different sectors, as well as other international organizations.
At the country level, this has produced some fuzzy results. For example, most Iraq country specialists could see serious deterioration in governance and security conditions prior to the rise of the self-declared Islamic State in 2014, but rankings such as the Fragile States Index and the WGI actually showed some improvement in Iraq in the same timeframe.
Likewise, the Early Warning Project, which focuses on mass violence episodes, also shows an improvement in Iraq prior to the 2019 protests. Granted, these improvements were small, and most caution against drawing too many conclusions from country level data. But considering the severity of the crisis in 2014, some deterioration should have been observable, at the very least. How could this be? The problem of measuring improvements in governance and capacity is undoubtedly vexing.
Measuring Capacity
In development, capacity falls within the wider analysis of governance, defined very generally by the OECD as “the ability of people, organisations and society as a whole to manage their affairs successfully.” When attempting to strengthen institutions, development practitioners are arguably engaging in a form of state building, defined by the OECD as “action to develop the capacity, institutions and legitimacy of the state in relation to effective political processes for negotiating the mutual demands between state and societal groups.” Clearly, “capacity” involves both technical and political processes.
Studies on capacity building efforts in extremely fragile states highlight a sisyphean task: any capacity building effort in this context will be besieged by systemic challenges including security problems, corruption risk and political intrigue requiring a level of preparedness and contextual understanding that will test even the most well-prepared program.
In Iraq, since a great deal of societal fragmentation occurs at the local level, it may make sense to look at the central government level first, where oil revenues flow into ministries that retain the legacy of a highly centralised political system. In other words, while there are severe capacity problems in the central government, local government has not had the chance in most cases to develop greater capacity. This is despite several capacity building efforts to decentralise government functions in Iraq.
One way of understanding government level capacity in Iraq would be what Hillel David Soifer and Juan Pablo Luna call “the state’s reach across territory.” For their study on state capacity in Chile, they looked at houses connected to the sewage system and the electrical grid, as well as the availability of government-run schools.
As a measure of Iraq’s state capacity, data on these sectors is readily available. A significant problem is the large presence of informal settlements across Iraq, through years of internal economic migration and internal refugee movement.
These settlements either lack access to state services or in some cases gain access to them informally, for example illegally tapping into water lines or re-connecting power lines. In 2017, there were over 3,600 of these communities with a population of over 3.2 million people. At the national level, data on state capacity is subsequently limited by this problem, a grey zone of population data comprising around 8% of Iraq’s inhabitants.
In such fragile contexts, progress in core sectors such as education or health is best observed over very long periods to flatten out the effects of reporting error, as William Easterly has argued in regard to infant mortality data in Ethiopia. By some accounts, progress in any sector cannot be attained without a high level of capacity at a country’s Ministry of Finance and Central Bank. According to the Effective States and Inclusive Development research centre, (ESID) these ministries are the “nodal points” when it comes to allocating resources, raising taxes and enacting monetary policy.
For Iraq’s “nodal points,” James Savage has documented the history of U.S. efforts to overhaul and strengthen Iraq’s budgetary process after 2003. Wracked by violence, Iraq only achieved a 22% budget execution rate in 2006. To Savage, the U.S. capacity building focus to rectify this, largely centred on the Ministries of Finance and Planning, while in many ways flawed, was justified. Auditing, contracting and procurement “reflect the state’s capacity to function,” he argues. The opposite of these processes being effective is less accountability, weaker institutions and more corruption.
Concurrently, the outputs of federal ministries can often produce a misleading appearance of capacity, a point Savage is keen to stress; in other words, gains such as higher budget execution rates may not be translating into sustainable or completed projects.
To Francis Fukuyama, the ability of a state to raise taxes, something almost absent in Iraq, is a key measure of state capacity, although he cautions that, “a given level of taxation does not necessarily translate into the efficient use of tax revenues.” For Iraq, non-oil tax revenue in 2018 was just 5% of non-oil GDP, according to the IMF.
Putting to one side indicators of central government capacity such as number of schools built, hospital beds per capita, electricity generation over a time period or tax revenues raised, other indicators look at government institutions internally, as well as perceptions of performance.
In this regard, the WGI “Government Effectiveness” indicator takes a survey based approach to six indicators, as well as incorporating a vast number of other indices from organizations such as Freedom House and the Economist Intelligence Unit.
As an example, survey data attempts to capture “perceptions of the quality of public services, the quality of the civil service and the degree of its independence from political pressures, the quality of policy formulation and implementation, and the credibility of the government’s commitment to such policies.” This qualitative approach may get around some of the problems of weak data quality, but has produced more than one peculiar result for Iraq.
The WGI Aggregate Indicator on Government Effectiveness for Iraq shows an improvement from 10.4% in 2011 to 13.9% in 2014, the year the Islamic State took control of one third of Iraq, prompting the U.N. to declare a Level 3 emergency. The WGI also shows a slight improvement in Rule of Law during this period. Perhaps this says much about what is not tangible in measuring governance: corruption in societies where transparency is a challenge may not always be apparent or may be under-reported. Either way, the indicators do flag Iraq as a highly fragile state.
Other efforts measuring capacity look at internal factors influencing public services provision. A 2010 UNDP report on capacity lists key questions for assessing capacity building efforts including how resilient the institution is to major shocks.
Under the heading, “efficiency,” the example the UNDP gives is the effectiveness of an Anti-Corruption Commission. The report suggests evidence of efficiency could include, “Number of cases per (year/month) investigated, leading to prosecution” and the “Average cost of investigations.”
For Iraq, a very obvious problem here is that an institution of this kind could be efficient (in Iraq’s case, the Commission of Integrity- CoI) but faces system-wide resistance. In Iraq, the CoI had 15,000 investigations on its books in 2017, yet reported that only 15% of them had been acted upon by authorities.
The UNDP suggests that “programmatic responses can be targeted at vulnerable spots” for example, mechanisms to control corruption and increase oversight, but as with the CoI example, this overlooks the wider problem, which is that the bigger threat to capacity may come from outside the ministry, the system itself.
The UNDP then discusses what they refer to as “Institutional Adaptability” where they highlight the importance of “proactive planning.” The example they give in this instance is a country’s Ministry of Health having the foresight to adapt for anticipated demographic change.
The problem here is that while this planning capacity could be present within a ministry, in Iraq’s case the main issue is that there has been a lack of political will to properly fund the Ministry. Iraq’s Ministry of Health has been well aware of the need for a huge increase in hospital bed capacity, needing as many as 10,000 new beds per year. But funding increases have not been forthcoming.
The debate on whether measuring capacity should be more quantitative (which may be more pleasing to some donors) or qualitative, which may more accurately reflect trends in political dynamics, is still not resolved in the case of Iraq. This is illustrated by some ongoing TVET projects which list numbers of people trained as a positive outcome despite a limited enabling environment for business. Perhaps a qualitative assessment of such programs prior to their launch would have concluded that a stronger enabling environment for employment should come before skills training.
Building System Capacity
Changes in the capacity of a specific institution are difficult to measure, but in the final analysis, it is may be impossible to know how effective a project is until long after its conclusion, a point made by a World Bank study in 2003:
“Technical assistance projects may have an initial positive impact on performance results, but as soon as the funding of these projects ends or foreign experts leave the country, performance indicators deteriorate.”
Time is therefore the enemy of most capacity building projects and this is the case both prior to, during and following implementation. Rachel Kleinfeld describes the challenge of measuring capacity building effectiveness, highlighting the importance of “nonlinear” political developments, particularly in factionalized, complex environments. She highlights a 2011 World Bank study that cautions that Rule of Law reform takes on average 41 years.
Perhaps inevitably, given the strong desire to tackle deeply troubling problems in conflict affected states such as Iraq, pressure to see results as soon as possible still impacts assessments. A report on capacity building from 2019 for example, celebrated a project which still showed positive results “after four years.”
Approaches that prioritize political economy analysis over simple technical metrics are now coming to the fore, as evidenced by the U.K. government’s own guidance on “stabilisation,” which critiques unsuccessful development efforts where problems were approached,
“as primarily or exclusively technical matters, e.g. building infrastructure, providing basic services, building the capacity of government agencies (including security and justice actors) through training and equipment,” while sidelining, “fundamental political issues which run throughout stabilisation.”
A detailed study on capacity building in 2017 by the Secure Livelihoods Research Consortium produced similar conclusions, framing the problem as one of efforts working on “units” amid systemic problems which constantly threaten progress:
“By focusing on individuals and organisations (‘the units’), capacity development is typically approached as a modular exercise, assuming micro-capacities naturally aggregate up to build better systems. This results in systems themselves receiving far less attention.”
Considering this, critics could point to the enormity of the challenge of effectively measuring progress and claim it is impossible, at least in terms of meaningful long term development: how can we build “system capacity?”
A forthcoming publication from IEI will seek to illustrate how some efforts have contributed to pockets of strong capacity (often called “islands of excellence” in development) which may have avoided even worse unrest than what has been seen in recent years.
An overarching problem for Iraq, and one widely analysed, is of course the chaotic nature of power sharing. In terms of capacity of institutions, this has translated into what Francis Fukuyama has called “multiple principles,” a reference to the principal-agent problem. In Fukuyama’s view, risk arises when multiple principles issue conflicting mandates to government units, for example state-owned utilities. Different political groups may require these bodies to cover:
“cost recovery, universal service to the poor, and efficient pricing to business clients, each promoted by a different part of the political system. These different mandates obviously cannot be simultaneously achieved, and generate bureaucratic dysfunctionality.”
The opposite of this arrangement is one where institutions have “autonomy,” which could make for a good proxy for capacity for those viewing the problem of capacity in more political terms. In future, this concept could make for an interesting starting point for efforts in Iraq.
Indeed, Iraq’s government appears to be aware of this issue. Iraq’s UNDP supported Voluntary National Review of its national development plans describes the challenge of “Institutional Confusion,” claiming:
“Despite the transfer of some powers in the education and health sectors to the local levels, they did not have a positive and tangible impact on the efforts of these institutions, the quality of services they provide and the coverage of their services for all the governorates’ people.”
All is not lost therefore. The very fact that Iraq’s government is at least cognizant of system-wide challenges in building capacity should become the starting point for sustained international efforts to rethink capacity building in Iraq. Finding ways to design projects that focus on problems such as “autonomy,” could add much needed realism to Iraq’s development goals.
