Every year, there are mixed reactions over the rankings and the efficacy of the Fund for Peace’s Failed States Index (FSI), the eighth edition of which was released in June. But this year, the criticism seems especially intense.

“Failed means there is no way back. Failed means a binary division between those countries that are salvageable and those beyond redemption. It is a word reserved for marriages and exams. It does not belong in a pragmatic debate,” wrote Claire Leigh for The Guardian in June.

Other critics find the index politicized. As Lionel Beehner and Joseph Young wrote on the World Policy Institute blog:

There’s a tendency to lump states deemed failing with states whose regimes the United States does not support. There are several states out there we find odious but are hardly in danger of failing, such as Iran (#34) and Uzbekistan (#39)…conversely, there are relatively weak states whose imperviousness to a coup or revolution remains iffy – e.g. Albania (#118) or Bahrain (#125) yet their low ranking makes them appear as secure as any Scandinavian country.

Individual countries too sometimes take umbrage at their ranking, often suggesting that the index fails to capture the nuances of governance including external influences and the underlying historical causes of some of their flaws.

This year, embassy spokesmen from Pakistan, Afghanistan, Sudan, Yemen, and Somalia wrote in to Foreign Policy to protest their rankings. Nadeem Hotiana, press attaché at the Embassy of Pakistan in Washington, DC, argued that “the true measure of a nation is not the number and magnitude of challenges it faces, but how it rises to meet them. Measured against that yardstick, Pakistan has hardly any equal.”

Indonesia’s minister of coordinating political, legal, and security affairs responded to its ranking (#63 – on the verge of failure) in The Jakarta Post:

Indonesia is a state that runs very well, from the central to local levels. We have recorded a remarkable 6.5 percent of economic growth and have more than US$115 billion in foreign exchange reserves. What kind of indicators did they use to determine a failed state?

A Failure of Communication?

The Fund for Peace has consistently responded to these criticisms.

Executive Director Krista Hendry wrote about the public debate in Indonesia on their website saying that “despite [the FSI’s] name, we are not calling any country a ‘failed state.’ We prefer to think of it as a barometer of the continued pressures on the state, and capacity of the state and the society to cope with and mitigate the pressures they experience.”

And the index is not designed to measure subtlety, argues Senior Associate Nate Haken, responding to comments on Foreign Policy. “As we often point out, the Failed States Index takes a broad brush to a complex problem. But sometimes a broad brush can be useful – and valid,” he writes.

In addition, “the FSI is not a forecasting tool,” he said in an interview with ECSP earlier this summer about the relative importance of the index’s 12 measured indicators. “But all this information, aggregated, normalized, scaled, and arranged in columns and rows, allows for the easy identification of risk factors and hotspots by location, time period, and sector.”

Similarly, Blake Hounshell, managing editor of Foreign Policy who publishes the index in partnership with the Fund for Peace, countered criticisms following the Arab Spring last year. “Clearly, the index is not a prediction model; it’s an annual portrait of state failure,” he noted, adding that “it’s not meant to be a crystal ball.”

The FSI does not provide deep and comprehensive analysis of the situation in every country, but it is not intended to, according to its authors. “Rather, it is a tool we hope government and civil society will use to perform more in-depth analyses of the issues we measure based on local knowledge,” writes Hendry.

“We are not saying that Pakistan is a ‘failed state,’” Haken responded last year to another indignant letter from the Embassy of Pakistan. “This index does not make that determination. Rather, it identifies pressures on states that put them at risk of failure, unless state institutions are sufficiently professional, representative, and legitimate enough to deal effectively with those pressures.”

Still, one can’t help but wonder: If it’s not measuring state failure, as Hendry and Haken suggest, perhaps the Failed States Index is simply misnamed. A framework that focuses less on “failure” and more on a spectrum of risk factors and capacities might nullify a lot of the criticisms thrown its way.