Three and a half years after the invasion of Iraq, American intelligence agencies have produced a devastating assessment of its effect on terrorism. According to the New York Times, they have found that it has given rise to new forms of Islamic extremism in that country.

The 16 separate bodies that contributed to the National Intelligence Estimate agreed that a core of al-Qa'eda members and affiliates had been joined by "self-generating" cells inspired by Osama bin Laden but without any direct connection to the network's leaders. They concluded that since 2001 the threat of global terrorism had heightened.

In that spies have an obvious interest in justifying their existence, their reports need to be treated with caution. However, two factors tend to confirm the veracity of the latest National Intelligence Estimate. First is the consensus reached by a broad range of frequently competing agencies. Second is the appalling state of Iraq, illustrated by the killing of 38 Shias in east Baghdad on Saturday. The holy month of Ramadan, in which the soul is supposed to be purged through fasting, threatens once again to become an excuse, by those who would pervert the Prophet's message, for an orgy of murder.

Against this, yesterday's appeal by the prime minister, Nouri al-Maliki, for Shias and Sunnis to use Ramadan to put aside their differences is likely to fall on deaf ears, so tenuous is the government's authority. As for the security services, the Iraqi army has benefited from foreign training but the police are still riddled with corruption and infiltrated by militias. Meanwhile, the economy hovers on the brink of collapse.

It would be dishonourable for America to walk away from a mess that it has helped to create. Instead, it must learn from the egregious mistakes of the occupation, from insufficient force levels to Abu Ghraib, and prepare for the long haul.

The overall plan, to guide Iraq from totalitarianism to democracy, is good. But the execution has too often been marked by crass arrogance.

Commentary: