Rather than trying to escape violence, human beings more often become habituated to it. History abounds with long conflicts – the Thirty Years’ War in early seventeenth-century Europe, the Time of Troubles in Russia, twentieth-century guerrilla conflicts – in which continuous slaughter has been accepted as normal. Famously adaptable, the human animal quickly learns to live with violence and soon comes to find satisfaction in it.

The modern world inherits the Christian view in which salvation is played out in history. In Christian myth human events follow a design known only to God; the history of humankind is an ongoing story of redemption. This is an idea that informs virtually all of western thought – not least when it is intensely hostile to religion. From Christianity onwards, human salvation would be understood (at least in the west) as involving movement through time. All modern philosophies in which history is seen as a process of human emancipation – whether through revolutionary change or incremental improvement – are garbled versions of this Christian narrative, itself a garbled version of the original message of Jesus.

For Leopardi evil is integral to the way the world works; but when he talks of evil he does not mean any kind of malign agency of the sort that Gnostics imagined. Evil is the suffering that is built into the scheme of things. ‘What hope is there when evil is ordinary?’ he asks. ‘I mean, in an order where evil is necessary?’ These rhetorical questions show why Leopardi had no interest in projects of revolution and reform. No type of human action – least of all the harlequinade of politics – could fundamentally alter a world in which evil was ordinary.

In Leopardi’s view, the universal claims of Christianity were a licence for universal savagery. Because it is directed to all of humanity, the Christian religion is usually praised, even by its critics, as an advance on Judaism. Leopardi – like Freud a hundred years later – did not share this view. The crimes of medieval Christendom were worse than those of antiquity, he believed, precisely because they could be defended as applying universal principles: the villainy introduced into the world by Christianity was ‘entirely new and more terrible … more horrible and more barbarous than that of antiquity’. Modern rationalism renews the central error of Christianity – the claim to have revealed the good life for all of humankind. Leopardi described the secular creeds that emerged in modern times as expressions of ‘half-philosophy’, a type of thinking with many of the defects of religion. What Leopardi called ‘the barbarism of reason’ – the project of remaking the world on a more rational model – was the militant evangelism of Christianity in a more dangerous form. Events have confirmed Leopardi’s diagnosis. As Christianity has waned, the intolerance it bequeathed to the world has only grown more destructive. From imperialism through communism and incessant wars launched to promote democracy and human rights, the most barbarous forms of violence have been promoted as means to a higher civilization.

In Kleist’s essay humans are caught between the graceful automatism of the puppet and the conscious freedom of a god. The jerky, stuttering quality of their actions comes from their feeling that they must determine the course of their lives. Other animals live without having to choose their path through life. Whatever uncertainty they may feel sniffing their way through the world is not a permanent condition; once they reach a place of safety, they are at rest. In contrast, human life is spent anxiously deciding how to live.

The idea of evil as it appears in modern secular thought is an inheritance from Christianity. To be sure, rationalists have repudiated the idea; but it is not long before they find they cannot do without it. What has been understood as evil in the past, they insist, is error – a product of ignorance that human beings can overcome. Here they are repeating a Zoroastrian theme, which was absorbed into later versions of monotheism: the belief that ‘as the “lord of creation” man is at the forefront of the contest between the powers of Truth and Untruth.’ But how to account for the fact that humankind is deaf to the voice of reason? At this point rationalists invoke sinister interests – wicked priests, profiteers from superstition, malignant enemies of enlightenment, secular incarnations of the forces of evil. As so often is the case, secular thinking follows a pattern dictated by religion while suppressing religion’s most valuable insights. Modern rationalists reject the idea of evil while being obsessed by it. Seeing themselves as embattled warriors in a struggle against darkness, it has not occurred to them to ask why humankind is so fond of the dark. They are left with the same problem of evil that faces religion. The difference is that religious believers know they face an insoluble difficulty, while secular believers do not. Aware of the evil in themselves, traditional believers know it cannot be expelled from the world by human action. Lacking this saving insight, secular believers dream of creating a higher species. They have not noticed the fatal flaw in their schemes: any such species will be created by actually existing human beings.

With new technologies of surveillance, economies of scale overcome problems of cost. Since all their electronic communications can be accessed, it is no longer necessary to segregate the inmates from one another. As there is no outside world, escape becomes unimaginable. Technological progress has brought into being a system of surveillance more far-reaching than any Bentham could have conceived. Enclosing the entire population in a virtual Panopticon might seem the ultimate invasion of freedom. But universal confinement need not be experienced as a privation. If they know nothing else, most are likely to accept it as normal. If the technology through which surveillance operates also provides continuous entertainment, they may soon find any other way of living intolerable.

The belief that there is some hidden cabal directing the course of events is a type of anthropomorphism – a way of finding agency in the entropy of history.

The first survey of my work which not only fully understands but is able to carry on my ideas beyond the point at which I left off.

Hayek watched the interwar collapse with horror, as Keynes did, and shared many of Keynes’s liberal values. What he failed to understand is that these values cannot be renewed by applying any formula or doctrine, or by trying to construct an ideal liberal regime in which freedom is insulated from the contingencies of politics.

Hayek may still have lessons to teach us. The policies he recommended during the Great Depression may have been badly flawed but his insight that prosperity cannot be restored by unending expansion of debt may have some value at a time when the limits of “Keynesian” quantitative easing are becoming clear. It is in any case far from obvious that Keynes would have supported a continuation of QE once a disastrous collapse had been averted. “Keynesianism” is a confection of Keynes’s more mechanical disciples, not an indication of how this mercurially brilliant mind would have responded to our present dilemmas. Again, Hayek’s claim that nothing can be done to mitigate the impact of free markets on social cohesion was dangerously misguided. But he was right to point out that capitalism cannot be remodelled to fit some conception of an ideally fair distribution of resources. Whether any kind of social democracy can be reconciled with the anarchic energies of global markets is an open question.

Hayek’s blind spot with regard to politics was clear in the early 1980s when the first Thatcher government, in an attempt to reduce inflation and bring the public finances closer to a balanced budget, was raising interest rates and cutting public spending. As he had done during the 1930s, Hayek attacked these policies as not being severe enough. It would be better, he told me in a conversation we had around this time, if Thatcher imposed a more drastic contraction on the economy so that the wage-setting power of the trade unions could be broken. He appeared unfazed by unemployment, which was already higher (more than three million people) than at any time since the 1930s, and would rise much further if his recommendations were accepted.

No doubt markets transmit information in the way that Hayek claimed. But what reason is there to believe that – unlike any other social institution – they have a built-in capacity to correct their mistakes? History hardly supports the supposition. Moods of irrational exuberance and panic can, and often do, swamp the price-discovery functions of markets.

The idea that consciousness is a mystery is a prejudice inherited from monotheism. The early seventeenth-century French philosopher René Descartes believed that animals other than humans are insensate machines. Obviously, this was a restatement in rationalist terms of the Christian belief that only humans have souls. Even if mind and matter were categorically distinct, that would not mean humans alone have minds. It was reported that in order to test his theories, Descartes used to throw animals out of the window and observe their reactions. Looking at behaviour of this kind, one might reasonably conclude that humans are the senseless machines.

How we come to have the world-views we do is an interesting question. No doubt reason plays a part, but human needs for meaning and purpose are usually more important. At times personal taste may be what decides the issue. There is nothing to say that, when all the work of reason is done, only one view of the world will remain. There may be many that fit everything that can be known. In that case you might as well choose the view of the world you find most interesting or beautiful. Adopting a world-view is more like selecting a painting to furnish a room than testing a scientific theory. The test is how it fits with your life.

Nothing carries so much authority today as science, but there is actually no such thing as ‘the scientific world-view’. Science is a method of inquiry, not a view of the world. Knowledge is growing at accelerating speed; but no advance in science will tell us whether materialism is true or false, or whether humans possess free will. The belief that the world is composed of matter is metaphysical speculation, not a testable theory. Science may succeed in explaining events in terms of causes and effects. In some accounts it may be able to formulate laws of nature. But what does it mean for something to cause something else and what is a law of nature? These are questions for philosophy or religion, not for science.

Human beings act, certainly. But none of them knows why they act as they do. There is a scattering of facts, which can be known and reported. Beyond these facts are the stories that are told. Human beings may behave like puppets, but no one is pulling the strings.

Many people today hold to a Gnostic view of things without realizing the fact. Believing that human beings can be fully understood in the terms of scientific materialism, they reject any idea of free will. But they cannot give up hope of being masters of their destiny. So they have come to believe that science will somehow enable the human mind to escape the limitations that shape its natural condition. Throughout much of the world, and particularly in western countries, the Gnostic faith that knowledge can give humans a freedom no other creature can possess has become the predominant religion.

In a traditional reading eating the apple was the original sin; but, as Gnostics understood the story, the two primordial humans were right to eat the apple. The God that commanded them not to do so was not the true God but only a demiurge, a tyrannical underling exulting in its power, while the serpent came to free them from slavery. True, when they ate the apple Adam and Eve fell from grace. This was indeed the Fall of Man – a fall into the dim world of everyday consciousness. But the Fall need not be final. Having eaten its fill from the Tree of Knowledge, humankind can then rise into a state of conscious innocence. When this happens, Herr C. declares, it will be ‘the final chapter in the history of the world'.

Science is not distinguished from myth by science being literally true and myth only a type of poetic analogy. While their aims are different, both are composed of symbols we use to deal with a slippery world.

Progress in civilization seems possible only in interludes when history is idling.

There are not two kinds of human being, savage and civilized. There is only the human animal, forever at war with itself.

As Malaparte saw it, Naples was a pagan city with an ancient sense of time. Christianity taught those who were converted to it to think of history as the unfolding of a single plot – a moral drama of sin and redemption. In the ancient world there was no such plot – only a multitude of stories that were forever being repeated. Inhabiting that ancient world, the Neapolitans did not expect any fundamental alteration in human affairs. Not having accepted the Christian story of redemption, they had not been seduced by the myth of progress. Never having believed civilization to be permanent, they were not surprised when it foundered.

For those who live inside a myth, it seems a self-evident fact. Human progress is a fact of this kind. If you accept it you have a place in the grand march of humanity. Humankind is, of course, not marching anywhere. ‘Humanity’ is a fiction composed from billions of individuals for each of whom life is singular and final. But the myth of progress is extremely potent. When it loses its power those who have lived by it are – as Conrad put it, describing Kayerts and Carlier – ‘like those lifelong prisoners who, liberated after many years, do not know what use to make of their freedoms’. When faith in the future is taken from them, so is the image they have of themselves. If they then opt for death, it is because without that faith they can no longer make sense of living.

Instead of enabling humans to improve their lot, science degrades the natural environment in which humans must live. Instead of enabling death to be overcome, it produces ever more powerful technologies of mass destruction. None of this is the fault of science; what it shows is that science is not sorcery. The growth of knowledge enlarges what humans can do. It cannot reprieve them from being what they are.

Cheating ageing by a low-calorie diet, uploading one’s mind into a super-computer, migrating into outer space … Longing for everlasting life, humans show that they remain the death-defined animal.

Before Christianity suicide was not in any way troubling. Our lives were our own, and when we tired of them we were at liberty to end them. One might think that as Christianity has declined, this freedom would be reclaimed. Instead secular creeds have sprung up, in which each person’s life belongs to everyone else. To hand back the gift of life because it does not please is still condemned as a kind of blasphemy, though the offended deity is now humanity instead of God.

Religion is not a primitive type of scientific theorizing, any more than science is a superior kind of belief-system. Just as rationalists have misunderstood myths as proto-versions of scientific theories, they have made the mistake of believing that scientific theories can be literally true. Both are systems of symbols, metaphors for a reality that cannot be rendered in literal terms. Every spiritual quest concludes in silence, and science also comes to a stop, if by another route. As George Santayana has written, ‘a really naked spirit cannot assume that the world is thoroughly intelligible. There may be surds, there may be hard facts, there may be dark abysses before which intelligence must be silent for fear of going mad.’ Science is like religion, an effort at transcendence that ends by accepting a world that is beyond understanding. All our inquiries come to rest in groundless facts. Just like faith, reason must at last submit; the final end of science is a revelation of the absurd.

Evangelical atheists preach the need for a scientific view of things, but a settled view does not go with scientific method. If we know anything it is that most of the theories that prevail at any one time are false. Scientific theories are not components of a world-view but tools we use to tinker with the world.

Contrary to generations of western progressives, it was not Russian backwardness or mistakes in applying Marxian theory that produced the society that Lyons observed. Similar regimes came into being wherever the communist project was attempted. Lenin’s Russia, Mao’s China, Ceausescu’s Romania and many more were variants of a single dictatorial model. From being a movement aiming for universal freedom, communism turned into a system of universal despotism. That is the logic of utopia. If 1984 is such a powerful myth, one reason is that it captures this truth. Yet there is a flaw in Orwell’s story, which emerges in his picture of the all-powerful interrogator. The dystopia of perpetual power is a fantasy, and so is O’Brien. Soviet torturers were sweating functionaries living in constant fear. Like their victims, they knew that they were resources that would be used up in the service of power. There was no inner-party elite safe from the contingencies of history.

Tyranny offers relief from the burden of sanity and a licence to enact forbidden impulses of hatred and violence. By acting on these impulses and releasing them in their subjects tyrants give people a kind of happiness, which as individuals they may be incapable of achieving.

Echoing the Christian faith in free will, humanists hold that human beings are – or may someday become – free to choose their lives. They forget that the self that does the choosing has not itself been chosen.

Through all of history and pre-history it has been accepted that there is something wrong with the human animal. Health may be the natural condition of other species, but in humans it is sickness that is normal. To be chronically unwell is part of what it means to be human. It is no accident that every culture has its own versions of therapy. Tribal shamans and modern psychotherapists answer the same needs and practise the same trade.

Human knowledge increases, while human irrationality stays the same. Scientific inquiry may be an embodiment of reason, but what such inquiry demonstrates is that humans are not rational animals. The fact that humanists refuse to accept the demonstration only confirms its truth.

As the Genesis story teaches, knowledge cannot save us from ourselves. If we know more than before, it means only that we have greater scope to enact our fantasies. But – as the Genesis myth also teaches – there is no way we can rid ourselves of what we know. If we try to regain a state of innocence, the result can only be a worse madness. The message of Genesis is that in the most vital areas of human life there can be no progress, only an unending struggle with our own nature.

The most important feature of natural selection is that it is a process of drift. Evolution has no end-point or direction, so if the development of society is an evolutionary process it is one that is going nowhere.

Humanists today, who claim to take a wholly secular view of things, scoff at mysticism and religion. But the unique status of humans is hard to defend, and even to understand, when it is cut off from any idea of transcendence. In a strictly naturalistic view – one in which the world is taken on its own terms, without reference to a creator or any spiritual realm – there is no hierarchy of value with humans at the top. There are simply multifarious animals, each with their own needs. Human uniqueness is a myth inherited from religion, which humanists have recycled into science.

The evidence of science and history is that humans are only ever partly and intermittently rational, but for modern humanists the solution is simple: human beings must in future be more reasonable. These enthusiasts for reason have not noticed that the idea that humans may one day be more rational requires a greater leap of faith than anything in religion. Since it requires a miraculous breach in the order of things, the idea that Jesus returned from the dead is not as contrary to reason as the notion that human beings will in future be different from how they have always been.

If there is anything unique about the human animal it is that it has the ability to grow knowledge at an accelerating rate while being chronically incapable of learning from experience. Science and technology are cumulative, whereas ethics and politics deal with recurring dilemmas. Whatever they are called, torture and slavery are universal evils; but these evils cannot be consigned to the past like redundant theories in science. They return under different names: torture as enhanced interrogation techniques, slavery as human trafficking. Any reduction in universal evils is an advance in civilization. But, unlike scientific knowledge, the restraints of civilized life cannot be stored on a computer disc. They are habits of behaviour, which once broken are hard to mend. Civilization is natural for humans, but so is barbarism.

Ichthyophils imagine that human beings want a life in which they can make their own choices. But what if they can be fulfilled only by a life in which they follow each other? The majority who obey the fashion of the day may be acting on a secret awareness that they lack the potential for a truly individual existence. Liberalism – the ichthyophil variety, at any rate – teaches that everyone yearns to be free. Herzen’s experience of the abortive European revolutions of 1848 led him to doubt that this was so. It was because of his disillusionment that he criticized Mill so sharply. But if it is true that Mill was deluded in thinking that everyone loves freedom, it may also be true that without this illusion there would be still less freedom in the world. The charm of a liberal way of life is that it enables most people to renounce their freedom unknowingly.

Though it is often assumed that naturalism must be hostile to religion, the opposite is true. Enemies of religion think of it as an intellectual error, which humanity will eventually grow out of. It is hard to square this view with Darwinian science – why should religion be practically universal, if it has no evolutionary value?

Liberals tend to regard being subjects of the Queen as an insult to their dignity. But at least the archaic structures by which we are ruled do not force us to define ourselves by blood, soil or faith, and we are protected from the poisonous politics of identity.

In ancient Europe, Stoics asserted that a slave could be freer than a master who suffers from self-division. In China, Daoists imagined a type of sage who responded to the flow of events without weighing alternatives. Disciples of monotheistic faiths have believed something similar: freedom, they say, is obeying God’s will. What those who follow these traditions want most is not any kind of freedom of choice. Instead, what they long for is freedom from choice.

The irony of the post-Cold War period is that the fall of communism was followed by the rise of another utopian ideology. In American and Britain, and to a lesser extent other Western countries, a type of market fundamentalism became the guiding philosophy. The collapse of American power that is underway is the predictable upshot.

To affirm that humans thrive in many different ways is not to deny that there are universal human values. Nor is it to reject the claim that there should be universal human rights. It is to deny that universal values can only be fully realized in a universal regime. Human rights can be respected in a variety of regimes, liberal and otherwise. Universal human rights are not an ideal constitution for a single regime throughout the world, but a set of minimum standards for peaceful coexistence among regimes that will always remain different.

I may not be as unambiguously hostile to capitalism as many people are, but what I don't like about it is the commodification of personal experiences, it turns everyone into actors.

What I liked was Thatcherism's Bolshevik aspect, which was to shake up the whole of Britain quite fundamentally, and if you read what I wrote in those years I think you might agree that in taking the view that I did then — that this was necessary and desirable — I never subscribed to the main delusion of the Thatcherites, which was that you could change everything and everything would remain the same. If what you wanted was a very anarchic, globalised, polyglot, mixed-up society in which most of the structures which had somehow been renewed from the Edwardian period to the Sixties were destroyed, then Thatcherism was what would do the job.

The core of the belief in progress is that human values and goals converge in parallel with our increasing knowledge. The twentieth century shows the contrary. Human beings use the power of scientific knowledge to assert and defend the values and goals they already have. New technologies can be used to alleviate suffering and enhance freedom. They can, and will, also be used to wage war and strengthen tyranny. Science made possible the technologies that powered the industrial revolution. In the twentieth century, these technologies were used to implement state terror and genocide on an unprecedented scale. Ethics and politics do not advance in line with the growth of knowledge — not even in the long run.

Terror is not now, if it ever was, something that comes to us from outside. It is a part of the society in which we live. Both liberals and neoconservatives believe terrorism can be dealt with by removing its causes. The truth is less reassuring. Al-Qaeda has mutated into a decentralised, often locally based type of apocalyptic terrorism and, in this new guise, seems to be acquiring a formidable momentum.

If you want to understand the beliefs that are shaping global politics, read the Book of Revelation.