Few ideologies have inspired such frequent premature eulogies as free-market capitalism. The economic system which has succeeded across cultures and geographies and millennia is berated by its critics as harsh, immoral and exploitative.

With each major technological development, the naysayers prophesise its collapse and replacement with some utopian economic system. Typically, these systems come with a worrying track record and a confused understanding of human nature.

The latest technological development to inspire predictions of the death of free-market capitalism is the widespread application of artificial intelligence (AI). The claims are sweeping: AI will allow us to efficiently run centralised economies; effectively manage markets and assign value; make vast swathes of careers redundant, and displace humans in all walks of employment and production.

These claims are so broad that if believed we would have to conclude that a post-AI revolution economy would be fundamentally different from any free-market society which has preceded it. Many on the left, awestruck by its promise, look on it as the missing piece in the puzzle of why control economies fail so consistently and dramatically.

But these reflect an overly optimistic faith in the potential of AI and an overly pessimistic understanding of the remarkable resilience of free-market capitalism.

AI refers to a group of methods which use large data sets to iteratively develop predictive algorithms and to recognise patterns. A problem is formulated, data is collected, AI programs are trained on that data, and the newly trained program is let loose on new data to attempt to make predictions.

The breadth of problems to which AI can be applied is staggering.

During the last decade, it has been embedded within our social media, financial institutions, governments, universities, military and healthcare. It is viewed by some as the unifying solution to all our predictive and analytic problems, and by extension our social woes.

It also has an air of mysticism – impenetrable algorithms, incomprehensible to human minds peering inside, yet able to extract previously unrecognised patterns in our past in order to gaze ahead into our future.

There is good reason to believe that over the next few decades, the economic disruption AI will produce will be vast. The leading players on the frontier of AI research and application are unsurprisingly big tech corporations. These have the money, freedom and incentive to discover the value AI can unlock. Amazon, in particular, has been incredibly successful here.

As with all technologies for which the limits are currently unknown, it has been met with a mix of misplaced fear and unjustified excitement. The prophecies have entered into the mainstream discourse – an Economist essay last year asked the question “Can technology plan economies and destroy democracy?”

To those on the left who continue to fantasise about centrally-controlled economies, AI is the final ingredient which will usher in an age of “fully automated luxury communism” and finally justify the interminable claims that communism “just hasn’t been properly tried yet.”

This betrays a misunderstanding of the limits of AI.

All current AI is narrow in its applications. We have yet to develop anything that would be described as a “artificial general intelligence,” (AGI) an AI capable of solving unrelated problems across different areas in the way a human brain can. The idea that even such an AGI could take over the judgement of humans in allocating resources based on their innumerable considerations is naïve.

Eventually, planned economies crumble as inefficiencies accumulate and unincentivized central planners fail to innovate. There is no magical ingredient in AI which would allow them to overcome their terminal flaws.

This is not to undersell AI. In many of the narrow problems it has been applied to, it can vastly outperform humans. But our lives are complex and difficult to prescribe. Our desires are fleeting and often unknowable.

AI relies on making predictions based on past data. In this sense, it is iterative, rather than revolutionary. Yet so many of the biggest ideas that have come to shape our lives have not been iterations. They have been new breaks which have not followed from previous paradigms.

Consider the release of the iPhone. If an AI had attempted to predict the future trend for mobile phone design in the early 2000s, it would likely have looked at historic trends and assumed the number of buttons would continue to increase. Yet Steve Jobs’ product was a creative revolution. It was not an iteration on previous designs. By definition, that is what makes a new idea revolutionary.

Current AI is unable to perform this role. It would require an AGI to perform creative leaps like this, and such leaps are vital to sustain an economy. Without them, it is doomed to stagnation.

As such, many that think AI will dominate sections of the economy are not envisioning AI in its current state. They instead picture an omniscient, omnipotent, creative and benevolent controller. Deus ex machina, if you will.

But the step from AI to AGI is an enormous one, and in the more than 70 years since the birth of the field, no-one has come close to implementing it. The difference between the two is the difference between a computer (which is essentially a glorified abacus) and a human being.

For the foreseeable future, AI will only be used in the narrow sense – as a specific solution for a specific problem. But that is not to say there are not other avenues through which AI could disrupt free markets.

Even if we are unlikely to produce an AGI to steer our economies any time soon, the widespread application of narrow AI to a multitude of fields could lead to large scale disruption and job displacement. This is the conclusion reached by many who look at the AI applications of big tech in horror.

These doomsayers point to the growing number of complex problems AI has been successfully applied to and argue that, as the trend continues, more workers (particularly unskilled or low-skilled workers, but increasingly high-skilled workers) will be vulnerable to job obsolescence in the AI revolution. We will iteratively put ourselves out of jobs.

Yet a consistent trend in the history of technological development has been that humans have little trouble creating new jobs as old ones are replaced by more efficient machines.

The industrial revolution is an important example. Those living in generations prior would have been horrified to learn of the impending mass redundancy it would produce for manual labourers and skilled craftsmen.

But the reality was that those workers were able to find different work as new possibilities rapidly expanded. Humans are perpetually attracted to the new and exciting. No amount of money or sustenance has been able to quench our ambition to build, buy and sell new products and ideas.

Every time a labour-saving device has proved successful in the past, those who previously held that occupational niche were rapidly drafted into a new opportunity which sprang up organically.

And that is, fundamentally, what current AI technology is: a labour-saving device. An extraordinarily exciting and useful one, but no replacement for human ingenuity and creativity.

We cannot expect that AI will be able to direct central economic control or permanently displace humans from the job market. In the first case, the iterative nature of AI and its narrow applicability to a given problem prevent it from being useful in the face of a challenge as vast as the economy. In the second case, each revolutionary labour-saving device has historically vastly expanded the job market, rather than shrunk it.

AI will certainly change our lives. But arguments that it will irreparably alter our free-market economy are overly optimistic about the utility of AI and overly pessimistic about the human capability for discovering new ventures and opportunities as markets evolve.

So, the question of whether the development of AI will usher in the demise of free-market capitalism depends on whether one is an optimist or a pessimist about the limits of human innovation. And we can find reassurance in the fact that there has yet to be a technological innovation which has fettered the human capacity to expand into new ventures.

Each innovation has been a stepping-stone to new ideas and products. There has never been a system of economics which has come near the efficiency of the free market in the allocation of resources. AI, for all its brilliance, will never be able to outplay the invisible hand of the market.

Rob Sutton is a Junior Doctor in Wales and a former Parliamentary Staffer. He is a recent graduate of the University of Oxford Medical School. Follow him on twitter: @DrRobSutton