Accessing pension pots early has become the new norm since freedoms were introduced in 2015, with most who do so taking lump sums rather than a regular income.
That is according to research published today by the Financial Conduct Authority (FCA), which shows 72% of pots accessed since then have been by people aged under 65.
It reveals that more than half of these pots have been fully withdrawn, with 52% of the customers not spending the money, but moving it to other savings or investments.
"The findings are extremely worrying and reinforce the view that the new freedoms were introduced too quickly," Barnett Waddingham senior consultant, Malcolm McLean, said.
"The fact that over half of fully withdrawn pots are not spent indicates not only a lack of trust in pensions, but the possibility of a deep misunderstanding of the tax and other consequences of mismanaging individuals' pension savings."
The pension freedoms were introduced on 6 April 2015, enabling consumers at a minimum pension age to take savings as cash, buy an annuity, use drawdown without any limits applied, or a combination of all three.
Today's findings reveal that twice as many pots are moving into drawdown than annuities, with 30% of consumers doing so without advice, compared with 5% before the freedoms.
It was also found that consumers that access their pots early without taking advice typically follow the 'path of least resistance', accepting drawdown from their current pension provider without shopping around.
In addition, the research shows that product innovation has been limited to date, particularly for the mass market, while providers are continuing to withdraw from the open annuity market, risking weakened competition over time.
"There is a clear need for greater regulatory intervention in a market that is not working as well as was expected, and a beefing-up of the advice and guidance services," McLean added.
The FCA announced that it would assess whether additional protections are needed for customers who buy drawdown without advice, and propose methods to make it easier to shop around.
In addition, it said that it will ask the government to consider proposals to enable customers early access to their savings without having to make a decision about the remainder of their pot.