A record £1.86bn was taken out of pensions in the UK between April and June this year, according to the latest figures from HM Revenue and Customs.
Analysis from Scottish Widows reveals that 62% of the withdrawals were made by people aged 55-60, as people increasingly take advantage of pension freedoms introduced in 2015.
However, it is thought that a significant amount of this is being held as cash at a time when savings rates are near historic lows, leading to fears that many will miss out on returns in the long-term.
"While it's good that more people are evidently becoming engaged with their pension at a younger age, it's essential that everyone is able to access help and information to make informed choices," Scottish Widows retirement planning expert, Catherine Stewart, said.
"People should feel confident they will have sufficient funds to last them for a comfortable retirement, and one of the best ways to do this is by getting good guidance and advice."
The pension freedoms, introduced on 6 April 2015, enable consumers at a minimum pension age to take savings as cash, buy an annuity, use drawdown without any limits applied, or a combination of all three.
However, the Financial Conduct Authority (FCA) released a report earlier this month revealing that more than half of pension pots accessed by people under 65 since then had been fully withdrawn.
It also showed that 52% of customers were not spending the money, but moving it to other savings or investments - suggesting a lack of trust in pensions.
In addition, it was found that consumers that access their pots early without taking advice typically follow the 'path of least resistance', accepting drawdown from their current pension provider without shopping around.
Barnett Waddingham senior consultant, Malcolm McLean, said the findings were "extremely worrying", and reinforce the view that the pension freedoms were introduced too quickly.
"The fact that over half of fully withdrawn pots are not spent indicates not only a lack of trust in pensions, but the possibility of a deep misunderstanding of the tax and other consequences of mismanaging individuals' pension savings," he added.
The FCA said it would assess whether additional protections are needed for customers who buy drawdown without advice, and propose methods to make it easier to shop around.