Following complaints from scheme members and participating employers, the auditor had concerns around late payment, difficulty in getting in touch with MyCSP and failure to provide "accurate and timely information" on pension entitlement to members.
The document revealed that when MyCSP took over pension payroll and certain administration services in September 2014, 14,703 retirees who lived overseas were paid up to seven days late and 99 were not paid at all in that month.
The NAO said this was because the administrator "did not fully understand" payment practices from the previous provider Capita, meaning it did not allow the extra time needed to make an international payment.
The review also found MyCSP could not cope with the increase in calls and emails following migration. Between September 2014 and March 2015, the firm failed to answer 99,408 calls.
In addition, a backlog of work grew as the administrator did not have enough staff to process the 14,000 items of work inherited from Capita as well as the 40,000 data issues that required attention caused by migration.
By September 2015, performance levels returned to where they were before migration, but the auditor argued MyCSP had received no financial penalty for its performance.
This is because the Cabinet Office, which owns 24% of MyCSP, told the auditor that it had suspended charges, which if imposed, would have amounted to around £90,000.
Amyas Morse, head of the NAO, said MyCSP's performance was now "back to a steady state," but added: "The underlying data problems have still not been fixed. This should now be a priority for the Cabinet Office, MyCSP and the employers."
The Cabinet Office was fully aware of the extent of the problem and instructed MyCSP to take action and track its progress, which "successfully cleared the backlog and brought performance back towards acceptance levels".
By September 2015, call-handling rate had improved to pre-migration levels at 96%. MyCSP still received a "high" number of complaints at 428 a month, but the auditor acknowledged the rate was falling.
However, members and employers continue to report problems with getting "accurate and timely" information about their pension entitlement, according to the NAO.
A spokesman at the Cabinet Office added: "We recognise that there are still some underlying data problems and we are developing a data improvement strategy as a matter of priority."
MyCSP said they and the Cabinet Office agreed a number of actions to manage and improve service levels, such as recruiting new members of staff including apprentices, increasing enquiry centre opening times, and establishing an in-house training academy.
A spokeswoman at MyCSP said: "Service to members remains our utmost priority and we will continue to work with the scheme manager - the Cabinet Office - and employers to deliver towards that aim."