Neither suspect is willing to speak to reporters, according to friends and neighbors in Battambang province, where they both keep homes.
The complaint follows the resignation earlier this month of UN investigating judge Siegfriend Blunk.
Patricia O’Brien, UN undersecretary-general for legal affairs, met with Cambodian officials to discuss concerns for the tribunal.
A growing number of court observers now say the court’s approach to cases 003 and 004 is derailing its credibility.
Medical experts said her current mental state would prevent her from fully participating in a trial, even by video link.
A UN spokesman said last week O’Brien plans to meet political and court officials, including Council Minister Sok An.
Peou Nam was taken twice for execution in a pit by the Khmer Rouge, and survived both times.
Chhang Youk, added in a later interview that the UN should not allow the resignation to go uninvestigated.
The hybrid court, the first of its kind, has spent $150 million and so far completed only one trial since 2006.
Government spokesman Khieu Kanharith dismissed allegations the government was interfering with the work of the court.
His resignation comes a week after Human Rights Watch said he and his Cambodian counterpart should quit.
Meas Muth, the former commander of the Khmer Rouge navy, is among five suspects in two cases before the UN-backed tribunal.