The official spoke on condition of anonymity because he was not authorized to discuss what Bergdahl has revealed about the conditions of his captivity. The New York Times first reported on the matter.

The official said it was difficult to verify the accounts Bergdahl has given since his release a week ago.

Bergdahl, now 28, was captured in June 2009 after he disappeared from his infantry unit. He was held for nearly five years by Taliban militants.

Taliban spokesmen could not be immediately reached for comment Sunday. On Friday, Taliban spokesman Zabihullah Mujahid told The Associated Press by telephone that Bergdahl was held under "good conditions." The claim could not be independently verified.

Military doctors at the Landstuhl Regional Medical Center say that while Bergdahl is physically able to travel he's not yet emotionally prepared to be reunited with his family. He has not yet spoken to his family.

It's unclear when he may get to go home.

The Pentagon said in a statement Sunday that it would not comment on Bergdahl's discussions with those caring for him.

Typically, a returned captive would spend from five days to three weeks in the phase of reintegration in which Bergdahl now finds himself, according to a Pentagon psychologist who is an expert in dealing with military members who have been released from captivity said this past week. The psychologist spoke to reporters Thursday on condition of anonymity under ground rules set by the Pentagon.

Once Bergdahl is considered ready to move on to the next phase of his decompression, he is expected to be flown to an Army medical center in San Antonio, where it is believed he will be reunited with his family.

Bergdahl was returned to the U.S. military in exchange for the release of five Taliban militants from the U.S. military prison at Guantanamo Bay, Cuba.

"It would have been offensive and incomprehensible to consciously leave an American behind, no matter what," Secretary of State John Kerry said.

Qatar, a tiny Gulf state, served as a go-between during the negotiations, and has ongoing role in ensuring the five released prisoners remain there for at least a year, under a memo of understanding with the U.S.

The Qataris aren't "the only ones keeping an on eye on them," Kerry said in an interview broadcast Sunday on CNN's "State of the Union."

He said the U.S. has confidence in the restrictions imposed on the former detainees, as a condition of their release.

"I am not telling you that they don't have some ability at some point to go back and get involved" in the terrorism fight against the United States, Kerry said.

"But they also have an ability to get killed doing that, and I don't think anybody should doubt the capacity of the United States of America to protect Americans. ... So these guys pick a fight with us in the future or now or at any time at enormous risk," Kerry said.

The deal, which the Obama White House pulled off without adhering to a law requiring 30 days' notice before the release of Guantanamo detainees, ignited a political firestorm that shows no signs of abating.

Lawmakers, both Republicans and Democrats, who initially praised Bergdahl's release backed off amid questions about whether he was a deserter who walked away from his post and an outcry over the exchange.

Some of Bergdahl's fellow soldiers maintain that Americans died during efforts to find and save him. Also, there is great concern that the high-level Taliban officials will resume activities with the Taliban and threaten members of the U.S. military in Afghanistan.

"I just think that's a lot of baloney," said Kerry, noting that the U.S. will be ending its combat role in Afghanistan and will have fewer people in positions of risk.

U.S. Sen. John McCain, who was held prisoner during the Vietnam War, agreed with Kerry that the U.S. should do all it can to win the release of any American being held, "but not at the expense of the lives or well-being of their fellow servicemen and women."

He told CNN that "when we join the military, we know we take certain risks, and among those risks are wounding, death, imprisonment."

On Wednesday, Bergdahl's hometown of Hailey, Idaho, abruptly canceled plans for a welcome-home celebration, citing security concerns. And on Saturday the FBI said Bergdahl's family had received threats that are being investigated by federal, state and local authorities.

The U.S. official told the AP that Bergdahl's parents were being harassed and threatened, including death threats.