Dr Singh was speaking to journalists at the end of the Nuclear Security Summit hosted by President Barack Obama in Washington.
Tharoor, who is locked in a fresh controversy around the Kochi IPL team which he helped set up, on Tuesday hit back at IPL boss Lalit Modi, alleging he had made attempts to pressure the winning consortium to abandon their bid for the Kerala outfit.
Tharoor also denied that he had told Modi not to ask details about owners of the consortium led by Rendezvous that bought the new franchisee.
When he was asked if Tharoor would be sacked over the Kochi IPL team controversy, the PM said, "I have heard about these things. I don't have all the facts before me and when I go back, I will get all the facts and in the light of these, if any action is necessary, I think that would be the proper way to proceed."
"I cannot, I think, go by hearsay or what appears in various columns of the newspaper," Dr Singh added.
Video: Paresh Gandhi