"On the question of my resignation, let me say I have quite a lot of work to do," Annan said citing preparations for next year's summit, the report he intends to release in March reviewing the Millennium Development Goals and tabling his views on the UN reforms.
He said the issue had "cast a shadow" over the UN and its relations with the US.
There are no personal allegations against Annan but some American Republican lawmakers had demanded his resignation as he was head of the organisation when alleged corruption took place which, they contended, showed lax control.
Addressing reporters Tuesday at the United Nations, Annan said "there's no doubt that this has been a particularly difficult year and I am relieved that this annus horribilis is coming to an end."
He said he has the confidence of the 191-member states, and the Bush administration has neither demanded his resignation nor raised the issue during the talks he had with Secretary of State Colin Powell and his successor Condoleezza Rice last week.
Asked about criticism of some Republican lawmakers, Annan said, current criticism and attacks have not been helpful for the relationship between UN and US, regardless of which quarters they came from.
"But we need to find a way of putting those acrimonious discussions behind us and move on. We have a very important agenda to tackle for the next two years and so and I think it is important that everyone focuses on that.
Responding to a question on US-UN relations, he said they have gone through tense periods periodically. "But the US needs UN and the UN needs the US. And we need to find a way of working together."
However, he did not support his son, Kojo, who during an interview with CNN described the attacks on the programme as a witchhunt from day one as part of a broader Republican political agenda.
I don't agree with that," was Annan's terse reply when asked about Kojo's remarks.
Kojo is under investigation for his relations with a Swiss firm which had a contract from UN to ensure the goods ordered by the Saddam Hussein regime reached their destination.
Annan declined to answer specific questions about the allegations, saying everyone should wait for the report of the independent inquiry led by former US Federal Reserve Chairman Paul Volcker.
Asked if some mistakes had been committed and he wished he had done things differently, Annan said, "When you run this sort of operation, it is inevitable that there may some mistakes and things that could have been done differently and things could have been done better."