"We'd rather it didn't go ahead. Obviously the Zimbabwe government would be pleased if it did," Downer told Melbourne radio station 3AW on Wednesday.
"We obviously have some concern about the security of the players. I would be very concerned about the political statement it would make. It would be devastating."
"It would send the wrong message to Zimbabwe and to southern Africa, that oh well, we are not too happy with President (Robert) Mugabe but it's not that bad and, you know, the games can go on."
"I think frankly that's a bit of a pity."
"They (Cricket Australia) will have to make a decision in the end and we will leave the decision to them."
The Commonwealth suspended Zimbabwe in 2002, saying President Mugabe had rigged his re-election. Last month Zimbabwe resigned from the 54-nation body.
CA and the Australian Cricketers' Association will conduct a pre-tour inspection of Zimbabwe in March and that is where the matter lies at the moment, CA chief executive James Sutherland told Australian Broadcasting Corp. Radio.
Asked if Downer's comments would affect CA's stance, Sutherland replied: "No, it won't. Our position is pretty well set and has been for a long time."
"We'll make a decision on any tour based on safety and security grounds. We don't see it as appropriate that we make judgements on those other (moral)
ENGLAND TOUR
England and Wales Cricket Board (ECB) chief executive Tim Lamb said on Sunday the British government had effectively instructed England to cancel their October tour of Zimbabwe.
Before last year's World Cup, the British government called on England to boycott their match against Zimbabwe in Harare in protest against Mugabe.
The team refused the request but then withdrew from the match because of security concerns. Australia went ahead with their World Cup game in Bulawayo without incident.
Sutherland said if England refused to tour Zimbabwe this year it would have no impact on Australia's plans.
"In the World Cup last year, the England side decided not to go to Zimbabwe and in the end the Australian team did, after working through all of those safety and security issues," he said.
"We all understand that inevitably there is going to be some overlap between sport and politics."
"We will continue to make our judgements on the things that we have a mandate to operate on, and that is the game of cricket, and obviously look out for the safety of our employees."
