D'Oliveira's health has been deteriorating for some time before he passed away in England, CSA said on its website without sharing further details.
Cape Town-born D'Oliveira emigrated to England aged 28, where he had a distinguished career, scoring 2,484 runs at an average of 40 and taking 47 wickets in 44 Tests from 1966-72.
D'Oliveira less successful in the four one-day internationals he played for England, managing 30 runs and three wickets.
England cancelled their 1968 South Africa tour when the ruling National Party objected to the presence of the non-white D'Oliveira in the visiting team.
No official team from any country subsequently toured South Africa until apartheid was abolished following Nelson Mandela's [ Images ] release from prison in 1990.
"'Dolly', as he was known around the world by an audience that went far beyond the game of cricket, was a true legend and a son of whom all South Africans can be extremely proud," CSA chief executive Gerald Majola said in a statement.
"He was a man of true dignity and a wonderful role model as somebody who overcame the most extreme prejudices and circumstances to take his rightful place on the world stage.
"The fact that he could have a test career batting average of 40 in 44 Tests and an economy rate of less than two with the ball on his way to 47 wickets was remarkable considering he was past his prime when he made his debut for England in his mid-30s.
"One can only imagine what he might have achieved had he made his debut as he should have done at the age of 20 on South Africa's tour of England in 1951," Majola added.