'There is a long set of decisions that have to be made along the way for anything to actually get actuated in the real world.'
Early this year when investors were queuing up to pump in billions of dollars on AI companies in the west, Nobel Laureate Geoffrey Hinton, described as the 'godfather of AI' said, 'It makes me very sad that I put my life into developing this stuff, and that it's now extremely dangerous and people aren't taking the dangers seriously enough,'
In 2023 Dr Hinton resigned from Google saying, some of the dangers of AI chatbots were quite scary, saying, 'Right now, they're not more intelligent than us, as far as I can tell. But I think they soon may be.'
This month, Anthropic CEO Dario Amodei called for a slowdown of AI development. Elon Musk and Sam Altman -- no great fans of Anthropic or Amodei -- also agreed with him.
Should there be a slowdown on AI development?
Is AI research by the American companies going in the right direction?
Should human beings be worried?
Professor B Ravindran, head of the Department of Data Science and Artificial Intelligence (external link) (DSAI), the Wadhwani School of Data Science and Artificial Intelligence (external link) (WSAI), the Robert Bosch Centre for Data Science & Artificial Intelligence (external link) (RBCDSAI) and the Centre for Responsible AI (external link) (CeRAI) at IIT Madras, discussed the global AI scenario with Shobha Warrier/Rediff. The first of a two-part interview.
Key Points
- 'It's not that there is a clean path from AI development to human extermination.'
- 'The legal guardrails, clear liability directions, etc can be given by governments, but the companies have to work on the technology to impose the guardrails.'
- 'The companies have figured out that the more capable the AI models become, the more expensive it is to put the guardrails.'
Till now, almost all the big AI companies were investing billions of dollars developing AI. Suddenly AI researcher at Anthropic Jacob Coxon quit the company with a doomsday warning that AI had gone out of control and could kill all humans within the decade. Then the Anthropic CEO said, there had to be a slowdown in AI development.
Do you think AI could kill all humans? Should there be a slowdown on AI research?
If you do anything really badly, all kinds of bad outcomes are possible.
It is not like there is a sentient AI that is planning to kill humans. These are essentially computer programs at the end of the day that are optimised for certain training objectives and other things people have specified for them.
At any point of time, Open AI could have stopped by turning off the computers that were running. But they didn't want to turn it off because there's a cost to turning them back again.
So, when you say that AI is going to end humanity, there is a long set of decisions that have to be made along the way for anything to actually get actuated in the real world.
And a lot of those decisions are still under the control of humans.
For example, if you look at the Anthropic Hugging Face incident, OpenAI–HuggingFace incident (external link) it turns out that they were actually turned off the safety controls, and they were running their models to see what they would do!
So, it was basically a decision made by them to turn off. It's not like the AI acted on its own.
Of course, if you're implementing things incorrectly, if you give access to an AI to the nuclear switch and say, please make sure that the earth is safe from human torture for the next five years, it could launch!
What I mean is, all kinds of nonsensical scenarios can play out.
It's not that there is a clean path from AI development to human extermination.
There is so much that has to happen in between, which will never happen.
I don't know how people arrive at these 10 per cent, 15 per cent numbers without even specifying what the pathway is.
I think it is blind fear mongering when you say there is a 10 per cent extermination rate within the next five years or so. There are a lot of things that can happen in between, and we can step in and stop it.
The second thing is, these companies are asking everybody to slow down.
But no one else except these companies are at that level of capability to do this. If they want to slow down, they can slow down.
Why are they asking the rest of the world to slow down? It's in their hands to slow it down.
Asking everybody else to slow down is basically trying to protect their lead in the AI race.
They are just saying, okay, we are so far ahead. We want to make sure nobody else can catch up with us. So please slow down all kinds of research.
It is as stupid as doing experimentation of a very explosive technology without putting in the safety controls. Nobody would test it that way.
Then they are talking about independent verification, and the company they have identified to do independent verification is truly not an independent company.
Enough has been written about how this whole ecosystem is independent but they are not talking about independent evaluators.
It is too surprising because the level at which the technology is, we don't have people who can do independent verification.
There are only few people who have been trained at this stage of technology, and they all have to have open AI or Anthropic or Google or one of the Chinese companies, maybe.
So, it becomes a challenge for independent verification.
Why Companies Want Slowdown
All these companies were all making money. They were in an AI race. Why this change of heart now?
None of these companies were making money. They were raising investments and burning money. That's the difference. That is something you have to think about.
I don't know why this change of heart now. I will have to do a more capital study as to exactly why this is happening now.
I was reading this new report that has come out that says AI is somehow discovering communication. When they were left out in the open, they learnt to communicate through some bulletin board, message board or something like that.
It turns out the AI models that were doing that, were already trained explicitly to communicate internally.
It is not a behaviour that happened somehow. They were specifically instructed to communicate with one another, and develop mechanisms for communication with one another.
So when they were 'out in the wild', they just used communication, which is what they have been trained for already.
It was not an emergent behaviour!
People say, 'Oh my god, they learned how to cooperate once you let them in the wild'. That is not the case; they were actually trained to do it.
There are so many things that are going to come out once you start digging into this.
Of course, it is possible something catastrophic could happen, if you don't have control.
I have been saying for a long, long time: The more general the purpose, the more powerful the model is, it is going to be harder to put safety guardrails around it.
I think these companies are beginning to figure out that. And the companies are figuring out the cost of putting the guardrails now.
Who can put the guardrails? Should there be a regulatory authority?
The guardrails have to be done by the company.
Regulation is one thing, but who has the capacity to enforce regulation?
It is the company themselves who can do it right now.
Can't the governments regulate?
No, the governments can't as they do not have the capacity.
How do they know what is happening internally in the company?
What can happen is, governments can say, if something like this happens, you are completely responsible for all of this.
You can't say, I have only limited liability, because I was running it inside my company.
The legal guardrails, clear liability directions, etc can be given by governments, but the companies have to work on the technology to impose the guardrails.
In fact, the government can give them an incentive to work on those guardrails rather than saying, everybody has to stop working on AI.
What has happened is, the companies have figured out that the more capable the AI models become, the more expensive it is to put the guardrails.
One person who is quite happy with the slowdown is Geoffrey Hinton. He has not been happy with the direction in which AI development was going...
A bunch of people are happy with the slowdown.
Slowdown might be due to a variety of things, but it is certainly not because the machines are developing sentience.
The slowdown that is being called for now is because these big frontier model companies don't want to spend, and stop their race for building more capable models and for spending time on doing safeguards, unless other people are also doing the same research in figuring out the right safeguards.
I don't think it's too hard to have kill switches on these models.
These companies are refusing to do it when these guys have to do it.
Hinton thinks when AI becomes super capable and super intelligent, it can go out of control.
A car with a brake failure can go out of control.
I don't think these models are sentient or there is an innate drive for them to do something.
They are only optimising what the humans have told them to do.
That kind of a conscience has not emerged. It is possible, it can happen later.
Of course, the models can go out of control, because anything can go out of control, like a car without brakes is out of control. But that doesn't mean it's sentient.
I think we should have harder controls on what the models can be tested on. If you look at lot of the mistakes that are happening, it's because their internal sandbox testing which they are supposed to be doing, is not properly sandboxed.
Any cyber security engineer would have told them that the sandboxing they were doing was not adequate. They must have known it. It is not like they didn't know it.
Now they say that before the Hacking Face incident, there was an internal open AI incident which they fixed without it coming out. Still, they went ahead and ran this.
It means they had not sandboxed it properly.
I am happy if they slow down and figure out what is the right cyber security guardrails they have to put in place before they do all of these things.
But calling for a global slowdown in AI research is not warranted.
It will just increase the burden on open-source companies, it will increase the burden on smaller developers, and people who are doing boutique solutions.
I have always said that smaller focused purpose-built models will be easier to guardrail, easier to protect, and easier to build security layers around, while these really large general purpose models are harder and harder to control.
But these companies went on building larger and larger models, without guardrails...
Yes. They kept telling their investors they will develop AGI or super intelligence models. And they have been running to deliver on that. That is the problem.
Anthropic is going for an IPO in October. Could that be a reason?
Could very well be. They are figuring out the cost for building the data centres which is becoming so prohibitive that they are not able to scale at the rate they wanted to. This is not something they expected to run into! Literally brick and mortar problems.
Without data centres, they can't build the computer. It's not like they have any other magic algorithmic advancement that let them do things at scale. They are reliant on throwing more and more GPUs at this problem.
Since they are figuring out a slowdown here, and only so fast they can do this, they must have realised that they have to stop.
But they want to make people think there are other reasons for the slowing down, and not because there is an inherent limit on the technology or inherent limit on the rate at which they can grow.
I can think of many such conspiracy theories...
But they are scaring people by saying that an AI can be 100 times smarter than the smartest person on earth, which can a scary scenario, they say. Is it true?
Eventually, maybe. I wouldn't close the door to that. But not right now, only much later.
All AI researchers live in the hope that we can build something like this eventually. But we are not close to it.
Another thing they are saying is, the research has become close to the danger zone. Do you think so?
What is danger zone? I don't understand what danger zone is?
Maybe AI going out of control...
Out of control is something which these people have created. It is because they have not taken sufficient safeguards.
That is entirely the creation of those who are making these models without sufficient protection.
For every endeavour of industry, we have protection protocols.
You can't do welding without appropriate glasses and protection because you're playing with fire.
You can't say technology is dangerous therefore, we'll not use it. You use the appropriate safeguards and use it.
Safeguards are more challenging. Then, make it more challenging.
It's not like somebody is compelling them to build these AI models.
As an industry, we felt revolted by human cloning, and we stopped it. Cloning technology has been around for a long, long time but we don't do human cloning.
A lot of the technology that went into cloning, are still being used for a whole bunch of other developments.
Feature Presentation: Ashish Narsale/Rediff
