Fulham [ Images ] were the other Premier League side to fall as they lost 1-0 to Championship (second division) Sheffield Wednesday.
West Brom had to come from behind, relying on two late goals to scrape past third-tier Yeovil Town 4-2.
James Collins scored a hat-trick -- including the winner in extra time -- as Swindon, managed by former West Ham United striker Paolo Di Canio, upset Stoke at the Britannia [ Get Quote ] Stadium.
Stoke came back from 2-0 down to force extra time and equalised for a third time through Peter Crouch after 111 minutes.
But Tony Pulis's side had no answer after Collins slid in to make it 4-3 in the 119th minute, prompting wild celebrations by Di Canio on the touchline.
At Hillsborough, Sheffield Wednesday, League Cup winners in 1991, accounted for Fulham with a Gary Madine penalty in the 50th minute after Chris Maguire was fouled by Stephen Kelly.
Goals from Belgian Yassine El Ghanassy and Shane Long in the final 17 minutes saved West Brom from embarrassment at Yeovil after the third tier side had taken the lead and then levelled at 2-2 through Reuben Reid.
At Villa Park, Paul Lambert enjoyed his first victory as Villa manager with a 3-0 defeat of third-tier Tranmere Rovers, in a game that marked England [ Images ] striker Darren Bent's first goal since February.
Nathaniel Knight-Percival's own goal gave Premier League Reading a 3-2 victory over second-tier club Peterborough after four goals in the first 19 minutes at the Madejski Stadium had left the sides level at 2-2.
Premier League clubs Wigan Athletic, Sunderland and West Ham had relatively straightforward wins.
Wigan brushed aside second-tier Nottingham Forest 4-1 at the City Ground, Sunderland beat fourth-tier Morecambe 2-0 and West Ham United eased past Crewe of the third division 2-0 at home.
Norwich City beat Scunthorpe United 2-1, Queens Park Rangers overcame Walsall 3-0, Southampton won 4-1 at Stevenage and Swansea City enjoyed a 3-1 victory over Barnsley.
Elsewhere, fourth-tier Burton Albion caused a surprise with a 4-2 victory at second-tier Leicester City.
Everton host Leyton Orient on Wednesday while the Premier League teams in European competition enter in the third round.