Goals from Gary Cahill and Ramires had Jose Mourinho's men cruising after 38 minutes of their fourth-round tie but John Stead pulled a goal back before the break and former Chelsea player Filipe Morais equalised in the second half.
Andy Halliday made it 3-2 before Mark Yeates' stoppage time goal sent the Bradford fans into raptures and left the manager of the Premier League leaders seething.
"Me and the players must feel ashamed," Mourinho told the BBC as his hopes of a quadruple this season for the Premier League leaders were extinguished in the most unexpected circumstances.
"I repeat a word I used before this match, it's a disgrace for a big team to lose to a small team from a lower (division). Frustration is not the right word, embarrassed would be more appropriate."
Premier League champions Manchester City could have few complaints as they were outplayed by second tier promotion hopefuls Middlesbrough for whom Chelsea loanee Patrick Bamford deflected in the opener and Spaniard Kike wrapped up victory.
"An unforgettable day for us and the fans. We went through some tough tests, the keeper kept us in it at times. The scruffy goals are the goals I love," Bamford said.
"As a team we were magnificent."
The surprises, that begun earlier in the day when top flight Swansea City lost 3-1 at second tier Blackburn Rovers and had two players sent off, continued when Southampton went down 3-2 at home to Crystal Palace, meaning the top three in the Premier League all crashed out.
Fourth-placed Manchester United could only draw 0-0 at fourth tier Cambridge United on Friday.
Spurs out
Tottenham Hotspur were also beaten 2-1 at home by the Premier League's bottom club Leicester City on Saturday.