City's Bernardo Silva and Mateo Kovacic both misfired in the shootout to put Madrid in the driving seat
Khedira collided with Cornella midfielder David Garcia shortly after halftime at the Bernabeu and after playing on for a few minutes the German international had to come off and was replaced by Jese.
"Following scans carried out on Sami Khedira by the Real Madrid medical services ... a concussion has been diagnosed," the European champions said on their website (www.realmadrid.com).
"He will remain under observation for the whole night."
Holders Real won the last-32, second leg game against the third-tier minnows 5-0 for a 9-1 aggregate victory.