The victor advanced to the quarterfinal to face Sweden, and have only Croatia and Russian Federation on the road ahead.
Fans wearing replica shirts chanted "football's coming home", the refrain which has always been adopted as an unofficial national anthem for the England football team, at various points during the match, most notably after Spurs striker Kane's opener, goalkeeper Jordan Pickford's penalty save, and substitute Eric Dier's decisive spot-kick.
There had been precious little drama - though much petulance - throughout most of normal time as a Colombia side shorn of injured playmaker James Rodriguez set out to defend. The (Round of 16, Match 8) match between Colombia vs England will be LIVE telecast on Sony Ten 2/Sony Ten 2 HD (English), Sony Ten 3/Sony Ten 3 HD (Hindi) and Sony ESPN (Bengali, Malayalam, Tamil, and Telugu) TV channels.
The Three Lions' travelling support has descended on Moscow, albeit in fewer numbers than their rival Colombia fans.
It's the first time in World Cup history to have three Round of 16 matches end in a shootout.
It brought up the Tottenham star's sixth goal of the tournament, and also saw England's captain equal Tommy Lawton's 79-year-old record.
Arsene Wenger may need to reconsider his prediction about Jordan Pickford after the Everton goalkeeper was England's hero against Colombia.
After England secured passage to the knockout stage, Southgate explained how that mentality had translated into his thinking on penalties.
However, England conceded a late equaliser from a determined Colombian attack, with Yerry Mina finally finding the back of the net in the third minute of extra time.
"Penalty shootouts are a lot about mentality, and obviously we know England haven't done great in the past, so it was nice to get that one off our back and it'll give us huge belief going forward". There were so many interruptions in the game. Colombia are usually a team that plays with a smile on its face, but too often in this game it was a nasty scowl. So if a team wants to play a certain style, those interruptions are not favorable.
Managers Jose Pekerman and Southgate move quickly to console the Colombian side after the dramatic finale. I know penalties are looming and that means one thing: England lose. But it was clear they no longer had the spark, and the extra 30 minutes seemed to evaporate. Colombia are also good.
"If we were down, it was because we were fouled", he said. He had to miss Tuesday's match, watching from the sidelines.
Southgate admitted the gruelling nature of the match had taken its toll on his players, with Colombia shown six yellow cards.
But Pickford, England's youngest-ever tournament goalkeeper, and Dier came to the fore to propel their country into the last eight of Russian Federation 2018.