Red Bull claimed consecutive wins for just the second time in the Hybrid Era, but it was Perez rather than Verstappen who led them to glory.
Verstappen was in complete control and set to head a Red Bull one-two when his left-rear tyre blew out in the closing stages. The same issue had ruled out Lance Stroll of Aston Martin.
That left the door open for title rival Lewis Hamilton to profit, but an error from the defending champion from the restart allowed Perez to race clear.
Red Bull boss Horner told Sky Sports: "I think I've aged about 20 years.
"We were staring down the barrel of our first one-two since 2016. Everything in control, we'd just checked in with Max, all okay, and then boom.
"The tyre went, and we don't know why. It looked like the wear was in good shape, so whether it's debris or something has happened.
"You can see it's a big place to have an accident. Thankfully, he's okay."
He added: "We were feeling at that point like the world had dropped from under us."
Worse might have been to come for Red Bull as Perez finished the race but then immediately broke down, receiving frantic messages over the radio as he celebrated his first victory for the team.
The Mexican, who has five consecutive top-five finishes for the first time in his career, said: "I'm so, so happy for today. Normally, Baku is pretty crazy.
"First of all, I have to say for Max I'm sorry. He did a tremendous race and really deserved the win, and it would have been incredible to get that one-two for the team.
"But at the end, it is a fantastic day for us. We were close to retiring the car but, luckily, we were able to finish the race. It was quite difficult all the way in the end."
Horner lauded Perez, whose victory moved Red Bull 26 points clear in the constructors' championship on a day Mercedes failed to score.
"He's a wily driver, a canny racer. He races really, really well," Horner said.
"We went from the despair of 'We've lost Max, we're going to lose the championship lead, Lewis could pass him at the restart and win the race'.
"It just shows the rollercoaster you go through in this sport.
"Suddenly it not being quite so bad, we've actually maintained the same lead in the drivers' championship leaving here and extended the constructors'."
Red Bull dedicated the victory to Mansour Ojjeh, the McLaren shareholder who passed away ahead of the race.
"He had the biggest heart and always carried the biggest smile," ex-McLaren man Hamilton posted on Twitter. "I am so grateful to have known such a man."
I carry a heavy heart into the race after the loss of a close friend, Mansour Ojjeh. He had the biggest heart and always carried the biggest smile. I am so grateful to have known such a man. This man loved unconditionally. Rest in peace brother, love you always🖤🙏🏾 pic.twitter.com/EGGFx2vtlM— Lewis Hamilton (@LewisHamilton) June 6, 2021