Doherty, who played Prue Halliwell, died in July at age 53 after years of living with cancer, and the cast remembered her at '90s Con Florida on Sept. 15

"I knew Shannen before the show, we met when I was 19," Combs said. "And we had actually gone to network with another person to be Phoebe and the network didn't like her. They didn't like me either. So they wanted to keep Shannen and recast the other two."

However, "Shannen and [executive producer] Aaron [Spelling] said we're not really the show without Holly," Combs continued. "It was a negotiation. I was supposed to be auditioning for Phoebe which would've never worked. Shannen was supposed to be Piper. We had to convince Aaron we were old enough to play the older characters -- he actually asked for our IDs. He saw her [Shannen] as a little girl still."

Doherty ended up playing Prue Halliwell, Combs played Piper Halliwell and Alyssa Milano played Phoebe Halliwell.

"There was an old school mentality in that they like an actor or actress and want to find a home, a spot for them. I was lucky to be in that stable," Fuller said. "I had done a Spelling pilot that didn't go anywhere, he was devastated about that, he came to me. Then I ended up being moved to a different WB show, a karate show. Kids after school doing like 21 Jump Street karate. [Spelling said] 'We've got this show for you, it's Charmed. We've written a role for you, we don't fully know where it's going to go yet.'"

McGowan admitted she didn't believe it when she got a call from Spelling, who died in 2006 at age 83, about doing Charmed.

"The phone rang in my hotel room. I answered it and it's, 'Hi, I'm Aaron Spelling.' I hung up," McGowan said. "The phone rings again and like, 'No, it's really me, Aaron Spelling.' 'Okay.' 'Come and meet me in L.A.' I was just finishing this project in Romania. I'm on the plane on the way home, I asked, 'Okay universe, I don't know this show, give me a sign, I'm meant to do this show.' I've been on a lot of planes and never since have I seen Charmed on any airplane, let alone the pilot episode. I'm on Romania Air and the only in flight entertainment on this flight back to the U.S. was the pilot episode of Charmed and if that's not a sign right there. "

The Brave author added, "I've been to Romania since then and it was not on the plane again."

Krause, 55, said he'd auditioned for Spelling "a million times before this" and nothing ever panned out.

"I was doing construction, I had a new baby at home, I just wasn't breaking through," Krause explained. "I got the audition, I came in and did it, asked if I could come back in the afternoon. I had to make 100 bucks today, my wife's gonna kill me. Come back at 2, come back at 6. I come in and do it for Leo and it's this soft, lovey kind of scene, nice chemistry, possible love interest type of thing."

"So I do it. I had met Aaron a million times. My strategy, I'm going to walk in and say, 'Mr. Spelling, it's good to see you again.' I walk in ... he looks down at my picture. 'Hello, um, Brian, it's nice to meet you.' I'm just like, f--- this s---. I did kind of an angry read, I finish, look up like, 'Anything else?' I walked out just pissed. I wasn't even halfway home, I got paged and 'you got it.'"

Related: Rose McGowan Opens Up About Her Biggest Regret in Her Friendship with the Late Shannen Doherty

"I'm not done with living. I'm not done with loving. I'm not done with creating. I'm not done with hopefully changing things for the better," she told PEOPLE. "I'm just not -- I'm not done."

Shortly after the news of Doherty's death, Combs shared a heartfelt video to Instagram that featured tons of photos of the pair over their decades-long friendship. As photos flashed across the screen in Combs' video, Faith Hill's song "There You'll Be" played in the background.

Never miss a story -- sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from celebrity news to compelling human interest stories.

"It's been one week and it feels like forever," Combs wrote in the caption of the post. "I spent all day looking for one particular picture and found just about every picture but that one."

"Shannen promised to haunt me but I thought she'd be occupied with a few others at first," she continued. "Alas I'm sure it will show up when and where I least expect it. My rock. 10:18."

90s Con Florida is occurring through Sunday, Sept. 15, at the Ocean Center in Daytona Beach, Fla.