Sean Astin on Mom Patty Duke’s Bipolar Disorder: ‘We Were All in the Storm Together’

Patty Duke sought to raise public awareness of mental health issues when she shared her bipolar diagnosis with the world in her 1987 memoir, Call Me Anna.

But for son Sean Astin, his late mother’s mental health struggle was deeply personal. And now, after her death Tuesday at 69, the actor tells PEOPLE that even through the darkest depths of her illness, there was never any doubt of her unrelenting affection.

“It wasn’t always pretty and it wasn’t always fun. But there was a lot of love,” says Astin, 45. “Even through her divorce and her attempts with her own life and horrible things, there was a never a moment of doubt that she loved me. We were all in it together. We were all in the storm together.”

The star born Anna Marie Duke was diagnosed with bipolar disorder in 1982, a revelation that finally explained years of erratic behavior that included bouts of anorexia and relationships in the midst of manic states.

“It saved my life and it gave me life,” Duke told PEOPLE in 1999. “Prior to that, I wasn’t able to make any long-term decisions.”

Astin says that even as a young child, he was aware of her struggle, which profoundly impacted those closest to her.

“Like any family it was an all-consuming part of our lives,” he says. “By the way, there were great times, when the mania is going and you are up. It can be fantastic – humor and fun and travel and spending. When it’s down, you don’t have to be older than 3, 4, 5 years old to feel the weight of someone’s experience.”

Now, Astin (who has set up the Patty Duke Mental Health Project, a fund in her honor) is proud that his mother leaves a legacy as a mental health pioneer – a star who had the courage to share her struggle in an era when it was still considered so taboo.

“Her whole point when she wrote her book and announced it to the world was to destigmatize mental health disorders,” he says. “Thirty years later, it has worked. It’s not perfect now; there is a lot to do. My mom was one of the first celebrities to take that initial step. It was really worth it to her.”