Hilary Duff's Husband Matthew Koma Playfully Teases Her Ex Joel Madden on His Birthday

Neilson Barnard/Getty Images; Gilbert Flores/Variety via Getty Images Hilary Duff and Matthew Koma (left) and Duff's ex Joel Madden (right)

Hilary Duff's husband Matthew Koma is wishing her ex Joel Madden a happy birthday with a few playful jokes on his Instagram Story.

Koma, 35, who married Duff in 2019, shared a few silly images on social media Saturday in honor of the special occasion for Madden, who dated Duff from 2004 to 2006 and has been married to Nicole Richie since 2010.

In one post, Koma shared a photo of Madden, 44, and Duff, 35, in the mid-2000s, alongside a funny caption. "It's not even weird that you dated my wife, it just makes me feel closer to you," he wrote.

In a follow-up post, Koma shared an image of himself with Richie, 41, where she can be seen sitting on the floor while he plays a pink guitar. "And now feels like a good time to tell you about me and your wife," Koma joked in the caption.

RELATED: Hilary Duff and Husband Matthew Koma Enjoy 'Date Night' with Her Ex Joel Madden and Nicole Richie

Elsewhere on his Instagram Story, Koma also shared pictures of Madden with his eyes closed, a photo of the pair walking their dogs and a throwback photo of the two with Madden's twin brother, Benji Madden.

He also shared an image of the words "Matt + Joel 4 eva" written in what appears to be red crayon.

Madden later reposted a few of Koma's birthday wishes, including the photo of him and Duff, adding, "We love you Matt."

During a January episode of Watch What Happens Live, Duff revealed that she and Madden have long been good friends, despite the fact that "the whole world's mind was blown for a hot second" over Koma having dinner with him and Richie.

"By the way, I have a good relationship with their whole family. I think people were [blown away]. I don't know. We're neighbors. And we actually hang out all the time," she explained at the time.

"I was just in her driveway, honking for her to come drink wine with us the other day," Duff added of Richie. "She wouldn't, she was asleep by 8."

Duff also shared that since they're all neighbors, they "hike with all the time," adding: "And you know what? It's lovely."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

RELATED: Hilary Duff Has 'Come Clean': She Never Watched Laguna Beach, Even Though Her Song Is the Theme

Duff shares daughters Banks Violet, 4, and Mae James, who is turning 2 years old this month, with Koma. She also co-parents son Luca Cruz, who is turning 11 this month, with retired NHL ex-husband Mike Comrie, to whom she was married from 2010 to 2016.

Madden and Richie, meanwhile, share daughter Harlow, 15, and 13-year-old son Sparrow.

Last February, Duff posted about a "date night" with Koma, Madden and Richie that also included Pete Wentz and his partner, Meagan Camper. They were similarly joined by Pulse Music Group founder Josh Abraham and his wife Gina.