“Ferg is right in the middle of shooting a video, so it’s going to be a pretty low-key Mother’s Day,” Duhamel, 43, told PEOPLE while attending the Kentucky Derby Saturday with G.H. Mumm champagne.
Want all the latest pregnancy and birth announcements, plus celebrity mom blogs? Click here to get those and more in the PEOPLE Babies newsletter.
Between both of their schedules, the actor says he and his pop star wife have been “crazy busy” for the last couple of weeks and are looking forward to enjoying some quiet family time with son Axl, 2.
“I [asked her], what do you want to do? We got the whole day off,” said Duhamel, 43. “She said: ‘I just want to order some food in, and be with you and Axl.’ We’re just going to sit at home and just be a family.”
“It’s going to be nice just to have a very low-key, at home, Mother’s Day,” Duhamel said. “We’ll probably order in from someplace fancy, but that’s about as extravagant as we’ll get.”
Of Axl, who will turn 3 in August, the actor marveled at how special it’s been to watch his son grow up before his very eyes.
A photo posted by Josh Duhamel (@joshduhamel) on Apr 17, 2016 at 12:27am PDT
“It’s pretty amazing to watch a newborn baby turn into an almost-3-year-old,” he said.”It’s like, wow. You hear everybody talking about that but to see it — and everyone says it just keeps getting better and better, and it truly does. Every day he’s blossoming right in front of us. It’s really beautiful. He’s a happy kid, too. He’s a happy boy and that’s really what we want.”
A photo posted by Josh Duhamel (@joshduhamel) on Mar 26, 2016 at 7:10pm PDT
And hopefully, Axl will be growing up with a little brother or sister by his side at some point.
“That’s the plan,” Duhamel said of expanding his family with Fergie. “It all depends on when we get a window of opportunity, you know. And it’ll happen. It’s definitely been our plan.”
In the meantime, Fergie, 41, is busy working on her upcoming album, while Duhamel is enjoying the big race in Louisville, Kentucky, for the first time in person.
“It’s crazy here,” he said. “It’s just endless, endless people everywhere you look, all dressed up to the nines, giant hats and flower dresses and dudes in suits.”
“I watch it every year on TV, and to see it in person and to see all the extravagant outfits and hats, all the pomp and circumstance that goes with it, is quite a sight to see,” he added.
And while he’ll “put a little money on a horse,” what he’s most excited about is all the people-watching: “It’s as much about people-watching as it is about watching the horses race! That’s my favorite pastime, people-watching.”
As for whether they might make it a family trip next year? Duhamel says he “would love to take [Fergie and Axl] to see this.”
“Ferg is jealous that I got to come and see it while she was working, but I told her next year we’ll come,” he said. “I think my boy would love it, too.”