Roger Federer may be one of the greatest tennis players of all time, but one thing he’s not? A pushy sports parent.

“No, I never pushed my kids into it,” he tells PEOPLE of getting his kids into the game. Of course, coming from such active parents — their mom is his wife and former Slovakian tennis pro Mirka Federer — they’re athletic kids nonetheless.

“They do play tennis, and we ski, and we’re all learning to golf. We recently tried scuba diving. But the girls were never passionate about tennis, and I was relieved about it,” Federer says of twin girls Charlene and Myla, 15, and twin boys Leo and Lenny, 10.

“The boys love their soccer. Leo is starting to show interest in getting serious about tennis. Lenny is much more creative, he’s very into drawing, reading.”

Federer says that while he’s not advocating for them to get into sports professionally, it’s fine if that’s what happens.

“I’ll jump on the court with them whenever they want,” he says.

In the new Prime Video documentary Federer: Twelve Final Days, about the two weeks leading up to him announcing his retirement (out on June 20), the usually private star opens up his home and allows viewers to see glimpses of his family life. He tells PEOPLE he’s loving how much time he gets to spend with the kids now that he’s retired.

“I used to have to sleep in in the mornings because of late matches. It’s completely different now,” he says. “I get to wake up when the kids do and help get them ready.” (He notes that he’s not the one flipping the pancakes — yet: “I’d love to learn to cook!”)

Once school drop-off is done, he’s often working on his Roger Federer Foundation and partnerships with brands like Uniqlo, Rolex and On athletic shoes.

“Then I get to connect with the kids again over their sports and homework,” he says. “It’s a very vital time for me to be around them, to help them and guide them and obviously let them fly on their own as well.”

And after so many years of his family traveling for his career, Federer says he wants to be sure his children now get their own time and space.

“They have their friends and need their playdates. We’re making sure their social aspect is a given.”

Not that the kids exactly minded tagging along. “The other day they were asking if we could go to Wimbledon because they wanted to walk the courts,” he says. “I’m just happy their life on the road was such a good memory.”