The insider notes that his six kids with ex Angelina Jolie have not spent "extensive time" with Pitt's parents ... every year for their birthdays and Christmas," says the source.