Jill Duggar Dillard and her family have been slowly distancing themselves from the rest of her supersized family for the past few years. Jill, who was once a regular cast member on the hit TLC reality show 19 Kids and Counting along with her parents, Jim Bob and Michelle Duggar, and her many siblings went on to star in the subsequent spin-off, Counting On. Jill and husband Derick Dillard ended up quitting the show and an apparent rift has formed between them and Jill's famous family. Now Jill is speaking out about why things have become strained between them.

"Our control to choose what jobs we were allowed to accept and even where we were allowed to live was taken away from us," Jill told PEOPLE in a new interview about why she and her husband chose to leave Counting On. Derick, who has been vocal about his feelings about the show as well as Jim Bob's control over their earnings, added, "The first few years of our marriage, we spent time and money working towards opportunities only to hit a dead end when we'd be told, 'Well, you're not allowed to do that.'"

The mother of two told the magazine that when they told her family about their decision to stop filming,  "It didn't go over very well with anyone. By that point, we'd had enough. We knew we had to pull out completely to reevaluate and get our bearings," she added.

Related: Anna Duggar May Have Taken A Shot Against Michelle & Jim Bob's Parenting

Jim Bob and Michelle Duggar released a statement to PEOPLE in response. "Every family has differences of opinion and perspective at times, but families work things out," it read. "We all love Jill, Derick, and their boys very much. It is our prayer that our relationship is healed and fully restored quickly!"

Fans of Jill have noticed that she and Derick have definitely broken away from many of the strict rules that Jill was brought up adhering to. While the Duggars prefer to homeschool and enforce a strict, modest dress code, Jill and Derick made the decision to enroll their older son, Israel in public school. Jill has also embraced her own sense of fashion and is often seen wearing pants and sporting a nose ring. Jill also admitted to using birth control in a recent Q&A session on the family blog, adding that the couple does want to have more children in the future. The use of birth control is highly frowned upon in the Duggar household.

Derick has been vocal in the past about the tense relationship between his family and the Duggars. He even commented on Instagram that they were not allowed to visit Jim Bob and Michelle's home if Jim Bob wasn't there.

Despite the obvious rift between the families, Jill told the magazine that she's hopeful that they will be close once again. "I never expected this to happen or for it to get to this point," she said. "But I'm realizing I can't put a timeline on healing. I love my family and they love me. I really just have to follow God's lead and take it one day at a time."

Read Next: Amy Duggar Says Jill Smiles "A Bit Brighter" These Days

Source: PEOPLE, Dillard Family