Were Joey Lawrence and Samantha Cope doomed from the start? That’s what the latest report seems to suggest.

As we’ve been following, Samantha filed for divorce from the Melissa & Joey alum earlier this month and is seeking sole custody of their 19-month-old daughter Dylan. Days later, reports surfaced alleging Joey has been involved in an ongoing affair with his Socked in for Christmas co-star Melina Alves, who is also apparently going through a divorce because of the infidelity. A source later revealed Samantha attempted therapy with Joey, but that he pretty much derailed any chance of patching up their marriage. It’s super sad stuff. You can read more about it HERE.

But now, an insider is claiming Joey’s alleged affair was NOT what ruined his and Samatha’s relationship!

On Sunday, a source told People the pair “had troubles from the beginning of their relationship.” Hmmm. Well it did seem rather rushed… We mean, it was just three months after Joey finalized his divorce from ex-wife Chandie Yawn-Nelson!

The source claimed:

“Infidelity isn’t the cause … not by a long shot.”

Inneresting… The source didn’t necessarily deny the infidelity — just the part about it being the reason he and Samantha are splitting up.

The insider added:

“[Joey] had gotten to a point where he had hit his limit with defending his kids, his mom and his family to Samantha.”

There weren’t many other details regarding that claim, specifically, but the Blossom star shares daughters Charleston, 18, and Liberty, 14, with ex-wife Chandie.

As for Samantha seeking sole custody of their daughter Dylan, the source claimed Joey is “torn up.” They explained:

“Joey loves his kids. He is a great dad and his daughters adore him. There’s nothing he wouldn’t do for those girls, and that includes Dylan too. He prides himself on being very involved with family and putting family first always. He loves Dylan. He’s torn up that it’s going to be difficult to co-parent Dylan with the demands Samantha has made. Joey is a man of integrity. He is a hard worker. He is a professional.”

A man of integrity… Who’s currently caught up in cheating rumors…

A the end of the day, though, the insider noted the 48-year-old is “relieved the marriage is over because he can breathe … He can be himself. He’s taking the time to make up for all of the lost time that he had with his children.”

So it sounds like his and Samantha’s marriage drove a wedge between his relationship with his oldest daughters. Well, the proximity of his relationship with her and their mom Chandie, alone, would likely be enough to ruffle feathers. But we guess we’ll just have to wait for more specific deets.