You’re not wrong, although I feel like the implication in this context was slightly more antagonistic.

i.e. “It’s only a matter of time before she leaves your sorry ass”