(1) Background: Gender differences between men’s and women’s parenting roles are well-documented as the “second shift”. We examined the main effects and interaction of work distress and parenting distress with energy (i.e., vigor) in a sample of 310 dual-income, different-sex couples with kids married for approximately nine years. (2) Methods: We used actor–partner interdependence modeling (APIM) to examine how spouses’ distress was associated with their energy. (3) Results: For both wives and husbands, there were negative associations between the actor’s parenting distress and their energy level and between the actor’s work distress and their energy level. However, only wives experienced a significant interaction of work and parenting distress such that high levels of both forms of distress were associated with low levels of energy, indicating that only wives experience this form of work–family conflict. (4) Conclusions: When women experience more strain at home than men, they may need more time to recover from their work and family duties. If they cannot do so, they will have less energy to carry out their responsibilities and may be at a higher risk of future adverse health outcomes.