This really comes down to your relationship with your wife, IMO.
My first wife, and the mother of my kids, only went backpacking once and hated it. We didn’t have kids for the first 8 years of our marriage, so she got used to me going on backpacking trips. We did go car camping a lot together.
When the two kids came, she was a stay at home mom, which is a full-time job. I would think that being at home all day with small kids would create more animosity towards a working spouse who wanted to go backpacking alone on the weekend. But she understood and never objected. However, we did a lot of camping as a family, and bought a tent trailer when the kids were very young. Plus we live where it was easy to go camping as a family.
I went backing often enough that when my kids went to church on Sundays and their friends would ask where their dad was, they would tell them dad wasn’t Catholic, but a backpacker. They would also tell their friends Mt San Jacinto was daddy’s mountain and Joshua Tree was daddy’s desert, since I spent a lot of time in those places.
The biggest challenge was when the kids got into school and joined activities, which meant that almost every weekend was taken up with sports, cheerleading, etc.
My second wife had never gone camping in her life. We started doing that together almost every weekend, as many here know. And I still did quite a bit of solo backpacking trips. My first wife told her that I was careful being alone, and it wasn’t something to worry about. Kinda of neat an ex-wife would have this kind of an attitude. Sometimes my present wife will suggest I go do a trip if it has been a while. So these are part of the marriage relationship.
Craig Wisner does many, many little trips alone. I don’t want to speak for Craig, and my observations might not be accurate, or even none of my business. Many of his solo trips are 24 hour trips or weekends. He has two kids. Craig and I have talked about this a lot and we first did a trip together when his kids were fairly young. I know his wife. His wife and I have talked about this a couple times while I was at their house waiting for Craig to come home. She is very supportive, knows how important it is for him, and like my two marriages, encourages him to go out, because it makes their marriage better when he gets back home.