No spare clothes for me, either, except for socks. My base layer is worn primarily in my sleeping bag and under my outer clothes in the morning. Even if it's cold (28*F) and windy, I get too hot hiking in my base layer, so it comes off and into the pack when I'm ready to hit the trail. My main concern is to keep the outside of the base layer clean so it protects the sleeping bag.
I try to follow the principle of taking no more clothing than I'd wear all at one time to keep me warm and dry in the worst possible conditions I might encounter. That, of course, varies considerably as to season of the year and where I'm backpacking. Above timberline in Wyoming's Wind Rivers calls for quite a bit more insulation than in the forested Cascades! Of course by late September or early October, the temps in the high Cascades more closely resemble the Wind Rivers!
The exception, of course, is that much prized extra pair of (hopefully) dry socks! Any other changes of clothes are in my car at the trailhead.
I do use either ziplocks or stuff sacks made of bug netting, nice and light! I try to keep them to a minimum, though–one for my first aid/"essentials" and one for my toiletries. Stuff like sunscreen, bug dope, hand sanitizer that I use during the day go into the hip belt pocket of my backpack.