Go for it!
Vests work well in an UL layering system that uses base/mid layers with wind and rain shells. A puffy layer is great for cold rest stops and camp, but with a long sleeve mid-layer and wind shell, you can get by without arm insulation above 20f of so and it will keep the cold fabric off your shoulders standing around in cold rain. I wouldn’t be adverse to throwing on a vest over my long sleeve base layer for a cool morning breakfast and breaking camp. I would sleep in it too.
I have a Patagonia Micro Puff vest that delivers great warmth for the weight and volume and would make a good model for your vest. The XL size shown is 25″ across the armpits and 30″ back length with a general barrel shape, snap neck, elastic hem, and the big bellows inside pocket that is self-stowing. Note the soft side of the Velcro tab on the pocket is on inside and out for use as a pocket or stuff sack. Quilting is on the inside only, leaving a smooth outer shell (sewn through would work as well I think and a zippered neck would be okay too). It weighs just 7.2 ounces, which I think is amazing for a 100g level fill vest. For comparison a 60g fill Nano Puff vest with a full zipper and hand pockets is 8.5oz in a medium.






