I would want the wicking stuff closest to my skin and the down outside, where it isn't compressed and the down layer has two layers of fabric for wind-blocking.
My strategy is:
Polyester LS wicking base layer
Power Stretch or R1 style midlayer hoodie (variable layer)
Windshirt or rain shell
-or-
Polyester LS wicking base layer
Power Stretch or R1 style midlayer hoodie (variable layer)
Synthetic puffy vest
Windshirt or rain shell
-or-
Polyester LS wicking base layer
Power Stretch or R1 style midlayer hoodie (variable layer)
Synthetic puffy hoodie
Rain shell if there is precip, colder yet, or cold wind.
By saying "variable layer," the fleecy midlayer can be worn or stowed as conditions merit. The wicking base layer could be varied in weight as conditions merit. In all cases, the puffy insulation layer is one of the outer layers to avoid compression and assuming it will be varied with the level of exertion. You always want good moisture transfer and avoiding saturation of the puffy layer with sweat, which is what would happen wearing it while hiking uphill with a winter load, snowshoeing, or x-country skiing. I would be hauling it out for an extended rest stop or camp as my body stabilized.
For much the same reasons I like midlayer and outer shells with full front zippers. You can ventilate and cool as needed while keeping your neck, shoulders, and upper arms from turning into cramped ice blocks. My pack keeps my back warm regardless of what I'm wearing and would make a sweaty compressed mess of a puffy layer, whether synthetic or down. It's not nice when you take your pack off and a cold wind hits your wet back. {{{{{{{{{{chatter}}}}}}}}}}