Basically, it is negligable. The interesting bit is why.
The insulation power of down lies in the surface film or boundary layer of air around every tiny tip or spicule of the down. That boundary layer does not move: it is 'attached' to the down itself. The bulk of the air in the bag is actually far more free to move around, and if there is enough space it will move, taking heat from the inside of a bag to the outside. That has several consequences.
First, a prime determinant for insulation is the quality of the down. The finer the down and the fewer feathers in it, the greater the number of tips present and the better the insulation. The interlocking of the down plumules also matters.
Second, another obvious determinant is the amount of down: the more down there is in the shell, the more insulation there is.
Third, the actual thickness of the loft is not such a good indicator of the insulation power of the down. We use it more as a guide to the amount of down inside the shell, as it is the amount of down which controls the insulation.
Fourth, if you have too much loft, then the bulk air between the bits of down starts to matter: that bulk air can transport heat (air) past the down plumules, making their insulation ineffective.
Compressing the down a limited amount does not change the boundary layer, and does not reduce the insulation power. This is why many people, more concerned with warmth than fanatically shaving the last few grams off, go for a slight 'overfill'. Yes, overfill can work, both by increasing the amount of boundary layer and by reducing the chances of bulk air flow.
Despite all this, there are a few other factors which influence the warmth of a down item – or the user.
First, if the shell fabric lets lots of air blow through, you are going to be cold. Cold air infiltrating deep into the down layer (or going right through it) may not alter the boundary layer, but it will alter the heat flow through the whole thing. This means a fine weave and calendaring of the fabric can be vital.
Second, if you crush the down layer by sitting on it or by stretching it tightly over a hip or elbow, then you can disrupt the boundary layer to the point where it ceases to work. Conduction starts to play a part. This is why a quilt is just as effective as a sleeping bag. Do remember: you sleep under a quilt at home, not in a bag.
Finally, something so many novices fail totally to understand. You might have a super down layer over the top of you, but if what you are sleeping on is cold, you will be cold. Your mat is just as important as your quilt. Those thin 1/8" layers of foam barely smooth out the bumps under you; they certainly will not keep you warm! (1/8" of foam over 3" of soft dry pine needle duff – another matter.)
So why do we go for UL shell fabric? To reduce the weight of the shell. No other reason.
Cheers