Augment a sleeping bag’s insulation?

Have any of you taken a lightweight bag and augmented the top side insulation (I am talking synthetic here) only?

Or is it not worth the money and trouble?

I am considering taking a lightweight 45 degree bag and adding some primaloft one to the top side of the bag. I was going to open up the seam, add in the insulation and lightly tack it in place in a few spots (to keep it from shifting) and then close it back up.

OR

I can take the lazy man's way and just wear another layer of clothes in the bag for the slightly more weight.

Any comments or suggestions?

Edit: The bag is not quilted and I could take out one long seam (on the side opposite the zipper). I could then go in and put a tack stitch every 6 inches so that would work.