This is not meant to be a definitive guide, nor a peer reviewed research paper, so i will not be providing many links, footnotes, etc. Consider it more an opinion piece and one based on reading and synthesizing various studies, investigation of different fabrics/textiles, and personal anecdotal experience.
For more truly scientific and researched based info, i recommend Richard Nisley's prior postings relating to Clo values in relation to garment insulation levels. But besides the intricacies of down, loft, density, etc, i think a more generalized, broad, holistic, and simplified approach might be helpful–especially for beginners and especially more so for woven garments. Everyone knows high quality down is so far the best insulator for the weight.
When it comes to insulation, particularly for that of woven fibers, there seems to be a hierarchy of more important to less important properties of the textiles and fibers involved. The most important factor seems to be fiber arrangement. This was found even in the early days of research with the researcher putting a thermometer wrapped in various textiles, warmed up to a certain temp, and measuring how long it took to get down to room temperature. To the surprise of such researchers, it mattered less the specific fibers and material involved, but more how they were arranged. In other words, simple, more flat woven materials were less insulating, whereas more bulky, thicker weaves and fiber arrangement were more insulating and this didn't matter if it was cotton, wool, etc.
Later studies indicated this was true, but also that fiber size and shape/structure also played an important role. Generally speaking, the smaller and finer the fibers, the more insulating the garment because it's trapping more air in between the weaves. Well, we knew that to some extent already because of the nature of down. One of down's interesting properties is that of many super, super fine fibrils that are like uber microfibers that trap a lot of air. Then it has the unusual property of having good recovery, resiliency, and elasticity when used as a loose insulation fill.
Along the lines of fiber structure/shape, i remember reading about a study that showed that triangular shaped fibers (as in silk naturally or manipulated synthetics) were more insulating than that of perfectly round fibers as is the often the shape of cheaper synthetic fibers. This same study, also found that large hollow fibers, actually insulated less than smaller solid ones in connection with woven garments. This was somewhat unexpected based on knowledge of loose, lofted insulation and the many hollow fiber based synthetics that are considered quite warm for their weight in relative comparison to many other materials (except down). However, logic dictates that two equally sized and shaped fibers made out of the same material, but one with a hollow core and one solid, the hollow core one will definitely be warmer because it's using more air as insulation rather than fiber material. This common sense logic is borne out in nature in various examples. Take the semi-hollow core Alpaca fiber and the solid Merino sheep fiber–Alpaca is known to be somewhat warmer per weight than Merino–provided there is not much of a difference in fiber size or arrangement.
Lastly, the material itself and it's thermal conductive properties are important to consider. Different materials have different thermal conductivity levels. Generally speaking, polypropylene, protein based fibers, and polyester are the top ones for being least thermally conductive materials. I've seen different comparisons between protein based fibers like Sheeps wool vs that of polyester, some have said polyester is less thermally conductive and some say Sheeps wool. I don't know for sure.
Knowing all the different factors that contribute to insulation capability, one can holistically synthesize the different factors and variables to come up with an idealized postulate in relation to a woven garment with greatest insulation for weight. Wovens innately will be heavier than loose insulations fills, but they do excel in their breathability and drying time over lofted loose insulation fills. These properties make them better suited to active conditions, wheras often lofted, loose insulations are better suited for slower pace or resting conditions wherein less moisture in the form of body vapor and perspiration is produced.
I suggest from a synthesis of above, that a Polypropylene based, microfiber pile fabric (see Polartecs thermal pro material, or natures abundant examples of animal fur) wherein the individual fibers are both hollow core and triangular shaped, would produce some of the warmest and most insulating garments for the weight. Polypropylene is the least thermally conductive materially commonly used in garment creation and is the lightest at the same time. It also has the added benefit of being the most hydrophobic, so per same thickness and density weave of fabric it will dry faster than all others. However, it's best suited as an mid layer rather than an outer or a baselayer. As an outer, it's more susceptible to UV damage and it's strength and wear properties are not as high as nylon or even polyester (but in woven forms, still much more durable than most natural fibers).
As a baselayer, it builds up odor quickly and extremely–the worst of any common garment material. I suggest a thin baselayer of merino or alpaca combined with some treated synthetic for the baselayer to act as a filter level for the polypro so it won't become so smelly so fast.
As a midlayer, with the proper combination such as using a lightweight windshirt and good baselayer, this hypothetical polypro midlayer would most likely be the most insulating woven fabric presently, easily, and cost effectively capable. Now, some companies make hollow cored polypro yarn and some companies make fleece versions. To my knowledge, no company so far is making pile (fur like) versions, nor triangular shaped fibers, let alone any combo of the latter.
Not directly related and more as an aside, but if Richard Nisley is reading, I'm curious if you have ever measured the Clo values of high quality (fine micron) Angora Rabbit fur used as a loose insulation fill before? I am extremely curious to know how this stacks in comparison to good quality Down. There are some practical issues to contend with as far as felting or matting, but it may be a possible substitute as in a "poor man's down", at least for garments like jackets or pants rather than quilts or sleeping bags.
What leads me to this conclusion are some studies that have been done on Kapok fiber. One Scandinavian study found that Kapok fiber was almost as insulating as average quality Duck down per slightly higher weight (about 15% if i remember correctly). However, Kapok has some issues, it's more brittle and less elastic than either down or angora rabbit fur so it cannot be compressed near as much. The average micron on Kapok fiber is actually relatively large compared to Angora Rabbit fur fiber and especially to Down's fibrils/fibers. I remember it's around 30 micron or so. Angora Rabbit fur on the other hand, can get down to less than 10 microns (10 to 16 seem to be the general average though)! Both are similar in the sense of being encapsulated hollow fibers. However, the protein structure of Angora Rabbit fur is going to be less thermally conductive than the primarily cellulose structure of Kapok fiber (though the waxy coating on the latter, may help in that regard). So, in other words, if Kapok fiber is almost as warm as say 500 fill Duck down per similar weight (slightly higher), imagine what Angora rabbit fur will be with it's much finer/smaller fiber size and protein structure. It's hard to say in any concrete way, but it may rival poor to decent Goose down for insulation properties per similar weight.
If you have the time and interest, i would be willing to send you a pound of Angora Rabbit fur fiber (no guarantee of high quality, but it's a start) to be tested in some way. If you would like it pre sewn into a mini quilt or the like, I can also do that. I would probably treat it first with a high quality water based DWR soak though as i suspect this may reduce matting or felting some.
