Yeah, the research is the hardest part. But I guess he's asking "What is the hardest part about packing?"
And I'd have to answer "Picking an appropriate amount of insulation."
Here out west conditions can vary. A lot. So if I'm going someplace new I spend a lot of time researching conditions, highs and lows, looking at forecasts, etc, so that I can decide if i need to bring a clothing insulation layer, if so how much, and which sleeping bag and pad is most appropriate. When Sarah asks "what is your biggest concern when preparing for a trip" this is my answer- I hate sleeping cold. That's not camping, it's enduring. Coming in a close second is over-packing, though, so this stuff is clearly a compromise.
After that the second hardest part is likely "Picking an appropriate rain strategy." I often only bring a water-resistant windshirt in the desert, for instance, but might need a full top/pants suit in other conditions.
Third is probably planning food, which i am bad at. Unlike others here i don't get out enough to realistically just keep stocks of camping food laying about, so i always end up at the grocery store picking out Idahoan instant mashed potatoes, instant rice, etc. I do have favorites, though, like Fritos Corn Chips and a nifty fruit bar brand that my wife found.
Actually getting all of the stuff into the pack is trivial- I just have to figure out which pack is the right size. I'm a believer in the "just cram it in" strategy. I don't do neat packing- too much effort. The closest I come to that is packing stuff I'm likely to need more often at the top, and heavier stuff closer to my back. Wet stuff (shelter, bivy rainsuit) goes in the front mesh pocket. Easy.