A few anecdotes on this subject:
Much like you, I left mine in a gear tote in the garage for a few nights about 2 months after its last use and 'shake-out', and it *may* have frozen on one of those nights. I contacted Sawyer and they told me that it needed to be replaced. I'm sure capitalism had some factor in this company statement, but I also got a decent explanation why, as outlined by above posters. Essentially the element pores get stretched out to something larger than the 0.1 micron it was created with.
FWIW, I have not replaced my 3-way, and am not sure I plan to for a 'maybe'
As for the water-in-the-filter issue, somewhere here on BPL a member posted about how they took a filter of another brand that became nonfunctional, and after a year of sitting on a shelf, cut it open for kicks to see inside. The ceramic element was, by his account, still wet. This just illustrates that a filter never *truly* dries out.
My personal opinion, if you aren't sure it got below freezing, and it was in the trunk of your car with other objects which potentially kept the trunk slightly warmer than the ambient air low temp outside the trunk, you're probably OK.
In the end, go with your gut, you're the one drinking the water.