Jenn,
As you know, blisters are caused by friction. Given that you don't use your poles a lot, that is probably the problem.
There are two strategies you can employ:
1. Toughen your hands
2. Reduce the friction
In the old days it was common practice for hikers to rub their feet with rubbing alcohol a couple times a day before a hike to toughen their feet. I can't validate that. Might work with your hands.
To reduce friction, in the old days with leather boots, many of use wore a silk liner sock under a wool sock. That worked well for many people. You might want to try a light but snug silk glove if you can find one. More breathable and lighter than a bike glove.
Another reduce friction strategy would be to grip your poles a little tighter, so your hand are moving as much to create friction. Too tight would not be good either. Rowers do hand exercises to strengthen their hands so they can grip the row tighter.
Don't know if this is helpful — if not, at least I tried :)