I've found all kinds of fishing gear at garage sales and thrift stores. Fly fishing gear can be outrageous, but casting gear is cheap, even when new. A two piece casting rod and a few lures will get you started. The last one I got was $5 with a reel. Of course, fly fishing has a romantic appeal to it. I do like the idea of tricking the fish with bits of fur and feathers.
I've always said that fishing tackle is designed to catch fishermen rather than fish. Your local tackle shop and/or fishing newspaper will have all kinds of advice on what to use for the local species.
I have a Popeil Pocket Fisherman that is great for hiking. You can actually catch fish with one, but I recommend a pole and reel. Study your knots– monofilament line is different.
http://www.popeilfamilystore.com/ppf.html
A Zebo closed-face reel and matching pole will work for lures and bait casting. If you can use eggs or worms with a bobber, it is a time-tested way to catch fish, but it bores the heck out of me– like watching golf :) Casting lures is more fun and very dramatic when you "hook one."
If you know what you are doing, you can catch fish with some very simple gear. It is more a process of understanding what and how fish eat, where they hide, etc. And then sometimes, they just aren't hungry or don't like your offering. I wouldn't count on having a certain catch for your meal plans!
The regulations vary from state to state as far as using bait, flies, or lures, and high mountain lakes and streams may have different regulations and seasons than lowland lakes– it is very much that way in Washington. Some lakes are "quality" fishing, where you may only keep a smaller limit of fish of a particular size– 3 fish over 12" or something like that. You can get a copy of the regs when you get your license.