Agreed, fresh is always better, particularly if you have a good farmers market and can get locally grown stuff. However -- beware of buying more than you can use in short order. Cooking with fresh produce is best when you buy today and cook no later than day after tomorrow. Some things, like squash, will hold longer; corn should be cooked the day it's bought (which hopefully is the day it's picked) for the best flavor. I advise starting slowly into the fresh produce realm; learn what you like how to cook it, and how to keep it. What should be refrigerated, what should stay out on the counter? Simplest preps with fresh produce are generally the best; let the flavor of the produce come through for you.

Depends on what you want to use it for. Fresh always tastes better, but some dried fruit, like dried apricots, figs, raisins, and cherries, are great in things like granola and oatmeal and are also really good in savory dishes, like stews, chutneys with grilled meats, etc. Dried mushrooms can be REALLY amazing if you reconstitute them in a flavorful liquid.