The vinegar that's usually used to preserve them is what makes pickles sour.