Preferably not imported from the UK.
Whole Foods has it, although it seems to sell quickly/they don't stock a lot so you may not always see it. You could also try Murray's cheese. Usually good cheese mongers have access to it.
Whole fFoods, Fairway organic dept, both have it.
In the Bay Area (Northern CA) we can find Meyenberg goat milk butter in most good grocery stores. It's both local and totally delicious. It comes in 8 oz. silver foil/purple packages. I think it beats cow's milk butter any day.
I do not know how much trouble you want to go to get this, nor do I know if they make butter. But below is web link for a goat dairy I am acquainted with in Hudson Valley. They list farmer markets they attend which include a couple in NYC.
So they are at the Union Street (14th and Broadway, easily accessible by subway)