Preheat the oven to 450º F. Line a baking sheet with parchment paper.
In a large bowl, whisk together the flour, baking powder, baking soda, salt, paprika, and cayenne (if using).
Cut the butter into the dry ingredients using a fork or pastry cutter until it's in mostly pea-sized chunks—some chunks can be slightly larger and some smaller, but don't overwork it.
Crumble the goat cheese into the bowl and stir to combine.
Add the milk, stir the dough with a fork until it is somewhat evenly moistened, then knead it a few times in the bowl so it mostly comes together in a ball but don't overwork it at all. It should not be cohesive and there should be chunks of drier areas and some wetter areas.
Turn the dough out onto the parchment-lined sheet, and fold it over onto itself until there aren't any dry spots remaining. Don't think of this as kneading: You want to handle it gently and as you fold, the wet/dry areas will disappear. Fold about 10 times, then gently press the dough down to a rectangle about 2" high.
Using a sharp knife, cut the dough into 2" squares and separate them slightly on the baking sheet.
Bake for about 12-15 minutes, or until golden brown. Let cool slightly, then eat!