Combine the sherry vinegar, brown sugar and peppercorns in a bowl and whisk to combine. (I like to first place the peppercorns on a cutting board and crush them with the back of a frying pan. They retain some of their integrity that way, rather than being ground to oblivion in a pepper mill.)
Slowly whisk in the olive in. Once it’s emulsified, whisk in the creme fraiche. Season with salt.