In a large pot, cover the potatoes with water and bring to a boil over high heat. Reduce the heat to medium and simmer until tender, about 15 minutes. Drain and let cool.
In a large bowl, whisk together the vinegar, mustard, and oil. Add the radishes, celery, and fennel. Slice the potatoes into rounds (removing the peel if you prefer) and add to the bowl. Stir together and season the salad with salt and pepper. Fold in the dill. Serve immediately or refrigerate for up to 1 day.