Pickle the eggs - Fill a quart jar with the vinegar the hot water, vinegar, garlic cloves, sugar, peppercorns, and salt. Shake until sugar is dissolved. Layer beets and eggs in the jar, and let jar cool on the counter. Refrigerate overnight.
Carefully slice each egg vertically, then carefully remove the yolks. Use a fork to mash the yolks completely, then add mayonnaise and dijon. Stir until creamy then fold in capers. Season with salt and pepper to taste, then scoop filling into the eggs. Top each egg with a caper. Serve at once or chill until serving.