Peel the lemons, taking care to avoid as much of the bitter white pith as possible. In a large bowl, combine the sugar with the lemon peels, then lightly muddle and let sit for 20 minutes to allow the citrus oils to infuse with the sugar. Boil 3 cups of water and steep the tea bags in it for 3 minutes. Set aside to cool slightly.
To the bowl, add the rum, sherry, lemon and lime juices, bitters, and Velvet Falernum. Stir to combine. Add the tea and let cool to room temperature. If serving immediately, add several large cubes to chill. Add lime and lemon wheels and grate some nutmeg over the punch. Ladle into punch cups over ice, garnish with mint sprigs, and serve. If making ahead, bottle and refrigerate. When serving, pour over ice and add grated nutmeg, a lemon or lime wheel, and mint sprigs to garnish.