Make the cupcakes: Preheat oven to 375F degrees. Line 14 muffin cups with liners.
Over high heat, bowl the water in a small saucepan. Pour the unsweetened cocoa powder into a medium sized bowl and mix with boiling water until smooth. Allow to cool to room temperature by placing in the refrigerator as you prepare the other ingredients.
Whisk together the flour, baking powder, and salt in a medium sized bowl. Set aside.
In a large bowl, using an electric hand or stand mixer, cream the softened butter and sugar until light and fluffy. Add the eggs, one at a time, beating until smooth. Scrape down the sides of the bowl as needed. Beat in the vanilla extract. Slowly add the flour mixture and beat only until *just* combined. In a steady stream, add the cooled cocoa mixture and stir until smooth.
Fill each muffin tin 3/4 of the way full with batter. drop teaspoon sized dollops of Nutella on top of the cupcakes. Do not push down. As the batter rises, it will close up on it’s on.
Bake for 16-19 minutes, keeping an eye on them to avoid even the slightest burning (which will lend dry cupcakes). Remove from oven and place on a wire rack to cool before frosting.
Mix the butter and nutella on medium-high until smooth.
Gradually add the sifted icing sugar until just incorporated. Scrape the sides of the bowl and mix in the vanilla extract. Optional: if you feel like the frosting is a bit too hard, add the milk or cream.