Preheat the oven to 400 degrees F. Individually wrap the bananas in foil, without peeling them, and roast for about 30 minutes, until soft.
Once the bananas are cool enough to handle, peel and mash in a bowl, adding the brown sugar and stirring until it dissolves. Transfer the mixture to a blender or food processor and add the remaining ingredients, plus a pinch of salt. Blend until very smooth.
Divide the puree among ice pop molds (or paper cups with wooden sticks) and freeze until solid. Lick.