Add the blueberries, sugar, lemon rind, and lemon juice to a small saucepan and heat to a boil. Cook until the blueberries burst and have broken down into a sauce - about 5-7 minutes. Remove from heat.
Set a fine mesh strainer over a small bowl. Pour the blueberry sauce through the strainer. Use the back of a wooden spoon to press on the blueberry solids, pushing them through the strainer. Do this until the only thing left in the strainer are the blueberry skins and the lemon rinds. Discard the solids. Cover and refrigerate the blueberry puree.
Place the egg yolks in a medium bowl and whisk them together.
Bring the cream to a simmer in a heavy medium-sized saucepan. When cream is hot, slowly whisk it into the egg yolks -- adding the cream a little at a time so that the eggs don't heat too quickly and scramble.
Pour the egg mixture back into the pot and stir constantly over medium low heat until custard thickens slightly. Remove from heat.
Place a fine mesh strainer over a medium bowl and pour the custard mixture through the strainer. Allow the custard to cool slightly, then stir in the sweetened condensed milk and key lime juice. Cover and refrigerate until cold or overnight.
Scoop out one quarter of the ice cream and add 2-3 tablespoons of blueberry puree. Use a spatula to fold the puree into the ice cream. One or two turns with the spatula is all you want -- so you get pretty swirls. If you work it too much -- the whole thing will turn purple. Transfer into a new loaf pan. Repeat this process with ice cream and puree until you've used all the ice cream. Note: you will have leftover puree -- which can be spooned over the top of ice cream before serving, or use it for other desserts or cocktails. (I had about 1 cup leftover).