Preheat oven to 325°F. Grease and flour two 9-inch cake rounds, tapping out any excess flour. In a medium bowl, whisk together flour, baking powder, and salt.
Using a stand mixer with a paddle attachment, cream butter and sugar on medium speed until pale and fluffy, 3-4 minutes. Add eggs one at a time, beating until each is incorporated. Scrape down the sides of the bowl as needed. Beat in lemon zest and vanilla. Add flour mixture in three batches, alternating with two additions of buttermilk and lemon juice, and beating until just combined after each.
Divide batter evenly among the prepared pans, filling each about 3/4 full. Bake, rotating half way through, until golden brown and a tooth pick inserted into the middle comes out clean, about 30-40 minutes. Transfer the cake pans to wire racks to cool completely before removing the cakes from the pans.
In a small saucepan over medium heat, heat the cream and lavender until it just comes to a boil. Remove from hear and cover. Let the cream infuse for 20 minutes. Once infused, strain the lavender out of the cream, and set the cream aside.
In a stand mixer with a paddle attachment, beat the butter until it is creamy, and add the powdered sugar 1/2 Cup at a time. Slowly add the infused cream until you get a consistency you like. If the mixture gets too runny, add more powdered sugar.
Remove the completely cooled cakes from their pans. Use a serrated knife and carefully slice the mound off of one of the cakes, so that the top is flat. (This cake will be the bottom layer.)
Frost the bottom cake, place the second cake on top, and frost all the way around. Slice and enjoy!