Bring 2 cups water, sugar, and lavender to a boil over medium heat. Stir until sugar has dissolved. Turn off heat, cover, and let steep 1 hour.
Combine remaining 3 cups water, lemon juice, and lime juice in a pitcher. Place a fine-mesh strainer over the top. Carefully strain the lavender simple syrup into the pitcher, discarding lavender. Stir to combine.
Serve in glasses over ice with lemon slices and a sprig of lavender for garnish. Also great with vodka!