What is the best way to incorporate a tea flavor into a cupcake recipe?

I have also found that with matcha (Japanese green tea powder), you lose a lot of favor after baking, so maybe frosting is the way to go, as long as the frosting is not disgusting sweet...

I agree with anita; you will have a more intense flavor by infusing into the frosting.

Do you want the flavor in the cake or the frosting? This is what I recommend for the most flavor: Melt the butter/shortening you are using for the frosting over low heat. Once it is melted and hot, add the tea leaves, turn off the heat and steep for about 10 minutes. Strain out the leaves and place the now infused butter in the fridge to re-solidify. Proceed making frosting as usual.

I learned this method from pastry chef Pichet Ong and used it to make this amazing thai tea cupcake at a restaurant I was consulting for! I have also done this with Earl Grey and it comes out amazing.

If you're trying for green tea use matcha powder mixed in with the flour. I made a lot of green tea cupcakes when I worked at starbucks and had access to the matcha powder they used for green tea lattes. I used about a tablespoon of powder per batch (approx 12 cupcakes).
I agree with boulangere about steeping black teas in whatever liquid your base recipe calls for, but you can steep it cold overnight in the fridge if you think ahead.
If you're using herbal teas be aware that anything with hibiscus will cause milk to curdle, but it'll taste fine.

If you do a search in British newspapers such the London Times, you'll find tons (well, several at least) of excellent recipes. They ran a contest for cake recipes at the time of the royal wedding last April. Lots were tea flavored!

Does the recipe call for milk? If so, heat it in a stainless steel pan, add the tea (in bags or a tea ball), let it steep, covered, for 15 minutes or so, remove the tea, and go on with your recipe. Lovely idea!