If you don't have powdered matcha, heat your cream and/or milk, seep some green tea leaves, and then strain out the tea. You don't want to add water because it will affect your ice cream's texture, making it a hard chunk (if you use more than a 1/4 cup water-based liquid).

I'd use the powder. Add the liquid may negatively affect the outcome.