Can this be made with almond or coconut milk?

Coconut milk will produce a richer pudding, due to it higher fat content than almond milk. Coconut milk will also, however, give the vanilla pudding a bit of coconut flavor, which you might like, or then again, might not. Coconut milk is less noticeable - but very pleasant, indeed - in chocolate pudding. ;o)

I haven’t made this one, but I have made chocolate pudding with almond milk, and it works well.