Would it work to sub in brown sugar for the topping vs white? I feel like this is the way crisps/crumbles typically are?

My guess is that it would be fine.

You could always do a little experiment and try both.

Combine all of the topping ingredients except the sugar, then split into two portions. Add white sugar to one batch and brown sugar to the other. Sprinkle one batch over one half of the dish, the other batch over the other half.

Bake and compare.