Can I swirl marshmallow into the cake batter before baking?
Hmmm.... I've never tried it that way, but don't see why you couldn't.
When I've made s'mores cake I've done it as layers with chocolate butter cake that I add some cinnamon to, and marshmallow frosting with crushed graham crackers between the layers. Frosted either with ganache or more marshmallow frosting.
Let us know how it works out for you if you give it a go!