Bring milk, coffee, chili powder, cinnamon, vanilla extract, and chocolate to a simmer (about 180 degrees). Meanwhile, combine egg yolks and sugar in the bowl with a wooden spoon, blend until smooth.
Temper the egg yolks: add the milk to the egg-yolk mixture in a slow but trickle-constant stream, whisking constantly until blended. Cook over low heat, stirring constantly (Do not boil). Pour in cream.
Preheat broiler. Spread marshmallows evenly on the bottom of a medium-sized ovenproof pan. Broil until tops of marshmallows are nicely browned. Pour the warm gelato mixture over the browned marshmallows. Mix together with a whisk. I did not completely mix the marshmallows so that the final gelato will have a marshmallow swirl.
Chill mixture for 1 hour before putting into ice cream machine. Freeze for 1 hour before serving.
Lay graham crackers on an oven-safe tray, adding a couple of marshmallows to each one. Broil in oven until golden brown. Top with gelato for a delicious sundae.