Cut loaf into 1 inch cubes and let sit overnight so the bread can dry out. Or you can cut it up just before and stick them in the oven on 250º F for about 20 minutes until they dry out a little but. Works just as well!
Whisk together milk, eggs, chicken broth, salt, pepper, olive oil in a large mixing bowl. Then add the nutmeg, and the lime zest and juice. Throw in the bread cubes, toss, and let it sit for a couple of minutes.
In the meantime, break the hard ends off of the asparagus, cut them into one inch pieces and throw them in. Then wash, stem and chop the mushrooms (however you wish) and throw them in too. Be careful not to toss too much since the bread tends to disintegrate.
Pour into your baking dish (I used a springform cheesecake pan that I lined with parchment paper so it wouldn't leak) and then sprinkle the gruyère on top, liberally coating the pudding.