My chicken broth (for chicken noodle soup) is too spicy. What can I do?

You could dilute your broth with some store bought vegetable or chicken stock.

Just a couple additional thoughts.
Yes, the addition of potato (or other starches) helps absorb or spread around some of the spiciness (as well as the saltiness Stephanie mentioned).
Another way to dilute the pepper is (if you have the ingredients, time & storage space) is to make another same-sized batch of the broth but totally WITHOUT pepper, then mix the two. If you do this, you may be able to get your typically more condensed texture you like, by cooking the larger double batch longer. But I would still taste as I cook, to make sure it doesn't get too concentrated again.
Or simply dilute with water each time you heat and serve.
Another approach is to NOT add pepper to anything else you would normally add to make the final soup...pasta, vegetables, more chicken.
Last, not for this batch but for futures, taste as you go to prevent or moderate a repeat of too-much-anything or too-little-anything as you cook...ingredients, heat, water, the cook all vary each time we go at a recipe.
Hope your batch is savable... please let us know what you do.

If you don’t have to use it right away, I’d let it sit and see if it mellows.

I'm not sure if this will work but when a broth or soup gets too salty you can add a couple pieces of raw potato.