The water under my bamboo steamer turned yellowish brown. Is this normal?

immediately after taking cold water water and putting it under the steamer? no.

after turning on the heat and having stream pass into the bamboo? yes, especially if there's food in the basket. It's like boiling food, the stream condenses on the food and bamboo, some of the colour from the bamboo and from the food gets dissolved in the water, and falls back down into the main water, changing the colour.

if this happens with just the bamboo basket (no food in it) and smells very bad, it might be a sign that your bamboo has rotted, and should be replaced. The colour isn't a sign of anything.