From home to the grocery store, I had whole fish at around room temperature for an hour. It's in the fridge, ready to cook tomorrow. It smells like fish (and the ocean!) but not fishy.
You should be fine, I think the danger starts as about 3 hours at room temp. Hopefully one of the experts will tune-in.
They way you store fish makes a huge difference in its outcome. Always store wrapped seafood on ice (that is in a plastic bag)in some non-leaking open- top container. Put this in the bottom rear corner of your frig- where it is the coldest. Since learning about this technique a few years ago, we have seen remarkably good results. We also store all raw poultry, meat in that lower rear corner.
I always rewrap the fish after bringing it home - before storing it in the fridge. Even if I'm cooking it the next day. And I put that package atop a bowl of ice water.