Specifically for Middle Eastern, Chinese and Thai cooking.
Amazon always has some great ingredients, and for spices I use Penzey's. Their quality and selection is unmatched. I've also used asianfoodgrocer.com and igourmet.com. If you have a Cost Plus near you they carry a lot of ingredients too.
Friedas (west coast produce company) for Asian produce, which will include many items in Chinese and Thai recipes.
Kalustyan's in NYC for dry spices and grains for all 3.
Bricks and mortar, you can often find M-E ingredients in Armenian, Indian or Pakistani stores (those Moghuls & Trade Route travellers carried dishes back and forth).
Also look in the back pages of cookbooks on these areas...they almost all have sources listed for ingredients & equipment, but the most recently published or updated will give you more live contacts.
Perhaps - and Im not sure where you reside, do you have any Asian/Middle-Eastern food shops? Ive got TONS of them where I live and their quality and prices are absolutely fantastic!
For those that are hard to find, I have found many reliable sources through Ebay for companies that sell spices/herbs/dried peppers.... Sometimes their prices on Ebay are CHEAPER than their Amazon account. Made a wicked purchase on Star Anise that was almost 20% cheaper through Ebay than Amazon...