When do you add the chopped potaoes?

I think you have it right. There would be no point to grating half and cutting half in small cubes if you were going to simply blend them all into the broth. I prefer to have a few bits to chew on besides the clams myself- so that's how I do it. As long as you like how it turns out when you make it, that's all that counts anyway!