used lemons, oranges and grapefruit... suspect grapefruit.
first taste is good, followed by too much bitterness to make it smooth
I'd blame the grapefruit, too! I don' t know that it's salvageable as vin d'orange. Typical things that will help reduce the perception of bitterness are increased sugar, salt, or fat. Obviously you are not adding fat to the mix. You could try adding extra sugar and/or a little small to a trial portion to see how it works. I'd also consider researching bitters to see if you could convert it to homemade bitters instead. http://spiritsandcocktails.wordpress.com/2008/04/22/how-to-make-bitters/