Are you using fresh anchovies or canned?

I use fresh anchovie as a tapa, "cooked" in lemon juice and olive oil served on toasts. For all other anchovie uses I use Italian, packed in oil in a glass jar. I stopped using the American tin can variety (where the opening "key" never works) once I discovered the taste difference. And, you can store the remainder of the glass jar in the fridge forever as long as they are covered in oil.
Many people swear by the salt packed - you need to rinse and when taking out of the can they break into pieces. I like the whole pieces from the jar.
But, at the end of the day - it is your choice.
Let your taste buds guide you.
A few pennies more and you get bigger flavor...

I have never seen or made bagna cauda with fresh anchovies. Always canned/jarred either packed in salt or oil (preferably oil). Save the fresh ones for other, more straight-forward preparations (left whole).