Heat 2 teaspoons of the olive oil over medium low heat in a large skillet. Add the onions and pinch of sugar and cook for about 20 minutes, stirring occasionally, until the onions are caramelized. Add in the mushrooms and cook for another 6 or so minutes or until the mushrooms are tender. Remove from heat and let cool slightly.
Put the ground turkey in a bowl and add the Worcestershire, Dijon, thyme, salt, pepper and nutmeg. Add the onion/mushroom mixture and mix well. Add in the Swiss and mix thoroughly. Form into 4 patties. Raise the heat under the skillet to the medium and add the remaining 2 teaspoons of olive oil. Cook the patties about 7-8 minutes per side or until done to your liking.