The wii fit will never weigh the same as the scales, especially when you tell it to minus 2 or 3 lbs for your clothes.
I just wear the same / similar clothes each time I use it, always choose -2lbs, and let it compare against previous weigh-ins on the wii fit, I don't compare it with the SW scales. Also, try to use it at the same time of day each time as your weight can fluctuate a lot over the course of a day.