I have both. An '05 S4 and an '04 M3.
They are very different cars but each great in their own way. Both are fast, no doubt.
The Audi is better built, hands down. Better interior, better fit and finish. It feels more refined, is more luxurious in feel compared to the M. AWD traction is impressive but it leaves no doubt that this is one heavy car which likes to plow into corners. An impressive overall car and I like it better for daily driving and long runs.
The M3 is more a 'drivers' car and 'sporty' feeling. It's easier and more fun to chuck around a track, which I do often. The suspension is tighter and the steering is indeed better. I think it looks cooler, certainly more muscular in appearance. And the aftermarket is much more supportive. Sounds good, too. This, also, is pretty much a daily driver for me as well, but with a more raw edge than the S4. And it gets better gas mileage than the S4!
IMO, you really can't go wrong with either. At least that's what I tell my wife every time I make the insurance payments!
