IMO, a lot of it has to do with the paint. As breakneckvtec said, yeah, it CAN be done, after all, we're just talking about abrading the paint down to a certain level by rubbing it with an abrasive. Whether your hand does it or a PC does it, it's basically the same process (rotaries are different).
But some paints (including many basecoat/clears) are VERY hard, too hard for most people to work by hand. Others, like old-style lacquer are VERY soft, and easy to work by hand.
There's a difference, IMO, between "wasting time" by not using a machine *IF* you can afford one and "wasting time" by doing what you can, how you can. PC vs. hand = time vs. money, great results vs. money too. But it's the "vs. MONEY" part that can be controlling. If you can't swing the cost of a PC, then *I* don't think you'd be wasting your time to do what you can by hand. Even if you don't remove ALL the marring, you'll make things a LOT better. But if you can afford the PC, then it's simple: get one.