Let's start with something close at hand. What do you grab hold of when you fly your sim? I got started with a pretty basic joystick. It works but isn't the same as a flight yoke. Maybe that's because almost all my real flight time is with a yoke. I have a total of maybe an hour with a stick. Or maybe the real reason is that I have a really cheap joystick and am not particularly coordinated.
A good portion of your interaction with the sim will be through the primary flight controls, a stick or yoke, collective and rudder/tail rotor pedals. If they don't feel real to you, you'll continually be reminded you're working with a balky simulation. It'll be pretty hard getting to that "suspended disbelief" state of mind. Buy a cheap joystick and you'll be writing stuff like me instead of flying your sim.
The bottom line is that it is important to invest sufficient time and/or money for a quality set of flight controls. If you plan on buying, you have a wide range of choices from Microsoft, Saitek, Thrustmaster, Logitech, CH Products and the more expensive, but higher quality, Precision Flight Controls, Inc., just to mention a few. By all means, take the time to read in-print and on-line reviews, and check out comments on the fight sim forums. (For instance, the one on AvSim ). You might also find some reviews at Tom's Hardware and CNET.