I started out with a RCBS Rockchucker Jr. If I'm not mistaken, they usually come with one set of dies bundled in.
Once you've got that, you need a powder thrower, a scale, a set of calipers and a reloading manual.
I do agree with the idea that a single stage press let's you start out slow. There is a lot going on with a progressive press and I occassionally catch my self looking like that re-run of I Love Lucy where she was working next to a conveyor belt in a factory.