As a new parent, my sister wanted to make sure she had everything for her baby. Baby registries lead you to believe that you should get everything for your baby. Only after she had my baby did she realize it's like buying a new home or car, you want it all right away, but you don't really NEED it right away or even at all. So she learned that just the essentials (ie. diapers, wipes, milk, some clothing) were all she needed to keep her baby content.
Things like: a Jolly Jumper, exersaucer, playard/playpen, Bumbo chair, bouncy chair, swing, high chair, even shoes, were extras. And a lot of those things are nice to have, but not must haves. What she did to save some money was borrow some of the bigger ticket items, look for sales, buy some gently used items, and even look on the local free cycle. It's amazing what you can get for a fraction of the cost! I know my parents didn't have that luxury when I was growing up! So to all of those soon-to-be parents, don't worry if you don't have it all, because your baby goes through stages and doesn't need much to be amused!