People have been looking up to the stars ever since we've been on Earth, wondering if someone on a distant planet or galaxy could be doing the same thing. Now that you're here, you've probably discovered the astronomer inside yourself just like I did a while ago. Now, onto the fun stuff.... :)
Right now you might be screaming out for a telescope, but it is important that we get some things said and done first. Get down to your local astronomy club or society and talk to everyone. More good advice can be obtained from 5 minutes talking with a seasoned stargazer than an hour on the internet. My local club, the Astronomical Society of Western Australia, can be reached here and it provides links to others all over Australia and NZ.
Secondly, get yourself a planisphere (a round piece of plastic or cardboard with a map of the heavens, detailing constellations and other objects) and spend a few nights to get to know the sky. If you're unable to come across one of these, check out Skymaps and print one out for the southern hemisphere. This is actually my preferred method because the site also gives you a page of all the objects easily seen with the naked eye, binoculars and a telescope.
Speaking of binoculars, you might as well take a look at some of the brighter celestial objects if you have a pair. It's best to start with the Orion nebula or Omega Centauri (a globular star cluster), which show up as nice, hazy shapes. There are also hundereds of star clusters in the sky which are extremely easy to spot - you just have to know where to look. You'll find everything and more on your star map.
It also helps to read up on the subject. At the local library you should find many books on space and astronomy to fill your mind with information and feed your interest. I suggest purchasing a 'beginners guide' which will tell you exactly what you should be doing. The book I have is detailed here and is strongly recommended.
OK. So now you've read up a bit, made friends with your local astro society's members and learned the constellations. If your interest is still flourishing this may be the time to think about a telescope! First it's important that you've gone through and completed the steps above. Before you go out and spend your saved up money, read the 'Telescope Buying Guide' on this site.