My advice is to not drive one until you're sure you can afford one. Why? Because you'll need one afterwards. :)
Get ready for a brain dump... I'm sure Brian Garfield will be on here at some point to fill in whatever I forget or fix what I got wrong.
For whatever reason autocross-only karts are usually cheaper than shifters used in sprint or road racing. Maybe lack of exposure? Most of the decent used packages are around $3500-5500, most of which is tied up in the engine and spare parts like chains, brake pads, gears, tools, seat, data acquisition, hubs, wheels, tires, axles, misc bolts+fasteners, etc. Getting below that mark and you'll start finding oddball stuff like orphan'd chassis, older engines that aren't legal/competitive/supported anymore, or karts that need a lot of work$$ to get back into good running shape (ask me how I know...). A local guy sold his 99 Margay CR125 that he road raced with literally a small garage full of spares and pretty top notch setup for <$6000. Another local is/was selling his 2002 Biesse with a TM Moto engine for $5500 with a respectable amount of spares, in good running shape, and with a small trailer. Two others were selling a pretty basic starter package of kart, motor, a few sprockets, and a pat on the back for $3500.
I'd personally be sticking with a CR125, having parts available down the street at any Honda Powersports dealer is nice and there's a ton of experience and support for those engines. Stock moto's will be somewhat down on power to modified powerplants and ICC engine'd kart, but will be (relatively) easier to drive and less maintenance intensive. A stock CR125 modified for karting from a reputable/national builder runs around $2500 and up and make around 35hp(?). ICC motors (purpose built kart racing engines from Europe) cost around $4500-5500 but make 40-45hp. You can have the CR125 modified to make similar power and cost somewhat less. Here's a good couple of places to go shopping
http://www.accelerationkarting.com/honda.html
http://www.fastech-racing.com/home.php?cat=75
More Info
http://www.kartweb.com/
Should you want to do it yourself, here's a handy guide for at least building a basically stock CR125: http://metal-matrix.com/stockmoto/
Note, I am not supported by or affiliated with any of the above sites, just pages I found useful in my search for info over the last year.
Keep in mind that pretty much everything on a kart is a wear item but autocrossing is really easy on these things compared to sprint racing. Personally, I lean towards supporting an American chassis builder (Coyote, Phantom, Margay, though the latter is the only one who really concentrates on 2-strokes) since they tend to last much much longer than the Italian chassis (basically everything else out there, TopKart, Birel, Biesse, TonyKart, CRG, KGB, Parolin, Sodi, and half a dozen others). When you are buying a used chassis, look for cracks on the frame, esp at the front spindles and around the engine mounts. Look for crash damage, evidence of the frame bottoming out, check the axle for straightness, make sure the hubs aren't cracking, brakes aren't leaking, etc. Most national level sprint racers move up to a new chassis every season, so there's always a bunch of last-years karts coming for sale in the silly season that are upward of half as much as a new kart. As with cars, buy the newest chassis you can afford. Especially with the Italian karts, remember that the chassis is the suspension so eventually it fatigues and loses its "springiness" so a kart thats a year old can be a second or more slower per lap than its replacement. This doesn't seem to affect US karts nearly as much, but fortunately the depreciation hit is the same for the frugal karter. :) Anything made overseas or after 1994 or 1995 from Margay will be metric (at least from the factory) and use metric fasteners.
You will need some safety equipment too. At minimum, start with a sturdy, abrasion resistent jacket and pants and gloves, driving shoes, closed face helmet, and neck roll. I knew I was sticking with this and went all the way and bought a Sparco kart suit ($150 on clearance), gloves ($80), and shoes ($110) locally from OG Racing. Buy locally, that way you can try everything on to make sure it all fits. I also recommend heavily that you get a rib protector, makes things much more comfortable for not a lot of money ($100). For autocrossing, the standard $30 neck brace is fine enough and helps keep your head level. I find mine very uncomfortable. Since I'm likely going sprint racing this year more than autocrossing (and I just bought a dirt bike), I'm probably going to pony up for something fancy like the Leatt Brace (basically its a HANS for karting and motorcycles).
2007 Toyota Tacoma
1998 Margay Brava, KT100
2000 Honda Civic Si - STS, sold
1998 Hyundai Accent L - FSP in '05 - SOLD
Never run out of ideas, traction, and real estate at the same time.