So you got some type of old portable record player then? I guess it depends on your budget... you could get something like a Pro-ject debut for $200 up to a $2000 Thorens. I use a mid range Stanton and have it connected up to my pc which has a good audio set up with audioengine n22 amp and gallo speakers.
Presumably speakers in portables are crap, but you could replace that with a better one, and the needle is the other big factor when it comes to record players. Though some may argue direct drive over belt and so on... and other things, but most of the things that make turntables expensive is style and gimmicks for audiophile suckers.
If I were in the market right now, I would probably just buy a Pro-ject. They are good quality and I really like thier aesthetics too. While not that important in sound, they aren't exatly small items so if its always going to be visable in the room it dosen't hurt to look nice.