What software does it happen in?
If it happens in everything from text editors to web browsers, then it's some deeper issue. But if it only happens in, say, MS Word, then it's a Word bug. I can't imagine it being a hardware problem. It could be a bad backspace key but then you could see the characters going one by one, not all at once.