Yeah, SF rents have gone crazy. The problem is that none of the little suburban towns in the peninsula/south bay are willing to build any denser housing (like, their whole towns are zoned for 1–2 story buildings with no more than 2 units per building and required reserved parking spaces, it’s absurd), but keep adding more office jobs. As a result there are increasing numbers of people commuting every day from SF to palo alto / mountain view / sunnyvale / cupertino / etc. There’s a housing shortage all over the Bay Area, and prices are rising commensurately.
The other big problem is that the mass transit infrastructure (while good compared to most of the USA) basically sucks compared to real cities like you’d find in Europe or East Asia.