I love Vice City. It really does have the best atmosphere, like various '80s movies whether it's day or night in the game. And it's fun to mess around in it like pretending Tommy is a hitman/wheelman... occasionally a cop with the uniform in the police station. The map is excellent. I think even the gangs (actually a good variety of them in Vice City) are more interesting, particularly the Cubans in Little Havana.
However, San Andreas introduced some good features like climbing, swimming, using a silenced pistol etc. Plus recruitable gang members. Graphics were kind of a mixed bag (better lighting in day/night cycle albeit lower textures) but still good 90s atmosphere, music, and a lot of content.
GTA IV massively improved the drive-by (whether solo or with Dwayne's homies) and also had sticky bombs. GTA V had no friend rewards, more or less putting all the cool stuff in GTA Online instead.