It's not that far-fetched, as this video from researchers at the Georgia Institute of Technology shows.
As the narrator explains, the video combines data from Google Earth with data from other sources, namely surveillance and traffic cameras, to synthesize images of traffic on highways, pedestrians walking down the street--even soccer and football games.
With the proliferation of public and private cameras, and the trend for private operators to share their video images with public authorities, this technology looks like a step toward something like a "Live" Google Earth, allowing the users to see what's happening in any place with surveillance cameras.
"We feel that this form of dynamic visualization opens up many exciting avenues," the voice-over says.
One likely addition that wouldn't even need cameras: real-time weather and lighting overlays, since in the world as shown on Google Earth it's always daytime and it never rains.