"Each photo is processed by computer vision algorithms to extract hundreds of distinctive features, like the corner of a window frame or a door handle. Photos that share features are then linked together in a web."
Photosynth has the ability to process large collections of pictures fast and also uses Seadragon technology to ensure a smooth zooming.
Microsoft thinks this technology will be available for mass consumption next year and photo sharing websites will be the first to use it.
You can see a live demo here, but it requires Internet Explorer and Windows XP SP2 (or Vista), as it uses an ActiveX control.