I remember reading that Photosynth takes a series of images as input; I always wondered why you couldn't feed it a movie. I'll be interested in seeing how you did this.
The process generally involved Play/pausing the video on Youtube, taking snapshots, importing snapshots into Photoshop, touching them up (cropping so only the screen was visible, skewing them into a rectangular shape, removing any ghost effects, removing the tops of people's heads, and fixing the light/colour balance), then throwing them all into PhotoSynth.Sadly, the more images I add, the more annoying Photosynth gets - for some reason, adding more images at the end seems to break the 'synthing' of some of the earlier images :(
If I remember right, GIMP has a feature that turns a video into a series of stills for you. Perhaps that would save some time.Then again, it could be my faulty memory acting up. Even if so - there's got to be something that can automate the process.