It's pretty obvious what he does. he watches movies and reads tons of books. The tenth doctor proved that with his references to Hitchhiker's guide to the galaxy, the lion king, Harry Potter, back to the future. etc.
He has probably read just about every book in The Library (not those particular vashta nerada infested copies, but you get the point.)