Apr 7, 2009

Crawling YouTube


I started crawling YouTube site (again!) to get video comments.  This time, I'm using Google Data API and properly approaching the site.  The API makes the code very short and I like that it runs fast. Here is a sample code for getting comments.

comment_feed=yt_service.GetYouTubeVideoCommentFeed(video_id=video_id)
for comment_entry in comment_feed.entry:
print comment_entry.ToString()

Strangely, the code gives me a subset of video comments (say 100), even when there are thousands of comments I can see in YouTube. I'll have to go through the documentation or switch back to wget and urlopen. 

PS: Crawling YouTube is rather distracting.  I ended up watching 20 cat videos and participated in viral spreading of those videos (i.e., sending spam video links to friends).  My favorite of the day: cat massage.

No comments: