很久就想學python了,但一直找不到合適的項目來練習,python的語法很有意思,寫起來很簡潔,今天有空邊找資料邊寫出來了這一個小項目。考慮到3.x的python庫還不是很多,大部分資料也是python2.x的,所以我用的python2.7來進行
之前就聽說python訪問網絡很easy,這次是真的體會到了。很簡單幾句話搞定,不像java那樣,再簡單的訪問都要裝飾幾層才能使用。
這次是拿糗事百科的網站,從上面抓取新鮮事並整理打印出來
不想多說了,以下上代碼:
import urllib2 import sgmllib class Entry: author='' content='' pic='' up = 0 down = 0 tag = '' comment = 0 def to_string(self): return '[Entry: author=%s content=%s pic=%s tag=%s up=%d down=%d comment=%d]'\ %(self.author,self.content,self.pic,self.tag,self.up,self.down,self.comment) class MyHTMLParser(sgmllib.SGMLParser): #所有用到的聲明 #note all the datas datas = [] # all the entries entries = [] #the entry now entry = Entry() #last Unclosed tag div_tag_unclosed = '' def start_div(self,attrs): for name,value in attrs: if name =='class' and value == 'content': self.div_tag_unclosed = 'content' elif name=='class' and value == 'tags' : self.div_tag_unclosed = 'tags' elif name=='class' and value=='up': self.div_tag_unclosed = 'up' elif name=='class' and value == 'down': self.div_tag_unclosed = 'down' elif name=='class' and value=='comment': self.div_tag_unclosed = 'comment' elif name=='class' and value=='author': self.div_tag_unclosed = 'author' self.entry = Entry() elif name=='class' and value=='thumb': self.div_tag_unclosed = 'thumb' def end_div(self): if self.div_tag_unclosed == 'content' : self.div_tag_unclosed ='' self.entry.content = self.datas.pop().strip() def start_a(self,attrs):pass def start_img(self,attrs): if self.div_tag_unclosed == 'thumb': for name,value in attrs: if name=='src': self.div_tag_unclosed ='' self.entry.img = value.strip() def end_img(self):pass def end_a(self): if self.div_tag_unclosed == 'author': self.div_tag_unclosed ='' self.entry.author = self.datas.pop().strip() if self.div_tag_unclosed == 'tags': self.div_tag_unclosed ='' self.entry.tag = self.datas.pop().strip() elif self.div_tag_unclosed == 'up': self.div_tag_unclosed ='' self.entry.up = int(self.datas.pop().strip()) elif self.div_tag_unclosed == 'down': self.div_tag_unclosed ='' self.entry.down = int(self.datas.pop().strip()) elif self.div_tag_unclosed == 'comment': self.div_tag_unclosed ='' self.entry.comment = int(self.datas.pop().strip()) self.entries.append(self.entry) def handle_data(self, data): # print 'data',data self.datas.append(data) #request the url response = urllib2.urlopen('http://www.qiushibaike.com/8hr') all = response.read() #parse HTML parser = MyHTMLParser() parser.feed(all) #print all the entries for entry in parser.entries: print entry.to_string()
整個程序很簡單,用到了urllib2來請求網絡,sgmllib來解析Html,由於第一次寫python程序,所以寫的時候效率很低,尤其是一直想在if后面加上小括號=-=