第一個python小程序--解析網頁


很久就想學python了,但一直找不到合適的項目來練習,python的語法很有意思,寫起來很簡潔,今天有空邊找資料邊寫出來了這一個小項目。考慮到3.x的python庫還不是很多,大部分資料也是python2.x的,所以我用的python2.7來進行

之前就聽說python訪問網絡很easy,這次是真的體會到了。很簡單幾句話搞定,不像java那樣,再簡單的訪問都要裝飾幾層才能使用。

這次是拿糗事百科的網站,從上面抓取新鮮事並整理打印出來

不想多說了,以下上代碼:

import urllib2
import sgmllib

class Entry:
    author=''
    content=''
    pic=''
    up = 0
    down = 0
    tag = ''
    comment = 0
    def to_string(self):
        return '[Entry: author=%s content=%s pic=%s tag=%s up=%d down=%d comment=%d]'\
            %(self.author,self.content,self.pic,self.tag,self.up,self.down,self.comment)

class MyHTMLParser(sgmllib.SGMLParser):
    #所有用到的聲明
    #note all the datas
    datas = []
    # all the entries
    entries = []
    #the entry now
    entry = Entry()
    #last Unclosed tag
    div_tag_unclosed = ''
    
    def start_div(self,attrs):
        for name,value in attrs:
            if name =='class' and value == 'content':
                self.div_tag_unclosed = 'content'
            elif name=='class' and value == 'tags' :
                self.div_tag_unclosed = 'tags'
            elif name=='class' and value=='up':
                self.div_tag_unclosed = 'up'
            elif name=='class' and value == 'down':
                self.div_tag_unclosed = 'down'
            elif name=='class' and value=='comment':
                self.div_tag_unclosed = 'comment'
            elif name=='class' and value=='author':
                self.div_tag_unclosed = 'author'
                self.entry = Entry()
            elif name=='class' and value=='thumb':
                self.div_tag_unclosed = 'thumb'
                
    def end_div(self):
        if self.div_tag_unclosed == 'content' :
            self.div_tag_unclosed =''
            self.entry.content =  self.datas.pop().strip()
    def start_a(self,attrs):pass
    def start_img(self,attrs):
        if self.div_tag_unclosed == 'thumb':
            for name,value in attrs:
                if name=='src':
                    self.div_tag_unclosed =''
                    self.entry.img = value.strip() 
    def end_img(self):pass
    def end_a(self):
        if self.div_tag_unclosed == 'author':
            self.div_tag_unclosed =''
            self.entry.author = self.datas.pop().strip()
        if self.div_tag_unclosed == 'tags':
            self.div_tag_unclosed =''
            self.entry.tag = self.datas.pop().strip()
        elif self.div_tag_unclosed == 'up':
            self.div_tag_unclosed =''
            self.entry.up = int(self.datas.pop().strip())
        elif self.div_tag_unclosed == 'down':
            self.div_tag_unclosed =''
            self.entry.down = int(self.datas.pop().strip())
        elif self.div_tag_unclosed == 'comment':
            self.div_tag_unclosed =''
            self.entry.comment = int(self.datas.pop().strip())
            self.entries.append(self.entry)
    def handle_data(self, data):
#        print 'data',data
        self.datas.append(data)

#request the url
response = urllib2.urlopen('http://www.qiushibaike.com/8hr')
all = response.read()

#parse HTML
parser = MyHTMLParser()
parser.feed(all)
#print all the entries
for entry in parser.entries:
    print entry.to_string()

整個程序很簡單,用到了urllib2來請求網絡,sgmllib來解析Html,由於第一次寫python程序,所以寫的時候效率很低,尤其是一直想在if后面加上小括號=-=


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM