Go語言正則表達式提取網頁文本


為了方便提取,我們會把正則表達式中要提取的數據使用命名方式來書寫正則表達式。這個技術在Go語言中如何實現,可以看下面這篇博客:

Using the Go Regexp Package
http://blog.kamilkisiel.net/blog/2012/07/05/using-the-go-regexp-package/

簡單期間,這里復制其中幾個例子的代碼:

我們期望在字符串  1000abcd123  中找出前后兩個數字。

例子1:匹配到這個字符串的例子 

 
 
 
         package main
 
import(
    "fmt"
    "regexp"
)
 
var digitsRegexp = regexp.MustCompile(`(\d+)\D+(\d+)`)
 
func main(){
    someString:="1000abcd123"
    fmt.Println(digitsRegexp.FindStringSubmatch(someString))
}
上面代碼輸出:

[1000abcd123 1000 123]

 
 
 
         

 

例子2:使用帶命名的正則表達式

 
 
 
         package main
 
import(
    "fmt"
    "regexp"
)
 
var myExp=regexp.MustCompile(`(?P<first>\d+)\.(\d+).(?P<second>\d+)`)
 
func main(){
    fmt.Printf("%+v",myExp.FindStringSubmatch("1234.5678.9"))
}

上面代碼輸出,所有匹配到的都輸出了: 

[1234.5678.9 1234 5678 9] 

這里的Named capturing groups  (?P<name>) 方式命名正則表達式是 python、Go語言特有的, java、c# 是 (?<name>) 命名方式。

參考: http://www.crifan.com/detailed_explanation_about_python_regular_express_match_named_group/

http://osdir.com/ml/go-language-discuss/2012-07/msg00605.html

Go 支持的正則表達式語法可以參看:https://code.google.com/p/re2/wiki/Syntax

 

例子3:對正則表達式類擴展一個獲得所有命名信息的方法,並使用它。

 
 
 
         package main
 
import(
    "fmt"
    "regexp"
)
 
//embed regexp.Regexp in a new type so we can extend it
type myRegexp struct{ 
    *regexp.Regexp
}
 
//add a new method to our new regular expression type
func(r *myRegexp)FindStringSubmatchMap(s string) map[string]string{
    captures:=make(map[string]string)
 
    match:=r.FindStringSubmatch(s)
    if match==nil{
        return captures
    }
 
    for i,name:=range r.SubexpNames(){
        //Ignore the whole regexp match and unnamed groups
        if i==0||name==""{
            continue
        }
 
        captures[name]=match[i]
 
    }
    return captures
}
 
//an example regular expression
var myExp=myRegexp{regexp.MustCompile(`(?P<first>\d+)\.(\d+).(?P<second>\d+)`)}
 
func main(){
    mmap:=myExp.FindStringSubmatchMap("1234.5678.9")
    ww:=mmap["first"]
    fmt.Println(mmap)
    fmt.Println(ww)
}
上面代碼的輸出結果:

map[first:1234 second:9]

1234

 
 
 
         

例子4,抓取限號信息,並記錄到一個Map中。 

 

 
 
 
         package main
 
import(
    "fmt"
    iconv "github.com/djimenez/iconv-go"
    "io/ioutil"
    "net/http"
    "os"
    "regexp"
)
 
// embed regexp.Regexp in a new type so we can extend it
type myRegexp struct{
    *regexp.Regexp
}
 
// add a new method to our new regular expression type
func(r *myRegexp)FindStringSubmatchMap(s string)[](map[string]string){
    captures:=make([](map[string]string),0)
 
    matches:=r.FindAllStringSubmatch(s,-1)
 
    if matches==nil{
        return captures
    }
 
    names:=r.SubexpNames()
 
    for _,match:=range matches{
 
        cmap:=make(map[string]string)
 
        for pos,val:=range match{
            name:=names[pos]
            if name==""{
                continue
            }
 
            /*
                fmt.Println("+++++++++")
                fmt.Println(name)
                fmt.Println(val)
            */
            cmap[name]=val
        }
 
        captures=append(captures,cmap)
 
    }
 
    return captures
}
 
// 抓取限號信息的正則表達式
var myExp=myRegexp{regexp.MustCompile(`自(?P<byear>[\d]{4})年(?P<bmonth>[\d]{1,2})月(?P<bday>[\d]{1,2})日至(?P<eyear>[\d]{4})年(?P<emonth>[\d]{1,2})月(?P<eday>[\d]{1,2})日,星期一至星期五限行機動車車牌尾號分別為:(?P<n11>[\d])和(?P<n12>[\d])、(?P<n21>[\d])和(?P<n22>[\d])、(?P<n31>[\d])和(?P<n32>[\d])、(?P<n41>[\d])和(?P<n42>[\d])、(?P<n51>[\d])和(?P<n52>[\d])`)}
 
func ErrorAndExit(err error){
    fmt.Fprintln(os.Stderr,err)
    os.Exit(1)
}
 
func main(){
    response,err:=http.Get("http://www.bjjtgl.gov.cn/zhuanti/10weihao/index.html")
    defer response.Body.Close()
 
    if err!=nil{
        ErrorAndExit(err)
    }
 
    input,err:=ioutil.ReadAll(response.Body)
    if err!=nil{
        ErrorAndExit(err)
    }
    body :=make([]byte,len(input))
    iconv.Convert(input,body,"gb2312","utf-8")
 
    mmap:=myExp.FindStringSubmatchMap(string(body))
 
    fmt.Println(mmap)
}

上述代碼輸出:

[map[n32:0 n22:9 emonth:7 n11:3 n41:1 n21:4 n52:7 bmonth:4 n51:2 bday:9 n42:6 byear:2012 eday:7 eyear:2012 n12:8 n31:5]
map[emonth:10 n41:5 n52:6 n31:4 byear:2012 n51:1 eyear:2012 n32:9 bmonth:7 n22:8 bday:8 n11:2 eday:6 n42:0 n21:3 n12:7]
map[bday:7 n51:5 n22:7 n31:3 eday:5 n32:8 byear:2012 bmonth:10 emonth:1 eyear:2013 n11:1 n12:6 n52:0 n21:2 n42:9 n41:4]
map[eyear:2013 byear:2013 n22:6 eday:10 bmonth:1 n41:3 n32:7 n31:2 n21:1 n11:5 bday:6 n12:0 n51:4 n42:8 emonth:4 n52:9]]

這段代碼首先下載北京市交管局的網頁;然后把這個gb2312的頁面轉換成utf-8編碼,然后用正則表達式提取其中的限號信息。 

 

參考資料:

marcinwyszynski / named_captures.go 
https://gist.github.com/marcinwyszynski/3889572/revisions

StefanSchroeder / Golang-Regex-Tutorial
https://github.com/StefanSchroeder/Golang-Regex-Tutorial

Using the Go Regexp Package
http://blog.kamilkisiel.net/blog/2012/07/05/using-the-go-regexp-package/ 

Go語言用正則表達式提取多個指定字符
http://blog.sina.com.cn/s/blog_4b51d46901019c3x.html 

用golang的正則regexp:去除HTML,CSS,SCRIPT代碼,僅保留頁面文字
http://bpbp.iteye.com/blog/1668869

re2
https://code.google.com/p/re2/wiki/Syntax

Go by Example: Regular Expressions
https://gobyexample.com/regular-expressions

Golang-Regex-Tutorial / 01-chapter0.markdown 
https://github.com/StefanSchroeder/Golang-Regex-Tutorial/blob/master/01-chapter0.markdown


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM