[Python正則表達式] 字符串中xml標簽的匹配

　　現在有一個需求，比如給定如下數據：

0-0-0 0:0:0 #### the 68th annual golden globe awards ####  the king s speech earns 7 nominations  ####  <LOCATION>LOS ANGELES</LOCATION> <ORGANIZATION>Dec Xinhua Kings Speech</ORGANIZATION> historical drama British king stammer beat competitors Tuesday grab seven nominations Golden Globe Awards nominations included best film drama nod contested award organizers said films competing best picture <ORGANIZATION>Social Network Black Swan Fighter Inception Kings Speech</ORGANIZATION> earned nominations best performance actor olin <PERSON>Firth</PERSON> best performance actress <PERSON>Helena Bonham</PERSON> arter best supporting actor <PERSON>Geoffrey Rush</PERSON> best director <PERSON>Tom Hooper</PERSON> best screenplay <PERSON>David Seidler</PERSON> best movie score <ORGANIZATION>Alexandre Desplat Social Network Fighter</ORGANIZATION> earned nods apiece Black Swan Inception Kids Right tied place movie race nominations best motion picture comedy musical category <ORGANIZATION>Alice Wonderland Burlesque Kids Right Red Tourist</ORGANIZATION> compete Nominated best actor motion picture olin <ORGANIZATION>Firth Kings Speech James Franco Hours Ryan Gosling Blue Valentine Mark Wahlberg Fighter Jesse Eisenberg Social Network</ORGANIZATION> best actress motion picture nominees <PERSON>Halle Berry Frankie Alice Nicole Kidman</PERSON> Rabbit Hole <PERSON>Jennifer Lawrence</PERSON> <ORGANIZATION>Winters Bone Natalie Portman Black Swan Michelle Williams Blue Valentine TV</ORGANIZATION> categories Glee nominee nods followed Rock Boardwalk Empire Dexter Good Wife Mad Men Modern Family Pillars Earth Temple <PERSON>Grandin</PERSON> tied nods apiece awards announced Jan 

 

　　要求按行把<></>標簽內的字符串中的空格替換成下划線_，並且將數據轉換形式，例：<X>A B C</X>需要轉換成A_B_C/X

　　由於正則表達式匹配是貪婪模式，即盡可能匹配到靠后，那么就非常麻煩，而且僅僅是用?是無法真正保證是非貪婪的。所以需要在正則匹配時給之前匹配好的字符串標一個名字。

python下，正則最終寫出來是這樣：

LABEL_PATTERN = re.compile('(<(?P<label>\S+)>.+?</(?P=label)>)')

　　接下來我們需要做是在原字符串中找出對應的子串，並且記下他們的位置，接下來就是預處理出需要替換成的樣子，再用一個正則就好了。

LABEL_CONTENT_PATTERN = re.compile('<(?P<label>\S+)>(.*?)</(?P=label)>')

　　對字符串集合做整次的map，對每一個字符串進行匹配，再吧這兩部分匹配結果zip在一起，就可以獲得一個start-end的tuple，大致這樣。

('<LOCATION>LOS ANGELES</LOCATION>', 'LOS_ANGELES/LOCATION')
('<ORGANIZATION>Dec Xinhua Kings Speech</ORGANIZATION>', 'Dec_Xinhua_Kings_Speech/ORGANIZATION')
('<ORGANIZATION>Social Network Black Swan Fighter Inception Kings Speech</ORGANIZATION>', 'Social_Network_Black_Swan_Fighter_Inception_Kings_Speech/ORGANIZATION')
('<PERSON>Firth</PERSON>', 'Firth/PERSON')
('<PERSON>Helena Bonham</PERSON>', 'Helena_Bonham/PERSON')
('<PERSON>Geoffrey Rush</PERSON>', 'Geoffrey_Rush/PERSON')
('<PERSON>Tom Hooper</PERSON>', 'Tom_Hooper/PERSON')
('<PERSON>David Seidler</PERSON>', 'David_Seidler/PERSON')
('<ORGANIZATION>Alexandre Desplat Social Network Fighter</ORGANIZATION>', 'Alexandre_Desplat_Social_Network_Fighter/ORGANIZATION')
('<ORGANIZATION>Alice Wonderland Burlesque Kids Right Red Tourist</ORGANIZATION>', 'Alice_Wonderland_Burlesque_Kids_Right_Red_Tourist/ORGANIZATION')
('<ORGANIZATION>Firth Kings Speech James Franco Hours Ryan Gosling Blue Valentine Mark Wahlberg Fighter Jesse Eisenberg Social Network</ORGANIZATION>', 'Firth_Kings_Speech_James_Franco_Hours_Ryan_Gosling_Blue_Valentine_Mark_Wahlberg_Fighter_Jesse_Eisenberg_Social_Network/ORGANIZATION')
('<PERSON>Halle Berry Frankie Alice Nicole Kidman</PERSON>', 'Halle_Berry_Frankie_Alice_Nicole_Kidman/PERSON')
('<PERSON>Jennifer Lawrence</PERSON>', 'Jennifer_Lawrence/PERSON')
('<ORGANIZATION>Winters Bone Natalie Portman Black Swan Michelle Williams Blue Valentine TV</ORGANIZATION>', 'Winters_Bone_Natalie_Portman_Black_Swan_Michelle_Williams_Blue_Valentine_TV/ORGANIZATION')
('<PERSON>Grandin</PERSON>', 'Grandin/PERSON')
('<LOCATION>BEIJING</LOCATION>', 'BEIJING/LOCATION')
('<ORGANIZATION>Xinhua Sanlu Group</ORGANIZATION>', 'Xinhua_Sanlu_Group/ORGANIZATION')
('<LOCATION>Gansu</LOCATION>', 'Gansu/LOCATION')
('<ORGANIZATION>Sanlu</ORGANIZATION>', 'Sanlu/ORGANIZATION')

　　處理的代碼如下：

def read_file(path):
    if not os.path.exists(path):
        print 'path : \''+ path + '\' not find.'
        return []
    content = ''
    try:
        with open(path, 'r') as fp:
            content += reduce(lambda x,y:x+y, fp)
    finally:
        fp.close()
    return content.split('\n')

def get_label(each):
    pair = zip(LABEL_PATTERN.findall(each),
                         map(lambda x: x[1].replace(' ', '_')+'/'+x[0], LABEL_CONTENT_PATTERN.findall(each)))
    return map(lambda x: (x[0][0], x[1]), pair)

src = read_file(FILE_PATH)
pattern = map(get_label, src)

 

　　接下來簡單處理以下就好：

for i in range(0, len(src)):
    for pat in pattern[i]:
        src[i] = re.sub(pat[0], pat[1], src[i])

 

　　所有代碼：

# -*- coding: utf-8 -*-
import re
import os

# FILE_PATH = '/home/kirai/workspace/sina_news_process/disworded_sina_news_attr_handled.txt'
FILE_PATH = '/home/kirai/workspace/sina_news_process/test.txt'
LABEL_PATTERN = re.compile('(<(?P<label>\S+)>.+?</(?P=label)>)')
LABEL_CONTENT_PATTERN = re.compile('<(?P<label>\S+)>(.*?)</(?P=label)>')

def read_file(path):
    if not os.path.exists(path):
        print 'path : \''+ path + '\' not find.'
        return []
    content = ''
    try:
        with open(path, 'r') as fp:
            content += reduce(lambda x,y:x+y, fp)
    finally:
        fp.close()
    return content.split('\n')

def get_label(each):
    pair = zip(LABEL_PATTERN.findall(each),
                         map(lambda x: x[1].replace(' ', '_')+'/'+x[0], LABEL_CONTENT_PATTERN.findall(each)))
    return map(lambda x: (x[0][0], x[1]), pair)

src = read_file(FILE_PATH)
pattern = map(get_label, src)

for i in range(0, len(src)):
    for pat in pattern[i]:
        src[i] = re.sub(pat[0], pat[1], src[i])