当使用正则表达式按复合类名称搜索时,BeautifulSoup返回空列表。
例:
import re from bs4 import BeautifulSoup bs = """ <a class="name-single name692" href="www.example.com"">Example Text</a> """ bsObj = BeautifulSoup(bs) # this returns the class found_elements = bsObj.find_all("a", class_= re.compile("^(name-single.*)$")) # this returns an empty list found_elements = bsObj.find_all("a", class_= re.compile("^(name-single name\d*)$"))
我需要对课程的选择非常精确。有任何想法吗?
不幸的是,当您尝试对包含多个类的类属性值进行正则表达式匹配时,BeautifulSoup会将正则表达式分别应用于每个单个类。
BeautifulSoup
这一切都是因为class是一个很特别的多值属性,每一次你解析HTML的一个BeautifulSoup的树建设者(取决于解析器选择)内部分裂从一个类的字符串值入类(报价列表HTMLTreeBuilder的docstring):
class
HTMLTreeBuilder
# The HTML standard defines these attributes as containing a # space-separated list of values, not a single value. That is, # class="foo bar" means that the 'class' attribute has two values, # 'foo' and 'bar', not the single value 'foo bar'. When we # encounter one of these attributes, we will parse its value into # a list of values if possible. Upon output, the list will be # converted back into a string.
有多种解决方法,但这是一种hack-ish解决方案- 我们将通过制作简单的自定义树生成器来要求BeautifulSoup不要将其class作为多值属性来处理:
import re from bs4 import BeautifulSoup from bs4.builder._htmlparser import HTMLParserTreeBuilder class MyBuilder(HTMLParserTreeBuilder): def __init__(self): super(MyBuilder, self).__init__() # BeautifulSoup, please don't treat "class" specially self.cdata_list_attributes["*"].remove("class") bs = """<a class="name-single name692" href="www.example.com"">Example Text</a>""" bsObj = BeautifulSoup(bs, "html.parser", builder=MyBuilder()) found_elements = bsObj.find_all("a", class_=re.compile(r"^name\-single name\d+$")) print(found_elements)
在这种情况下,正则表达式将class整体应用于属性值。
或者,您可以仅分析xml启用了功能的HTML (如果适用):
xml
soup = BeautifulSoup(data, "xml")
您还可以使用CSS选择器,将所有元素与name-singleclass和以“ name”开头的类进行匹配:
name-single
soup.select("a.name-single,a[class^=name]")
然后,您可以根据需要手动应用正则表达式:
pattern = re.compile(r"^name-single name\d+$") for elm in bsObj.select("a.name-single,a[class^=name]"): match = pattern.match(" ".join(elm["class"])) if match: print(elm)