Peppertext 0.1.3

Description:

Peppertext 0.1.3

Declarative hypertext client

Installation
pip install peppertext


Overview
from peppertext import Hypertext, SimpleURLField, resolve, register, selector

@register
class GoogleBlogPage(Hypertext):
url = SimpleURLField(
"https://googleblog.blogspot.kr/{year}/{month}/{title}.html"
)

title = selector.find(".title[itemprop=name]").text()
body = selector.find(".post-body").text()
It resolves given headers, url and query string to hypertext object.
>>> p = resolve("https://googleblog.blogspot.kr/2015/11/google-gobble-thanksgiving-trends-on.html")
>>> p
<GoogleBlogPage at 0x108a4d1f0 >

>>> p.fetch()
>>> p['title']
'Google gobble: Thanksgiving trends on Search'
>>> p['body']
'In just a few hours, people across the U.S. will be settling...'
You can create GoogleBlogPage object with profile variables which are declared as
fields in GoogleBlogPage class.
>>> p = GoogleBlogPage(
... year="2015",
... month="11",
... title="google-gobble-thanksgiving-trends-on"
... )

>>> p.fetch()
>>> p['title']
'Google gobble: Thanksgiving trends on Search'


Selectors
class GoogleBlogPage(Hypertext):
# ...
title = selector.find(".title[itemprop=name]").text()
# ...
Selectors process a document which is returned from server as response.
In the GoogleBlogPage example above, title selector parses document and
find an element specified with “.title[itemprop=name]” css selector.
You can access the value title with subscribing the GoogleBlogPage object
with selector name.
document = pq("""<div>
<a href="http://example.com">Link1</a>
<a href="http://example.com/dahokan">Link2</a>
<a href="http://example.com/manoha">Link3</a>
</div>""")

find_selector = selector.find('a')
selected_els = find_selector.select(document)
self.assertEqual( [pq(el).attr["href"] for el in selected_els],
[
"http://example.com",
"http://example.com/dahokan",
"http://example.com/manoha"
]
)

find
Select html elements which match to given css selector string.


attribute
Get an element’s attribute value with given attribute name.


text
Select the html element’s inner text value.


at
Get an item on index


sub
sub_selector = selector.sub(pattern="\d+", repl="")
Do regex substitution.


cast
int_cast_selector = selector.cast(int)
Pass the data to the function given as a parameter.



Compatibility
Peppertext supports Python 2.7 and 3.


Features in developing

Interface for parse error handling
Polymorphic access to page selectors
Interface to resolve and traverse links in a page