commit | 9ed3dce6b0517b51d03bb10e7edc1d6b1c719e97 | [log] [tgz] |
---|---|---|
author | Martin Angers <martin.n.angers@gmail.com> | Mon Sep 17 15:14:03 2012 |
committer | Martin Angers <martin.n.angers@gmail.com> | Mon Sep 17 15:14:03 2012 |
tree | 94f44e9f31fedc88dd8240cd6b7fcd0a025207b0 | |
parent | 1f8447feb698dc961a040f0d5cbabc1f28b73aa4 [diff] |
forgot changelog
GoQuery brings a syntax and a set of features similar to jQuery to the Go language. It is based on the experimental html package and the CSS Selector library cascadia. Since the experimental html parser returns tokens (nodes), and not a full-featured DOM object, jQuery's manipulation and modification functions have been left off (no point in modifying data in the parsed tree of the HTML, it has no effect).
Supported functions are query-oriented features (hasClass()
, attr()
and the likes), as well as traversing functions that make sense given what we have to work with. This makes GoQuery a great library for scraping web pages.
Syntax-wise, it is as close as possible to jQuery, with the same function names when possible, and that warm and fuzzy chainable interface. jQuery being the ultra-popular library that it is, I felt that writing a similar HTML-manipulating library was better to follow its API than to start anew (in the same spirit as Go's fmt
package), even though some of its methods are less than intuitive (looking at you, index()...).
Since this library (and cascadia) depends on the experimental branch, this package must be installed first. Both GoQuery and Cascadia expect to find the experimental library with the "exp/html"
import statement. To install it at this location, please follow this guide.
Once this is done, install GoQuery:
go get github.com/PuerkitoBio/goquery
To run unit tests, run this command in goquery's source directory ($GOPATH/src/github.com/PuerkitoBio/goquery
):
go test
To run benchmarks, run this command in goquery's source directory:
go test -bench=".*"
GoQuery exposes two classes, Document
and Selection
. Unlike jQuery, which is loaded as part of a DOM document, and thus acts on its containing document, GoQuery doesn‘t know which HTML document to act upon. So it needs to be told, and that’s what the Document
class is for. It holds the root document node as the initial Selection object to manipulate.
jQuery often has many variants for the same function (no argument, a selector string argument, a jQuery object argument, a DOM element argument, ...). Instead of exposing the same features in GoQuery as a single method with variadic empty interface arguments, I use statically-typed signatures following this naming convention:
Prev()
), and the version with a selector string argument is called XxxFiltered()
(e.g.: PrevFiltered()
)Is()
)XxxSelection()
and take a *Selection
object as argument (e.g.: FilterSelection()
)XxxNodes()
and take a variadic argument of type *html.Node
(e.g.: FilterNodes()
)XxxFunction()
and take a function as argument (e.g.: FilterFunction()
)GoQuery's complete godoc reference documentation can be found here.
Please note that Cascadia's selectors do NOT necessarily match all supported selectors of jQuery (Sizzle). See the cascadia project for details.
Taken from example_test.go:
import ( "fmt" // In real use, this import would be required (not in this example, since it // is part of the goquery package) //"github.com/PuerkitoBio/goquery" "strconv" ) // This example scrapes the 10 reviews shown on the home page of MetalReview.com, // the best metal review site on the web :) (and no, I'm not affiliated to them!) func ExampleScrape_MetalReview() { // Load the HTML document (in real use, the type would be *goquery.Document) var doc *Document var e error if doc, e = NewDocument("http://metalreview.com"); e != nil { panic(e.Error()) } // Find the review items (the type of the Selection would be *goquery.Selection) doc.Root.Find(".slider-row:nth-child(1) .slider-item").Each(func(i int, s *Selection) { var band, title string var score float64 // For each item found, get the band, title and score, and print it band = s.Find("strong").Text() title = s.Find("em").Text() if score, e = strconv.ParseFloat(s.Find(".score").Text(), 64); e != nil { // Not a valid float, ignore score fmt.Printf("Review %d: %s - %s.\n", i, band, title) } else { // Print all, including score fmt.Printf("Review %d: %s - %s (%2.1f).\n", i, band, title, score) } }) // To see the output of the Example while running the test suite (go test), simply // remove the leading "x" before Output on the next line. This will cause the // example to fail (all the "real" tests should pass). // xOutput: voluntarily fail the Example output. }
Closest()
? Other missing functions?Slice()
, like jQuery.The BSD 3-Clause license, the same as the Go language. Cascadia's license is here.