Clone this repo:
  1. 05908fb Update to latest protocol definitions by Mathias Fredriksson · 2 days ago master
  2. ff59efa Update to latest protocol definitions by Mathias Fredriksson · 5 days ago v0.14.2
  3. 4b80152 protocol: Enable json.RawMessage messages in the Target domain by Mathias Fredriksson · 6 days ago
  4. 0f27aa1 cmd/cdpgen: Change Target mesages to json.RawMessage by Mathias Fredriksson · 6 days ago
  5. c3ead3b Update to latest protocol definitions by Mathias Fredriksson · 4 weeks ago v0.14.1

cdp

Build Status Coverage Status Go Report Card GoDoc

Package cdp provides type-safe bindings for the Chrome Debugging Protocol (CDP), written in the Go programming language. The bindings are generated (by cdpgen) from the latest tip-of-tree (tot) protocol definitions and are mainly intended for use with Google Chrome or Chromium, however, they can be used with any debug target (Node.js, Edge, Safari, etc.) that implement the protocol.

This package can be used for any kind of browser automation, scripting or debugging via the Chrome Debugging Protocol.

A big motivation for cdp is to expose the full functionality of the Chrome Debugging Protocol and provide it in a discoverable and self-documenting manner.

Providing high-level browser automation is a non-goal for this project. That being said, cdp hopes to improve the ergonomics of working with the protocol by providing primitives better suited for Go and automating repetitive tasks.

Features

  • Discoverable API for the Chrome Debugging Protocol (GoDoc, autocomplete friendly)
  • Contexts as a first-class citizen (for timeouts and cancellation)
  • Simple and synchronous event handling (no callbacks)
  • Concurrently safe
  • No silent or hidden errors
  • Do what the user expects
  • Match CDP types to Go types wherever possible
  • Separation of concerns (avoid mixing CDP and RPC)

Installation

$ go get -u github.com/mafredri/cdp

Documentation

See API documentation for package, API descriptions and examples. Examples can also be found in this repository, see the simple, advanced and logging examples.

Usage

The main packages are cdp and rpcc, the former provides the CDP bindings and the latter handles the RPC communication with the debugging target.

To connect to a debug target, a WebSocket debugger URL is needed. For example, if Chrome is running with --remote-debugging-port=9222 the debugger URL can be found at localhost:9222/json. The devtool package can also be used to query the DevTools JSON API (see example below).

Here is an example of using cdp:

package main

import (
	"context"
	"fmt"
	"io/ioutil"
	"log"
	"time"

	"github.com/mafredri/cdp"
	"github.com/mafredri/cdp/devtool"
	"github.com/mafredri/cdp/protocol/dom"
	"github.com/mafredri/cdp/protocol/page"
	"github.com/mafredri/cdp/rpcc"
)

func main() {
	err := run(5 * time.Second)
	if err != nil {
		log.Fatal(err)
	}
}

func run(timeout time.Duration) error {
	ctx, cancel := context.WithTimeout(context.Background(), timeout)
	defer cancel()

	// Use the DevTools HTTP/JSON API to manage targets (e.g. pages, webworkers).
	devt := devtool.New("http://127.0.0.1:9222")
	pt, err := devt.Get(ctx, devtool.Page)
	if err != nil {
		pt, err = devt.Create(ctx)
		if err != nil {
			return err
		}
	}

	// Initiate a new RPC connection to the Chrome Debugging Protocol target.
	conn, err := rpcc.DialContext(ctx, pt.WebSocketDebuggerURL)
	if err != nil {
		return err
	}
	defer conn.Close() // Leaving connections open will leak memory.

	c := cdp.NewClient(conn)

	// Open a DOMContentEventFired client to buffer this event.
	domContent, err := c.Page.DOMContentEventFired(ctx)
	if err != nil {
		return err
	}
	defer domContent.Close()

	// Enable events on the Page domain, it's often preferrable to create
	// event clients before enabling events so that we don't miss any.
	if err = c.Page.Enable(ctx); err != nil {
		return err
	}

	// Create the Navigate arguments with the optional Referrer field set.
	navArgs := page.NewNavigateArgs("https://www.google.com").
		SetReferrer("https://duckduckgo.com")
	nav, err := c.Page.Navigate(ctx, navArgs)
	if err != nil {
		return err
	}

	// Wait until we have a DOMContentEventFired event.
	if _, err = domContent.Recv(); err != nil {
		return err
	}

	fmt.Printf("Page loaded with frame ID: %s\n", nav.FrameID)

	// Fetch the document root node. We can pass nil here
	// since this method only takes optional arguments.
	doc, err := c.DOM.GetDocument(ctx, nil)
	if err != nil {
		return err
	}

	// Get the outer HTML for the page.
	result, err := c.DOM.GetOuterHTML(ctx, &dom.GetOuterHTMLArgs{
		NodeID: &doc.Root.NodeID,
	})
	if err != nil {
		return err
	}

	fmt.Printf("HTML: %s\n", result.OuterHTML)

	// Capture a screenshot of the current page.
	screenshotName := "screenshot.jpg"
	screenshotArgs := page.NewCaptureScreenshotArgs().
		SetFormat("jpeg").
		SetQuality(80)
	screenshot, err := c.Page.CaptureScreenshot(ctx, screenshotArgs)
	if err != nil {
		return err
	}
	if err = ioutil.WriteFile(screenshotName, screenshot.Data, 0644); err != nil {
		return err
	}

	fmt.Printf("Saved screenshot: %s\n", screenshotName)

	return nil
}

For more information, consult the documentation.

Acknowledgements

The Go implementation of gRPC (grpc-go) has been a source of inspiration for some of the design descisions made in the cdp and rpcc packages. Some ideas have also been borrowed from the net/rpc package from the standard library.

Resources