blob: b9f3bafd01fc88df83333716db3058717e0ea4c8 [file] [log] [blame]
LUCI/gae: App Engine pseudo-SDK
Fast Tests, Slick API, Happy Hacking
18:00 12 Jul 2016
Tags: appengine, datastore, testing
Robbie Iannucci
Professional Software Wrestler, Google
Chrome Infrastructure Team
* Go on App Engine kicks ass!
: Hello! Robbie Iannucci, work CIT. Use Go/GAE to build CI microservices.
: QUICK GAE overview (PaaS, Datastore K/V, Memcache, load balancing, scaling, etc.)
: Go/GAE is BEST App Engine
: We made testing Go/GAE super fast+fun
* Trouble in Paradise?
- Requires SDK+Python (!!)
- Slow (3-4s minumum overhead per datastore instance)
- Accidental serialization (non-concurrent, race detector ~useless)
- automatic object memcaching
- hooks
- embedded IDs
- 'mockable' services
: Go/GAE 2GTTTGT, but hurdles because Go is recently added
: Uses same Python devserver (impl. speed, maint)
: We miss NDB
: So, as any good engineers would do...
* Solution
.background lightning/all_the_things.png
* Solution
- Concurrent
- Accurate Indexes, Query semantics, Query execution
- Accurate Transaction behavior (including transactional Task Queue)
- Fast tests
- Embedded object identity
- Flexible datatypes/encoding
: So we built a thing! In-memory pure-Go implementation of the datastore (as well
: other App Engine services). Library lets you access prod services with nice
: API, and get our really fast in-memory fakes in your tests.
* Code + Numbers
: So enough run-up, let's take a look at some lightning-style code and numbers.
* Native way (aetest)
.code lightning/native_test.go /START OMIT/,/END OMIT/
: Duplication ("Model")
: Key management
: Takes ~8 seconds == yawn.
~7.5 seconds
* luci/gae way
.code lightning/gae_test.go /START OMIT/,/END OMIT/
: Shorter! Model, ID part of model. ID properly typed.
: LUCI/gae can dynamically compute ID too (say if it was a composition of A+B)
: 50 Milliseconds.
~0.056 seconds
* >100x speedup! Less redundancy!
: Hooray! I can run 100 times as many tests in the same amount of time!
* But wait, there's more!
* Features
700 tests (parallel, clean-slate) run in ~35 seconds.
- Transparent caching (based on [[]])
- Recursive buffered transactions
- *<your*logic*here>*
Allows explicit testing around Eventual Consistency states.
Classic GAE, Managed VMs, Cloud Datastore, In-Memory
* Thanks
This couldn't have been done without amazing Go packages like:
- [[]] 😀
- [[]]
- [[]]
- [[]]