Group items tagged

 * Implements background tasks for * <a href="http://code.google.com/appengine/docs/java/overview.html">Google App * Engine for Java</a>, based on the * <a href="http://code.google.com/appengine/articles/deferred.html">Python 'deferred' * library</a>; simplifies use of the <a href="http://code.google.com/appengine/docs/java/taskqueue/overview.html"> * Task Queue Java API</a> by automatically handling the serialization and * deserializtion of complex task arguments.

to write large batch processing jobs without having to think about the plumbing details.

it is a very easy way to perform some operation on every single Entity of a given Kind in your datastore in parallel

What would you have to build for yourself if Mapper weren’t available? Begin querying over every Entity in chained Task Queues Store beginning and end cursors (introduced in 1.3.5) Create tasks to work with chunks of your datastore Write the code to manipulate your data Build an interface to control your batch jobs Build a callback system for your multitudes of parallelized workers to call when the entire task has completed It’s certainly not a trivial amount of work

An end cursor allows you to specify when you want a query to stop. A cursor represents a query start position - an end cursor is another position. This lets you take, say, 10k entities, break them up into 100 "chunks", then do task queue or other background operations on them. Unlike using an offset (cursor +100), this method solves the case of new Entities being inserted, since you're not working with a cursor +100 Entities. You're working with all the Entities between startCursor and endCursor.