Group items tagged

What do I mean by life cycle events? Events like entity creation, entity update and entity deletion. Mainstream ORM systems popularised callbacks like oncreate, onupdate, ondelete. Introducing such callbacks in the Java and Python APIs may be easy, but things get messy when you consider the ecosystem of alternative language implementations based on the Java API: developers using alternative languages would be forced to use Java to write the callbacks. There is a more robust solution though. Google App Engine already leverages the power of webhooks in such APIs as taskqueue, email, xmpp and more. Webhooks can elegantly solve the life cycle management problem as well: when an entity is created, updated or deleted through the Datastore viewer a corresponding webhook is triggered. Let's say the user is playing with Article entities, the webhooks uris could be: http://myapp.com/_ah/admin/datastore/le/Article/create/{key} http://myapp.com/_ah/admin/datastore/le/Article/update/{key} http://myapp.com/_ah/admin/datastore/le/Article/delete/{key} Slightly more work than callbacks, but still simple and effective. If there is an even better solution, I would love to hear about it in the comments section.

Thanks to the Task Queue API released in SDK 1.2.3, it's easier than ever to do work 'offline', separate from user serving requests. In some cases, however, setting up a handler for each distinct task you want to run can be cumbersome, as can serializing and deserializing complex arguments for the task - particularly if you have many diverse but small tasks that you want to run on the queue. Fortunately, a new library in release 1.2.5 of the SDK makes these ad-hoc tasks much easier to write and execute. This library is found in google.appengine.ext.deferred, and from here on in we'll refer to it as the 'deferred' library. The deferred library lets you bypass all the work of setting up dedicated task handlers and serializing and deserializing your parameters by exposing a simple function, deferred.defer().

To demonstrate how powerful the deferred library can be, we're going to reprise an example from the remote_api article - the Mapper class. Like the example in the remote_api article, this class will make it easy to iterate over a large set of entities, making changes or calculating totals. Unlike the remote_api version, though, our version won't require an external computer to run it on, and it'll be more efficient to boot!

Task Queue items are limited to 10kb of associated data. This means that when the deferred library serializes the details of your call, it must amount to less than 10 kilobytes in order to fit on the Task Queue directly. No need to panic, though: If you try to enqueue a task that is too big to fit on the queue by itself, the deferred library will automatically create a new Entity in the datastore to hold information about the task, and will delete the entity once the task has been run. This means that in practice, your function call can be up to 1MB once serialized.

Yes, you will need to use the federated login (OpenID) stuff. The long-and-short of it is that you pass the federated_identity parameter to users.create_login_url.  You'll need to setup a page for users to tell you what goes in federated_identity somehow, perhaps by clicking a google logo or entering an apps domain. For Google accounts:   users.create_login_url(federated_identity='google.com/accounts/o8/id')   or   users.create_login_url(federated_identity='gmail.com') For an Apps account:   users.create_login_url(federated_identity='google.com/accounts/o8/site-xrds?hd=yourappsdomain.com') There is a little info here:   http://code.google.com/appengine/docs/python/users/overview.html And Wesley has a nice article about it here:   http://code.google.com/appengine/articles/openid.html Some info on Google Apps domains and OpenID:   http://groups.google.com/group/google-federated-login-api/web/openid-...

to enable both Maven and GPE for the same project. This article explains how to do that.

Mavenizing an existing GPE project To use Maven with an existing GPE project, follow these steps:

The most obvious use case is data import. A developer looking to import large amounts of data would take the following steps: Create a CSV file containing the data you want to import. The assumption here is that each line of data corresponds to a datastore entity you want to create Upload the CSV file to the blobstore. You’ll need billing to be enabled for this to work. Create your Mapper, push it live and run your job importing your data. This isn’t meant to be a replacement for the bulk uploader tool; merely an alternative. This method requires a good amount more programmatic changes for custom data transforms. The advantage of this method is that the work is done on the server side, whereas the bulk uploader makes use of the remote API to get work done. Let’s get started on each of the steps.

to build Mappers that map across some large, contiguous piece of data as opposed to Entities in the datastore

you can only expect to update any single entity or entity group about five times a second. That is an estimate and the actual update rate for an entity is dependent on several attributes of the entity, including how many properties it has, how large it is, and how many indexes need updating. While a single entity or entity group has a limit on how quickly it can be updated, App Engine excels at handling many parallel requests distributed across distinct entities, and we can take advantage of this by using sharding.

 If you're following the how-to article on unit testing ( http://code.google.com/appengine/docs/java/howto/unittesting.html) you'll need to update your TestEnvironment class to look like this one: http://code.google.com/p/datanucleus-appengine/source/browse/branches... (lines 34 - 66) We're working on getting the docs updated right now.

The remote_api module consists of two parts: A 'handler', which you install on the server to handle remote datastore requests, and a 'stub', which you set up on the client to translate datastore requests into calls to the remote handler. remote_api works at the lowest level of the datastore, so once you've set up the stub, you don't have to worry about the fact that you're operating on a remote datastore: With a few caveats, it works exactly the same as if you were accessing the datastore directly.

Note that the handler specifies "login: admin". This is extremely important, since we don't want to give just anyone unfettered access to our datastore!

Since you're accessing the datastore over HTTP, there's a bit more overhead and latency than when you access it locally. In order to speed things up and decrease load, try to limit the number of round-trips you do by batching gets and puts, and fetching batches of entities from queries. This is good advice not just for remote_api, but for using the datastore in general, since a batch operation is only considered to be a single Datastore operation