Group items tagged

You can tell if it's using sqlite by executing 'sqlite3 /path/to/datastore' on the command line, and seeing if sqlite recognizes it as a valid DB. The code is essentially the same as that which I announced on my blog, so there shouldn't be significant performance differences.

import_transform A single-argument function that returns the correct value and type data based on the external_name or import_template strings. Examples include the built-in Python conversion operators (such as float), any of several helper functions provided in transform, such as get_date_time or generate_foreign_key, a function provided in your own library, or an in-line lambda function. Or, a two-argument function with the keyword argument bulkload_state, which on return contains useful information about the entity: bulkload_state.current_entity, which is the current entity being processed; bulkload_state.current_dictionary, the current export dictionary, and bulkload_state.filename, the --filename argument that was passed to appcfg.py.

import_template Specifies multiple dictionary items for a single property, using Python string interpolation.

Actually, remote_api now supports almost all App Engine's APIs. An exception is the Users API, which is fairly useless over remote_api. :)

Just set up remote_api for your app using the directions here - notably the tip: Tip: If you have a Java app, you can use the Python bulkloader.py tool by installing the Java version of the remote_api handler, which is included with the Java runtime environment. The handler servlet class is com.google.apphosting.utils.remoteapi.RemoteApiServlet. Then, use the Python bulkloader with --dump or --restore.

With the Task Queue API, applications can perform work outside of a user request but initiated by a user request. If an app needs to execute some background work, it may use the Task Queue API to organize that work into small, discrete units, called Tasks. The app then inserts these Tasks into one or more Queues. App Engine automatically detects new Tasks and executes them when system resources permit.

i had to do half an hour of googling to find the easiest solution; http://localhost:8080/_ah/admin/datastore imo it would be useful to have this information on http://code.google.com/appengine/docs/python/datastore/overview.html and related pages.

Try this: SELECT * FROM PreparedTransaction WHERE __key__=KEY('agdwYXllbGV4cjkLEhNQcmVwYXJlZFRyYW5zYWN0aW9uIiAwMDAwMGNjMjMwYzg2MTFjZTFhOWZjZDJkZDEzMWMyNww') This is documented here: http://code.google.com/appengine/docs/python/datastore/gqlreference.html

Tip: If you have a Java app, you can use the Python bulkloader.py tool by installing the Java version of the remote_api handler, which is included with the Java runtime environment. The handler servlet class is com.google.apphosting.utils.remoteapi.RemoteApiServlet.

The property map consists of a set of 'property' entries, each of which specifies how to handle a particular property of the model on import and on export. for our Permission kind, the bulkloader has identified 4 properties, plus the __key__ pseudo-property. Each has an 'external_name', and optional import and export transforms, which specify how to translate between the App Engine datastore representation and an external representation.

All we had to do here was to remove some of the boilerplate and the extraneous invite_nonce entry, and fill in the kind names for the two reference properties, and we're sorted.

we didn't have to write a single line of Python code, or set up an app.yaml, or anything else Python-specific in order to achieve it! Further, the bulkloader took care of generating a mostly-finished configuration file for us, in a format that ensures the data we download can be re-uploaded again without loss of fidelity.

If you are trying to connect to a local server you will need to provide -s <server> arg Otherwise it will try to connect to <appid>.appspot.com

You could create an entity group for this. Post:    data    category    sort UserPost:    user

if you can form the key_name for post and userpost like this Key('Post', '<data>') Key('Post', '<data>', 'UserPost', '<user>') Then you can perform a key_only query for a userpost and using the keys parents perform a get to retrieve the relevant userposts Adding new users incrementally to a Post is very simple and light weight. Deleting a Post would also require you to delete the UserPost children. These being in an entity group would provide for performing transactions... generally you wouldn't need to use them anyway... maybe when performing a full delete.

Thanks to the Task Queue API released in SDK 1.2.3, it's easier than ever to do work 'offline', separate from user serving requests. In some cases, however, setting up a handler for each distinct task you want to run can be cumbersome, as can serializing and deserializing complex arguments for the task - particularly if you have many diverse but small tasks that you want to run on the queue. Fortunately, a new library in release 1.2.5 of the SDK makes these ad-hoc tasks much easier to write and execute. This library is found in google.appengine.ext.deferred, and from here on in we'll refer to it as the 'deferred' library. The deferred library lets you bypass all the work of setting up dedicated task handlers and serializing and deserializing your parameters by exposing a simple function, deferred.defer().

To demonstrate how powerful the deferred library can be, we're going to reprise an example from the remote_api article - the Mapper class. Like the example in the remote_api article, this class will make it easy to iterate over a large set of entities, making changes or calculating totals. Unlike the remote_api version, though, our version won't require an external computer to run it on, and it'll be more efficient to boot!

Task Queue items are limited to 10kb of associated data. This means that when the deferred library serializes the details of your call, it must amount to less than 10 kilobytes in order to fit on the Task Queue directly. No need to panic, though: If you try to enqueue a task that is too big to fit on the queue by itself, the deferred library will automatically create a new Entity in the datastore to hold information about the task, and will delete the entity once the task has been run. This means that in practice, your function call can be up to 1MB once serialized.

Well, in AppEngine your primary key is the key for the object(Entity) and it can either be a generated ID or a unique string your application provides.  "Google App Engine" by Dan Sanderson has a lot of examples on this.  If you don't use a key_name property, then when you save with a put(), an ID is generated, but if you pass in key_name='thisIsMyKey' into the constructor, then you manually set the key for the object.  You can use the method, id_or_name() to return either the object's key name or its ID, which ever one it has and has_id_or_name() would return a boolean about it, and if it's not saved and your not using key_name, then the ID would not exists yet.  Also you can get the Entity(Object) from the datastore by using the get(k) method where k is a key object and the key object has two parts: kind and ID or key_name.  Additionally you can fetch an object from the datastore with get_by_id() and get_by_key_name()

An end cursor allows you to specify when you want a query to stop. A cursor represents a query start position - an end cursor is another position. This lets you take, say, 10k entities, break them up into 100 "chunks", then do task queue or other background operations on them. Unlike using an offset (cursor +100), this method solves the case of new Entities being inserted, since you're not working with a cursor +100 Entities. You're working with all the Entities between startCursor and endCursor.