Group items tagged

public void setProperty(java.lang.String propertyName, java.lang.Object value) Sets the property named, propertyName, to value. As the value is stored in the datastore, it is converted to the datastore's native type. This may include widening, such as converting a Short to a Long. All Collections are prone to losing their sort order and their original types as they are stored in the datastore. For example, a TreeSet may be returned as a List from getProperty(java.lang.String), with an arbitrary re-ordering of elements. Overrides any existing value for this property, whether indexed or unindexed. Note that Blob and Text property values are never indexed by the built-in single property indexes. To store other types without being indexed, use #setUnindexedProperty. Parameters:value - may be one of the supported datatypes, a heterogenous Collection of one of the supported datatypes, or an UnindexedValue wrapping one of the supported datatypes. Throws: java.lang.IllegalArgumentException - If the value is not of a type that the data store supports.

you could leverage Entity's getProperties() method, in combination with JSON-lib's groovy-friendliness and ability to serialize maps as JSON. So, say you have an Entity: def person = new Entity("person") person.name = "Guillaume Laforge" def jsonString = person.properties as JSONObject Have a look at JSON-lib and the Entity JavaDoc. And I think it'll solve your problem pretty neatly!

It would need to be something like Transaction txn = dataStore.beginTransaction(); Key key = dataStore.put(txn, question); Entity a1 = new Entity("Answer", key); Entity a2 = new Entity("Answer", key); dataStore.put(txn, a1); dataStore.put(txn, a2); txn.commit();

An end cursor allows you to specify when you want a query to stop. A cursor represents a query start position - an end cursor is another position. This lets you take, say, 10k entities, break them up into 100 "chunks", then do task queue or other background operations on them. Unlike using an offset (cursor +100), this method solves the case of new Entities being inserted, since you're not working with a cursor +100 Entities. You're working with all the Entities between startCursor and endCursor.

You can optimise it further, however: For each favorite, create a 'UserFavorite' entity as a child entity of the relevant Story entry (or equivalently, as a child entity of a UserInfo entry), with the key name set to the user's unique ID. This way, you can determine if a user has favorited a story with a simple get: UserFavorite.get_by_name(user_id, parent=a_story) get operations are 3 to 5 times faster than queries, so this is a substantial improvement.

A join model is a data model that models the relationship between two other models. For example, you might have Person entities and Group entities, and you want Persons to be in Groups and Groups to "have" Persons. The relationship between Persons and Groups is one of Memberships. A Person is "in" a Group "through" a Membership. They may belong to multiple Groups, i.e. have multiple Memberships.

class Membership(db.Model): person = db.ReferenceProperty(Person) group = db.ReferenceProperty(Group)

If certain properties of your joined entities (Person and Group in this example) don't change much, but get queried often via the join model, you may find it worth caching those properties on the join model itself.

Thanks to the Task Queue API released in SDK 1.2.3, it's easier than ever to do work 'offline', separate from user serving requests. In some cases, however, setting up a handler for each distinct task you want to run can be cumbersome, as can serializing and deserializing complex arguments for the task - particularly if you have many diverse but small tasks that you want to run on the queue. Fortunately, a new library in release 1.2.5 of the SDK makes these ad-hoc tasks much easier to write and execute. This library is found in google.appengine.ext.deferred, and from here on in we'll refer to it as the 'deferred' library. The deferred library lets you bypass all the work of setting up dedicated task handlers and serializing and deserializing your parameters by exposing a simple function, deferred.defer().

To demonstrate how powerful the deferred library can be, we're going to reprise an example from the remote_api article - the Mapper class. Like the example in the remote_api article, this class will make it easy to iterate over a large set of entities, making changes or calculating totals. Unlike the remote_api version, though, our version won't require an external computer to run it on, and it'll be more efficient to boot!

Task Queue items are limited to 10kb of associated data. This means that when the deferred library serializes the details of your call, it must amount to less than 10 kilobytes in order to fit on the Task Queue directly. No need to panic, though: If you try to enqueue a task that is too big to fit on the queue by itself, the deferred library will automatically create a new Entity in the datastore to hold information about the task, and will delete the entity once the task has been run. This means that in practice, your function call can be up to 1MB once serialized.

The remote_api module consists of two parts: A 'handler', which you install on the server to handle remote datastore requests, and a 'stub', which you set up on the client to translate datastore requests into calls to the remote handler. remote_api works at the lowest level of the datastore, so once you've set up the stub, you don't have to worry about the fact that you're operating on a remote datastore: With a few caveats, it works exactly the same as if you were accessing the datastore directly.

Note that the handler specifies "login: admin". This is extremely important, since we don't want to give just anyone unfettered access to our datastore!

Since you're accessing the datastore over HTTP, there's a bit more overhead and latency than when you access it locally. In order to speed things up and decrease load, try to limit the number of round-trips you do by batching gets and puts, and fetching batches of entities from queries. This is good advice not just for remote_api, but for using the datastore in general, since a batch operation is only considered to be a single Datastore operation

You might want to instead use datastore's built in ability to   efficiently create unique long ids and then when sending them to a   client "muddle" them using a reversible hash function so they don't   appear predictable.  You will find that using long ids results in   shorter Keys which saves space in the datastore. You can only guarantee a unique value in the datastore by creating an   Entity with the value in its key.  This does not need to be your   "main" entity - just a special UniqueName type.  Then you can

db.get('ag1iYXRjaC1nZW5lcmljchcLEgxCYXRjaGVzTW9kZWwiBUpvYiAyDA')

The most obvious use case is data import. A developer looking to import large amounts of data would take the following steps: Create a CSV file containing the data you want to import. The assumption here is that each line of data corresponds to a datastore entity you want to create Upload the CSV file to the blobstore. You’ll need billing to be enabled for this to work. Create your Mapper, push it live and run your job importing your data. This isn’t meant to be a replacement for the bulk uploader tool; merely an alternative. This method requires a good amount more programmatic changes for custom data transforms. The advantage of this method is that the work is done on the server side, whereas the bulk uploader makes use of the remote API to get work done. Let’s get started on each of the steps.

to build Mappers that map across some large, contiguous piece of data as opposed to Entities in the datastore

import_transform A single-argument function that returns the correct value and type data based on the external_name or import_template strings. Examples include the built-in Python conversion operators (such as float), any of several helper functions provided in transform, such as get_date_time or generate_foreign_key, a function provided in your own library, or an in-line lambda function. Or, a two-argument function with the keyword argument bulkload_state, which on return contains useful information about the entity: bulkload_state.current_entity, which is the current entity being processed; bulkload_state.current_dictionary, the current export dictionary, and bulkload_state.filename, the --filename argument that was passed to appcfg.py.

import_template Specifies multiple dictionary items for a single property, using Python string interpolation.

Because A is the entity group parent of B, the key of an instance of B must contain information about its entity group parent instance of A. Instead of using b.key.getId() in   // We cannot retrieve this object. Why?   B newB = pm.getObjectById(B.class,b.key.getId()); you might care to use the KeyFactory.Builder class as detailed in "http://code.google.com/intl/en/appengine/docs/java/datastore/ creatinggettinganddeletingdata.html#Creating_and_Using_Keys" in order to build a key instance which contains information about both your B instance and its parental A instance.

Tip: If the app creates a new object and gives it the same string ID as another object of the same kind (and the same entity group parent), saving the new object overwrites the other object in the datastore. To detect whether a string ID is already in use prior to creating a new object, you can use a transaction to attempt to get an entity with a given ID, then create one if it doesn't exist. See Transactions.

There are 4 types of primary key fields:

As I understand it, the process of performing a single fetch (call to get())  from the dastastore using a key basically involves finding the host housing the entity, opening a socket, fetching the data, and then cleaning up the connection.  So to fetch something like 30 entities from the datastore, you're repeating the process 30 times over in serial, each time incurring whatever overhead is involved.  I also read that if you perform bulk fetches, (ie passing multiple keys at once) you can eliminate a great deal of that overhead.  In one of the videos I watched from Google I/0 2009, the presenter (whose name I forget - d'oh) said that performing a bulk fetch actually performs the fetches in parallel from the data store and you shoudl see requests noticeably faster.

You could create an entity group for this. Post:    data    category    sort UserPost:    user

if you can form the key_name for post and userpost like this Key('Post', '<data>') Key('Post', '<data>', 'UserPost', '<user>') Then you can perform a key_only query for a userpost and using the keys parents perform a get to retrieve the relevant userposts Adding new users incrementally to a Post is very simple and light weight. Deleting a Post would also require you to delete the UserPost children. These being in an entity group would provide for performing transactions... generally you wouldn't need to use them anyway... maybe when performing a full delete.

You can't in a query, the indexes only index values that exist,  you would have to explicitly set properties to None or some other sentinal value by default to find objects that haven't had a property set.

> How does one test for the presence or absence of a particular property > in an Entity?

Well, in AppEngine your primary key is the key for the object(Entity) and it can either be a generated ID or a unique string your application provides.  "Google App Engine" by Dan Sanderson has a lot of examples on this.  If you don't use a key_name property, then when you save with a put(), an ID is generated, but if you pass in key_name='thisIsMyKey' into the constructor, then you manually set the key for the object.  You can use the method, id_or_name() to return either the object's key name or its ID, which ever one it has and has_id_or_name() would return a boolean about it, and if it's not saved and your not using key_name, then the ID would not exists yet.  Also you can get the Entity(Object) from the datastore by using the get(k) method where k is a key object and the key object has two parts: kind and ID or key_name.  Additionally you can fetch an object from the datastore with get_by_id() and get_by_key_name()