Group items tagged

A join model is a data model that models the relationship between two other models. For example, you might have Person entities and Group entities, and you want Persons to be in Groups and Groups to "have" Persons. The relationship between Persons and Groups is one of Memberships. A Person is "in" a Group "through" a Membership. They may belong to multiple Groups, i.e. have multiple Memberships.

class Membership(db.Model): person = db.ReferenceProperty(Person) group = db.ReferenceProperty(Group)

If certain properties of your joined entities (Person and Group in this example) don't change much, but get queried often via the join model, you may find it worth caching those properties on the join model itself.

Entity groups are a simple concept. Entities in App Engine have ancestors. For e.g. you could model Books and Authors. Author: name->X, and Author: name->Y are two entities in the Author KIND. Book is another KIND (KIND is just a logical grouping of entities). The relationship between Books and Authors can be modeled as a ancestor-child relationship. E.g. Book: name->B1, B2 could have been written by Author: name->X. So you modeled them as: Author: name->X is a parent of both Book: name->B1, B2. Similarly, Book: name->B3 is written by Author: name->Y and so could model that relationship as Author:name->Y is a parent of Book:name->B3. When you try to transact on Books kind, you cannot transact on B1,B3 and B3 together. As they participate in different ancestor-child relationships. Every ancestor child relationship is an Entity group. You can only "lock" on one entity group at a time. Hope this makes things a little clear.

One option is to create a model User in your own application, one of the things it contains would be the google user object.  In this case, you have your own ID generated by each instance of your Model, for each User. EJ: class User(db.Model):         googleUser = db.UserProperty()

You can optimise it further, however: For each favorite, create a 'UserFavorite' entity as a child entity of the relevant Story entry (or equivalently, as a child entity of a UserInfo entry), with the key name set to the user's unique ID. This way, you can determine if a user has favorited a story with a simple get: UserFavorite.get_by_name(user_id, parent=a_story) get operations are 3 to 5 times faster than queries, so this is a substantial improvement.

Basically, all you have to do is create a subclass of bulkloader.Loader and implement (at a minimum) the generate_records method, which should yield lists of strings. This same strategy would work for loading data from XML files or ROT13-encrypted files or whatever. Note that the list of strings yielded by the generate_records method must match up (in length and order) with the "properties" list you provide when you initialize the loader (ie, the second argument to the AlbumLoader.__init__ method in this example). This approach actually provides a lot of flexibility: We're overriding the __init__ method on our JSONLoader implementation and automatically determining the kind of model we're loading and its list of properties to provide to the bulkloader.Loader parent class.

allocate_ids(model_key, count)

allocate_id_range(model, start, end, **kwargs)

The property map consists of a set of 'property' entries, each of which specifies how to handle a particular property of the model on import and on export. for our Permission kind, the bulkloader has identified 4 properties, plus the __key__ pseudo-property. Each has an 'external_name', and optional import and export transforms, which specify how to translate between the App Engine datastore representation and an external representation.

All we had to do here was to remove some of the boilerplate and the extraneous invite_nonce entry, and fill in the kind names for the two reference properties, and we're sorted.

we didn't have to write a single line of Python code, or set up an app.yaml, or anything else Python-specific in order to achieve it! Further, the bulkloader took care of generating a mostly-finished configuration file for us, in a format that ensures the data we download can be re-uploaded again without loss of fidelity.

In a relational world duplication is removed in order to prevent update anomalies. Error prevention is the driving force in relational modeling. Normalization is a kind of ethical system for data.

BigTable data ethics are more Mardi Gras than dinner with the in-laws. Data just wants to have fun. BigTable won’t stop you from hurting yourself. And to get the best results you may have to engage in some conventionally risky behaviors.

Task Queue is defined as a mechanism to synchronously distribute a sequence of tasks among parallel threads of execution. The global problem is broken down into tasks and the tasks are enqueued onto the queue. Parallel threads of execution pull tasks from the queue and perform computations on the tasks. The runtime system is responsible for managing thread accesses to the tasks in the queue as well as ensuring proper queue usage (i.e. dequeuing from an empty queue should not be allowed).

GAE Task Queue provides a push model. Instead of having an arbitrary number of worker processes constantly polling for available tasks, GAE Task Queue instead pushes work to workers when tasks are available. This work is then processed by the existing auto-scaling GAE infrastructure, allowing you to not have to worry about scaling up and down workers. You simply define the maximum rates of processing and GAE takes care of farming out the tasks to workers appropriately.

What is also compelling about GAE Task Queue is its use of web hooks as the description of a task unit. When you break it down, an individual task consists of the code to execute for that task as well as the individual data input for that specific task.

Just my 2 cents: I was "forced" to use an "unowned relationship" for my application because I had very large objects: My parent object "Journey" contained thousands of "JourneyPoints" in a List element. A call to "makePersistent" constantly hit the 30s deadline. By modeling it as an unowned relationship and distributing the parent key by hand to the JourneyPoints everything was fine again and the performance was very good.

About loading request/performance, there're lots of discussions that you can find in the groups, please just try google it. Here are some notes based on reading those. Latency causes by 1- time to start new JVM 2- time to load your application To reduce load time by 1) others star request to a) pay to reserve JVM b) request Google to load your app before start dispatch request to that instance c) accept the situation For 2) we try to a- try to use/replace frameworks with light-weight ones: datastore access framework, MVC framework,... b- try to limit calculation in your index page to alleviate the impact of loading request c- design your object model based on your need, so that you do calculation at insert time, not at query time. For example, in my app, if I want to report on year and quarter, then I have 5 summary "record" for those, instead of querying and computing those d- caching result. For example, If I know 1 one 5 piece of data above would be read frequently, then I will read those 5 all, and put into memcache for later use ... So it's application-specific, I don't know if each of above can help you. But only you who can know if which one of your code can be cached and how...

I propose two possiblities: Duplicate Line_Item. Line_Item appointment_key name price finished ... A Line_Item should have a finished state, when the item was finished or not by the employee. If an employee hadn't finished all line items, mark them as unfinished, create a new appointment and copy all items that were unfinished. You can index on the appointment_key field on all Line_Items, which is a Good Thing. However, the duplicated data may be a problem. Dynamic fields for Line_Item: Line_Item duplicate_key appointment_key name price finished ... Create a new field, duplicate_key, for Line_Item which points to another Line_Item or to null (reserve this key!). Null means that the Line_Item is original, any other value means that this Line_Item is a duplicate of the Line_Item the field points to. All fields of Line_Item marked as a duplicate inherit the fields of the original Line_Item, except the appointment_key: so it will take less storage. Also this solution should have appointment_key indexed, to speed up lookup times. This requires one additional query per duplicated Line_Item, which may be a problem. Now, it's a clear choice: either better speed or better storage. I would go for the first, as it reduces complexity of your model, and storage is never a problem with modern systems. Less complexity generally means less bugs and less development/testing costs, which justifies the cost of the storage requirement.

Have you considered doing both? Then you could quickly get a list of computers a student owns by key OR use a query which returns results in some sorted order. I don't think maintaining a list of keys on the student model is as intimidating as you think.