Google AppEngine

You can't in a query, the indexes only index values that exist,  you would have to explicitly set properties to None or some other sentinal value by default to find objects that haven't had a property set.

> How does one test for the presence or absence of a particular property > in an Entity?

You will only find "unowned relationships" terminology in Google docs. This "unowned relationships" in the GAE/J docs is where you have a field that is a "Key" or a Collection/Map/array of keys. So no *real* relation, just some implicit relation by that "key". You have to manage these keys yourself, and you tie your code to GAE/J by using them.

Don't bother with GQL for key-based retrieval -- make a key object from the string: k = db.Key('aght52oobW1hZHIOCxIHTWVzc2FnZRiyAQw') and just db.get(k). If you insist on GQL, btw, that k -- a suitably constructed instance of db.Key, NOT a string object!-) -- is also what you need to substitute into the GQL query (by :1 or whatrever).

keys are SERIOUSLY unique -- from a key you can get the entity's .app() (so uniqueness across apps is a given), .kind() (so uniqueness WITHIN the app's no problem either), etc.

Thanks to the Task Queue API released in SDK 1.2.3, it's easier than ever to do work 'offline', separate from user serving requests. In some cases, however, setting up a handler for each distinct task you want to run can be cumbersome, as can serializing and deserializing complex arguments for the task - particularly if you have many diverse but small tasks that you want to run on the queue. Fortunately, a new library in release 1.2.5 of the SDK makes these ad-hoc tasks much easier to write and execute. This library is found in google.appengine.ext.deferred, and from here on in we'll refer to it as the 'deferred' library. The deferred library lets you bypass all the work of setting up dedicated task handlers and serializing and deserializing your parameters by exposing a simple function, deferred.defer().

To demonstrate how powerful the deferred library can be, we're going to reprise an example from the remote_api article - the Mapper class. Like the example in the remote_api article, this class will make it easy to iterate over a large set of entities, making changes or calculating totals. Unlike the remote_api version, though, our version won't require an external computer to run it on, and it'll be more efficient to boot!

Task Queue items are limited to 10kb of associated data. This means that when the deferred library serializes the details of your call, it must amount to less than 10 kilobytes in order to fit on the Task Queue directly. No need to panic, though: If you try to enqueue a task that is too big to fit on the queue by itself, the deferred library will automatically create a new Entity in the datastore to hold information about the task, and will delete the entity once the task has been run. This means that in practice, your function call can be up to 1MB once serialized.

The remote_api module consists of two parts: A 'handler', which you install on the server to handle remote datastore requests, and a 'stub', which you set up on the client to translate datastore requests into calls to the remote handler. remote_api works at the lowest level of the datastore, so once you've set up the stub, you don't have to worry about the fact that you're operating on a remote datastore: With a few caveats, it works exactly the same as if you were accessing the datastore directly.

Note that the handler specifies "login: admin". This is extremely important, since we don't want to give just anyone unfettered access to our datastore!

Since you're accessing the datastore over HTTP, there's a bit more overhead and latency than when you access it locally. In order to speed things up and decrease load, try to limit the number of round-trips you do by batching gets and puts, and fetching batches of entities from queries. This is good advice not just for remote_api, but for using the datastore in general, since a batch operation is only considered to be a single Datastore operation

All you can do is fetch the entities of the type you want from the datastore and delete them one by one. (call datastore.delete(Iterable<Entity>))This way you can delete about 1000 entities at each request... There is no way to delete all entities of a Kind by a single api call.

> > If there is a way to list current Kinds or select all entities of any > > kind I could manage this.

One thing you get used to on appengine is that any bulk data work requires the task queue.  You can use a little bit of framework and make all of these transforms (including deleting data) a question of just writing a simple task class and firing it off.  You'll want a copy of the Deferred servlet: http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlant... Fair warning:  I found that I needed to change the code to make it perform base64 encoding all the time, not just on the dev instance.

A Java app sets up queues using a configuration file named queue.xml, in the WEB-INF/ directory inside the WAR. See Java Task Queue Configuration. If an app does not have a queue.xml file, it has a queue named default with some default settings. To enqueue a task, you get a Queue using the QueueFactory, then call its add() method. You can get a named queue specified in the queue.xml file using the getQueue() method of the factory, or you can get the default queue using getDefaultQueue(). You can call the Queue's add() method with a TaskOptions instance (produced by TaskOptions.Builder), or you can call it with no arguments to create a task with the default options for the queue.

Although a queue defines a general FIFO ordering, tasks are not executed entirely serially. Multiple tasks from a single queue may be executed simultaneously by the scheduler, so the usual locking and transaction semantics need to be observed for any work performed by a task.

One thing you get used to on appengine is that any bulk data work requires the task queue.  You can use a little bit of framework and make all of these transforms (including deleting data) a question of just writing a simple task class and firing it off.  You'll want a copy of the Deferred servlet: http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlant... Fair warning:  I found that I needed to change the code to make it perform base64 encoding all the time, not just on the dev instance.

 * Implements background tasks for * <a href="http://code.google.com/appengine/docs/java/overview.html">Google App * Engine for Java</a>, based on the * <a href="http://code.google.com/appengine/articles/deferred.html">Python 'deferred' * library</a>; simplifies use of the <a href="http://code.google.com/appengine/docs/java/taskqueue/overview.html"> * Task Queue Java API</a> by automatically handling the serialization and * deserializtion of complex task arguments.

Google app engine uses the concept of static files. This is a performance optimization. Using the file appengine-web.xml you can configure the way google handles static files. You can include and exclude certain files using their extension or name. More information can be found here at google. This all works nice in the online version, however there seems to be a problem with the development server. Some solutions try to configure the local version as well, still that did not work for me. I decided to look for a servlet that serves static files.

That is it, now you can test your stuff locally and all your scripts, images, styles are loaded by your application. Of course you have to remove this servlet before uploading your application. Hope it helps people with their local debugging of jquery scripts or other javascript things.

To get the development environment working the plugin also requires access to the unzipped SDK as packaged by Google. The plugin tries to help you set this up (“gae:unpack”) but that failed for me. I was able to get stuff working by manually unzipping the SDK artifact downloaded directly from Google to the following directory: ~/.m2/repository/com/google/appengine/appengine-java-sdk/1.3.0/appengine-java-sdk-1.3.0

You’ll also notice in our appengine-web.xml that we substitute in our application name. By default this comes from the properties section of the pom.xml file (line 11).

3. Adds a new gae:start goal. This is identical to gae:run, except that it does not automatically execute the package phase before starting the server. The intent is to use this in a project POM, but it can also be used to start the server quickly when you don't want to rebuild the project. Right now I just copied the RunGoal completely, except for the '@execute phase="package"' declaration, because I didn't want to introduce conflicts in case someone else is editing RunGoal.java. A better thing to do in the future would be to have RunGoal extend StartGoal with an empty class that just adds the '@execute phase="package"' annotation (it can't be the other way around, with StartGoal extending Run goal, because there's no way to override a mojo with an @execute annotation with one that doesn't have it---it gets inherited automatically).

The idea behind all of these changes is to make it so that you can use the maven-gae-plugin in a project's POM for automated integration tests. The gae:start goal automatically binds to the pre-integration-test phase by default, and the gae:stop goal binds to post-integration-test. This is most useful when combined with something like the maven-failsafe-plugin and a functional testing library like HtmlUnit that you can use to simulate web requests. It would also be useful for Selenium testing.

An Entity is an individual record that gets stored and retrieved from the datastore. The Kind is the unique string identifier of the type of entity. For example, "Joe" is an Entity with age=42, dob=10-12-2000, and Kind "Person".

Please support the ability to server static files from the runtime based on regex patterns similar to the current Python runtime. Currently the only way to simulate this functionality is with a servlet. This is not ideal peformance, as evidenced by existing special handling of static files. The most compelling use case is for versioning static files with the app's version ID so that browsers can maximally cache static files without experiencing stale caches later when the app is updated.

queue.add(     DatastoreServiceFactory.getDatastoreService().getCurrentTransaction(),     TaskOptions.Builder.url("/path/to/my/worker"));

You have three options at the moment, when it comes to a 'multi-tenant' app such as you describe: You can have a single app that your customers add to their domains. Your app will have a single datastore, but you can use the Host header to determine which customer is accessing the app, and segregate the datastore entries based on that. Easy to deploy and upgrade Easy for customers to install Users have to have Google accounts, not Apps accounts, to log in. You can deploy a fresh app instance for each customer. Harder to deploy and upgrade More customer involvement required to install Provides firm separation of data Users can log in with their Apps credentials You can work with Google to create a new Apps Marketplace app All the benefits of point 1 and 2, above Requires Google involvement No certain release date yet

You should never store a password in plain text. Use a ir-reversable data hashing algorithm, like sha or md5 Here is how you can create a hash in python: from hashlib import sha256from random import randomrandom_key = random()sha256('%s%s%s'%('YOUR SECRET KEY',random_key,password)) You should also store the random key and hash the user supplied password similarly.

It would need to be something like Transaction txn = dataStore.beginTransaction(); Key key = dataStore.put(txn, question); Entity a1 = new Entity("Answer", key); Entity a2 = new Entity("Answer", key); dataStore.put(txn, a1); dataStore.put(txn, a2); txn.commit();

You create a java project only with the shared files (NOT a GAE project). You DON'T copy any libraries into the lib folder. You should add the required third-party components into your main project. Then you rightclick on the shared project's *source* folder, go to "Build Path -> Remove from build path" item in the context menu. You may, by the way, rename the source folder, so that it has a name different from the default source folder name. Next you rightclick on the main project's folder, go to "Build Path -> Link source".