Group items tagged

'm guessing your tests were run locally because the counter in the local datastore does indeed have datastore scope.  The scope of the counter in the prod datastore, however, is parent key + kind.  This is described here: http://code.google.com/appengine/docs/java/datastore/creatinggettinga... If it's important to you that the scope of the generated ids match between dev and and prod please file an issue.

                StringBuffer sb = new StringBuffer();                 KeyRange range = ds.allocateIds("a", 2);                 for (Key key : range) {                         sb.append("\n a " + key.toString());                 }                 range = ds.allocateIds("b", 2);                 for (Key key : range) {                         sb.append("\n b " + key.toString());                 }                 Key parentKey = KeyFactory.createKey("c", 1);                 sb.append("\n c " + parentKey.toString());                 range = ds.allocateIds(parentKey, "d", 2);                 for (Key key : range) {                         sb.append("\n d " + key.toString());                 }                 System.out.println(sb.toString());

The URL I posted earlier in the thread explains it, but here's a little bit more detail: A parent entity plus a kind defines an id-space, so entities with the same parent and the same kind are guaranteed to have unique ids.  For example, if you have an Entity with Parent:A Kind: Person Id: 10 you are guaranteed that no other entity with Parent A and Kind Person will be assigned an Id of 10.  However, an entity with a different Parent and Kind Person or an entity with Parent A and a different Kind _can_ be assigned an Id of 10.  The datastore pre-allocates batches of ids across multiple servers under-the-hood, so you can't make any assumptions about the Id that will get assigned in terms of contiguousness.  The only safe assumption is that the id will be unique for that Parent/Kind combination.

public final class Keyextends java.lang.Objectimplements java.io.Serializable, java.lang.Comparable<Key> The primary key for a datastore entity. A datastore GUID. A Key instance uniquely identifies an entity across all apps, and includes all information necessary to fetch the entity from the datastore with DatastoreService.get(Key). You can create Key objects directly by using KeyFactory.createKey(java.lang.String, long) or getChild(java.lang.String, long). You can also retrieve the Key automatically created when you create a new Entity, or serialize Key objects, or use KeyFactory to convert them to and from websafe String values.

equals public boolean equals(java.lang.Object object) Compares two Key objects by comparing ids, kinds, parent and appIdNamespace. If both keys are assigned names rather than ids, compares names instead of ids. If neither key has an id or a name, the keys are only equal if they reference the same object.

compareTo public int compareTo(Key other) Compares two Key objects. The algorithm proceeds as follows: Turn each Key into an iterator where the first element returned is the top-most ancestor, the next element is the child of the previous element, and so on. The last element will be the Key we started with. Once we have assembled these two iterators (one for 'this' and one for the Key we're comparing to), consume them in parallel, comparing the next element from each iterator. If at any point the comparison of these two elements yields a non-zero result, return that as the result of the overall comparison. If we exhaust the iterator built from 'this' before we exhaust the iterator built from the other Key, we return less than. An example: app1.type1.4.app1.type2.9 < app1.type1.4.app1.type2.9.app1.type3.2 If we exhaust the iterator built from the other Key before we exhaust the iterator built from 'this', we return greater than. An example: app1.type1.4.app1.type2.9.app1.type3.2 > app1.type1.4.app1.type2.9 The relationship between individual Key Keys is performed by comparing app followed by kind followed by id. If both keys are assigned names rather than ids, compares names instead of ids. If neither key has an id or a name we return an arbitrary but consistent result. Assuming all other components are equal, all ids are less than all names.

I have an Entity with an id type of Long, and the id gets filled in after I call makePersistent(). Here is what the code looks like:     GameEntity game = new GameEntity();    log.warning("before makePersistent id is " + game.getId());    pm.makePersistent(game);    log.warning("after makePersistent id is " + game.getId()); Here is a snippet of the GameEntity class: @PersistenceCapable(identityType = IdentityType.APPLICATION)public class GameEntity {    @PrimaryKey    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)    private Long id; And the output shows what you'd expect: WARNING 6428 - before makePersistent id is nullWARNING 6444 - after makePersistent id is 6 UPDATE: It occurred to me belatedly that you might want an actual Key object. You can create that yourself if you have the id: public Key getKey() {    return KeyFactory.createKey(GameEntity.class.getSimpleName(), id);}

Every entity has a key that is unique over all entities in App Engine. A complete key includes several pieces of information, including the application ID, the kind, and an entity ID. (Keys also contain information about entity groups; see Transactions for more information.) An object's key is stored in a field on the instance. You identify the primary key field using the @PrimaryKey annotation. The app can provide the ID portion of the key as a string when the object is created, or it can allow the datastore to generate a numeric ID automatically. The complete key must be unique across all entities in the datastore. In other words, an object must have an ID that is unique across all objects of the same kind and with the same entity group parent (if any). You select the desired behavior of the key using the type of the field and annotations. If the class is used as a "child" class in a relationship, the key field must be of a type capable of representing an entity group parent: either a Key instance, or a Key value encoded as a string. See Transactions for more information on entity groups, and Relationships for more information on relationships. Tip: If the app creates a new object and gives it the same string ID as another object of the same kind (and the same entity group parent), saving the new object overwrites the other object in the datastore. To detect whether a string ID is already in use prior to creating a new object, you can use a transaction to attempt to get an entity with a given ID, then create one if it doesn't exist. See Transactions. There are 4 types of primary key fields:

 * Implements background tasks for * <a href="http://code.google.com/appengine/docs/java/overview.html">Google App * Engine for Java</a>, based on the * <a href="http://code.google.com/appengine/articles/deferred.html">Python 'deferred' * library</a>; simplifies use of the <a href="http://code.google.com/appengine/docs/java/taskqueue/overview.html"> * Task Queue Java API</a> by automatically handling the serialization and * deserializtion of complex task arguments.

Well, in AppEngine your primary key is the key for the object(Entity) and it can either be a generated ID or a unique string your application provides.  "Google App Engine" by Dan Sanderson has a lot of examples on this.  If you don't use a key_name property, then when you save with a put(), an ID is generated, but if you pass in key_name='thisIsMyKey' into the constructor, then you manually set the key for the object.  You can use the method, id_or_name() to return either the object's key name or its ID, which ever one it has and has_id_or_name() would return a boolean about it, and if it's not saved and your not using key_name, then the ID would not exists yet.  Also you can get the Entity(Object) from the datastore by using the get(k) method where k is a key object and the key object has two parts: kind and ID or key_name.  Additionally you can fetch an object from the datastore with get_by_id() and get_by_key_name()

public final class KeyRangeextends java.lang.Objectimplements java.lang.Iterable<Key>, java.io.Serializable Represents a range of unique datastore identifiers from getStart().getId() to getEnd().getId() inclusive. The Keys returned by an instance of this class have been consumed in the datastore's id-space and are guaranteed never to be reused. This class can be used to construct Entities with Keys that have specific id values without fear of the datastore creating new records with those same ids at a later date. This can be helpful as part of a data migration or large bulk upload where you may need to preserve existing ids and relationships between entities. This class is threadsafe but the Iterators returned by iterator() are not.

IDs aren't even unique within a kind; they're unique within a kind and entity group. The reason they're not sequential is that an instance is allocated a large block of IDs, and unused ones aren't assigned to later instances

If you use the RemoteDatastore you have complete control in Java of   what you key is. http://code.google.com/p/remote-datastore/ e.g. this code puts an entity with the id set in your remote datastore   from your local machine:      RemoteDatastore.install();      RemoteDatastore.divert("http://myVersion.latest.myApp.appspot.com/remote-datastore ", "myApp", "myVersion");      DatastoreService service =   DatastoreServiceFactory.getDatastoreService();      Key key = KeyFactory.createKey("MyKindName, 35);      Entity entity1 = new Entity(key);      entity1.setProperty("property1", "hello");      datastore.put(Arrays.asList(entity1, entity2);

You can do this now using the RemoteDatastore Java utility http://code.google.com/p/remote-datastore/ For example, this code runs on your desktop and creates a single   entity in your live datastore:     // divert datastore operations to live application     RemoteDatastore.install();     RemoteDatastore.divert("http://myVersion.latest.myApp.appspot.com/remote-datastore ", "myApp", "myVersion");     // create an entity with a numeric key     Key key = KeyFactory.createKey("MyKindName, 35);     Entity entity1 = new Entity(key);     entity1.setProperty("property1", "hello");     // put entity to the remote datastore     DatastoreService service =   DatastoreServiceFactory.getDatastoreService();     datastore.put(entity1); This also works for bulk puts

this won't be available until 1.3.6. You should be able to do something like this:  - property: __key__    external_name: CityId    export_transform: datastore.Key.id    import_transform: lambda value: datastore.Key.from_path('City', int(value))

I’m trying to use the bulkuploader for a java program but am running into an interesting issue. My PrimaryKey property is a Long, and in java I can explicitly give them id numbers and they show in the data store as “id=xxx”. When I download the data via the appcfg.py I get a reasonably looking data file. If I reupload the same file it actually inserts things into the data store with key “name=xxx” and therefore doubles every one of my entries.

create a custom uploader using the file upload example provided on appengine’s java FAQ.

App Engine’s datastore is schemaless. That is – it is possible to have Entities of the same Kind with completely different sets of properties. Most of the time, this is a good thing. MySQL, for instance, requires a table lock to do a schema update. By being schema free, migrations can happen lazily, and application developers can check at runtime for whether a Property exists on a given Entity, then create or set the value as needed. But there are times when this isn’t sufficient. One use case is if we want to change a default value on Entities and grandfather older Entities to the new default value, but we also want the default value to possibly be null.

Tip: If the app creates a new object and gives it the same string ID as another object of the same kind (and the same entity group parent), saving the new object overwrites the other object in the datastore. To detect whether a string ID is already in use prior to creating a new object, you can use a transaction to attempt to get an entity with a given ID, then create one if it doesn't exist. See Transactions.

There are 4 types of primary key fields:

disabling session id in code will not stop the cookie from saving session id. When you enable session in app engine xml file than you are telling the framework that you don't want to save sessions in cookies (client side), instead you want to save them on server i.e. using _ah_session. Cookies JSession id is default behavior of Java session mgmt. And it is nothing to do with APP engine framework.*

If you use the RemoteDatastore you have complete control in Java of   what you key is. http://code.google.com/p/remote-datastore/ e.g. this code puts an entity with the id set in your remote datastore   from your local machine:      RemoteDatastore.install();      RemoteDatastore.divert("http://myVersion.latest.myApp.appspot.com/remote-datastore ", "myApp", "myVersion");      DatastoreService service =   DatastoreServiceFactory.getDatastoreService();      Key key = KeyFactory.createKey("MyKindName, 35);      Entity entity1 = new Entity(key);      entity1.setProperty("property1", "hello");      datastore.put(Arrays.asList(entity1, entity2);

It was one of the other ways to do it, I just wanted to avoid more code in my architecture just to do this (MVP has the disadvantage of added boilerplate for simple tasks like this one). Anyway I found a workaroud, passing the URL in an invisible div in my page and then reading and populating another field in my page using GWT RootPanel.get(id), using different ids for various parts of the page.

You might want to instead use datastore's built in ability to   efficiently create unique long ids and then when sending them to a   client "muddle" them using a reversible hash function so they don't   appear predictable.  You will find that using long ids results in   shorter Keys which saves space in the datastore. You can only guarantee a unique value in the datastore by creating an   Entity with the value in its key.  This does not need to be your   "main" entity - just a special UniqueName type.  Then you can

Now you can choose to put this in an existing version of your applications config.ru. Or you can choose to put it in a new version (ie. version just for bulk upload/download without your application code). Either way once a version with the RemoteApiServlet is uploaded run this command to download your data: bulkloader.py --dump --app_id="application id" --url="url_to_remote_api_servlet" --filename="file to download data to" Here is an example from one of my apps: bulkloader.py --dump --app_id=jsm277 --url=http://bulkmove.latest.jsm277.appspot.com/remote_api --filename=test_download_data This command will download every object in your applications Datastore. If you only want to download the objects of one kind then simple add the --kind= option. Another example: bulkloader.py --dump --app_id=jsm277 --url=http://bulkmove.latest.jsm277.appspot.com/remote_api --filename=test_download_data --kind=Posts In order to restore the data that you have just downloaded use this command: bulkloader.py --restore --url="url_to_remote_api_servlet" --filename="file name from dump command" --app_id="application id" Here is an example command: bulkloader.py --restore --url=http://bulkmove.latest.railsturbinetest.appspot.com/remote_api --filename=test_download_data --app_id=railsturbinetest There are several important thing to keep in mind when restoring the Datastore objectes. The restore command simple restores whatever is in the filename that you provide. So it does not matter if you dump your entire Datastore or just one kind you use the same command to restore them both. You can restore Datastore objects to a different application then the one that they were downloaded from. However, is important to keep in mind that the object's key is used to restore each object. So if there already esists an object in the Datastore with the same key as an object that is being restored the object in the Datastore will be overwritten The bulkloader.py is a good tool for moving data between applications on the Google App Engine and for locally backing up your data. However, it is not good for data conversion or manipulation in anyway since the data is stored in a binary form. So happy data moving.

How do I handle multipart form data? or How do I handle file uploads to my app? You can obtain the uploaded file data from a multipart form post using classes from the Apache Commons FileUpload package. Specifically you may want to use FileItemStream, FileItermIterator and ServletFileUpload as illustrated below. If you see a java.lang.NoClassDefFoundError after starting your application, make sure that the Apache Commons FileUpload JAR file has been copied to your war/WEB-INF/lib directory and added to your build path. import org.apache.commons.fileupload.FileItemStream;import org.apache.commons.fileupload.FileItemIterator;import org.apache.commons.fileupload.servlet.ServletFileUpload;import java.io.InputStream;import java.io.IOException;import java.util.logging.Logger;import javax.servlet.ServletException;import javax.servlet.http.HttpServlet;import javax.servlet.http.HttpServletRequest;import javax.servlet.http.HttpServletResponse;public class FileUpload extends HttpServlet {  private static final Logger log =      Logger.getLogger(FileUpload.class.getName());  public void doPost(HttpServletRequest req, HttpServletResponse res)      throws ServletException, IOException {    try {      ServletFileUpload upload = new ServletFileUpload();      res.setContentType("text/plain");      FileItemIterator iterator = upload.getItemIterator(req);      while (iterator.hasNext()) {        FileItemStream item = iterator.next();        InputStream stream = item.openStream();        if (item.isFormField()) {          log.warning("Got a form field: " + item.getFieldName());        } else {          log.warning("Got an uploaded file: " + item.getFieldName() +                      ", name = " + item.getName());          // You now have the filename (item.getName() and the          // contents (which you can read from stream).  Here we just          // print them back out to the servlet output stream, but you          // will probably want to do something more interesting (for          // example, wrap them in a Blob and commit them to the          // datastore).          int len;          byte[] buffer = new

One option is to create a model User in your own application, one of the things it contains would be the google user object.  In this case, you have your own ID generated by each instance of your Model, for each User. EJ: class User(db.Model):         googleUser = db.UserProperty()