Here is the approach I would probably take unless you have many hundreds of child tasks. I would keep a list of keys of all children of each project/task. (You can then do a db.get and get all immediate children with single call) and each child has a reference to it's parent. In addition in each item keep a list of all parents keys back to the root. That way you can search for all children in part of the tree/subtree (This approach is especially useful if you have multiple child types) as back ref sets are kind specific. Keep a  running count of complete children at each level, rather than trying to calculate the current state in a single big function,  (keep these subsidiary values up to date through tasks)

Every entity has a key that is unique over all entities in App Engine. A complete key includes several pieces of information, including the application ID, the kind, and an entity ID. (Keys also contain information about entity groups; see Transactions for more information.) An object's key is stored in a field on the instance. You identify the primary key field using the @PrimaryKey annotation. The app can provide the ID portion of the key as a string when the object is created, or it can allow the datastore to generate a numeric ID automatically. The complete key must be unique across all entities in the datastore. In other words, an object must have an ID that is unique across all objects of the same kind and with the same entity group parent (if any). You select the desired behavior of the key using the type of the field and annotations. If the class is used as a "child" class in a relationship, the key field must be of a type capable of representing an entity group parent: either a Key instance, or a Key value encoded as a string. See Transactions for more information on entity groups, and Relationships for more information on relationships. Tip: If the app creates a new object and gives it the same string ID as another object of the same kind (and the same entity group parent), saving the new object overwrites the other object in the datastore. To detect whether a string ID is already in use prior to creating a new object, you can use a transaction to attempt to get an entity with a given ID, then create one if it doesn't exist. See Transactions. There are 4 types of primary key fields:

public final class Keyextends java.lang.Objectimplements java.io.Serializable, java.lang.Comparable<Key> The primary key for a datastore entity. A datastore GUID. A Key instance uniquely identifies an entity across all apps, and includes all information necessary to fetch the entity from the datastore with DatastoreService.get(Key). You can create Key objects directly by using KeyFactory.createKey(java.lang.String, long) or getChild(java.lang.String, long). You can also retrieve the Key automatically created when you create a new Entity, or serialize Key objects, or use KeyFactory to convert them to and from websafe String values.

equals public boolean equals(java.lang.Object object) Compares two Key objects by comparing ids, kinds, parent and appIdNamespace. If both keys are assigned names rather than ids, compares names instead of ids. If neither key has an id or a name, the keys are only equal if they reference the same object.

compareTo public int compareTo(Key other) Compares two Key objects. The algorithm proceeds as follows: Turn each Key into an iterator where the first element returned is the top-most ancestor, the next element is the child of the previous element, and so on. The last element will be the Key we started with. Once we have assembled these two iterators (one for 'this' and one for the Key we're comparing to), consume them in parallel, comparing the next element from each iterator. If at any point the comparison of these two elements yields a non-zero result, return that as the result of the overall comparison. If we exhaust the iterator built from 'this' before we exhaust the iterator built from the other Key, we return less than. An example: app1.type1.4.app1.type2.9 < app1.type1.4.app1.type2.9.app1.type3.2 If we exhaust the iterator built from the other Key before we exhaust the iterator built from 'this', we return greater than. An example: app1.type1.4.app1.type2.9.app1.type3.2 > app1.type1.4.app1.type2.9 The relationship between individual Key Keys is performed by comparing app followed by kind followed by id. If both keys are assigned names rather than ids, compares names instead of ids. If neither key has an id or a name we return an arbitrary but consistent result. Assuming all other components are equal, all ids are less than all names.

Entity is the fundamental unit of data storage. It has an immutable identifier (contained in the Key) object, a reference to an optional parent Entity, a kind (represented as an arbitrary string), and a set of zero or more typed properties.

setProperty public void setProperty(java.lang.String propertyName, java.lang.Object value) Sets the property named, propertyName, to value. As the value is stored in the datastore, it is converted to the datastore's native type. This may include widening, such as converting a Short to a Long. All Collections are prone to losing their sort order and their original types as they are stored in the datastore. For example, a TreeSet may be returned as a List from getProperty(java.lang.String), with an arbitrary re-ordering of elements. Overrides any existing value for this property, whether indexed or unindexed. Note that Blob and Text property values are never indexed by the built-in single property indexes. To store other types without being indexed, use #setUnindexedProperty. Parameters:value - may be one of the supported datatypes, a heterogenous Collection of one of the supported datatypes, or an UnindexedValue wrapping one of the supported datatypes. Throws: java.lang.IllegalArgumentException - If the value is not of a type that the data store supports.

public final class KeyRangeextends java.lang.Objectimplements java.lang.Iterable<Key>, java.io.Serializable Represents a range of unique datastore identifiers from getStart().getId() to getEnd().getId() inclusive. The Keys returned by an instance of this class have been consumed in the datastore's id-space and are guaranteed never to be reused. This class can be used to construct Entities with Keys that have specific id values without fear of the datastore creating new records with those same ids at a later date. This can be helpful as part of a data migration or large bulk upload where you may need to preserve existing ids and relationships between entities. This class is threadsafe but the Iterators returned by iterator() are not.

the necessary changes to appengine-web.xml to get Sitebricks to run in development mode:         <sessions-enabled>true</sessions-enabled>         <system-properties>                 <property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>                 <property name="mvel2.disable.jit" value="true"/>         </system-properties>

As I understand it, the process of performing a single fetch (call to get())  from the dastastore using a key basically involves finding the host housing the entity, opening a socket, fetching the data, and then cleaning up the connection.  So to fetch something like 30 entities from the datastore, you're repeating the process 30 times over in serial, each time incurring whatever overhead is involved.  I also read that if you perform bulk fetches, (ie passing multiple keys at once) you can eliminate a great deal of that overhead.  In one of the videos I watched from Google I/0 2009, the presenter (whose name I forget - d'oh) said that performing a bulk fetch actually performs the fetches in parallel from the data store and you shoudl see requests noticeably faster.

the necessary changes to appengine-web.xml to get Sitebricks to run in development mode:         <sessions-enabled>true</sessions-enabled>         <system-properties>                 <property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>                 <property name="mvel2.disable.jit" value="true"/>         </system-properties>

the necessary changes to appengine-web.xml to get Sitebricks to run in development mode:         <sessions-enabled>true</sessions-enabled>         <system-properties>                 <property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>                 <property name="mvel2.disable.jit" value="true"/>         </system-properties>

We had a GaeFlashCache built specifically for appengine. Try doing: bind(FlashCache.class).to(GaeFlashCache.class).in(Singleton.class);

The next problem that arises is that if the database has grown too big, then the csv generation will be slow(as it has to jump a lot of times). So the next idea is , instead of generating all cursors in the main cgi , the main cgi generates only first few set of page links, and then triggers a task queue entry to continue from there. By the time the user would have read the first page and clicks on "next page", the task would have generated few more entries. As the set of cursors for pagination is in memcache, you can use it even in your first page, if you are using AJAX (guessing, not that sure.)