Group items tagged

"HDF (Hierarchical Data Format) technologies are relevant when the data challenges being faced push the limits of what can be addressed by traditional database systems, XML documents, or in-house data formats. Leveraging the powerful HDF products and the expertise of The HDF Group, organizations realize substantial cost savings while solving challenges that seemed intractable using other data management technologies. Many HDF adopters have very large datasets, very fast access requirements, or very complex datasets. Others turn to HDF because it allows them to easily share data across a wide variety of computational platforms using applications written in different programming languages. Some use HDF to take advantage of the many open-source and commercial tools that understand HDF. Similar to XML documents, HDF files are self-describing and allow users to specify complex data relationships and dependencies. In contrast to XML documents, HDF files can contain binary data (in many representations) and allow direct access to parts of the file without first parsing the entire contents. HDF, not surprisingly, allows hierarchical data objects to be expressed in a very natural manner, in contrast to the tables of relational database. Whereas relational databases support tables, HDF supports n-dimensional datasets and each element in the dataset may itself be a complex object. Relational databases offer excellent support for queries based on field matching, but are not well-suited for sequentially processing all records in the database or for subsetting the data based on coordinate-style lookup."

Data.js is a data representation framework for Javascript. It is being developed in the context of Substance, a web-based document authoring and publishing engine. It took some inspiration from various existing libraries such as the Google Visualization API or Underscore.js.  You can report bugs and discuss features on the GitHub issues page, on Freenode IRC in the #_substance chann el, post questions to the Google Group, or send tweets to @_substance. With Data.js you can: Model your domain data using a simple graph-based object model that can be serialized to JSON. Traverse your graph, including relationships using a simple API. Manipulate and query data on the client (browser) or on the server (Node.js) by using exactly the same API. 

"Saddle is a data manipulation library for Scala that provides array-backed, indexed, one- and two-dimensional data structures that are judiciously specialized on JVM primitives to avoid the overhead of boxing and unboxing. Saddle offers vectorized numerical calculations, automatic alignment of data along indices, robustness to missing (N/A) values, and facilities for I/O. Saddle draws inspiration from several sources, among them the R programming language & statistical environment, the numpy and pandas Python libraries, and the Scala collections library."

"Bandicoot is an open source programming system with a new set-based programming language, persistency capabilities, and run-time environment. The language is similar to general purpose programming languages where you write functions/methods and access data through variables. Though, in Bandicoot, you always manipulate data in sets using a small set-based algebra (the relational algebra)." "Here are the main features:   - functions are automatically exposed via HTTP using CSV for data, e.g. /List, /Append  - supports persistency via global variables (with transactions and ACID)  - can run on multiple computers to scale up the read throughput  - built in operators from the relational algebra with a simple syntax, e.g. "+" (union), "-" (minus)  - small binary ~100KB"

"In incoming Play2.1 version, a huge re-thinking has been done about JSON API provided by Play2.0.x which provides some great features but is clearly just the tip of the iceberg… Here is a first presentation of those evolutions aimed at unleashing your JSON usage in Play2 and revealing new forms of manipulation of web dataflows from/to external data systems. A usecase of this is manipulating DB structures directly using Json without any class models for document oriented structures such as MongoDB"

"Datomic is a distributed database designed to enable scalable, flexible and intelligent applications, running on next-generation cloud architectures. It does this by: Bringing declarative data manipulation into the application, and the data with it Getting time, process and perception right Process (writes) require coordination Perception (reads) require none The past doesn't change Leveraging immutability, and a sound model of state Datomic has: ACID Transactions Joins A sound data model A logical query language - Datalog Thus, Datomic avoids the compromises and losses of many NoSQL solutions. In addition, it offers flexibility and power over the traditional model in supporting: Hierarchy Multi-valued attributes Minimal schema Reliable operation on unreliable, ephemeral cloud instances Time Datomic avoids manual caching and replication, complex configuration, sharding (automatic or manual), logging, locking, latching and disk management of traditional servers."

"Unlike most client libraries dealing with HAL REST APIs, Hybind provides a high-level approach similar to what Object Relational Mapping (ORM) frameworks are for databases. When using Spring Data REST in the server, it is amazing how the amount of code to write is reduced to a minimum. However, a significant amount of repeated boilerplate is still required in the JavaScript client to manipulate the resources and map them to the client-side model. That's why this library exists. It enriches plain JavaScript objects with a convenient API so that performing REST requests is as easy as calling methods directly on the model objects. It is optimized for Spring Data REST, but should work with other HAL APIs following similar conventions."

"Figaro is a probabilistic programming language that supports development of very rich probabilistic models and provides reasoning algorithms that can be applied to models to draw useful conclusions from evidence. Both model representation and reasoning algorithm development can be challenging tasks. Figaro makes it possible to express probabilistic models using the power of programming languages, giving the modeler the expressive tools to create a wide variety of models. Figaro comes with a number of built-in reasoning algorithms that can be applied automatically to new models. In addition, Figaro models are data structures in the Scala programming language, which is interoperable with Java, and can be constructed, manipulated, and used directly within any Scala or Java program."

"CockroachDB is a distributed SQL database built on a transactional and strongly-consistent key-value store. It scales horizontally; survives disk, machine, rack, and even datacenter failures with minimal latency disruption and no manual intervention; supports strongly-consistent ACID transactions; and provides a familiar SQL API for structuring, manipulating, and querying data. "

"q is a command line tool that allows direct execution of SQL-like queries on CSVs/TSVs (and any other tabular text files)."