Group items tagged

Una descripción del stack server side de twitter bastante profunda y útil.

"Abstract algebra for Scala. This code is targeted at building aggregation systems (via Scalding or Storm). It was originally developed as part of Scalding's Matrix API, where Matrices had values which are elements of Monoids, Groups, or Rings. Subsequently, it was clear that the code had broader application within Scalding and on other projects within Twitter."

"Zipkin is a distributed tracing system that we created to help us gather timing data for all the disparate services involved in managing a request to the Twitter API."

"What is Aurora? Platforms like Twitter operate across tens of thousands of machines, with hundreds of engineers deploying software daily. In this type of environment, automation is critical. Aurora is software that keeps services running in the face of many types of failure, and provides engineers a convenient, automated way to create and update these services. To accomplish this, Aurora leverages the Apache Mesos cluster manager, which provides information about the state of the cluster. Aurora uses that knowledge to make scheduling decisions. For example, when a machine experiences failure Aurora automatically reschedules those previously-running services onto a healthy machine in order to keep them running."

Importantísimo de ver... sobre todo para diferenciar bien entre un simple publicador de recursos, estilo openshift, de un verdadero administrador de recursos distribuidos (léase scheduler), como mesos.

A collection of MapReduce tasks translated (from Pig, Hive, MapReduce streaming, etc.) into Scalding for comparison.

"Scalding is a Scala library that makes it easy to specify Hadoop MapReduce jobs. Scalding is built on top of Cascading, a Java library that abstracts away low-level Hadoop details. Scalding is comparable to Pig, but offers tight integration with Scala, bringing advantages of Scala to your MapReduce jobs."

Simple and flexible HTML, CSS, and Javascript for popular user interface components and interactions.

"Efficient, scalable, rock-solid, and flexible Java implementation of the Short Messaging Peer to Peer Protocol (SMPP)."

Passport is authentication middleware for Node.js. Extremely flexible and modular, Passport can be unobtrusively dropped in to any Express-based web application. A comprehensive set of strategies support authentication using a username and password, Facebook, Twitter, and more."

Ostrich is a library for servers that makes it easy to:  * load & reload per-environment configuration  * collect runtime statistics (counters, gauges, metrics, and labels)  * report those statistics through a simple web interface (optionally with graphs) or into log files  * interact with the server over HTTP to check build versions or shut it down

"CoreOS is a Linux distribution that has been rearchitected to provide features needed to run modern clustered infrastructure stacks. The strategies and architectures that influence CoreOS allow companies like Google, Facebook and Twitter to run their services at scale with high resilience. We've implemented them correctly so you don't have to endure the slow, learn-as-you-go infrastructure building process. CoreOS can run on your existing hardware or on most cloud providers. Clustering works across platforms, making it easy to migrate parts of your gear over to CoreOS, or to switch cloud providers while running CoreOS."

"Twitter's collection of LZO and Protocol Buffer-related Hadoop, Pig, Hive, and HBase code."

ble Vector collections provides a referentially transparent interface to arrays); and use buffers instead of direct sequence cons

"A great resource (PDF document) about big data, originally posted on CTOvision.com."

"In this section we'll use Finagle to build a very simple HTTP server that is also an HTTP client - an HTTP proxy." Proxy reverso reactivo en un puñado de líneas en scala.