Group items tagged

"Faunus is a property graph analytics engine based on Hadoop. A breadth-first version of the graph traversal language Gremlin operates on a vertex-centric property graph data structure. Faunus can be extended with new operations written using MapReduce and Blueprints."

If you've been tasked with the job of maintaining large and complex Hadoop clusters, or are about to be, this book is a must. You'll learn the particulars of Hadoop operations, from planning, installing, and configuring the system to providing ongoing maintenance.

"Welcome to the VirtualBox® Images project website. This project provides virtual machines for Sun XVM VirtualBox® sporting several free and/or open-source operating systems, such as GNU/Linux or Free/Net/OpenBSD for testing, security and/or entertainment purposes."

"You may have noticed some new OS releases: CoreOS (August 2013), Project Atomic (April 2014), and more recently Snappy Ubuntu Core (December 2014). These are all minimalist OSs designed to host Docker applications and simplify your infrastructure.  If you are interested in the distinguishing features of each and how they differ from traditional Linux distributions, read on."

"Breaking distributed systems so you don't have to. Jepsen is a Clojure library. A test is a Clojure program which uses the Jepsen library to set up a distributed system, run a bunch of operations against that system, and verify that the history of those operations makes sense. Jepsen has been used to verify everything from eventually-consistent commutative databases to linearizable coordination systems to distributed task schedulers. It can also generate graphs of performance and availability, helping you characterize how a system responds to different faults. See jepsen.io for examples of the sorts of analyses you can carry out with Jepsen."

"Amazon EC2 Container Service (ECS) is a highly scalable, high performance container management service that supports Docker containers and allows you to easily run applications on a managed cluster of Amazon EC2 instances. Amazon ECS eliminates the need for you to install, operate, and scale your own cluster management infrastructure. With simple API calls, you can launch and stop container-enabled applications, query the complete state of your cluster, and access many familiar features like security groups, Elastic Load Balancing, EBS volumes, and IAM roles. You can use Amazon ECS to schedule the placement of containers across your cluster based on your resource needs and availability requirements. You can also integrate your own scheduler or third-party schedulers to meet business or application specific requirements."

"OpenStack and Docker will continue to dominate the open source cloud discussion. But Docker may prove to gain the most as it is also breeding a diverse ecosystem of open source projects. OpenStack is primarily contained (no pun intended) to the development of its own cloud operating system. It does integrate with OpenShift, for example, but for the most part the different groups within OpenStack do the lion's share of development. Docker's influence is such that it is affecting the overall open source community. The projects that are closely tied to Docker, such as Ansible, will continue to grow as developers seek tools to use with the fast growing container technology."

DRBL (Diskless Remote Boot in Linux) is a solution to managing the deployment of the GNU/Linux operating system across many clients. Imagine the time required to install GNU/Linux on 40, 30, or even 10 client machines individually! DRBL allows for the configuration all of your client computers by installing just one server machine. DRBL provides a diskless or systemless environment for client machines. It works on Debian, Ubuntu, Mandriva, Red Hat, Fedora, CentOS and SuSE. DRBL uses distributed hardware resources and makes it possible for clients to fully access local hardware.

"Here You can find ready to download virtual machines with preinstalled different Open Source Unix and Linux operating systems. Each virtual machine on this website has been prepared using VMware Player technology."

"VirtualBoxImages.com is the worlds leading source for pre-installed virtualbox virtual computers. No other single source provides more virtual computer choices. This is the mother lode of VirtualBox virtualization software. VDI images of pre-installed "Open Source" Operating System distros. Pre-installed virtualbox images ready for you to explore and play with. VirtualBox runs on SunOS, OpenSolaris, Mac OS X, Windows, OS/2 and Linux. It's a computer inside your computer."

"In 2010 we led the seed round for Docker (formerly known as dotCloud) for one simple reason: devops means that the way applications are packaged, deployed, and run is fundamentally changing (though Docker's business model has evolved since its early days as a PaaS vendor, the fundamental premise is the same).  Rather than requiring custom configurations and painstaking management, Docker "containerizes" applications components such that every container is lightweight and behaves consistently.  Applications and their underlying components can be programmatically deployed, managed and moved on ever-changing cloud infrastructure without a hint of operating system or hardware configuration.  In a pre-Docker world, companies with tremendous and evolving application demands looked to virtualization as a way of abstracting their infrastructure, but paid a tax in dollars and performance for doing so. In the future we think of Docker will take the mantle as the VMware of the devops world, with containers as the ultimate devops platform."

"Flynn is two things: A "distribution" of components that out-of-the-box gives companies a reasonable starting point for an internal "platform" for running their applications and services. The banner for a collection of independent projects that together make up a toolkit or loose framework for building distributed systems. Flynn is both a whole and many parts, depending on what is most useful for you. The common goal is to democratize years of experience and best practices in building distributed systems. It is the software layer between operators and developers that makes both their lives easier. Unlike most PaaS's, Flynn can run stateful services as well as 12 factor apps. This includes built-in database appliances (just Postgres to start). Flynn is modular so users can easily modify, upgrade, and replace components. "

"Fabric is a Python (2.5-2.7) library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks. It provides a basic suite of operations for executing local or remote shell commands (normally or via sudo) and uploading/downloading files, as well as auxiliary functionality such as prompting the running user for input, or aborting execution."

"Internet Routing: It's a program (well, a daemon, as you are going to discover in a moment) which works as a dynamic router in an Internet type network (that is, in a network running either the IPv4 or the IPv6 protocol). Routers are devices which forward packets between interconnected networks in order to allow hosts not connected directly to the same local area network to communicate with each other. They also communicate with the other routers in the Internet to discover the topology of the network which allows them to find optimal (in terms of some metric) rules for forwarding of packets (which are called routing tables) and to adapt themselves to the changing conditions such as outages of network links, building of new connections and so on. Most of these routers are costly dedicated devices running obscure firmware which is hard to configure and not open to any changes (on the other hand, their special hardware design allows them to keep up with lots of high-speed network interfaces, better than general-purpose computer does). Fortunately, most operating systems of the UNIX family allow an ordinary computer to act as a router and forward packets belonging to the other hosts, but only according to a statically configured table."

"Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions. Configuration files describe to Terraform the components needed to run a single application or your entire datacenter. Terraform generates an execution plan describing what it will do to reach the desired state, and then executes it to build the described infrastructure. As the configuration changes, Terraform is able to determine what changed and create incremental execution plans which can be applied. The infrastructure Terraform can manage includes low-level components such as compute instances, storage, and networking, as well as high-level components such as DNS entries, SaaS features, etc. The key features of Terraform are: Infrastructure as Code: Infrastructure is described using a high-level configuration syntax. This allows a blueprint of your datacenter to be versioned and treated as you would any other code. Additionally, infrastructure can be shared and re-used. Execution Plans: Terraform has a "planning" step where it generates an execution plan. The execution plan shows what Terraform will do when you call apply. This lets you avoid any surprises when Terraform manipulates infrastructure. Resource Graph: Terraform builds a graph of all your resources, and parallelizes the creation and modification of any non-dependent resources. Because of this, Terraform builds infrastructure as efficiently as possible, and operators get insight into dependencies in their infrastructure. Change Automation: Complex changesets can be applied to your infrastructure with minimal human interaction. With the previously mentioned execution plan and resource graph, you know exactly what Terraform will change and in what order, avoiding many possible human errors."

"Introduction In the modern era, software is commonly delivered as a service: called web apps, or software-as-a-service. The twelve-factor app is a methodology for building software-as-a-service apps that: Use declarative formats for setup automation, to minimize time and cost for new developers joining the project; Have a clean contract with the underlying operating system, offering maximum portability between execution environments; Are suitable for deployment on modern cloud platforms, obviating the need for servers and systems administration; Minimize divergence between development and production, enabling continuous deployment for maximum agility; And can scale up without significant changes to tooling, architecture, or development practices. The twelve-factor methodology can be applied to apps written in any programming language, and which use any combination of backing services (database, queue, memory cache, etc). Background The contributors to this document have been directly involved in the development and deployment of hundreds of apps, and indirectly witnessed the development, operation, and scaling of hundreds of thousands of apps via our work on the Heroku platform. This document synthesizes all of our experience and observations on a wide variety of software-as-a-service apps in the wild. It is a triangulation on ideal practices for app development, paying particular attention to the dynamics of the organic growth of an app over time, the dynamics of collaboration between developers working on the app's codebase, and avoiding the cost of software erosion. Our motivation is to raise awareness of some systemic problems we've seen in modern application development, to provide a shared vocabulary for discussing those problems, and to offer a set of broad conceptual solutions to those problems with accompanying terminology. The format is inspired by Martin Fowler's books Patterns of Enterprise Application Architecture and Refactoring. Who should

Bueno. Eso. Compartí el que me di cuenta que puso antes Pablo en vez del original por error, pero la idea entre ambos, si la obviedad es tolerable, es idéntica :) Está muy bien estructurado en cuanto que cada factor depende de los demás a la vez que los promueve. Permite un enfoque general que incluye prácticas de arquitectura - y de armado cotidiano de productos - que posibilitan llegar donde yo entiendo - según me voy enterando - que es el lugar a donde llegar. Sin embargo, creo que ni éste departamento en sus sistemas más nuevos cumple todos y cada uno de aquellos factores. Esto, lejos de ser una crítica, es una invitación para que revisemos si es el único método posible - cosa improbabilísima - o el mejor método - también bastante improblable - a seguir. Lo que sí sostengo como un absoluto - quien no lo haría - es que es un método practicable. Mi aporte mínimo es defenderlo como uno bueno.

"Splunk Enterprise is the industry-leading platform for operational intelligence. Collect and index any machine data from virtually any source in real time. Search, monitor, analyze and visualize your data to gain new insights and intelligence. Index everything for deep visibility, forensics and troubleshooting. Work smarter as you and your team share searches and add knowledge specific to your organization. Create ad hoc reports to identify trends or prove compliance controls. Create interactive dashboards to monitor for security incidents, service levels and other key performance metrics. Analyze user transactions, customer behavior, machine behavior, security threats and fraudulent activity, all in real time."

"developers own the containers, operations own the platform" ftw!

"What is Aurora? Platforms like Twitter operate across tens of thousands of machines, with hundreds of engineers deploying software daily. In this type of environment, automation is critical. Aurora is software that keeps services running in the face of many types of failure, and provides engineers a convenient, automated way to create and update these services. To accomplish this, Aurora leverages the Apache Mesos cluster manager, which provides information about the state of the cluster. Aurora uses that knowledge to make scheduling decisions. For example, when a machine experiences failure Aurora automatically reschedules those previously-running services onto a healthy machine in order to keep them running."