Group items tagged

"Consul has multiple components, but as a whole, it is a tool for discovering and configuring services in your infrastructure. It provides several key features: Service Discovery: Clients of Consul can provide a service, such as api or mysql, and other clients can use Consul to discover providers of a given service. Using either DNS or HTTP, applications can easily find the services they depend upon. Health Checking: Consul clients can provide any number of health checks, either associated with a given service ("is the webserver returning 200 OK"), or with the local node ("is memory utilization below 90%"). This information can be used by an operator to monitor cluster health, and it is used by the service discovery components to route traffic away from unhealthy hosts. Key/Value Store: Applications can make use of Consul's hierarchical key/value store for any number of purposes including: dynamic configuration, feature flagging, coordination, leader election, etc. The simple HTTP API makes it easy to use. Multi Datacenter: Consul supports multiple datacenters out of the box. This means users of Consul do not have to worry about building additional layers of abstraction to grow to multiple regions. Consul is designed to be friendly to both the DevOps community and application developers, making it perfect for modern, elastic infrastructures."

"NIST recently published its four-volume SP800-63b Digital Identity Guidelines. Among other things, it makes three important suggestions when it comes to passwords: Stop it with the annoying password complexity rules. They make passwords harder to remember. They increase errors because artificially complex passwords are harder to type in. And they don't help that much. It's better to allow people to use pass phrases. Stop it with password expiration. That was an old idea for an old way we used computers. Today, don't make people change their passwords unless there's indication of compromise. Let people use password managers. This is how we deal with all the passwords we need."

"Leaflet is a modern open-source JavaScript library for mobile-friendly interactive maps. It is developed by Vladimir Agafonkin with a team of dedicated contributors. Weighing just about 31 KB of JS, it has all the features most developers ever need for online maps. Leaflet is designed with simplicity, performance and usability in mind. It works efficiently across all major desktop and mobile platforms out of the box, taking advantage of HTML5 and CSS3 on modern browsers while still being accessible on older ones. It can be extended with many plugins, has a beautiful, easy to use and well-documented API and a simple, readable source code that is a joy to contribute to."

"Configuration library for JVM languages. Overview implemented in plain Java with no dependencies extensive test coverage supports files in three formats: Java properties, JSON, and a human-friendly JSON superset merges multiple files across all formats can load from files, URLs, or classpath good support for "nesting" (treat any subtree of the config the same as the whole config) users can override the config with Java system properties, java -Dmyapp.foo.bar=10 supports configuring an app, with its framework and libraries, all from a single file such as application.conf parses duration and size settings, "512k" or "10 seconds" converts types, so if you ask for a boolean and the value is the string "yes", or you ask for a float and the value is an int, it will figure it out. JSON superset features: comments includes substitutions ("foo" : ${bar}, "foo" : Hello ${who}) properties-like notation (a.b=c) less noisy, more lenient syntax substitute environment variables This library limits itself to config files. If you want to load config from a database or something, you would need to write some custom code. The library has nice support for merging configurations so if you build one from a custom source it's easy to merge it in."

"OpenStack and Docker will continue to dominate the open source cloud discussion. But Docker may prove to gain the most as it is also breeding a diverse ecosystem of open source projects. OpenStack is primarily contained (no pun intended) to the development of its own cloud operating system. It does integrate with OpenShift, for example, but for the most part the different groups within OpenStack do the lion's share of development. Docker's influence is such that it is affecting the overall open source community. The projects that are closely tied to Docker, such as Ansible, will continue to grow as developers seek tools to use with the fast growing container technology."

The ss command is used to show socket statistics. It can display stats for PACKET sockets, TCP sockets, UDP sockets, DCCP sockets, RAW sockets, Unix domain sockets, and much more. It allows showing information similar to netstat command. It can display more TCP and state information than other tools. It is a new, incredibly useful and faster (as compare to netstat) tool for tracking TCP connections and sockets. SS can provide information about: All TCP sockets. All UDP sockets. All established ssh / ftp / http / https connections. All local processes connected to X server. Filtering by state (such as connected, synchronized, SYN-RECV, SYN-SENT,TIME-WAIT), addresses and ports. All the tcp sockets in state FIN-WAIT-1 and much more. Most Linux distributions are shipped with ss and many monitoring tools. Being familiar with this tool helps enhance your understand of what's going on in the system sockets and helps you find the possible causes of a performance problem.

"CFEngine is an IT infrastructure automation framework that helps engineers, system administrators and other stakeholders in an IT organization manage and understand IT infrastructure throughout its lifecycle. CFEngine takes systems from Build to Deploy, Manage and Audit."

"Polipo is a small and fast caching web proxy (a web cache, an HTTP proxy, a proxy server). While Polipo was designed to be used by one person or a small group of people, there is nothing that prevents it from being used by a larger group. Polipo has some features that are, as far as I know, unique among currently available proxies: Polipo will use HTTP/1.1 pipelining if it believes that the remote server supports it, whether the incoming requests are pipelined or come in simultaneously on multiple connections (this is more than the simple usage of persistent connections, which is done by e.g. Squid); Polipo will cache the initial segment of an instance if the download has been interrupted, and, if necessary, complete it later using Range requests; Polipo will upgrade client requests to HTTP/1.1 even if they come in as HTTP/1.0, and up- or downgrade server replies to the client's capabilities (this may involve conversion to or from the HTTP/1.1 chunked encoding); Polipo has complete support for IPv6 (except for scoped (link-local) addresses). Polipo can optionally use a technique known as Poor Man's Multiplexing to reduce latency even further. In short, Polipo uses a plethora of techniques to make web browsing (seem) faster."

"Comdb2 is a relational database built in-house at Bloomberg L.P. over the last 14 years or so. It started with a modest goal of replacing an older home-grown system to allow databases to stay in sync easier. SQL was added early in its development, and it quickly started replacing other relational databases in addition to its original goal. Comdb2 today holds a good chunk of Bloomberg's data, and is continually developed by a dedicated team."

"ScalaTags is a small XML/HTML construction library for Scala. The core functionality of Scalatags is less than 200 lines of code, and yet it provides all the functionality of large frameworks like Python's Jinja2 or C#'s Razor. It does this by leveraging the functionality of the Scala language to do almost everything. A lot of different language constructs can be used to help keep your templates concise and DRY, and why re-invent them all yourself when you have someone else who has done it before you."

"Complete and usable DNS library. All widely used Resource Records are supported, including the DNSSEC types. It follows a lean and mean philosophy. If there is stuff you should know as a DNS programmer there isn't a convenience function for it. Server side and client side programming is supported, i.e. you can build servers and resolvers with it."

Rich Hickey discusses simplicity, why it is important, how to achieve it in design and how to recognize its absence in the tools, language constructs and libraries.

"Apache Hama is a pure BSP (Bulk Synchronous Parallel) computing framework on top of HDFS (Hadoop Distributed File System) for massive scientific computations such as matrix, graph and network algorithms. Today, many practical data processing applications require a more flexible programming abstraction model that is compatible to run on highly scalable and massive data systems (e.g., HDFS, HBase, etc). A message passing paradigm beyond Map-Reduce framework would increase its flexibility in its communication capability. Bulk Synchronous Parallel (BSP) model fills the bill appropriately. Some of its significant advantages over MapReduce and MPI are: * Supports message passing paradigm style of application development * Provides a flexible, simple, and easy-to-use small APIs * Enables to perform better than MPI for communication-intensive applications * Guarantees impossibility of deadlocks or collisions in the communication mechanisms"

"Datomic is a distributed database designed to enable scalable, flexible and intelligent applications, running on next-generation cloud architectures. It does this by: Bringing declarative data manipulation into the application, and the data with it Getting time, process and perception right Process (writes) require coordination Perception (reads) require none The past doesn't change Leveraging immutability, and a sound model of state Datomic has: ACID Transactions Joins A sound data model A logical query language - Datalog Thus, Datomic avoids the compromises and losses of many NoSQL solutions. In addition, it offers flexibility and power over the traditional model in supporting: Hierarchy Multi-valued attributes Minimal schema Reliable operation on unreliable, ephemeral cloud instances Time Datomic avoids manual caching and replication, complex configuration, sharding (automatic or manual), logging, locking, latching and disk management of traditional servers."

"HPaste unlocks the rich functionality of HBase for a Scala audience. In so doing, it attempts to achieve the following goals: Provide a strong, clear syntax for querying and filtration Perform as fast as possible while maintaining idiomatic Scala client code -- the abstractions should not show up in a profiler! Re-articulate HBase's data structures rather than force it into an ORM-style atmosphere. A rich set of base classes for writing MapReduce jobs in hadoop against HBase tables. Provide a maximum amount of code re-use between general Hbase client usage, and operation from within a MapReduce job. Use Scala's type system to its advantage--the compiler should verify the integrity of the schema. Be a verbose DSL--minimize boilerplate code, but be human readable!"

"The new Docker Registry 2.0 was released on April 16th, 2015. It was completely rewritten in Go with added support for the new Docker Registry HTTP API V2 (thus only working with Docker 1.6+), promising to provide faster and more secure distribution of images. If you work with Docker and for some reason decided not to use the public Docker Hub, a private Docker Registry is an essential part of your architecture. But even if you don't have private images, you will likely need to use your own registry in production/testing for efficiency. The default installation, however, runs without encryption and authentication. I was wondering what's involved in securing it. There is an official tutorial on how to configure TLS on a registry server. TLS/SSL is absolutely necessary for any secure setup, but I also wanted to enable an authentication mechanism. The Configuration Reference document describes two authentication options supported by Docker Registry itself: so-called silly and token solutions. The silly one is apparently only useful for very limited development use-cases. The token solution seems to be more serious, but because of the lack of documentation (at the time of writing), I decided to find an alternative approach to secure it. In this article I'm going to show you how to set up the Docker Registry 2.0 with username/password authentication and SSL using the official Docker Registry image and a custom configured nginx as a proxy server."

Pacto judges the contracts between consumers and providers of RESTful services. It can aid in designing realistic test doubles, by ensuring the double complies with the same contract as the real service. It can also aid with service evolution patterns, like Consumer-Driven Contracts or Documentation-Driven Contracts. Pacto ensures consumers meet their contractual obligations: Send the required HTTP request headers Send an appropriate request body (when required) Pacto also ensures providers meet their contractual obligations: Send an appropriate response code Send the required HTTP response headers Send an appropriate response body Pacto can also ensure the provider and consumer collaborate appropriately. It can ensure that for a given scenario: The consumer calls the expected service(s) with a valid request The provider sends a valid response No unexpected services were called

"Rust is a curly-brace, block-structured expression language. It visually resembles the C language family, but differs significantly in syntactic and semantic details. Its design is oriented toward concerns of "programming in the large", that is, of creating and maintaining boundaries - both abstract and operational - that preserve large-system integrity, availability and concurrency. It supports a mixture of imperative procedural, concurrent actor, object-oriented and pure functional styles. Rust also supports generic programming and metaprogramming, in both static and dynamic styles."

"Cloudbreak is a RESTful Hadoop as a Service API. Once it is deployed in your favourite servlet container exposes a REST API allowing to span up Hadoop clusters of arbitrary sizes on your selected cloud provider. Provisioning Hadoop has never been easier. Cloudbreak is built on the foundation of cloud providers API (Amazon AWS, Microsoft Azure, Google Cloud Compute...), Apache Ambari, Docker containers, Serf and dnsmasq."