Group items tagged

Amazon VPC is the networking layer for Amazon EC2.

A virtual private cloud (VPC) is a virtual network dedicated to your AWS account. It is logically isolated from other virtual networks in the AWS Cloud.

Instances can connect to the internet over IPv6 through an internet gateway

IPv6 traffic is separate from IPv4 traffic; your route tables must include separate routes for IPv6 traffic.

You can optionally connect your VPC to your own corporate data center using an IPsec AWS managed VPN connection, making the AWS Cloud an extension of your data center.

A VPN connection consists of a virtual private gateway attached to your VPC and a customer gateway located in your data center.

AWS PrivateLink is a highly available, scalable technology that enables you to privately connect your VPC to supported AWS services, services hosted by other AWS accounts (VPC endpoint services)

Traffic between your VPC and the service does not leave the Amazon network

To use AWS PrivateLink, create an interface VPC endpoint for a service in your VPC. This creates an elastic network interface in your subnet with a private IP address that serves as an entry point for traffic destined to the service.

create your own AWS PrivateLink-powered service (endpoint service) and enable other AWS customers to access your service.

{% ... %} for Statements

{{ ... }} for Expressions to print to the template output

use a dot (.) to access attributes of a variable

the outer double-curly braces are not part of the variable, but the print statement.

if an object has an item and attribute with the same name. Additionally, the attr() filter only looks up attributes.

Variables can be modified by filters. Filters are separated from the variable by a pipe symbol (|) and may have optional arguments in parentheses.

Multiple filters can be chained

Tests can be used to test a variable against a common expression.

add is plus the name of the test after the variable.

to find out if a variable is defined, you can do name is defined, which will then return true or false depending on whether name is defined in the current template context.

strip whitespace in templates by hand. If you add a minus sign (-) to the start or end of a block (e.g. a For tag), a comment, or a variable expression, the whitespaces before or after that block will be removed

not add whitespace between the tag and the minus sign

mark a block raw

Template inheritance allows you to build a base “skeleton” template that contains all the common elements of your site and defines blocks that child templates can override.

The {% extends %} tag is the key here. It tells the template engine that this template “extends” another template.

access templates in subdirectories with a slash

can’t define multiple {% block %} tags with the same name in the same template

put the name of the block after the end tag for better readability

if the block is replaced by a child template, a variable would appear that was not defined in the block or passed to the context.

setting the block to “scoped” by adding the scoped modifier to a block declaration

Jinja2 functions (macros, super, self.BLOCKNAME) always return template data that is marked as safe.

With the default syntax, control structures appear inside {% ... %} blocks.

the dictsort filter

loop.cycle

use loops recursively

whether the value changed at all,

use it to test if a variable is defined, not empty and not false

Macros are comparable with functions in regular programming languages.

If a macro name starts with an underscore, it’s not exported and can’t be imported.

pass a macro to another macro

caller()

a single trailing newline is stripped if present

other whitespace (spaces, tabs, newlines etc.) is returned unchanged

caller(user)

call(user)

This is a simple dialog rendered by using a macro and a call block.

Filter sections allow you to apply regular Jinja2 filters on a block of template data.

Assignments at top level (outside of blocks, macros or loops) are exported from the template like top level macros and can be imported by other templates.

using namespace objects which allow propagating of changes across scopes

use block assignments to capture the contents of a block into a variable name.

The extends tag can be used to extend one template from another.

Blocks are used for inheritance and act as both placeholders and replacements at the same time.

The include statement is useful to include a template and return the rendered contents of that file into the current namespace

Included templates have access to the variables of the active context by default.

putting often used code into macros

imports are cached and imported templates don’t have access to the current template variables, just the globals by default.

Macros and variables starting with one or more underscores are private and cannot be imported.

By default, included templates are passed the current context and imported templates are not.

Integers and floating point numbers are created by just writing the number down

Everything between two brackets is a list.

Tuples are like lists that cannot be modified (“immutable”).

A dict in Python is a structure that combines keys and values.

// Divide two numbers and return the truncated integer result

(expr) group an expression.

The is and in operators support negation using an infix notation

in Perform a sequence / mapping containment test.

| Applies a filter.

~ Converts all operands into strings and concatenates them.

use inline if expressions.

always an attribute is returned and items are not looked up.

default(value, default_value=u'', boolean=False)¶ If the value is undefined it will return the passed default value, otherwise the value of the variable

dictsort(value, case_sensitive=False, by='key', reverse=False)¶ Sort a dict and yield (key, value) pairs.

format(value, *args, **kwargs)¶ Apply python string formatting on an object

indent(s, width=4, first=False, blank=False, indentfirst=None)¶ Return a copy of the string with each line indented by 4 spaces. The first line and blank lines are not indented by default.

join(value, d=u'', attribute=None)¶ Return a string which is the concatenation of the strings in the sequence.

map()¶ Applies a filter on a sequence of objects or looks up an attribute.

pprint(value, verbose=False)¶ Pretty print a variable. Useful for debugging.

reject()¶ Filters a sequence of objects by applying a test to each object, and rejecting the objects with the test succeeding.

replace(s, old, new, count=None)¶ Return a copy of the value with all occurrences of a substring replaced with a new one.

round(value, precision=0, method='common')¶ Round the number to a given precision

even if rounded to 0 precision, a float is returned.

select()¶ Filters a sequence of objects by applying a test to each object, and only selecting the objects with the test succeeding.

sort(value, reverse=False, case_sensitive=False, attribute=None)¶ Sort an iterable. Per default it sorts ascending, if you pass it true as first argument it will reverse the sorting.

striptags(value)¶ Strip SGML/XML tags and replace adjacent whitespace by one space.

tojson(value, indent=None)¶ Dumps a structure to JSON so that it’s safe to use in <script> tags.

trim(value)¶ Strip leading and trailing whitespace.

unique(value, case_sensitive=False, attribute=None)¶ Returns a list of unique items from the the given iterable

urlize(value, trim_url_limit=None, nofollow=False, target=None, rel=None)¶ Converts URLs in plain text into clickable links.

defined(value)¶ Return true if the variable is defined

in(value, seq)¶ Check if value is in seq.

mapping(value)¶ Return true if the object is a mapping (dict etc.).

number(value)¶ Return true if the variable is a number.

sameas(value, other)¶ Check if an object points to the same memory address than another object

undefined(value)¶ Like defined() but the other way round.

A joiner is passed a string and will return that string every time it’s called, except the first time (in which case it returns an empty string).

namespace(...)¶ Creates a new container that allows attribute assignment using the {% set %} tag

The with statement makes it possible to create a new inner scope. Variables set within this scope are not visible outside of the scope.

activate and deactivate the autoescaping from within the templates

With both trim_blocks and lstrip_blocks enabled, you can put block tags on their own lines, and the entire block line will be removed when rendered, preserving the whitespace of the contents

the abstraction of the infrastructure layer, which is now considered code. Deployment of a new application may require the deployment of new infrastructure code as well.

"big bang" deployments update whole or large parts of an application in one fell swoop.

Big bang deployments required the business to conduct extensive development and testing before release, often associated with the "waterfall model" of large sequential releases.

Rollbacks are often costly, time-consuming, or even impossible.

In a rolling deployment, an application’s new version gradually replaces the old one.

new and old versions will coexist without affecting functionality or user experience.

Each container is modified to download the latest image from the app vendor’s site.

two identical production environments work in parallel.

Once the testing results are successful, application traffic is routed from blue to green.

In a blue-green deployment, both systems use the same persistence layer or database back end.

You can use the primary database by blue for write operations and use the secondary by green for read operations.

Blue-green deployments rely on traffic routing.

long TTL values can delay these changes.

The main challenge of canary deployment is to devise a way to route some users to the new application.

Using an application logic to unlock new features to specific users and groups.

With CD, the CI-built code artifact is packaged and always ready to be deployed in one or more environments.

Use Build Automation tools to automate environment builds

Use configuration management tools

Enable automated rollbacks for deployments

An application performance monitoring (APM) tool can help your team monitor critical performance metrics including server response times after deployments.

language images

service images

A language image should be listed first under the docker key in your configuration, making it the primary container during execution.

For example, if you want to add browsers to the circleci/golang:1.9 image, use the circleci/golang:1.9-browsers image.

Service images are convenience images for services like databases

should be listed after language images so they become secondary service containers.

To speed up builds using RAM volume, add the -ram suffix to the end of a service image tag

All convenience images have been extended with additional tools.

all images include the following packages, installed via apt-get

Most CircleCI convenience images are Debian Jessie- or Stretch-based images, however some extend Ubuntu-based images.

The following packages are installed via curl

A certificate is considered a duplicate of an earlier certificate if they contain the exact same set of hostnames, ignoring capitalization and ordering of hostnames.

a Renewal Exemption to the Certificates per Registered Domain limit.

The Duplicate Certificate limit and the Renewal Exemption ignore the public key and extensions requested

You can issue 20 certificates in week 1, 20 more certificates in week 2, and so on, while not interfering with renewals of existing certificates.

Revoking certificates does not reset rate limits

If you’ve hit a rate limit, we don’t have a way to temporarily reset it.

get a list of certificates issued for your registered domain by searching on crt.sh

If you have a large number of pending authorization objects and are getting a rate limiting error, you can trigger a validation attempt for those authorization objects by submitting a JWS-signed POST to one of its challenges, as described in the ACME spec.

If you do not have logs containing the relevant authorization URLs, you need to wait for the rate limit to expire.

having a large number of pending authorizations is generally the result of a buggy client

can overwrite the default label by passing it to the input method

configure the html of any of them

disable labels, hints or error

add a hint, an error, or even a placeholder

add an inline label

pass any html attribute straight to the input, by using the :input_html option

use the :defaults option in simple_form_fo

Simple Form generates a wrapper div around your label and input by default, you can pass any html attribute to that wrapper as well using the :wrapper_html option,

By default all inputs are required

the required property of any input can be overwritten

lets you overwrite the default input type it creates

can also render boolean attributes using as: :select to show a dropdown.

give the :disabled option to Simple Form, and it'll automatically mark the wrapper as disabled with a CSS class

Any extra option passed to these methods will be rendered as html option.

use label, hint, input_field, error and full_error helpers

to strip away all the div wrappers around the <input> field

is to use f.input_field

changing boolean_style from default value :nested to :inline

overriding the :collection option

Collections can be arrays or ranges, and when a :collection is given the :select input will be rendered by default

label_method

value_method

Both of these options also accept lambda/procs

define a to_label method on your model as Simple Form will search for and use :to_label as a :label_method first if it is found

create grouped collection selects, that will use the html optgroup tags

Grouped collection inputs accept the same :label_method and :value_method options

group_method

group_label_method

configured with a default value to be used on the site through the SimpleForm.country_priority and SimpleForm.time_zone_priority helpers.

association

association

render a :select input for choosing the :company, and another :select input with :multiple option for the :roles

all options available to :select, :radio_buttons and :check_boxes are also available to association

f.input

f.select

create a <button> element

simple_fields_for

Creates a collection of radio inputs with labels associated

Creates a collection of checkboxes with labels associated

collection_radio_buttons

collection_check_boxes

associations in your model

Role.all

the html element you will get for each attribute according to its database definition

redefine existing Simple Form inputs by creating a new class with the same name

Simple Form uses all power of I18n API to lookup labels, hints, prompts and placeholders

specify defaults for all models under the 'defaults' key

Simple Form will always look for a default attribute translation under the "defaults" key if no specific is found inside the model key

Simple Form will fallback to default human_attribute_name from Rails when no other translation is found for labels.

Simple Form will only do the lookup for options if you give a collection composed of symbols only.

"Add %{model}"

translations for labels, hints and placeholders for a namespaced model, e.g. Admin::User, should be placed under admin_user, not under admin/user

configure how your components will be rendered using the wrappers API

optional

unless_blank

By default, Simple Form will generate input field types and attributes that are supported in HTML5

The HTML5 extensions include the new field types such as email, number, search, url, tel, and the new attributes such as required, autofocus, maxlength, min, max, step.

If you want to have all other HTML 5 features, such as the new field types, you can disable only the browser validation

add novalidate to a specific form by setting the option on the form itself

the Simple Form configuration file

passing the html5 option

as: :date, html5: true

contexts can be layered within one another

The children contexts can override these values at will

Nginx will error out on reading a configuration file with directives that are declared in the wrong context.

The most general context is the "main" or "global" context

Any directive that exist entirely outside of these blocks is said to inhabit the "main" context

The main context represents the broadest environment for Nginx configuration.

The "events" context is contained within the "main" context. It is used to set global options that affect how Nginx handles connections at a general level.

Nginx uses an event-based connection processing model, so the directives defined within this context determine how worker processes should handle connections.

the connection processing method is automatically selected based on the most efficient choice that the platform has available

a worker will only take a single connection at a time

When configuring Nginx as a web server or reverse proxy, the "http" context will hold the majority of the configuration.

fine-tune the TCP keep alive settings (keepalive_disable, keepalive_requests, and keepalive_timeout)

multiple declarations

each instance defines a specific virtual server to handle client requests

Each client request will be handled according to the configuration defined in a single server context, so Nginx must decide which server context is most appropriate based on details of the request.

listen: The ip address / port combination that this server block is designed to respond to.

server_name: This directive is the other component used to select a server block for processing.

configure files to try to respond to requests (try_files)

issue redirects and rewrites (return and rewrite)

set arbitrary variables (set)

Location contexts share many relational qualities with server contexts

multiple location contexts can be defined, each location is used to handle a certain type of client request, and each location is selected by virtue of matching the location definition against the client request through a selection algorithm

Location blocks live within server contexts and, unlike server blocks, can be nested inside one another.

The request URI is the portion of the request that comes after the domain name or IP address/port combination.

New directives at this level allow you to reach locations outside of the document root (alias), mark the location as only internally accessible (internal), and proxy to other servers or locations (using http, fastcgi, scgi, and uwsgi proxying).

These can then be used to do A/B testing by providing different content to different hosts.

configures Perl handlers for the location they appear in

set the value of a variable depending on the value of another variable

used to map MIME types to the file extensions that should be associated with them.

this context defines a named pool of servers that Nginx can then proxy requests to

The upstream context can then be referenced by name within server or location blocks to pass requests of a certain type to the pool of servers that have been defined.

function as a high performance mail proxy server

The mail context is defined within the "main" or "global" context (outside of the http context).

Nginx has the ability to redirect authentication requests to an external authentication server

the if directive in Nginx will execute the instructions contained if a given test returns "true".

The limit_except context is used to restrict the use of certain HTTP methods within a location context.

The result of the above example is that any client can use the GET and HEAD verbs, but only clients coming from the 192.168.1.1/24 subnet are allowed to use other methods.

Many directives are valid in more than one context

Declaring at higher levels provides you with a sane default

Nginx already engages in a well-documented selection algorithm for things like selecting server blocks and location blocks.

incorrect requests can get by with a redirect rather than a rewrite, which should execute with lower overhead.

Docker is developed in the Go language and utilizes LXC, cgroups, and the Linux kernel itself. Since it’s based on LXC, a Docker container does not include a separate operating system; instead it relies on the operating system’s own functionality as provided by the underlying infrastructure.

a VE there is no preloaded emulation manager software as in a VM.

LXC will boast bare metal performance characteristics because it only packages the needed applications.

a VM, which packages the entire OS and machine setup, including hard drive, virtual processors and network interfaces. The resulting bloated mass usually takes a long time to boot and consumes a lot of CPU and RAM.

don’t offer some other neat features of VM’s such as IaaS setups and live migration.

LXC as supercharged chroot on Linux. It allows you to not only isolate applications, but even the entire OS.

Libvirt, which allows the use of containers through the LXC driver by connecting to 'lxc:///'.

'LXC', is not compatible with libvirt, but is more flexible with more userspace tools.

Portable deployment across machines

Versioning: Docker includes git-like capabilities for tracking successive versions of a container

Component reuse: Docker allows building or stacking of already created packages.

Shared libraries: There is already a public registry (http://index.docker.io/ ) where thousands have already uploaded the useful containers they have created.

Docker taking the devops world by storm since its launch back in 2013.

LXC, while older, has not been as popular with developers as Docker has proven to be

LXC having a focus on sys admins that’s similar to what solutions like the Solaris operating system, with its Solaris Zones, Linux OpenVZ, and FreeBSD, with its BSD Jails virtualization system

it started out being built on top of LXC, Docker later moved beyond LXC containers to its own execution environment called libcontainer.

LXC tooling sticks close to what system administrators running bare metal servers are used to

The LXC command line provides essential commands that cover routine management tasks, including the creation, launch, and deletion of LXC containers.

Docker containers aim to be even lighter weight in order to support the fast, highly scalable, deployment of applications with microservice architecture.

With backing from Canonical, LXC and LXD have an ecosystem tightly bound to the rest of the open source Linux community.

Docker Swarm

Docker Trusted Registry

Docker Compose

Docker Machine

Kubernetes facilitates the deployment of containers in your data center by representing a cluster of servers as a single system.

Swarm is Docker’s clustering, scheduling and orchestration tool for managing a cluster of Docker hosts. 

rkt is a security minded container engine that uses KVM for VM-based isolation and packs other enhanced security features. 

Apache Mesos can run different kinds of distributed jobs, including containers. 

Elastic Container Service is Amazon’s service for running and orchestrating containerized applications on AWS

LXC offers the advantages of a VE on Linux, mainly the ability to isolate your own private workloads from one another. It is a cheaper and faster solution to implement than a VM, but doing so requires a bit of extra learning and expertise.

A build system, git, and development headers for many popular libraries, so that the most popular Ruby, Python and Node.js native extensions can be compiled without problems.

Nginx 1.18. Disabled by default

production-grade features, such as process monitoring, administration and status inspection.

Redis 5.0. Not installed by default.

The image has an app user with UID 9999 and home directory /home/app. Your application is supposed to run as this user.

Passenger works like a mod_ruby, mod_nodejs, etc. It changes Nginx into an application server and runs your app from Nginx.

placing a .conf file in the directory /etc/nginx/sites-enabled

The best way to configure Nginx is by adding .conf files to /etc/nginx/main.d and /etc/nginx/conf.d

To preserve these variables, place an Nginx config file ending with *.conf in the directory /etc/nginx/main.d, in which you tell Nginx to preserve these variables.

By default, Phusion Passenger sets all of the following environment variables to the value production

PASSENGER_APP_ENV environment variable

passenger-docker autogenerates an Nginx configuration file (/etc/nginx/conf.d/00_app_env.conf) during container boot.

The configuration file is in /etc/redis/redis.conf. Modify it as you see fit, but make sure daemonize no is set.

You can add additional daemons to the image by creating runit entries.

the shell script must run the daemon without letting it daemonize/fork it.

We use RVM to install and to manage Ruby interpreters.

Pod-to-Service communications

External-to-Service communications

sharing machines requires ensuring that two applications do not try to use the same ports.

Dynamic port allocation brings a lot of complications to the system

do not need to explicitly create links between Pods

almost never need to deal with mapping container ports to host ports.

pods on a node can communicate with all pods on all nodes without NAT

agents on a node (e.g. system daemons, kubelet) can communicate with all pods on that node

pods in the host network of a node can communicate with all pods on all nodes without NAT

containers within a Pod share their network namespaces - including their IP address

containers within a Pod must coordinate port usage

“IP-per-pod” model.

request ports on the Node itself which forward to your Pod (called host ports), but this is a very niche operation

The Pod itself is blind to the existence or non-existence of host ports.

AOS is an Intent-Based Networking system that creates and manages complex datacenter environments from a simple integrated platform.

Cisco Application Centric Infrastructure offers an integrated overlay and underlay SDN solution that supports containers, virtual machines, and bare metal servers.

AOS Reference Design currently supports Layer-3 connected hosts that eliminate legacy Layer-2 switching problems.

The AWS VPC CNI offers integrated AWS Virtual Private Cloud (VPC) networking for Kubernetes clusters.

users can apply existing AWS VPC networking and security best practices for building Kubernetes clusters.

Using this CNI plugin allows Kubernetes pods to have the same IP address inside the pod as they do on the VPC network.

The CNI allocates AWS Elastic Networking Interfaces (ENIs) to each Kubernetes node and using the secondary IP range from each ENI for pods on the node.

Big Cloud Fabric is a cloud native networking architecture, designed to run Kubernetes in private cloud/on-premises environments.

Cilium is L7/HTTP aware and can enforce network policies on L3-L7 using an identity based security model that is decoupled from network addressing.

CNI-Genie is a CNI plugin that enables Kubernetes to simultaneously have access to different implementations of the Kubernetes network model in runtime.

CNI-Genie also supports assigning multiple IP addresses to a pod, each from a different CNI plugin.

cni-ipvlan-vpc-k8s contains a set of CNI and IPAM plugins to provide a simple, host-local, low latency, high throughput, and compliant networking stack for Kubernetes within Amazon Virtual Private Cloud (VPC) environments by making use of Amazon Elastic Network Interfaces (ENI) and binding AWS-managed IPs into Pods using the Linux kernel’s IPvlan driver in L2 mode.

to be straightforward to configure and deploy within a VPC

Contiv provides configurable networking

Contrail, based on Tungsten Fabric, is a truly open, multi-cloud network virtualization and policy management platform.

DANM is a networking solution for telco workloads running in a Kubernetes cluster.

Flannel is a very simple overlay network that satisfies the Kubernetes requirements.

Any traffic bound for that subnet will be routed directly to the VM by the GCE network fabric.

sysctl net.ipv4.ip_forward=1

Jaguar provides overlay network using vxlan and Jaguar CNIPlugin provides one IP address per pod.

Knitter is a network solution which supports multiple networking in Kubernetes.

Kube-OVN is an OVN-based kubernetes network fabric for enterprises.

Kube-router provides a Linux LVS/IPVS-based service proxy, a Linux kernel forwarding-based pod-to-pod networking solution with no overlays, and iptables/ipset-based network policy enforcer.

If you have a “dumb” L2 network, such as a simple switch in a “bare-metal” environment, you should be able to do something similar to the above GCE setup.

Multus is a Multi CNI plugin to support the Multi Networking feature in Kubernetes using CRD based network objects in Kubernetes.

NSX-T can provide network virtualization for a multi-cloud and multi-hypervisor environment and is focused on emerging application frameworks and architectures that have heterogeneous endpoints and technology stacks.

NSX-T Container Plug-in (NCP) provides integration between NSX-T and container orchestrators such as Kubernetes

Nuage uses the open source Open vSwitch for the data plane along with a feature rich SDN Controller built on open standards.

OpenVSwitch is a somewhat more mature but also complicated way to build an overlay network

OVN is an opensource network virtualization solution developed by the Open vSwitch community.

Project Calico is an open source container networking provider and network policy engine.