Group items tagged

Serverless can also mean applications where some amount of server-side logic is still written by the application developer but unlike traditional architectures is run in stateless compute containers that are event-triggered, ephemeral (may only last for one invocation), and fully managed by a 3rd party.

‘Functions as a service

AWS Lambda is one of the most popular implementations of FaaS at present,

A good example is Auth0 - they started initially with BaaS ‘Authentication as a Service’, but with Auth0 Webtask they are entering the FaaS space.

a typical ecommerce app

a backend data-processing service

with zero administration.

FaaS offerings do not require coding to a specific framework or library.

Horizontal scaling is completely automatic, elastic, and managed by the provider

Functions in FaaS are triggered by event types defined by the provider.

from a deployment-unit point of view FaaS functions are stateless.

allowed the client direct access to a subset of our database

deleted the authentication logic in the original application and have replaced it with a third party BaaS service

The client is in fact well on its way to becoming a Single Page Application.

implement a FaaS function that responds to http requests via an API Gateway

port the search code from the Pet Store server to the Pet Store Search function

replaced a long lived consumer application with a FaaS function that runs within the event driven context

server applications - is a key difference when comparing with other modern architectural trends like containers and PaaS

the only code that needs to change when moving to FaaS is the ‘main method / startup’ code, in that it is deleted, and likely the specific code that is the top-level message handler (the ‘message listener interface’ implementation), but this might only be a change in method signature

With FaaS you need to write the function ahead of time to assume parallelism

Most providers also allow functions to be triggered as a response to inbound http requests, typically in some kind of API gateway

you should assume that for any given invocation of a function none of the in-process or host state that you create will be available to any subsequent invocation.

FaaS functions are either naturally stateless

certain classes of long lived task are not suited to FaaS functions without re-architecture

if you were writing a low-latency trading application you probably wouldn’t want to use FaaS systems at this time

An API Gateway is an HTTP server where routes / endpoints are defined in configuration and each route is associated with a FaaS function.

API Gateway will allow mapping from http request parameters to inputs arguments for the FaaS function

API Gateways may also perform authentication, input validation, response code mapping, etc.

the Serverless Framework makes working with API Gateway + Lambda significantly easier than using the first principles provided by AWS.

Apex - a project to ‘Build, deploy, and manage AWS Lambda functions with ease.'

'Serverless' to mean the union of a couple of other ideas - 'Backend as a Service' and 'Functions as a Service'.

A chart is organized as a collection of files inside of a directory.

values.yaml # The default configuration values for this chart

charts/ # A directory containing any charts upon which this chart depends.

version: A SemVer 2 version (required)

apiVersion: The chart API version, always "v1" (required)

Every chart must have a version number. A version must follow the SemVer 2 standard.

When generating a package, the helm package command will use the version that it finds in the Chart.yaml as a token in the package name.

the appVersion field is not related to the version field. It is a way of specifying the version of the application.

appVersion: The version of the app that this contains (optional). This needn't be SemVer.

If the latest version of a chart in the repository is marked as deprecated, then the chart as a whole is considered to be deprecated.

deprecated: Whether this chart is deprecated (optional, boolean)

one chart may depend on any number of other charts.

the preferred method of declaring dependencies is by using a requirements.yaml file inside of your chart.

A requirements.yaml file is a simple file for listing your dependencies.

The repository field is the full URL to the chart repository.

helm dependency update and it will use your dependency file to download all the specified charts into your charts/ directory for you.

When helm dependency update retrieves charts, it will store them as chart archives in the charts/ directory.

All charts are loaded by default.

The condition field holds one or more YAML paths (delimited by commas). If this path exists in the top parent’s values and resolves to a boolean value, the chart will be enabled or disabled based on that boolean value.

The tags field is a YAML list of labels to associate with this chart.

all charts with tags can be enabled or disabled by specifying the tag and a boolean value.

The --set parameter can be used as usual to alter tag and condition values.

The first condition path that exists wins and subsequent ones for that chart are ignored.

The keys containing the values to be imported can be specified in the parent chart’s requirements.yaml file using a YAML list. Each item in the list is a key which is imported from the child chart’s exports field.

specifying the key data in our import list, Helm looks in the exports field of the child chart for data key and imports its contents.

To access values that are not contained in the exports key of the child chart’s values, you will need to specify the source key of the values to be imported (child) and the destination path in the parent chart’s values (parent).

To drop a dependency into your charts/ directory, use the helm fetch command

A dependency can be either a chart archive (foo-1.2.3.tgz) or an unpacked chart directory.

a single release is created with all the objects for the chart and its dependencies.

When Helm renders the charts, it will pass every file in that directory through the template engine.

Chart users may supply a YAML file that contains values. This can be provided on the command line with helm install.

Template files follow the standard conventions for writing Go templates

{{default "minio" .Values.storage}}

Values that are supplied via a values.yaml file (or via the --set flag) are accessible from the .Values object in a template.

Release.Name: The name of the release (not the chart)

Release.IsUpgrade: This is set to true if the current operation is an upgrade or rollback.

Chart: The contents of the Chart.yaml

Files: A map-like object containing all non-special files in the chart.

Files can be accessed using {{index .Files "file.name"}} or using the {{.Files.Get name}} or {{.Files.GetString name}} functions.

access the contents of the file as []byte using {{.Files.GetBytes}}

Any unknown Chart.yaml fields will be dropped

Chart.yaml cannot be used to pass arbitrarily structured data into the template.

A values file is formatted in YAML.

A chart may include a default values.yaml file

accessible inside of templates using the .Values object

Values files can declare values for the top-level chart, as well as for any of the charts that are included in that chart’s charts/ directory.

Charts at a higher level have access to all of the variables defined beneath.

lower level charts cannot access things in parent charts

Values are namespaced, but namespaces are pruned.

Helm supports special “global” value.

a way of sharing one top-level variable with all subcharts, which is useful for things like setting metadata properties like labels.

global variables of parent charts take precedence over the global variables from subcharts.

helm lint

A chart repository is an HTTP server that houses one or more packaged charts

Any HTTP server that can serve YAML files and tar files and can answer GET requests can be used as a repository server.

Helm does not provide tools for uploading charts to remote repository servers.

the only way to add a chart to $HELM_HOME/starters is to manually copy it there.

Helm provides a hook mechanism to allow chart developers to intervene at certain points in a release’s life cycle.

Hooks are declared as an annotation in the metadata section of a manifest

pre-install

post-install: Executes after all resources are loaded into Kubernetes

pre-delete

post-delete: Executes on a deletion request after all of the release’s resources have been deleted.

pre-upgrade

post-upgrade

pre-rollback

post-rollback: Executes on a rollback request after all resources have been modified.

crd-install

test-success: Executes when running helm test and expects the pod to return successfully (return code == 0).

test-failure: Executes when running helm test and expects the pod to fail (return code != 0).

Hooks allow you, the chart developer, an opportunity to perform operations at strategic points in a release lifecycle

Tiller then loads the hook with the lowest weight first (negative to positive)

Tiller returns the release name (and other data) to the client

If the resources is a Job kind, Tiller will wait until the job successfully runs to completion.

good practice to add a hook weight, and set it to 0 if weight is not important.

leave the hook resource alone.

To destroy such resources, you need to either write code to perform this operation in a pre-delete or post-delete hook or add "helm.sh/hook-delete-policy" annotation to the hook template file.

Hooks are just Kubernetes manifest files with special annotations in the metadata section

One resource can implement multiple hooks

no limit to the number of different resources that may implement a given hook.

Hook weights can be positive or negative numbers but must be represented as strings.

sort those hooks in ascending order.

Hook deletion policies

"before-hook-creation" specifies Tiller should delete the previous hook before the new hook is launched.

By default Tiller will wait for 60 seconds for a deleted hook to no longer exist in the API server before timing out.

Custom Resource Definitions (CRDs) are a special kind in Kubernetes.

The crd-install hook is executed very early during an installation, before the rest of the manifests are verified.

A common reason why the hook resource might already exist is that it was not deleted following use on a previous install/upgrade.

Helm uses Go templates for templating your resource files.

two special template functions: include and required

include function allows you to bring in another template, and then pass the results to other template functions.

The required function allows you to declare a particular values entry as required for template rendering.

Quote Strings, Don’t Quote Integers

when working with integers do not quote the values

env variables values which are expected to be string

to include a template, and then perform an operation on that template’s output, Helm has a special include function

The above includes a template called toYaml, passes it $value, and then passes the output of that template to the nindent function.

Go provides a way for setting template options to control behavior when a map is indexed with a key that’s not present in the map

The required function gives developers the ability to declare a value entry as required for template rendering.

The tpl function allows developers to evaluate strings as templates inside a template.

Rendering a external configuration file

(.Files.Get "conf/app.conf")

Image pull secrets are essentially a combination of registry, username, and password.

Automatically Roll Deployments When ConfigMaps or Secrets change

configmaps or secrets are injected as configuration files in containers

a restart may be required should those be updated with a subsequent helm upgrade

The sha256sum function can be used to ensure a deployment’s annotation section is updated if another file changes

helm upgrade --recreate-pods

"helm.sh/resource-policy": keep

resources that should not be deleted when Helm runs a helm delete

create some reusable parts in your chart

In the templates/ directory, any file that begins with an underscore(_) is not expected to output a Kubernetes manifest file.

The current best practice for composing a complex application from discrete parts is to create a top-level umbrella chart that exposes the global configurations, and then use the charts/ subdirectory to embed each of the components.

SAP’s Converged charts: These charts install SAP Converged Cloud a full OpenStack IaaS on Kubernetes. All of the charts are collected together in one GitHub repository, except for a few submodules.

Deis’s Workflow: This chart exposes the entire Deis PaaS system with one chart. But it’s different from the SAP chart in that this umbrella chart is built from each component, and each component is tracked in a different Git repository.

YAML is a superset of JSON

any valid JSON structure ought to be valid in YAML.

As a best practice, templates should follow a YAML-like syntax unless the JSON syntax substantially reduces the risk of a formatting issue.

There are functions in Helm that allow you to generate random data, cryptographic keys, and so on.

a chart repository is a location where packaged charts can be stored and shared.

A chart repository is an HTTP server that houses an index.yaml file and optionally some packaged charts.

Because a chart repository can be any HTTP server that can serve YAML and tar files and can answer GET requests, you have a plethora of options when it comes down to hosting your own chart repository.

It is not required that a chart package be located on the same server as the index.yaml file.

A valid chart repository must have an index file. The index file contains information about each chart in the chart repository.

The Helm project provides an open-source Helm repository server called ChartMuseum that you can host yourself.

$ helm repo index fantastic-charts --url https://fantastic-charts.storage.googleapis.com

A repository will not be added if it does not contain a valid index.yaml

add the repository to their helm client via the helm repo add [NAME] [URL] command with any name they would like to use to reference the repository.

Helm has provenance tools which help chart users verify the integrity and origin of a package.

Integrity is established by comparing a chart to a provenance record

The provenance file contains a chart’s YAML file plus several pieces of verification information

Chart repositories serve as a centralized collection of Helm charts.

Chart repositories must make it possible to serve provenance files over HTTP via a specific request, and must make them available at the same URI path as the chart.

We don’t want to be “the certificate authority” for all chart signers. Instead, we strongly favor a decentralized model, which is part of the reason we chose OpenPGP as our foundational technology.

The Keybase platform provides a public centralized repository for trust information.

A chart contains a number of Kubernetes resources and components that work together.

A test in a helm chart lives under the templates/ directory and is a pod definition that specifies a container with a given command to run.

The pod definition must contain one of the helm test hook annotations: helm.sh/hook: test-success or helm.sh/hook: test-failure

helm test

nest your test suite under a tests/ directory like <chart-name>/templates/tests/

deployment.yaml: A basic manifest for creating a Kubernetes deployment

using the suffix .yaml for YAML files and .tpl for helpers.

helm get manifest

The helm get manifest command takes a release name (full-coral) and prints out all of the Kubernetes resources that were uploaded to the server. Each file begins with --- to indicate the start of a YAML document

Names should be unique to a release

The name: field is limited to 63 characters because of limitations to the DNS system.

release names are limited to 53 characters

{{ .Release.Name }}

The values that are passed into a template can be thought of as namespaced objects, where a dot (.) separates each namespaced element.

The Release object is one of the built-in objects for Helm

Using --dry-run will make it easier to test your code, but it won’t ensure that Kubernetes itself will accept the templates you generate.

Objects are passed into a template from the template engine.

create new objects within your templates

Objects can be simple, and have just one value. Or they can contain other objects or functions.

Release is one of the top-level objects that you can access in your templates.

Release.Namespace: The namespace to be released into (if the manifest doesn’t override)

Chart: The contents of the Chart.yaml file.

Files: This provides access to all non-special files in a chart.

Files.Get is a function for getting a file by name

Files.GetBytes is a function for getting the contents of a file as an array of bytes instead of as a string. This is useful for things like images.

Template: Contains information about the current template that is being executed

BasePath: The namespaced path to the templates directory of the current chart

Go’s naming convention

use only initial lower case letters in order to distinguish local names from those built-in.

If this is a subchart, the values.yaml file of a parent chart

Individual parameters passed with --set

While structuring data this way is possible, the recommendation is that you keep your values trees shallow, favoring flatness.

If you need to delete a key from the default values, you may override the value of the key to be null, in which case Helm will remove the key from the overridden values merge.

Kubernetes would then fail because you can not declare more than one livenessProbe handler.

When injecting strings from the .Values object into the template, we ought to quote these strings.

quote

Template functions follow the syntax functionName arg1 arg2...

While we talk about the “Helm template language” as if it is Helm-specific, it is actually a combination of the Go template language, some extra functions, and a variety of wrappers to expose certain objects to the templates.

Drawing on a concept from UNIX, pipelines are a tool for chaining together a series of template commands to compactly express a series of transformations.

pipelines are an efficient way of getting several things done in sequence

The repeat function will echo the given string the given number of times

default DEFAULT_VALUE GIVEN_VALUE. This function allows you to specify a default value inside of the template, in case the value is omitted.

all static default values should live in the values.yaml, and should not be repeated using the default command

Operators are implemented as functions that return a boolean value.

To use eq, ne, lt, gt, and, or, not etcetera place the operator at the front of the statement followed by its parameters just as you would a function.

if and

if or

with to specify a scope

range, which provides a “for each”-style loop

block declares a special kind of fillable template area

A pipeline is evaluated as false if the value is: a boolean false a numeric zero an empty string a nil (empty or null) an empty collection (map, slice, tuple, dict, array)

incorrect YAML because of the whitespacing

When the template engine runs, it removes the contents inside of {{ and }}, but it leaves the remaining whitespace exactly as is.

{{- (with the dash and space added) indicates that whitespace should be chomped left, while -}} means whitespace to the right should be consumed.

Newlines are whitespace!

an * at the end of the line indicates a newline character that would be removed

the indent function

Scopes can be changed. with can allow you to set the current scope (.) to a particular object.

range

The range function will “range over” (iterate through) the pizzaToppings list.

Just like with sets the scope of ., so does a range operator.

The toppings: |- line is declaring a multi-line string.

not a YAML list. It’s a big string.

the data in ConfigMaps data is composed of key/value pairs, where both the key and the value are simple strings.

The |- marker in YAML takes a multi-line string.

range can be used to iterate over collections that have a key and a value (like a map or dict).

In Helm templates, a variable is a named reference to another object. It follows the form $name

Variables are assigned with a special assignment operator: :=

{{- $relname := .Release.Name -}}

capture both the index and the value

the integer index (starting from zero) to $index and the value to $topping

For data structures that have both a key and a value, we can use range to get both

one variable that is always global - $ - this variable will always point to the root context.

$.

$.

Helm template language is its ability to declare multiple templates and use them together.

A named template (sometimes called a partial or a subtemplate) is simply a template defined inside of a file, and given a name.

If you declare two templates with the same name, whichever one is loaded last will be the one used.

naming convention is to prefix each defined template with the name of the chart: {{ define "mychart.labels" }}

Drawing on a concept from UNIX, pipelines are a tool for chaining together a series of template commands to compactly express a series of transformations.

the default function: default DEFAULT_VALUE GIVEN_VALUE

all static default values should live in the values.yaml, and should not be repeated using the default command (otherwise they would be redundant).

the default command is perfect for computed values, which can not be declared inside values.yaml.

When lookup returns an object, it will return a dictionary.

The synopsis of the lookup function is lookup apiVersion, kind, namespace, name -> resource or resource list

When no object is found, an empty value is returned. This can be used to check for the existence of an object.

The lookup function uses Helm's existing Kubernetes connection configuration to query Kubernetes.

Helm is not supposed to contact the Kubernetes API Server during a helm template or a helm install|update|delete|rollback --dry-run, so the lookup function will return an empty list (i.e. dict) in such a case.

the operators (eq, ne, lt, gt, and, or and so on) are all implemented as functions. In pipelines, operations can be grouped with parentheses ((, and )).

One popular naming convention is to prefix each defined template with the name of the chart: {{ define "mychart.labels" }}

using the specific chart name as a prefix we can avoid any conflicts

But files whose name begins with an underscore (_) are assumed to not have a manifest inside.

The define action allows us to create a named template inside of a template file.

include it with the template action

a define does not produce output unless it is called with a template

define functions should have a simple documentation block ({{/* ... */}}) describing what they do.

A popular naming convention is to prefix each defined template with the name of the chart

When a named template (created with define) is rendered, it will receive the scope passed in by the template call.

Note that we pass . at the end of the template call. We could just as easily pass .Values or .Values.favorite or whatever scope we want

the template that is substituted in has the text aligned to the left. Because template is an action, and not a function, there is no way to pass the output of a template call to other functions; the data is simply inserted inline.

use indent to indent

Don't need to disable Docker's iptables and let Docker to manage it's network.

The public network cannot access ports that published by Docker.

In a very convenient way to allow/deny public networks to access container ports without additional software and extra configurations

Enable Docker's iptables feature. Remove all changes like --iptables=false , including configuration file /etc/docker/daemon.json

Modify the UFW configuration file /etc/ufw/after.rules

There may be some unknown reasons cause the UFW rules will not take effect after restart UFW, please reboot servers.

If we publish a port by using option -p 8080:80, we should use the container port 80, not the host port 8080

allow the private networks to be able to visit each other.

The following rules block connection requests initiated by all public networks, but allow internal networks to access external networks.

Since the UDP protocol is stateless, it is not possible to block the handshake signal that initiates the connection request as TCP does.

For GNU/Linux we can find the local port range in the file /proc/sys/net/ipv4/ip_local_port_range. The default range is 32768 60999

It not only exposes ports of containers but also exposes ports of the host.

Cannot expose services running on hosts and containers at the same time by the same command.

Functions perform operations on and within strings

the {{timestamp}} function can be used in any string to generate the current timestamp.

pwd - The working directory while executing Packer.

template_dir - The directory to the template for the build.

uuid - Returns a random UUID.

user - Specifies a user variable.

Template variables are special variables automatically set by Packer at build time.

{% ... %} for Statements

{{ ... }} for Expressions to print to the template output

use a dot (.) to access attributes of a variable

the outer double-curly braces are not part of the variable, but the print statement.

if an object has an item and attribute with the same name. Additionally, the attr() filter only looks up attributes.

Variables can be modified by filters. Filters are separated from the variable by a pipe symbol (|) and may have optional arguments in parentheses.

Multiple filters can be chained

Tests can be used to test a variable against a common expression.

add is plus the name of the test after the variable.

to find out if a variable is defined, you can do name is defined, which will then return true or false depending on whether name is defined in the current template context.

strip whitespace in templates by hand. If you add a minus sign (-) to the start or end of a block (e.g. a For tag), a comment, or a variable expression, the whitespaces before or after that block will be removed

not add whitespace between the tag and the minus sign

mark a block raw

Template inheritance allows you to build a base “skeleton” template that contains all the common elements of your site and defines blocks that child templates can override.

The {% extends %} tag is the key here. It tells the template engine that this template “extends” another template.

access templates in subdirectories with a slash

can’t define multiple {% block %} tags with the same name in the same template

put the name of the block after the end tag for better readability

if the block is replaced by a child template, a variable would appear that was not defined in the block or passed to the context.

setting the block to “scoped” by adding the scoped modifier to a block declaration

Jinja2 functions (macros, super, self.BLOCKNAME) always return template data that is marked as safe.

With the default syntax, control structures appear inside {% ... %} blocks.

the dictsort filter

loop.cycle

use loops recursively

whether the value changed at all,

use it to test if a variable is defined, not empty and not false

Macros are comparable with functions in regular programming languages.

If a macro name starts with an underscore, it’s not exported and can’t be imported.

pass a macro to another macro

caller()

a single trailing newline is stripped if present

other whitespace (spaces, tabs, newlines etc.) is returned unchanged

caller(user)

call(user)

This is a simple dialog rendered by using a macro and a call block.

Filter sections allow you to apply regular Jinja2 filters on a block of template data.

Assignments at top level (outside of blocks, macros or loops) are exported from the template like top level macros and can be imported by other templates.

using namespace objects which allow propagating of changes across scopes

use block assignments to capture the contents of a block into a variable name.

The extends tag can be used to extend one template from another.

Blocks are used for inheritance and act as both placeholders and replacements at the same time.

The include statement is useful to include a template and return the rendered contents of that file into the current namespace

Included templates have access to the variables of the active context by default.

putting often used code into macros

imports are cached and imported templates don’t have access to the current template variables, just the globals by default.

Macros and variables starting with one or more underscores are private and cannot be imported.

By default, included templates are passed the current context and imported templates are not.