Group items tagged

language images

service images

A language image should be listed first under the docker key in your configuration, making it the primary container during execution.

For example, if you want to add browsers to the circleci/golang:1.9 image, use the circleci/golang:1.9-browsers image.

Service images are convenience images for services like databases

should be listed after language images so they become secondary service containers.

To speed up builds using RAM volume, add the -ram suffix to the end of a service image tag

All convenience images have been extended with additional tools.

all images include the following packages, installed via apt-get

Most CircleCI convenience images are Debian Jessie- or Stretch-based images, however some extend Ubuntu-based images.

The following packages are installed via curl

Modifiers and matchers

Matcher rules determine if a particular request should be forwarded to a backend

if any rule matches

if all rules match

Use a *Prefix* matcher if your backend listens on a particular base path but also serves requests on sub-paths. For instance, PathPrefix: /products would match /products but also /products/shoes and /products/shirts. Since the path is forwarded as-is, your backend is expected to listen on /products

Use Path if your backend listens on the exact path only. For instance, Path: /products would match /products but not /products/shoes.

A backend is responsible to load-balance the traffic coming from one or more frontends to a set of http servers

wrr: Weighted Round Robin

drr: Dynamic Round Robin: increases weights on servers that perform better than others.

A circuit breaker can also be applied to a backend, preventing high loads on failing servers.

To proactively prevent backends from being overwhelmed with high load, a maximum connection limit can also be applied to each backend.

Sticky sessions are supported with both load balancers.

When sticky sessions are enabled, a cookie is set on the initial request.

The check is defined by a path appended to the backend URL and an interval (given in a format understood by time.ParseDuration) specifying how often the health check should be executed (the default being 30 seconds). Each backend must respond to the health check within 5 seconds.

The static configuration is the global configuration which is setting up connections to configuration backends and entrypoints.

We only need to enable watch option to make Træfik watch configuration backend changes and generate its configuration automatically.

a comma-separated key/value pair where both key and value must be literals.

namespacing of your backends happens on the basis of hosts in addition to paths

Modifiers will be applied in a pre-determined order regardless of their order in the rule configuration section.

customize priority

Custom headers can be configured through the frontends, to add headers to either requests or responses that match the frontend's rules.

Security related headers (HSTS headers, SSL redirection, Browser XSS filter, etc) can be added and configured per frontend in a similar manner to the custom headers above.

Servers are simply defined using a url. You can also apply a custom weight to each server (this will be used by load-balancing).

Maximum connections can be configured by specifying an integer value for maxconn.amount and maxconn.extractorfunc which is a strategy used to determine how to categorize requests in order to evaluate the maximum connections.

Another style of <none>:<none> images are the dangling images which can cause disk space problems.

In programming languages like Java or Golang a dangling block of memory is a block that is not referenced by any piece of code.

a dangling file system layer in Docker is something that is unused and is not being referenced by any images.

intermediate images

do docker images and see <none>:<none> images in the list, these are dangling images and needs to be pruned.

These dangling images are produced as a result of docker build or pull command.

docker rmi $(docker images -f "dangling=true" -q)

A chart is organized as a collection of files inside of a directory.

values.yaml # The default configuration values for this chart

charts/ # A directory containing any charts upon which this chart depends.

version: A SemVer 2 version (required)

apiVersion: The chart API version, always "v1" (required)

Every chart must have a version number. A version must follow the SemVer 2 standard.

When generating a package, the helm package command will use the version that it finds in the Chart.yaml as a token in the package name.

the appVersion field is not related to the version field. It is a way of specifying the version of the application.

appVersion: The version of the app that this contains (optional). This needn't be SemVer.

If the latest version of a chart in the repository is marked as deprecated, then the chart as a whole is considered to be deprecated.

deprecated: Whether this chart is deprecated (optional, boolean)

one chart may depend on any number of other charts.

the preferred method of declaring dependencies is by using a requirements.yaml file inside of your chart.

A requirements.yaml file is a simple file for listing your dependencies.

The repository field is the full URL to the chart repository.

helm dependency update and it will use your dependency file to download all the specified charts into your charts/ directory for you.

When helm dependency update retrieves charts, it will store them as chart archives in the charts/ directory.

All charts are loaded by default.

The condition field holds one or more YAML paths (delimited by commas). If this path exists in the top parent’s values and resolves to a boolean value, the chart will be enabled or disabled based on that boolean value.

The tags field is a YAML list of labels to associate with this chart.

all charts with tags can be enabled or disabled by specifying the tag and a boolean value.

The --set parameter can be used as usual to alter tag and condition values.

The first condition path that exists wins and subsequent ones for that chart are ignored.

The keys containing the values to be imported can be specified in the parent chart’s requirements.yaml file using a YAML list. Each item in the list is a key which is imported from the child chart’s exports field.

specifying the key data in our import list, Helm looks in the exports field of the child chart for data key and imports its contents.

To access values that are not contained in the exports key of the child chart’s values, you will need to specify the source key of the values to be imported (child) and the destination path in the parent chart’s values (parent).

To drop a dependency into your charts/ directory, use the helm fetch command

A dependency can be either a chart archive (foo-1.2.3.tgz) or an unpacked chart directory.

a single release is created with all the objects for the chart and its dependencies.

When Helm renders the charts, it will pass every file in that directory through the template engine.

Chart users may supply a YAML file that contains values. This can be provided on the command line with helm install.

Template files follow the standard conventions for writing Go templates

{{default "minio" .Values.storage}}

Values that are supplied via a values.yaml file (or via the --set flag) are accessible from the .Values object in a template.

Release.Name: The name of the release (not the chart)

Release.IsUpgrade: This is set to true if the current operation is an upgrade or rollback.

Chart: The contents of the Chart.yaml

Files: A map-like object containing all non-special files in the chart.

Files can be accessed using {{index .Files "file.name"}} or using the {{.Files.Get name}} or {{.Files.GetString name}} functions.

access the contents of the file as []byte using {{.Files.GetBytes}}

Any unknown Chart.yaml fields will be dropped

Chart.yaml cannot be used to pass arbitrarily structured data into the template.

A values file is formatted in YAML.

A chart may include a default values.yaml file

accessible inside of templates using the .Values object

Values files can declare values for the top-level chart, as well as for any of the charts that are included in that chart’s charts/ directory.

Charts at a higher level have access to all of the variables defined beneath.

lower level charts cannot access things in parent charts

Values are namespaced, but namespaces are pruned.

Helm supports special “global” value.

a way of sharing one top-level variable with all subcharts, which is useful for things like setting metadata properties like labels.

global variables of parent charts take precedence over the global variables from subcharts.

helm lint

A chart repository is an HTTP server that houses one or more packaged charts

Any HTTP server that can serve YAML files and tar files and can answer GET requests can be used as a repository server.

Helm does not provide tools for uploading charts to remote repository servers.

the only way to add a chart to $HELM_HOME/starters is to manually copy it there.

Helm provides a hook mechanism to allow chart developers to intervene at certain points in a release’s life cycle.

Hooks are declared as an annotation in the metadata section of a manifest

pre-install

post-install: Executes after all resources are loaded into Kubernetes

pre-delete

post-delete: Executes on a deletion request after all of the release’s resources have been deleted.

pre-upgrade

post-upgrade

pre-rollback

post-rollback: Executes on a rollback request after all resources have been modified.

crd-install

test-success: Executes when running helm test and expects the pod to return successfully (return code == 0).

test-failure: Executes when running helm test and expects the pod to fail (return code != 0).

Hooks allow you, the chart developer, an opportunity to perform operations at strategic points in a release lifecycle

Tiller then loads the hook with the lowest weight first (negative to positive)

Tiller returns the release name (and other data) to the client

If the resources is a Job kind, Tiller will wait until the job successfully runs to completion.

good practice to add a hook weight, and set it to 0 if weight is not important.

leave the hook resource alone.

To destroy such resources, you need to either write code to perform this operation in a pre-delete or post-delete hook or add "helm.sh/hook-delete-policy" annotation to the hook template file.

Hooks are just Kubernetes manifest files with special annotations in the metadata section

One resource can implement multiple hooks

no limit to the number of different resources that may implement a given hook.

Hook weights can be positive or negative numbers but must be represented as strings.

sort those hooks in ascending order.

Hook deletion policies

"before-hook-creation" specifies Tiller should delete the previous hook before the new hook is launched.

By default Tiller will wait for 60 seconds for a deleted hook to no longer exist in the API server before timing out.

Custom Resource Definitions (CRDs) are a special kind in Kubernetes.

The crd-install hook is executed very early during an installation, before the rest of the manifests are verified.

A common reason why the hook resource might already exist is that it was not deleted following use on a previous install/upgrade.

Helm uses Go templates for templating your resource files.

two special template functions: include and required

include function allows you to bring in another template, and then pass the results to other template functions.

The required function allows you to declare a particular values entry as required for template rendering.

Quote Strings, Don’t Quote Integers

when working with integers do not quote the values

env variables values which are expected to be string

to include a template, and then perform an operation on that template’s output, Helm has a special include function

The above includes a template called toYaml, passes it $value, and then passes the output of that template to the nindent function.

Go provides a way for setting template options to control behavior when a map is indexed with a key that’s not present in the map

The required function gives developers the ability to declare a value entry as required for template rendering.

The tpl function allows developers to evaluate strings as templates inside a template.

Rendering a external configuration file

(.Files.Get "conf/app.conf")

Image pull secrets are essentially a combination of registry, username, and password.

Automatically Roll Deployments When ConfigMaps or Secrets change

configmaps or secrets are injected as configuration files in containers

a restart may be required should those be updated with a subsequent helm upgrade

The sha256sum function can be used to ensure a deployment’s annotation section is updated if another file changes

helm upgrade --recreate-pods

"helm.sh/resource-policy": keep

resources that should not be deleted when Helm runs a helm delete

create some reusable parts in your chart

In the templates/ directory, any file that begins with an underscore(_) is not expected to output a Kubernetes manifest file.

The current best practice for composing a complex application from discrete parts is to create a top-level umbrella chart that exposes the global configurations, and then use the charts/ subdirectory to embed each of the components.

SAP’s Converged charts: These charts install SAP Converged Cloud a full OpenStack IaaS on Kubernetes. All of the charts are collected together in one GitHub repository, except for a few submodules.

Deis’s Workflow: This chart exposes the entire Deis PaaS system with one chart. But it’s different from the SAP chart in that this umbrella chart is built from each component, and each component is tracked in a different Git repository.

YAML is a superset of JSON

any valid JSON structure ought to be valid in YAML.

As a best practice, templates should follow a YAML-like syntax unless the JSON syntax substantially reduces the risk of a formatting issue.

There are functions in Helm that allow you to generate random data, cryptographic keys, and so on.

a chart repository is a location where packaged charts can be stored and shared.

A chart repository is an HTTP server that houses an index.yaml file and optionally some packaged charts.

Because a chart repository can be any HTTP server that can serve YAML and tar files and can answer GET requests, you have a plethora of options when it comes down to hosting your own chart repository.

It is not required that a chart package be located on the same server as the index.yaml file.

A valid chart repository must have an index file. The index file contains information about each chart in the chart repository.

The Helm project provides an open-source Helm repository server called ChartMuseum that you can host yourself.

$ helm repo index fantastic-charts --url https://fantastic-charts.storage.googleapis.com

A repository will not be added if it does not contain a valid index.yaml

add the repository to their helm client via the helm repo add [NAME] [URL] command with any name they would like to use to reference the repository.

Helm has provenance tools which help chart users verify the integrity and origin of a package.

Integrity is established by comparing a chart to a provenance record

The provenance file contains a chart’s YAML file plus several pieces of verification information

Chart repositories serve as a centralized collection of Helm charts.

Chart repositories must make it possible to serve provenance files over HTTP via a specific request, and must make them available at the same URI path as the chart.

We don’t want to be “the certificate authority” for all chart signers. Instead, we strongly favor a decentralized model, which is part of the reason we chose OpenPGP as our foundational technology.

The Keybase platform provides a public centralized repository for trust information.

A chart contains a number of Kubernetes resources and components that work together.

A test in a helm chart lives under the templates/ directory and is a pod definition that specifies a container with a given command to run.

The pod definition must contain one of the helm test hook annotations: helm.sh/hook: test-success or helm.sh/hook: test-failure

helm test

nest your test suite under a tests/ directory like <chart-name>/templates/tests/

Vegeta was designed to be run on the command line; it reads from stdin a list of HTTP transactions to generate, and sends results in binary format to stdout,

Vegeta is a really strong tool that caters to people who want a tool to test simple, static URLs (perhaps API end points) but also want a bit more functionality.

Vegeta can even be used as a Golang library/package if you want to create your own load testing tool.

Wrk is so damn fast

being fast and measuring correctly is about all that Wrk does

k6 is scriptable in plain Javascript

k6 is average or better. In some categories (documentation, scripting API, command line UX) it is outstanding.

Jmeter is a huge beast compared to most other tools.

Siege is a simple tool, similar to e.g. Apachebench in that it has no scripting and is primarily used when you want to hit a single, static URL repeatedly.

A good way of testing the testing tools is to not test them on your code, but on some third-party thing that is sure to be very high-performing.

use a tool like e.g. top to keep track of Nginx CPU usage while testing. If you see just one process, and see it using close to 100% CPU, it means you could be CPU-bound on the target side.

If you see multiple Nginx processes but only one is using a lot of CPU, it means your load testing tool is only talking to that particular worker process.

Network delay is also important to take into account as it sets an upper limit on the number of requests per second you can push through.

If, say, the Nginx default page requires a transfer of 250 bytes to load, it means that if the servers are connected via a 100 Mbit/s link, the theoretical max RPS rate would be around 100,000,000 divided by 8 (bits per byte) divided by 250 => 100M/2000 = 50,000 RPS. Though that is a very optimistic calculation - protocol overhead will make the actual number a lot lower so in the case above I would start to get worried bandwidth was an issue if I saw I could push through max 30,000 RPS, or something like that.

Wrk managed to push through over 50,000 RPS and that made 8 Nginx workers on the target system consume about 600% CPU.

exporter前面我们已经学习了，是用来提供专门提供metrics数据接口的工具

Operator 会自动创建4个 CRD 资源对象