Group items tagged

A chart is organized as a collection of files inside of a directory.

values.yaml # The default configuration values for this chart

charts/ # A directory containing any charts upon which this chart depends.

version: A SemVer 2 version (required)

apiVersion: The chart API version, always "v1" (required)

Every chart must have a version number. A version must follow the SemVer 2 standard.

When generating a package, the helm package command will use the version that it finds in the Chart.yaml as a token in the package name.

the appVersion field is not related to the version field. It is a way of specifying the version of the application.

appVersion: The version of the app that this contains (optional). This needn't be SemVer.

If the latest version of a chart in the repository is marked as deprecated, then the chart as a whole is considered to be deprecated.

deprecated: Whether this chart is deprecated (optional, boolean)

one chart may depend on any number of other charts.

the preferred method of declaring dependencies is by using a requirements.yaml file inside of your chart.

A requirements.yaml file is a simple file for listing your dependencies.

The repository field is the full URL to the chart repository.

helm dependency update and it will use your dependency file to download all the specified charts into your charts/ directory for you.

When helm dependency update retrieves charts, it will store them as chart archives in the charts/ directory.

All charts are loaded by default.

The condition field holds one or more YAML paths (delimited by commas). If this path exists in the top parent’s values and resolves to a boolean value, the chart will be enabled or disabled based on that boolean value.

The tags field is a YAML list of labels to associate with this chart.

all charts with tags can be enabled or disabled by specifying the tag and a boolean value.

The --set parameter can be used as usual to alter tag and condition values.

The first condition path that exists wins and subsequent ones for that chart are ignored.

The keys containing the values to be imported can be specified in the parent chart’s requirements.yaml file using a YAML list. Each item in the list is a key which is imported from the child chart’s exports field.

specifying the key data in our import list, Helm looks in the exports field of the child chart for data key and imports its contents.

To access values that are not contained in the exports key of the child chart’s values, you will need to specify the source key of the values to be imported (child) and the destination path in the parent chart’s values (parent).

To drop a dependency into your charts/ directory, use the helm fetch command

A dependency can be either a chart archive (foo-1.2.3.tgz) or an unpacked chart directory.

a single release is created with all the objects for the chart and its dependencies.

When Helm renders the charts, it will pass every file in that directory through the template engine.

Chart users may supply a YAML file that contains values. This can be provided on the command line with helm install.

Template files follow the standard conventions for writing Go templates

{{default "minio" .Values.storage}}

Values that are supplied via a values.yaml file (or via the --set flag) are accessible from the .Values object in a template.

Release.Name: The name of the release (not the chart)

Release.IsUpgrade: This is set to true if the current operation is an upgrade or rollback.

Chart: The contents of the Chart.yaml

Files: A map-like object containing all non-special files in the chart.

Files can be accessed using {{index .Files "file.name"}} or using the {{.Files.Get name}} or {{.Files.GetString name}} functions.

access the contents of the file as []byte using {{.Files.GetBytes}}

Any unknown Chart.yaml fields will be dropped

Chart.yaml cannot be used to pass arbitrarily structured data into the template.

A values file is formatted in YAML.

A chart may include a default values.yaml file

accessible inside of templates using the .Values object

Values files can declare values for the top-level chart, as well as for any of the charts that are included in that chart’s charts/ directory.

Charts at a higher level have access to all of the variables defined beneath.

lower level charts cannot access things in parent charts

Values are namespaced, but namespaces are pruned.

Helm supports special “global” value.

a way of sharing one top-level variable with all subcharts, which is useful for things like setting metadata properties like labels.

global variables of parent charts take precedence over the global variables from subcharts.

helm lint

A chart repository is an HTTP server that houses one or more packaged charts

Any HTTP server that can serve YAML files and tar files and can answer GET requests can be used as a repository server.

Helm does not provide tools for uploading charts to remote repository servers.

the only way to add a chart to $HELM_HOME/starters is to manually copy it there.

Helm provides a hook mechanism to allow chart developers to intervene at certain points in a release’s life cycle.

Hooks are declared as an annotation in the metadata section of a manifest

pre-install

post-install: Executes after all resources are loaded into Kubernetes

pre-delete

post-delete: Executes on a deletion request after all of the release’s resources have been deleted.

pre-upgrade

post-upgrade

pre-rollback

post-rollback: Executes on a rollback request after all resources have been modified.

crd-install

test-success: Executes when running helm test and expects the pod to return successfully (return code == 0).

test-failure: Executes when running helm test and expects the pod to fail (return code != 0).

Hooks allow you, the chart developer, an opportunity to perform operations at strategic points in a release lifecycle

Tiller then loads the hook with the lowest weight first (negative to positive)

Tiller returns the release name (and other data) to the client

If the resources is a Job kind, Tiller will wait until the job successfully runs to completion.

good practice to add a hook weight, and set it to 0 if weight is not important.

leave the hook resource alone.

To destroy such resources, you need to either write code to perform this operation in a pre-delete or post-delete hook or add "helm.sh/hook-delete-policy" annotation to the hook template file.

Hooks are just Kubernetes manifest files with special annotations in the metadata section

One resource can implement multiple hooks

no limit to the number of different resources that may implement a given hook.

Hook weights can be positive or negative numbers but must be represented as strings.

sort those hooks in ascending order.

Hook deletion policies

"before-hook-creation" specifies Tiller should delete the previous hook before the new hook is launched.

By default Tiller will wait for 60 seconds for a deleted hook to no longer exist in the API server before timing out.

Custom Resource Definitions (CRDs) are a special kind in Kubernetes.

The crd-install hook is executed very early during an installation, before the rest of the manifests are verified.

A common reason why the hook resource might already exist is that it was not deleted following use on a previous install/upgrade.

Helm uses Go templates for templating your resource files.

two special template functions: include and required

include function allows you to bring in another template, and then pass the results to other template functions.

The required function allows you to declare a particular values entry as required for template rendering.

Quote Strings, Don’t Quote Integers

when working with integers do not quote the values

env variables values which are expected to be string

to include a template, and then perform an operation on that template’s output, Helm has a special include function

The above includes a template called toYaml, passes it $value, and then passes the output of that template to the nindent function.

Go provides a way for setting template options to control behavior when a map is indexed with a key that’s not present in the map

The required function gives developers the ability to declare a value entry as required for template rendering.

The tpl function allows developers to evaluate strings as templates inside a template.

Rendering a external configuration file

(.Files.Get "conf/app.conf")

Image pull secrets are essentially a combination of registry, username, and password.

Automatically Roll Deployments When ConfigMaps or Secrets change

configmaps or secrets are injected as configuration files in containers

a restart may be required should those be updated with a subsequent helm upgrade

The sha256sum function can be used to ensure a deployment’s annotation section is updated if another file changes

helm upgrade --recreate-pods

"helm.sh/resource-policy": keep

resources that should not be deleted when Helm runs a helm delete

create some reusable parts in your chart

In the templates/ directory, any file that begins with an underscore(_) is not expected to output a Kubernetes manifest file.

The current best practice for composing a complex application from discrete parts is to create a top-level umbrella chart that exposes the global configurations, and then use the charts/ subdirectory to embed each of the components.

SAP’s Converged charts: These charts install SAP Converged Cloud a full OpenStack IaaS on Kubernetes. All of the charts are collected together in one GitHub repository, except for a few submodules.

Deis’s Workflow: This chart exposes the entire Deis PaaS system with one chart. But it’s different from the SAP chart in that this umbrella chart is built from each component, and each component is tracked in a different Git repository.

YAML is a superset of JSON

any valid JSON structure ought to be valid in YAML.

As a best practice, templates should follow a YAML-like syntax unless the JSON syntax substantially reduces the risk of a formatting issue.

There are functions in Helm that allow you to generate random data, cryptographic keys, and so on.

a chart repository is a location where packaged charts can be stored and shared.

A chart repository is an HTTP server that houses an index.yaml file and optionally some packaged charts.

Because a chart repository can be any HTTP server that can serve YAML and tar files and can answer GET requests, you have a plethora of options when it comes down to hosting your own chart repository.

It is not required that a chart package be located on the same server as the index.yaml file.

A valid chart repository must have an index file. The index file contains information about each chart in the chart repository.

The Helm project provides an open-source Helm repository server called ChartMuseum that you can host yourself.

$ helm repo index fantastic-charts --url https://fantastic-charts.storage.googleapis.com

A repository will not be added if it does not contain a valid index.yaml

add the repository to their helm client via the helm repo add [NAME] [URL] command with any name they would like to use to reference the repository.

Helm has provenance tools which help chart users verify the integrity and origin of a package.

Integrity is established by comparing a chart to a provenance record

The provenance file contains a chart’s YAML file plus several pieces of verification information

Chart repositories serve as a centralized collection of Helm charts.

Chart repositories must make it possible to serve provenance files over HTTP via a specific request, and must make them available at the same URI path as the chart.

We don’t want to be “the certificate authority” for all chart signers. Instead, we strongly favor a decentralized model, which is part of the reason we chose OpenPGP as our foundational technology.

The Keybase platform provides a public centralized repository for trust information.

A chart contains a number of Kubernetes resources and components that work together.

A test in a helm chart lives under the templates/ directory and is a pod definition that specifies a container with a given command to run.

The pod definition must contain one of the helm test hook annotations: helm.sh/hook: test-success or helm.sh/hook: test-failure

helm test

nest your test suite under a tests/ directory like <chart-name>/templates/tests/

prometheus 通过 HTTP 的方式来暴露的它本身的监控数据

需要配置 rbac 认证，因为我们需要在 prometheus 中去访问 Kubernetes 的相关信息

要获取的资源信息，在每一个 namespace 下面都有可能存在，所以我们这里使用的是 ClusterRole 的资源对象，值得一提的是我们这里的权限规则声明中有一个nonResourceURLs的属性，是用来对非资源型 metrics 进行操作的权限声明

PromQL其实就是 prometheus 便于数据聚合展示开发的一套 ad hoc 查询语言的，你想要查什么找对应函数取你的数据好了。

A pom.xml file contains everything needed to describe your Java project.

Java source code is to be meant to live in the "/src/main/java" folder

Maven will put compiled Java classes into the "target/classes" folder

Maven will also build a .jar or .war file, depending on your project, that lives in the "target" folder.

Maven has the concept of a build lifecycle, which is made up of different phases.

clean is not part of Maven’s default lifecycle, you end up with commands like mvn clean install or mvn clean package. Install or package will trigger all preceding phases, but you need to specify clean in addition.

Maven will always download your project dependencies into your local maven repository first and then reference them for your build.

local repositories (in your user’s home directory: ~/.m2/)

clean: deletes the /target folder.

mvn clean package

mvn clean install

package: Converts your .java source code into a .jar/.war file and puts it into the /target folder.

install: First, it does a package(!). Then it takes that .jar/.war file and puts it into your local Maven repository, which lives in ~/.m2/repository.

calling 'mvn install' would be enough if Maven was smart enough to do reliable, incremental builds.

developers got it ingrained to always call 'mvn clean install' (even though this increases build time a lot in bigger projects).

make sure that Maven always tries to download the latest snapshot dependency versions

DevOps is a culture, a movement, a philosophy.

a firm handshake between development and operations

DevOps isn’t magic, and transformations don’t happen overnight.

Culture is the #1 success factor in DevOps.

Building a culture of shared responsibility, transparency and faster feedback is the foundation of every high performing DevOps team.

 'not our problem' mentality

DevOps is that change in mindset of looking at the development process holistically and breaking down the barrier between Dev and Ops.

Speed is everything.

Open communication helps Dev and Ops teams swarm on issues, fix incidents, and unblock the release pipeline faster.

Unplanned work is a reality that every team faces–a reality that most often impacts team productivity.

“cross-functional collaboration.”

All the tooling and automation in the world are useless if they aren’t accompanied by a genuine desire on the part of development and IT/Ops professionals to work together.

Forming project- or product-oriented teams to replace function-based teams is a step in the right direction.

sharing a common goal and having a plan to reach it together

join sprint planning sessions, daily stand-ups, and sprint demos.

DevOps culture across every department

open channels of communication, and talk regularly

automation eliminates repetitive manual work, yields repeatable processes, and creates reliable systems.

continuous delivery: the practice of running each code change through a gauntlet of automated tests, often facilitated by cloud-based infrastructure, then packaging up successful builds and promoting them up toward production using automated deploys.

automated deploys alert IT/Ops to server “drift” between environments, which reduces or eliminates surprises when it’s time to release.

when DevOps uses automated deploys to send thoroughly tested code to identically provisioned environments, “Works on my machine!” becomes irrelevant.

A DevOps mindset sees opportunities for continuous improvement everywhere.

regular retrospectives

A/B testing

failure is inevitable. So you might as well set up your team to absorb it, recover, and learn from it (some call this “being anti-fragile”).

Postmortems focus on where processes fell down and how to strengthen them – not on which team member f'ed up the code.

Our engineers are responsible for QA, writing, and running their own tests to get the software out to customers.

How long did it take to go from development to deployment? 

How long does it take to recover after a system failure?

service level agreements (SLAs)

DevOps is big on the idea that the same people who build an application should be involved in shipping and running it.

developers and operators pair with each other in each phase of the application’s lifecycle.

Cluster-level logging architectures require a separate backend to store, analyze, and query logs

use kubectl logs --previous to retrieve logs from a previous instantiation of a container.

A container engine handles and redirects any output generated to a containerized application's stdout and stderr streams

An important consideration in node-level logging is implementing log rotation, so that logs don't consume all available storage on the node

You can also set up a container runtime to rotate an application's logs automatically.

The two kubelet flags container-log-max-size and container-log-max-files can be used to configure the maximum size for each log file and the maximum number of files allowed for each container respectively.

The kubelet and container runtime do not run in containers.

On machines with systemd, the kubelet and container runtime write to journald. If systemd is not present, the kubelet and container runtime write to .log files in the /var/log directory.

Kubernetes does not provide a native solution for cluster-level logging

implement cluster-level logging by including a node-level logging agent on each node.

the logging agent must run on every node, it is recommended to run the agent as a DaemonSet

Node-level logging creates only one agent per node and doesn't require any changes to the applications running on the node.

Containers write stdout and stderr, but with no agreed format. A node-level agent collects these logs and forwards them for aggregation.

Each sidecar container prints a log to its own stdout or stderr stream.

It is not recommended to write log entries with different formats to the same log stream

writing logs to a file and then streaming them to stdout can double disk usage.

it's recommended to use stdout and stderr directly and leave rotation and retention policies to the kubelet.

The control plane's components make global decisions about the cluster

Control plane components can be run on any machine in the cluster.

for simplicity, set up scripts typically start all control plane components on the same machine, and do not run user containers on this machine

The API server is the front end for the Kubernetes control plane.

kube-apiserver is designed to scale horizontally—that is, it scales by deploying more instances. You can run several instances of kube-apiserver and balance traffic between those instances.

Kubernetes cluster uses etcd as its backing store, make sure you have a back up plan for those data.

watches for newly created Pods with no assigned node, and selects a node for them to run on.

Factors taken into account for scheduling decisions include: individual and collective resource requirements, hardware/software/policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference, and deadlines.

each controller is a separate process, but to reduce complexity, they are all compiled into a single binary and run in a single process.

Node controller

Job controller

Endpoints controller

Service Account & Token controllers

The cloud controller manager lets you link your cluster into your cloud provider's API, and separates out the components that interact with that cloud platform from components that only interact with your cluster.

If you are running Kubernetes on your own premises, or in a learning environment inside your own PC, the cluster does not have a cloud controller manager.

An agent that runs on each node in the cluster. It makes sure that containers are running in a Pod.

The kubelet takes a set of PodSpecs that are provided through various mechanisms and ensures that the containers described in those PodSpecs are running and healthy.

kube-proxy is a network proxy that runs on each node in your cluster, implementing part of the Kubernetes Service concept.

kube-proxy maintains network rules on nodes. These network rules allow network communication to your Pods from network sessions inside or outside of your cluster.

Kubernetes supports several container runtimes: Docker, containerd, CRI-O, and any implementation of the Kubernetes CRI (Container Runtime Interface).

Addons use Kubernetes resources (DaemonSet, Deployment, etc) to implement cluster features

all Kubernetes clusters should have cluster DNS,

Cluster DNS is a DNS server, in addition to the other DNS server(s) in your environment, which serves DNS records for Kubernetes services.

Container Resource Monitoring records generic time-series metrics about containers in a central database, and provides a UI for browsing that data.

A cluster-level logging mechanism is responsible for saving container logs to a central log store with search/browsing interface.

In-use IP addresses

run one or two control plane instances per failure zone, scaling those instances vertically first and then scaling horizontally after reaching the point of falling returns to (vertical) scale.

Kubernetes nodes do not automatically steer traffic towards control-plane endpoints that are in the same failure zone

store Event objects in a separate dedicated etcd instance.

start and configure additional etcd instance

Kubernetes resource limits help to minimize the impact of memory leaks and other ways that pods and containers can impact on other components.

Addons' default limits are typically based on data collected from experience running each addon on small or medium Kubernetes clusters.

Many addons scale horizontally - you add capacity by running more pods

The VerticalPodAutoscaler can run in recommender mode to provide suggested figures for requests and limits.

Some addons run as one copy per node, controlled by a DaemonSet: for example, a node-level log aggregator.

VerticalPodAutoscaler is a custom resource that you can deploy into your cluster to help you manage resource requests and limits for pods.

The cluster autoscaler integrates with a number of cloud providers to help you run the right number of nodes for the level of resource demand in your cluster.

The addon resizer helps you in resizing the addons automatically as your cluster's scale changes.