Group items tagged

You can access the service on port 443 of any swarm node. Docker sends the requests to the node which is running the service.

--publish published=443,target=443

The most important aspect is that a load balanced cluster of registries must share the same resources

S3 or Azure, they should be accessing the same resource and share an identical configuration.

you must make sure you are properly sending the X-Forwarded-Proto, X-Forwarded-For, and Host headers to their “client-side” values. Failure to do so usually makes the registry issue redirects to internal hostnames or downgrading from https to http.

A properly secured registry should return 401 when the “/v2/” endpoint is hit without credentials

registries should always implement access restrictions.

REGISTRY_AUTH=htpasswd

REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd

The registry also supports delegated authentication which redirects users to a specific trusted token server. This approach is more complicated to set up, and only makes sense if you need to fully configure ACLs and need more control over the registry’s integration into your global authorization and authentication systems.

a volume outlives any Containers that run within the Pod, and data is preserved across Container restarts.

Kubernetes supports many types of volumes, and a Pod can use any number of them simultaneously.

To use a volume, a Pod specifies what volumes to provide for the Pod (the .spec.volumes field) and where to mount those into Containers (the .spec.containers.volumeMounts field).

Each Container in the Pod must independently specify where to mount each volume

localnfs

cephfs

awsElasticBlockStore

glusterfs

vsphereVolume

An awsElasticBlockStore volume mounts an Amazon Web Services (AWS) EBS Volume into your Pod.

an EBS volume can be pre-populated with data, and that data can be “handed off” between Pods.

create an EBS volume using aws ec2 create-volume

the nodes on which Pods are running must be AWS EC2 instances

EBS only supports a single EC2 instance mounting a volume

check that the size and EBS volume type are suitable for your use!

A cephfs volume allows an existing CephFS volume to be mounted into your Pod.

the contents of a cephfs volume are preserved and the volume is merely unmounted.

have your own Ceph server running with the share exported

The configMap resource provides a way to inject configuration data into Pods

When referencing a configMap object, you can simply provide its name in the volume to reference it

volumeMounts: - name: config-vol mountPath: /etc/config volumes: - name: config-vol configMap: name: log-config items: - key: log_level path: log_level

create a ConfigMap before you can use it.

An emptyDir volume is first created when a Pod is assigned to a Node, and exists as long as that Pod is running on that node.

By default, emptyDir volumes are stored on whatever medium is backing the node - that might be disk or SSD or network storage, depending on your environment.

you can set the emptyDir.medium field to "Memory" to tell Kubernetes to mount a tmpfs (RAM-backed filesystem)

volumeMounts: - mountPath: /cache name: cache-volume volumes: - name: cache-volume emptyDir: {}

An fc volume allows an existing fibre channel volume to be mounted in a Pod.

configure FC SAN Zoning to allocate and mask those LUNs (volumes) to the target WWNs beforehand so that Kubernetes hosts can access them.

Flocker is an open-source clustered Container data volume manager. It provides management and orchestration of data volumes backed by a variety of storage backends.

emptyDir

flocker

A flocker volume allows a Flocker dataset to be mounted into a Pod

have your own Flocker installation running

A gcePersistentDisk volume mounts a Google Compute Engine (GCE) Persistent Disk into your Pod.

Using a PD on a Pod controlled by a ReplicationController will fail unless the PD is read-only or the replica count is 0 or 1

A glusterfs volume allows a Glusterfs (an open source networked filesystem) volume to be mounted into your Pod.

have your own GlusterFS installation running

a powerful escape hatch for some applications

access to Docker internals; use a hostPath of /var/lib/docker

allowing a Pod to specify whether a given hostPath should exist prior to the Pod running, whether it should be created, and what it should exist as

specify a type for a hostPath volume

the files or directories created on the underlying hosts are only writable by root.

hostPath: # directory location on host path: /data # this field is optional type: Directory

An iscsi volume allows an existing iSCSI (SCSI over IP) volume to be mounted into your Pod.

have your own iSCSI server running

A feature of iSCSI is that it can be mounted as read-only by multiple consumers simultaneously.

A local volume represents a mounted local storage device such as a disk, partition or directory.

Compared to hostPath volumes, local volumes can be used in a durable and portable manner without manually scheduling Pods to nodes, as the system is aware of the volume’s node constraints by looking at the node affinity on the PersistentVolume.

PersistentVolume spec using a local volume and nodeAffinity

PersistentVolume volumeMode can now be set to “Block” (instead of the default value “Filesystem”) to expose the local volume as a raw block device.

When using local volumes, it is recommended to create a StorageClass with volumeBindingMode set to WaitForFirstConsumer

An nfs volume allows an existing NFS (Network File System) share to be mounted into your Pod.

have your own NFS server running with the share exported

A persistentVolumeClaim volume is used to mount a PersistentVolume into a Pod.

PersistentVolumes are a way for users to “claim” durable storage (such as a GCE PersistentDisk or an iSCSI volume) without knowing the details of the particular cloud environment.

A projected volume maps several existing volume sources into the same directory.

All sources are required to be in the same namespace as the Pod. For more details, see the all-in-one volume design document.

Each projected volume source is listed in the spec under sources

A Container using a projected volume source as a subPath volume mount will not receive updates for those volume sources.

RBD volumes can only be mounted by a single consumer in read-write mode - no simultaneous writers allowed

A secret volume is used to pass sensitive information, such as passwords, to Pods

create a secret in the Kubernetes API before you can use it

StorageOS runs as a Container within your Kubernetes environment, making local or attached storage accessible from any node within the Kubernetes cluster.

StorageOS provides block storage to Containers, accessible via a file system.

A vsphereVolume is used to mount a vSphere VMDK Volume into your Pod.

supports both VMFS and VSAN datastore.

create VMDK using one of the following methods before using with Pod.

share one volume for multiple uses in a single Pod.

volumeMounts: - name: workdir1 mountPath: /logs subPathExpr: $(POD_NAME)

env: - name: POD_NAME valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.name

Use the subPathExpr field to construct subPath directory names from Downward API environment variables

enable the VolumeSubpathEnvExpansion feature gate

emptyDir and hostPath volumes will be able to request a certain amount of space using a resource specification, and to select the type of media to use, for clusters that have several media types.

the Container Storage Interface (CSI) and Flexvolume. They enable storage vendors to create custom storage plugins without adding them to the Kubernetes repository.

Container Storage Interface (CSI) defines a standard interface for container orchestration systems (like Kubernetes) to expose arbitrary storage systems to their container workloads.

Once a CSI compatible volume driver is deployed on a Kubernetes cluster, users may use the csi volume type to attach, mount, etc. the volumes exposed by the CSI driver.

This feature requires CSIInlineVolume feature gate to be enabled:--feature-gates=CSIInlineVolume=true

In-tree plugins that support CSI Migration and have a corresponding CSI driver implemented are listed in the “Types of Volumes” section above.

Mount propagation of a volume is controlled by mountPropagation field in Container.volumeMounts.

HostToContainer - This volume mount will receive all subsequent mounts that are mounted to this volume or any of its subdirectories.

Bidirectional - This volume mount behaves the same the HostToContainer mount. In addition, all volume mounts created by the Container will be propagated back to the host and to all Containers of all Pods that use the same volume.

Edit your Docker’s systemd service file. Set MountFlags as follows:MountFlags=shared

all nodes are monitoring and communicating with all the other ProxySQL nodes. When another node detects a change in the checksum and version (both at the same time), each node will get a copy of the table that was modified, make the same changes locally, and apply the new config to RUNTIME to refresh the new config, make it visible to the applications connected and automatically save it to DISK for persistence.

a “synchronous cluster” so any changes to these 4 tables on any ProxySQL server will be replicated to all other ProxySQL nodes.

Name servers host a domain’s DNS information in a text file called a zone file.

Start of Authority (SOA) records

specifying DNS records, which match domain names to IP addresses.

Every domain’s zone file contains the domain administrator’s email address, the name servers, and the DNS records.

Your ISP’s DNS resolver queries a root nameserver for the proper TLD nameserver. In other words, it asks the root nameserver, *Where can I find the nameserver for .com domains?*

In actuality, ISPs cache a lot of DNS information after they’ve looked it up the first time.

An A record points your domain or subdomain to your Linode’s IP address,

use an asterisk (*) as your subdomain

An AAAA record is just like an A record, but for IPv6 IP addresses.

An AXFR record is a type of DNS record used for DNS replication

DNS Certification Authority Authorization uses DNS to allow the holder of a domain to specify which certificate authorities are allowed to issue certificates for that domain.

A CNAME record or Canonical Name record matches a domain or subdomain to a different domain.

Some mail servers handle mail oddly for domains with CNAME records, so you should not use a CNAME record for a domain that gets email.

MX records cannot reference CNAME-defined hostnames.

A DKIM record or DomainKeys Identified Mail record displays the public key for authenticating messages that have been signed with the DKIM protocol

DKIM records are implemented as text records.

An MX record or mail exchanger record sets the mail delivery destination for a domain or subdomain.

An MX record should ideally point to a domain that is also the hostname for its server.

Priority allows you to designate a fallback server (or servers) for mail for a particular domain. Lower numbers have a higher priority.

NS records or name server records set the nameservers for a domain or subdomain.

You can also set up different nameservers for any of your subdomains

Primary nameservers get configured at your registrar and secondary subdomain nameservers get configured in the primary domain’s zone file.

A PTR record or pointer record matches up an IP address to a domain or subdomain, allowing reverse DNS queries to function.

opposite service an A record does

PTR records are usually set with your hosting provider. They are not part of your domain’s zone file.

An SOA record or Start of Authority record labels a zone file with the name of the host where it was originally created.

Minimum TTL: The minimum amount of time other servers should keep data cached from this zone file.

An SPF record or Sender Policy Framework record lists the designated mail servers for a domain or subdomain.

An SPF record for your domain tells other receiving mail servers which outgoing server(s) are valid sources of email so they can reject spoofed mail from your domain that has originated from unauthorized servers.

Make sure your SPF records are not too strict.

An SRV record or service record matches up a specific service that runs on your domain or subdomain to a target domain.

Service: The name of the service must be preceded by an underscore (_) and followed by a period (.)

Protocol: The name of the protocol must be proceeded by an underscore (_) and followed by a period (.)

Port: The TCP or UDP port on which the service runs.

Target: The target domain or subdomain. This domain must have an A or AAAA record that resolves to an IP address.

A TXT record or text record provides information about the domain in question to other resources on the internet.

When you run Docker without using swarm mode, you execute container commands.

You can run swarm services and standalone containers on the same Docker instances.

A node is an instance of the Docker engine participating in the swarm

You can run one or more nodes on a single physical computer or cloud server

To deploy your application to a swarm, you submit a service definition to a manager node.

Manager nodes also perform the orchestration and cluster management functions required to maintain the desired state of the swarm.

Worker nodes receive and execute tasks dispatched from manager nodes.

service is the definition of the tasks to execute on the worker nodes

When you create a service, you specify which container image to use and which commands to execute inside running containers.

replicated services model, the swarm manager distributes a specific number of replica tasks among the nodes based upon the scale you set in the desired state.

global services, the swarm runs one task for the service on every available node in the cluster.

A task carries a Docker container and the commands to run inside the container

Manager nodes assign tasks to worker nodes according to the number of replicas set in the service scale.

If you do not specify a port, the swarm manager assigns the service a port in the 30000-32767 range.

Swarm mode has an internal DNS component that automatically assigns each service in the swarm a DNS entry.

決定每個 shard 要被分配到哪個 data node 上

為 cluster 設置多個 master node

一旦發現被選中的 master node 出現問題，就會選出新的 master node

處理 request 的 node 稱為 Coordinating Node，其功能是將 request 轉發到合適的 node 上

所有的 node 都預設是 Coordinating Node

可以保存資料的 node，每個 node 啟動後都會預設是 data node，可以透過設定 node.data: false 停用 data node 功能

由 master node 決定如何把分片分發到不同的 data node 上

每個 node 上都保存了 cluster state

只有 master 才可以修改 cluster state 並負責同步給其他 node

每個 node 都會詳細紀錄本身的狀態資訊

shard 是 Elasticsearch 分散式儲存的基礎，包含 primary shard & replica shard

每一個 shard 就是一個 Lucene instance

primary shard 功能是將一份被索引後的資料，分散到多個 data node 上存放，實現儲存方面的水平擴展

當 primary shard 遺失時，replica shard 就可以被 promote 成 primary shard 來保持資料完整性

replica shard 數量可以動態調整，讓每個 data node 上都有完整的資料

ES 7.0 開始，primary shard 預設為 1，replica shard 預設為 0

replica shard 若設定過多，會降低 cluster 整體的寫入效能

所有的 primary shard 可以在同一個 data node 上

透過 GET _cluster/health/<target> 可以取得目前 cluster 的健康狀態

Yellow：表示 primary shard 可以正常分配，但 replica shard 分配有問題

透過 GET /_cat/shards/<target> 可以取得目前的 shard 狀態

replica shard 無法被分配，因此 cluster 健康狀態為黃色

若是擔心 reboot 機器造成 failover 動作開始執行，可以設定將 replication 延遲一段時間後再執行(透過調整 settings 中的 index.unassigned.node_left.delayed_timeout 參數)，避免無謂的 data copy 動作 (此功能稱為 delay allocation)

集群變紅，代表有 primary shard 丟失，這個時候會影響讀寫。

如果 node 重新回來，會從 translog 中恢復沒有寫入的資料

設定 index settings 之後，primary shard 數量無法隨意變更

不建議直接發送請求到master節點，雖然也會工作，但是大量請求發送到 master，會有潛在的性能問題

shard 是 ES 中最小的工作單元

shard 是一個 Lucene 的 index

將 Index Buffer 中的內容寫入 Segment，而這寫入的過程就稱為 Refresh

當 document 被 refresh 進入到 segment 之後，就可以被搜尋到了

在進行 refresh 時先將 segment 寫入 cache 以開放查詢

將 document 進行索引時，同時也會寫入 transaction log，且預設都會寫入磁碟中

每個 shard 都會有對應的 transaction log

由於 transaction log 都會寫入磁碟中，因此當 node 從故障中恢復時，就會優先讀取 transaction log 來恢復資料

The control plane's components make global decisions about the cluster

Control plane components can be run on any machine in the cluster.

for simplicity, set up scripts typically start all control plane components on the same machine, and do not run user containers on this machine

The API server is the front end for the Kubernetes control plane.

kube-apiserver is designed to scale horizontally—that is, it scales by deploying more instances. You can run several instances of kube-apiserver and balance traffic between those instances.

Kubernetes cluster uses etcd as its backing store, make sure you have a back up plan for those data.

watches for newly created Pods with no assigned node, and selects a node for them to run on.

Factors taken into account for scheduling decisions include: individual and collective resource requirements, hardware/software/policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference, and deadlines.

each controller is a separate process, but to reduce complexity, they are all compiled into a single binary and run in a single process.

Node controller

Job controller

Endpoints controller

Service Account & Token controllers

The cloud controller manager lets you link your cluster into your cloud provider's API, and separates out the components that interact with that cloud platform from components that only interact with your cluster.

If you are running Kubernetes on your own premises, or in a learning environment inside your own PC, the cluster does not have a cloud controller manager.

An agent that runs on each node in the cluster. It makes sure that containers are running in a Pod.

The kubelet takes a set of PodSpecs that are provided through various mechanisms and ensures that the containers described in those PodSpecs are running and healthy.

kube-proxy is a network proxy that runs on each node in your cluster, implementing part of the Kubernetes Service concept.

kube-proxy maintains network rules on nodes. These network rules allow network communication to your Pods from network sessions inside or outside of your cluster.

Kubernetes supports several container runtimes: Docker, containerd, CRI-O, and any implementation of the Kubernetes CRI (Container Runtime Interface).

Addons use Kubernetes resources (DaemonSet, Deployment, etc) to implement cluster features

all Kubernetes clusters should have cluster DNS,

Cluster DNS is a DNS server, in addition to the other DNS server(s) in your environment, which serves DNS records for Kubernetes services.

Container Resource Monitoring records generic time-series metrics about containers in a central database, and provides a UI for browsing that data.

A cluster-level logging mechanism is responsible for saving container logs to a central log store with search/browsing interface.

Ephemeral containers differ from other containers in that they lack guarantees for resources or execution, and they will never be automatically restarted, so they are not appropriate for building applications.

distroless images enable you to deploy minimal container images that reduce attack surface and exposure to bugs and vulnerabilities.