Group items tagged

Communication between pods is more complicated, however, and requires a separate networking component that can transparently route traffic from a pod on one node to a pod on another.

pod network plugins. For this cluster, you will use Flannel, a stable and performant option.

Passing the argument --pod-network-cidr=10.244.0.0/16 specifies the private subnet that the pod IPs will be assigned from.

kubectl apply -f descriptor.[yml|json] is the syntax for telling kubectl to create the objects described in the descriptor.[yml|json] file.

deploy Nginx using Deployments and Services

NodePort, a scheme that will make the pod accessible through an arbitrary port opened on each node of the cluster

load balancing requests to multiple pods

Pods are ubiquitous in Kubernetes, so understanding them will facilitate your work

how controllers such as deployments work since they are used frequently in stateless applications for scaling and the automated healing of unhealthy applications.

Understanding the types of services and the options they have is essential for running both stateless and stateful applications.

Packer needs to decide on a port to use for VNC when building remotely.

vnc_disable_password - This must be set to "true" when using VNC with ESXi 6.5 or 6.7

remote_type (string) - The type of remote machine that will be used to build this VM rather than a local desktop product. The only value accepted for this currently is esx5. If this is not set, a desktop product will be used. By default, this is not set.

HTTP is meant to run over a bidirectional tunnel for arbitrary binary data; when that tunnel is an SSL/TLS connection, then the whole is called "HTTPS".

"SSL" means "Secure Sockets Layer".

"TLS" means "Transport Layer Security".

not just Internet-based sockets

"HTTPS" is supposed to mean "HyperText Transfer Protocol Secure",

Other protocol acronyms have been built the same way, e.g. SMTPS, IMAPS, FTPS... all of them being a bare protocol that "got secured" by running it within some SSL/TLS.

To make the confusing perfect: SSL (secure socket layer) often refers to the old protocol variant which starts with the handshake right away and therefore requires another port for the encrypted protocol such as 443 instead of 80.

obviating the need for servers and systems administration

scale up without significant changes to tooling, architecture, or development practices

Ops engineers who deploy or manage such applications.

developer building applications which run as a service

One codebase

many deploys

in the environment

services as attached resources

Explicitly declare

separate build and run stages

stateless processes

Export services via port binding

Scale out

fast startup and graceful shutdown

as similar as possible

logs as event streams

admin/management tasks as one-off processes

load balancing, connection failover and decoupling of the application tier from the underlying database topologies.

ProxySQL as a Kubernetes service (centralized deployment)

running as a service makes ProxySQL pods live independently from the applications and can be easily scaled and clustered together with the help of Kubernetes ConfigMap.

ProxySQL's multi-layer configuration system makes pod clustering possible with ConfigMap.

create ProxySQL pods and attach a Kubernetes service to be accessed by the other pods within the Kubernetes network or externally.

Default to 6033 for MySQL load-balanced connections and 6032 for ProxySQL administration console.

separated by "---" delimiter

deploy two ProxySQL pods as a ReplicaSet that matches containers labelled with "app=proxysql,tier=frontend".

A Kubernetes service is an abstraction layer which defines the logical set of pods and a policy by which to access them

The range of valid ports for NodePort resource is 30000-32767.

ConfigMap - To store ProxySQL configuration file as a volume so it can be mounted to multiple pods and can be remounted again if the pod is being rescheduled to the other Kubernetes node.

Nginx is built to handle many concurrent connections at the same time.

provides you with flexibility in easily adding backend servers or taking them down as needed for maintenance

Proxying in Nginx is accomplished by manipulating a request aimed at the Nginx server and passing it to other servers for the actual processing

When a request matches a location with a proxy_pass directive inside, the request is forwarded to the URL given by the directive

The request coming from Nginx on behalf of a client will look different than a request coming directly from a client

Nginx gets rid of any empty headers

Nginx, by default, will consider any header that contains underscores as invalid. It will remove these from the proxied request

The "Host" header is re-written to the value defined by the $proxy_host variable.

The upstream should not expect this connection to be persistent

Headers with empty values are completely removed from the passed request.

by default, this will be set to the value of $proxy_host, a variable that will contain the domain name or IP address and port taken directly from the proxy_pass definition

This is selected by default as it is the only address Nginx can be sure the upstream server responds to

$http_host: Sets the "Host" header to the "Host" header from the client request.

preference to: the host name from the request line itself

set the "Host" header to the $host variable. It is the most flexible and will usually provide the proxied servers with a "Host" header filled in as accurately as possible

This variable takes the value of the original X-Forwarded-For header retrieved from the client and adds the Nginx server's IP address to the end.

Once defined, this name will be available for use within proxy passes as if it were a regular domain name

By default, this is just a simple round-robin selection process (each request will be routed to a different host in turn)

Specifies that new connections should always be given to the backend that has the least number of active connections.

distributes requests to different servers based on the client's IP address.

mainly used with memcached proxying

As for the hash method, you must provide the key to hash against

Server Weight

Nginx's buffering and caching capabilities

Without buffers, data is sent from the proxied server and immediately begins to be transmitted to the client.

With buffers, the Nginx proxy will temporarily store the backend's response and then feed this data to the client

Nginx defaults to a buffering design

can be set in the http, server, or location contexts.

the sizing directives are configured per request, so increasing them beyond your need can affect your performance

When buffering is "off" only the buffer defined by the proxy_buffer_size directive will be used

A high availability (HA) setup is an infrastructure without a single point of failure, and your load balancers are a part of this configuration.

multiple load balancers (one active and one or more passive) behind a static IP address that can be remapped from one server to another.

Nginx also provides a way to cache content from backend servers

proxy_cache backcache;

The proxy_cache_bypass directive is set to the $http_cache_control variable. This will contain an indicator as to whether the client is explicitly requesting a fresh, non-cached version of the resource

For private content, you should set the Cache-Control header to "no-cache", "no-store", or "private" depending on the nature of the data

其實 Auto DevOps 就是一份官方寫好的 gitlab-ci.yml，在啟動 Auto DevOps 的專案裡，如果找不到 gitlab-ci.yml 檔，那就會直接用官方 gitlab-ci.yml 去跑 CI / CD 流程。

Pod 是 K8S 中可以被部署的最小元件，一個 Pod 是由一到多個 Container 組成，同個 Pod 的不同 Container 之間彼此共享網路資源。

Node 又分成 Worker Node 和 Master Node 兩種

Helm 透過參數 (parameter) 跟模板 (template) 的方式，讓我們可以在只修改參數的方式重複利用模板。

為了要有 CI CD 的功能我們會把 .gitlab-ci.yml 放在專案的根目錄裡， GitLab 會依造 .gitlab-ci.yml 的設定產生 CI/CD Pipeline，每個 Pipeline 裡面可能有多個 Job，這時候就會需要有 GitLab Runner 來執行這些 Job 並把執行的結果回傳給 GitLab 讓它知道這個 Job 是否有正常執行。

把專案打包成 Docker Image 這工作又或是 helm 的操作都會在 Container 內執行

CI/CD Pipeline 是由 stage 還有 job 組成的，stage 是有順序性的，前面的 stage 完成後才會開始下一個 stage。

每個 stage 裡面包含一到多個 Job

Auto Devops 裡也會大量用到這種在指定 Container 內運行的工作。

可以通過 health checks

開 private 的話還要注意使用 Container Registry 的權限問題

申請好的 wildcard 的 DNS

Auto Devops 也提供只要設定環境變數就能一定程度客製化的選項

特別注意 namespace 有沒有設定對，不然會找不到資料喔

Auto Devops，如果想要進一步的客製化，而且是改 GitLab 環境變數都無法實現的客製化，這時候還是得回到 .gitlab-ci.yml 設定檔

在 Docker in Docker 的環境用 Dockerfile 打包 Image

用 helm upgrade 把 chart 部署到 K8S 上

把專案打包成 Docker Image 首先需要在專案下新增一份 Dockerfile

在 Runner 的環境中是沒有 docker 指令可以用的，所以這邊啟動一個 Docker Container 在裡面執行就可以用 docker 指令了。

其中 $CI_COMMIT_SHA $CI_COMMIT_BEFORE_SHA 這兩個都是 GitLab 預設環境變數，代表這次 commit 還有上次 commit 的 SHA 值。

dind 則是直接啟動 docker daemon，此外 dind 還會自動產生 TLS certificates

為了在 Docker Container 內運行 Docker，會把 Host 上面的 Docker API 分享給 Container。

docker:stable 有執行 docker 需要的執行檔，他裡面也包含要啟動 docker 的程式(docker daemon)，但啟動 Container 的 entrypoint 是 sh

docker:dind 繼承自 docker:stable，而且它 entrypoint 就是啟動 docker 的腳本，此外還會做完 TLS certificates

Container 要去連 Host 上的 Docker API 。但現在連線失敗卻是找 http://docker:2375，現在的 dind 已經不是被當做 services 來用了，而是要直接在裡面跑 Docker，所以他應該是要 unix:///var/run/docker.sock 用這種連線，於是把環境變數 DOCKER_HOST 從 tcp://docker:2375 改成空字串，讓 docker daemon 走預設連線就能成功囉！

auto-deploy preparationhelm init 建立 helm 專案設定 tiller 在背景執行設定 cluster 的 namespace

auto-deploy deploy使用 helm upgrade 部署 chart 到 K8S 上透過 --set 來設定要注入 template 的參數

set -x，這樣就能在執行前，顯示指令內容。

用 helm repo list 看看現在有註冊哪些 Chart Repository

helm fetch gitlab/auto-deploy-app --untar

nohup 可以讓你在離線或登出系統後，還能夠讓工作繼續進行

在不特別設定 CI_APPLICATION_REPOSITORY 的情況下，image_repository 的值就是預設環境變數 CI_REGISTRY_IMAGE/CI_COMMIT_REF_SLUG

A:-B 的意思是如果有 A 就用它，沒有就用 B

研究 Auto Devops 難度最高的地方就是太多工具整合在一起，搞不清楚他們之間的關係，出錯也不知道從何查起

the ELB also performs a vital role in improving the fault tolerance of the services which it fronts.

he Open Systems Interconnection Model, or OSI Model, is a conceptual model which is used to facilitate communications between different computing systems.

Layer 1 is the physical layer, and represents the physical medium across which the request is sent.

Layer 2 describes the data link layer

Layer 3 (the network layer)

Layer 7, which serves the application layer.

A network device, of which the Classic ELB is an example, reads the protocol and port of the incoming request, and then routes it to one or more backend servers.

Whereas a request to a specific URL backed by a Classic ELB would only enable routing to a particular pool of homogeneous servers, the ALB can route based on the content of the URL, and direct to a specific subgroup of backing servers existing in a heterogeneous collection registered with the load balancer.

The Classic ELB is a simple load balancer, is easy to configure

As organizations move towards microservice architecture or adopt a container-based infrastructure, the ability to merely map a single address to a specific service becomes more complicated and harder to maintain.

oute traffic to different services based on either the host or the content of the path contained within that URL.

Some system unit configuration options do not work in the rootless container

it's better to create an override.conf drop-in that sets PrivateNetwork=no

Difficult to use additional stores for sharing content

Can not use overlayfs driver, but does support fuse-overlayfs

No CNI Support

Making device nodes within a container fails, even when running --privileged.

ngress controller helps to consolidate routing rules of multiple applications into one entity.

cloud load balancers are not necessary. Load balancer can also be implemented with MetalLB, which can be deployed in the same Kubernetes cluster.

Installing NGINX using NodePort is the most simple example for Ingress Controller as we can avoid the load balancer dependency. NodePort is used for exposing the NGINX Ingress to the external network.