Group items tagged

docker node inspect self --pretty

docker node update --availability drain node

use node labels in service constraints

The labels you set for nodes using docker node update apply only to the node entity within the swarm

node labels can be used to limit critical tasks to nodes that meet certain requirements

promote a worker node to the manager role

demote a manager node to the worker role

If the last manager node leaves the swarm, the swarm becomes unavailable requiring you to take disaster recovery measures.

DevOps is a culture, a movement, a philosophy.

a firm handshake between development and operations

DevOps isn’t magic, and transformations don’t happen overnight.

Culture is the #1 success factor in DevOps.

Building a culture of shared responsibility, transparency and faster feedback is the foundation of every high performing DevOps team.

 'not our problem' mentality

DevOps is that change in mindset of looking at the development process holistically and breaking down the barrier between Dev and Ops.

Speed is everything.

Open communication helps Dev and Ops teams swarm on issues, fix incidents, and unblock the release pipeline faster.

Unplanned work is a reality that every team faces–a reality that most often impacts team productivity.

“cross-functional collaboration.”

All the tooling and automation in the world are useless if they aren’t accompanied by a genuine desire on the part of development and IT/Ops professionals to work together.

Forming project- or product-oriented teams to replace function-based teams is a step in the right direction.

sharing a common goal and having a plan to reach it together

join sprint planning sessions, daily stand-ups, and sprint demos.

DevOps culture across every department

open channels of communication, and talk regularly

automation eliminates repetitive manual work, yields repeatable processes, and creates reliable systems.

continuous delivery: the practice of running each code change through a gauntlet of automated tests, often facilitated by cloud-based infrastructure, then packaging up successful builds and promoting them up toward production using automated deploys.

automated deploys alert IT/Ops to server “drift” between environments, which reduces or eliminates surprises when it’s time to release.

when DevOps uses automated deploys to send thoroughly tested code to identically provisioned environments, “Works on my machine!” becomes irrelevant.

A DevOps mindset sees opportunities for continuous improvement everywhere.

regular retrospectives

A/B testing

failure is inevitable. So you might as well set up your team to absorb it, recover, and learn from it (some call this “being anti-fragile”).

Postmortems focus on where processes fell down and how to strengthen them – not on which team member f'ed up the code.

Our engineers are responsible for QA, writing, and running their own tests to get the software out to customers.

How long did it take to go from development to deployment? 

How long does it take to recover after a system failure?

service level agreements (SLAs)

DevOps is big on the idea that the same people who build an application should be involved in shipping and running it.

developers and operators pair with each other in each phase of the application’s lifecycle.

決定每個 shard 要被分配到哪個 data node 上

為 cluster 設置多個 master node

一旦發現被選中的 master node 出現問題，就會選出新的 master node

處理 request 的 node 稱為 Coordinating Node，其功能是將 request 轉發到合適的 node 上

所有的 node 都預設是 Coordinating Node

可以保存資料的 node，每個 node 啟動後都會預設是 data node，可以透過設定 node.data: false 停用 data node 功能

由 master node 決定如何把分片分發到不同的 data node 上

每個 node 上都保存了 cluster state

只有 master 才可以修改 cluster state 並負責同步給其他 node

每個 node 都會詳細紀錄本身的狀態資訊

shard 是 Elasticsearch 分散式儲存的基礎，包含 primary shard & replica shard

每一個 shard 就是一個 Lucene instance

primary shard 功能是將一份被索引後的資料，分散到多個 data node 上存放，實現儲存方面的水平擴展

當 primary shard 遺失時，replica shard 就可以被 promote 成 primary shard 來保持資料完整性

replica shard 數量可以動態調整，讓每個 data node 上都有完整的資料

ES 7.0 開始，primary shard 預設為 1，replica shard 預設為 0

replica shard 若設定過多，會降低 cluster 整體的寫入效能

所有的 primary shard 可以在同一個 data node 上

透過 GET _cluster/health/<target> 可以取得目前 cluster 的健康狀態

Yellow：表示 primary shard 可以正常分配，但 replica shard 分配有問題

透過 GET /_cat/shards/<target> 可以取得目前的 shard 狀態

replica shard 無法被分配，因此 cluster 健康狀態為黃色

若是擔心 reboot 機器造成 failover 動作開始執行，可以設定將 replication 延遲一段時間後再執行(透過調整 settings 中的 index.unassigned.node_left.delayed_timeout 參數)，避免無謂的 data copy 動作 (此功能稱為 delay allocation)

集群變紅，代表有 primary shard 丟失，這個時候會影響讀寫。

如果 node 重新回來，會從 translog 中恢復沒有寫入的資料

設定 index settings 之後，primary shard 數量無法隨意變更

不建議直接發送請求到master節點，雖然也會工作，但是大量請求發送到 master，會有潛在的性能問題

shard 是 ES 中最小的工作單元

shard 是一個 Lucene 的 index

將 Index Buffer 中的內容寫入 Segment，而這寫入的過程就稱為 Refresh

當 document 被 refresh 進入到 segment 之後，就可以被搜尋到了

在進行 refresh 時先將 segment 寫入 cache 以開放查詢

將 document 進行索引時，同時也會寫入 transaction log，且預設都會寫入磁碟中

每個 shard 都會有對應的 transaction log

由於 transaction log 都會寫入磁碟中，因此當 node 從故障中恢復時，就會優先讀取 transaction log 來恢復資料

The generate attribute is used to inform Terragrunt to generate the Terraform code for configuring the backend.

The find_in_parent_folders() helper will automatically search up the directory tree to find the root terragrunt.hcl and inherit the remote_state configuration from it.

Unlike the backend configurations, provider configurations support variables,

if you needed to modify the configuration to expose another parameter (e.g session_name), you would have to then go through each of your modules to make this change.

instructs Terragrunt to create the file provider.tf in the working directory (where Terragrunt calls terraform) before it calls any of the Terraform commands

large modules should be considered harmful.

Large modules are slow, insecure, hard to update, hard to code review, hard to test, and brittle (i.e., you have all your eggs in one basket).

Terragrunt allows you to define your Terraform code once and to promote a versioned, immutable “artifact” of that exact same code from environment to environment.

You can use role-based access control (RBAC) and other security mechanisms to make sure that users and workloads can get access to the resources they need, while keeping workloads, and the cluster itself, secure. You can set limits on the resources that users and workloads can access by managing policies and container resources.

you need to plan how to scale to relieve increased pressure from more requests to the control plane and worker nodes or scale down to reduce unused resources.

Managed control plane: Let the provider manage the scale and availability of the cluster's control plane, as well as handle patches and upgrades.

The simplest Kubernetes cluster has the entire control plane and worker node services running on the same machine.

You can deploy a control plane using tools such as kubeadm, kops, and kubespray.

Secure communications between control plane services are implemented using certificates.

Separate and backup etcd service: The etcd services can either run on the same machines as other control plane services or run on separate machines

Create multiple control plane systems: For high availability, the control plane should not be limited to a single machine

Some deployment tools set up Raft consensus algorithm to do leader election of Kubernetes services. If the primary goes away, another service elects itself and take over.

Groups of zones are referred to as regions.

if you installed with kubeadm, there are instructions to help you with Certificate Management and Upgrading kubeadm clusters.

Production-quality workloads need to be resilient and anything they rely on needs to be resilient (such as CoreDNS).

Add nodes to the cluster: If you are managing your own cluster you can add nodes by setting up your own machines and either adding them manually or having them register themselves to the cluster’s apiserver.

Set up node health checks: For important workloads, you want to make sure that the nodes and pods running on those nodes are healthy.

Authentication: The apiserver can authenticate users using client certificates, bearer tokens, an authenticating proxy, or HTTP basic auth.

Authorization: When you set out to authorize your regular users, you will probably choose between RBAC and ABAC authorization.

Role-based access control (RBAC): Lets you assign access to your cluster by allowing specific sets of permissions to authenticated users. Permissions can be assigned for a specific namespace (Role) or across the entire cluster (ClusterRole).

Attribute-based access control (ABAC): Lets you create policies based on resource attributes in the cluster and will allow or deny access based on those attributes.

Set limits on workload resources

Set namespace limits: Set per-namespace quotas on things like memory and CPU

Prepare for DNS demand: If you expect workloads to massively scale up, your DNS service must be ready to scale up as well.