Group items tagged

其實 Auto DevOps 就是一份官方寫好的 gitlab-ci.yml，在啟動 Auto DevOps 的專案裡，如果找不到 gitlab-ci.yml 檔，那就會直接用官方 gitlab-ci.yml 去跑 CI / CD 流程。

Pod 是 K8S 中可以被部署的最小元件，一個 Pod 是由一到多個 Container 組成，同個 Pod 的不同 Container 之間彼此共享網路資源。

Node 又分成 Worker Node 和 Master Node 兩種

Helm 透過參數 (parameter) 跟模板 (template) 的方式，讓我們可以在只修改參數的方式重複利用模板。

為了要有 CI CD 的功能我們會把 .gitlab-ci.yml 放在專案的根目錄裡， GitLab 會依造 .gitlab-ci.yml 的設定產生 CI/CD Pipeline，每個 Pipeline 裡面可能有多個 Job，這時候就會需要有 GitLab Runner 來執行這些 Job 並把執行的結果回傳給 GitLab 讓它知道這個 Job 是否有正常執行。

把專案打包成 Docker Image 這工作又或是 helm 的操作都會在 Container 內執行

CI/CD Pipeline 是由 stage 還有 job 組成的，stage 是有順序性的，前面的 stage 完成後才會開始下一個 stage。

每個 stage 裡面包含一到多個 Job

Auto Devops 裡也會大量用到這種在指定 Container 內運行的工作。

可以通過 health checks

開 private 的話還要注意使用 Container Registry 的權限問題

申請好的 wildcard 的 DNS

Auto Devops 也提供只要設定環境變數就能一定程度客製化的選項

特別注意 namespace 有沒有設定對，不然會找不到資料喔

Auto Devops，如果想要進一步的客製化，而且是改 GitLab 環境變數都無法實現的客製化，這時候還是得回到 .gitlab-ci.yml 設定檔

在 Docker in Docker 的環境用 Dockerfile 打包 Image

用 helm upgrade 把 chart 部署到 K8S 上

把專案打包成 Docker Image 首先需要在專案下新增一份 Dockerfile

在 Runner 的環境中是沒有 docker 指令可以用的，所以這邊啟動一個 Docker Container 在裡面執行就可以用 docker 指令了。

其中 $CI_COMMIT_SHA $CI_COMMIT_BEFORE_SHA 這兩個都是 GitLab 預設環境變數，代表這次 commit 還有上次 commit 的 SHA 值。

dind 則是直接啟動 docker daemon，此外 dind 還會自動產生 TLS certificates

為了在 Docker Container 內運行 Docker，會把 Host 上面的 Docker API 分享給 Container。

docker:stable 有執行 docker 需要的執行檔，他裡面也包含要啟動 docker 的程式(docker daemon)，但啟動 Container 的 entrypoint 是 sh

docker:dind 繼承自 docker:stable，而且它 entrypoint 就是啟動 docker 的腳本，此外還會做完 TLS certificates

Container 要去連 Host 上的 Docker API 。但現在連線失敗卻是找 http://docker:2375，現在的 dind 已經不是被當做 services 來用了，而是要直接在裡面跑 Docker，所以他應該是要 unix:///var/run/docker.sock 用這種連線，於是把環境變數 DOCKER_HOST 從 tcp://docker:2375 改成空字串，讓 docker daemon 走預設連線就能成功囉！

auto-deploy preparationhelm init 建立 helm 專案設定 tiller 在背景執行設定 cluster 的 namespace

auto-deploy deploy使用 helm upgrade 部署 chart 到 K8S 上透過 --set 來設定要注入 template 的參數

set -x，這樣就能在執行前，顯示指令內容。

用 helm repo list 看看現在有註冊哪些 Chart Repository

helm fetch gitlab/auto-deploy-app --untar

nohup 可以讓你在離線或登出系統後，還能夠讓工作繼續進行

在不特別設定 CI_APPLICATION_REPOSITORY 的情況下，image_repository 的值就是預設環境變數 CI_REGISTRY_IMAGE/CI_COMMIT_REF_SLUG

A:-B 的意思是如果有 A 就用它，沒有就用 B

研究 Auto Devops 難度最高的地方就是太多工具整合在一起，搞不清楚他們之間的關係，出錯也不知道從何查起

Docker is developed in the Go language and utilizes LXC, cgroups, and the Linux kernel itself. Since it’s based on LXC, a Docker container does not include a separate operating system; instead it relies on the operating system’s own functionality as provided by the underlying infrastructure.

a VE there is no preloaded emulation manager software as in a VM.

LXC will boast bare metal performance characteristics because it only packages the needed applications.

a VM, which packages the entire OS and machine setup, including hard drive, virtual processors and network interfaces. The resulting bloated mass usually takes a long time to boot and consumes a lot of CPU and RAM.

don’t offer some other neat features of VM’s such as IaaS setups and live migration.

LXC as supercharged chroot on Linux. It allows you to not only isolate applications, but even the entire OS.

Libvirt, which allows the use of containers through the LXC driver by connecting to 'lxc:///'.

'LXC', is not compatible with libvirt, but is more flexible with more userspace tools.

Portable deployment across machines

Versioning: Docker includes git-like capabilities for tracking successive versions of a container

Component reuse: Docker allows building or stacking of already created packages.

Shared libraries: There is already a public registry (http://index.docker.io/ ) where thousands have already uploaded the useful containers they have created.

Docker taking the devops world by storm since its launch back in 2013.

LXC, while older, has not been as popular with developers as Docker has proven to be

LXC having a focus on sys admins that’s similar to what solutions like the Solaris operating system, with its Solaris Zones, Linux OpenVZ, and FreeBSD, with its BSD Jails virtualization system

it started out being built on top of LXC, Docker later moved beyond LXC containers to its own execution environment called libcontainer.

LXC tooling sticks close to what system administrators running bare metal servers are used to

The LXC command line provides essential commands that cover routine management tasks, including the creation, launch, and deletion of LXC containers.

Docker containers aim to be even lighter weight in order to support the fast, highly scalable, deployment of applications with microservice architecture.

With backing from Canonical, LXC and LXD have an ecosystem tightly bound to the rest of the open source Linux community.

Docker Swarm

Docker Trusted Registry

Docker Compose

Docker Machine

Kubernetes facilitates the deployment of containers in your data center by representing a cluster of servers as a single system.

Swarm is Docker’s clustering, scheduling and orchestration tool for managing a cluster of Docker hosts. 

rkt is a security minded container engine that uses KVM for VM-based isolation and packs other enhanced security features. 

Apache Mesos can run different kinds of distributed jobs, including containers. 

Elastic Container Service is Amazon’s service for running and orchestrating containerized applications on AWS

LXC offers the advantages of a VE on Linux, mainly the ability to isolate your own private workloads from one another. It is a cheaper and faster solution to implement than a VM, but doing so requires a bit of extra learning and expertise.

that /var/lib/docker should be a volume. This is important, because the filesystem of a container is an AUFS mountpoint, composed of multiple branches; and those branches have to be “normal” filesystems (i.e. not AUFS mountpoints).

since the private Docker instances run in privileged mode, they can easily escalate to the host, and you probably don’t want this! If you really want to run something like this and expose it to the public, you will have to fine-tune the LXC template file, to restrict the capabilities and devices available to the Docker instances.

When you are inside a privileged container, you can always nest one more level

the LXC tools cannot start nested containers if the devices control group is not in its own hierarchy.

if you use AppArmor, you need a special policy to support nested containers.

After the block type keyword and any labels, the block body is delimited by the { and } characters

Identifiers can contain letters, digits, underscores (_), and hyphens (-). The first character of an identifier must not be a digit, to avoid ambiguity with literal numbers.

The # single-line comment style is the default comment style and should be used in most cases.

he idiomatic style is to use the Unix convention

Indent two spaces for each nesting level.

align their equals signs

Use empty lines to separate logical groups of arguments within a block.

Use one blank line to separate the arguments from the blocks.

"meta-arguments" (as defined by the Terraform language semantics)

Avoid separating multiple blocks of the same type with other blocks of a different type, unless the block types are defined by semantics to form a family.

Resource names must start with a letter or underscore, and may contain only letters, digits, underscores, and dashes.

By convention, resource type names start with their provider's preferred local name.

Most publicly available providers are distributed on the Terraform Registry, which also hosts their documentation.

The Terraform language defines several meta-arguments, which can be used with any resource type to change the behavior of resources.

use precondition and postcondition blocks to specify assumptions and guarantees about how the resource operates.

Some resource types provide a special timeouts nested block argument that allows you to customize how long certain operations are allowed to take before being considered to have failed.

Timeouts are handled entirely by the resource type implementation in the provider

Most resource types do not support the timeouts block at all.

A resource block declares that you want a particular infrastructure object to exist with the given settings.

Destroy resources that exist in the state but no longer exist in the configuration.

Destroy and re-create resources whose arguments have changed but which cannot be updated in-place due to remote API limitations.

Expressions within a Terraform module can access information about resources in the same module, and you can use that information to help configure other resources. Use the <RESOURCE TYPE>.<NAME>.<ATTRIBUTE> syntax to reference a resource attribute in an expression.

resources often provide read-only attributes with information obtained from the remote API; this often includes things that can't be known until the resource is created, like the resource's unique random ID.

data sources, which are a special type of resource used only for looking up information.

some dependencies cannot be recognized implicitly in configuration.

local-only resource types exist for generating private keys, issuing self-signed TLS certificates, and even generating random ids.

The count meta-argument accepts a whole number, and creates that many instances of the resource or module.

the count value must be known before Terraform performs any remote resource actions. This means count can't refer to any resource attributes that aren't known until after a configuration is applied

Within nested provisioner or connection blocks, the special self object refers to the current resource instance, not the resource block as a whole.

Don't need to disable Docker's iptables and let Docker to manage it's network.

The public network cannot access ports that published by Docker.

In a very convenient way to allow/deny public networks to access container ports without additional software and extra configurations

Enable Docker's iptables feature. Remove all changes like --iptables=false , including configuration file /etc/docker/daemon.json

Modify the UFW configuration file /etc/ufw/after.rules

There may be some unknown reasons cause the UFW rules will not take effect after restart UFW, please reboot servers.

If we publish a port by using option -p 8080:80, we should use the container port 80, not the host port 8080

allow the private networks to be able to visit each other.

The following rules block connection requests initiated by all public networks, but allow internal networks to access external networks.

Since the UDP protocol is stateless, it is not possible to block the handshake signal that initiates the connection request as TCP does.

For GNU/Linux we can find the local port range in the file /proc/sys/net/ipv4/ip_local_port_range. The default range is 32768 60999

It not only exposes ports of containers but also exposes ports of the host.

Cannot expose services running on hosts and containers at the same time by the same command.