Group items tagged

Auto DevOps works with any Kubernetes cluster;

using the Docker or Kubernetes executor, with privileged mode enabled.

Base domain (needed for Auto Review Apps and Auto Deploy)

Kubernetes (needed for Auto Review Apps, Auto Deploy, and Auto Monitoring)

Prometheus (needed for Auto Monitoring)

scrape your Kubernetes cluster.

project level as a variable: KUBE_INGRESS_BASE_DOMAIN

A wildcard DNS A record matching the base domain(s) is required

Once set up, all requests will hit the load balancer, which in turn will route them to the Kubernetes pods that run your application(s).

review/ (every environment starting with review/)

staging

production

need to define a separate KUBE_INGRESS_BASE_DOMAIN variable for all the above based on the environment.

Continuous deployment to production: Enables Auto Deploy with master branch directly deployed to production.

Continuous deployment to production using timed incremental rollout

Automatic deployment to staging, manual deployment to production

Auto Build creates a build of the application using an existing Dockerfile or Heroku buildpacks.

If a project’s repository contains a Dockerfile, Auto Build will use docker build to create a Docker image.

Each buildpack requires certain files to be in your project’s repository for Auto Build to successfully build your application.

Auto Test automatically runs the appropriate tests for your application using Herokuish and Heroku buildpacks by analyzing your project to detect the language and framework.

Auto Code Quality uses the Code Quality image to run static analysis and other code checks on the current code.

Static Application Security Testing (SAST) uses the SAST Docker image to run static analysis on the current code and checks for potential security issues.

Dependency Scanning uses the Dependency Scanning Docker image to run analysis on the project dependencies and checks for potential security issues.

License Management uses the License Management Docker image to search the project dependencies for their license.

Vulnerability Static Analysis for containers uses Clair to run static analysis on a Docker image and checks for potential security issues.

Review Apps are temporary application environments based on the branch’s code so developers, designers, QA, product managers, and other reviewers can actually see and interact with code changes as part of the review process. Auto Review Apps create a Review App for each branch. Auto Review Apps will deploy your app to your Kubernetes cluster only. When no cluster is available, no deployment will occur.

The Review App will have a unique URL based on the project ID, the branch or tag name, and a unique number, combined with the Auto DevOps base domain.

Your apps should not be manipulated outside of Helm (using Kubernetes directly).

Dynamic Application Security Testing (DAST) uses the popular open source tool OWASP ZAProxy to perform an analysis on the current code and checks for potential security issues.

Auto Browser Performance Testing utilizes the Sitespeed.io container to measure the performance of a web page.

add the paths to a file named .gitlab-urls.txt in the root directory, one per line.

After a branch or merge request is merged into the project’s default branch (usually master), Auto Deploy deploys the application to a production environment in the Kubernetes cluster, with a namespace based on the project name and unique project ID

Auto Deploy doesn’t include deployments to staging or canary by default, but the Auto DevOps template contains job definitions for these tasks if you want to enable them.

For internal and private projects a GitLab Deploy Token will be automatically created, when Auto DevOps is enabled and the Auto DevOps settings are saved.

If the GitLab Deploy Token cannot be found, CI_REGISTRY_PASSWORD is used. Note that CI_REGISTRY_PASSWORD is only valid during deployment.

If present, DB_INITIALIZE will be run as a shell command within an application pod as a helm post-install hook.

a post-install hook means that if any deploy succeeds, DB_INITIALIZE will not be processed thereafter.

DB_MIGRATE will be run as a shell command within an application pod as a helm pre-upgrade hook.

Once your application is deployed, Auto Monitoring makes it possible to monitor your application’s server and response metrics right out of the box.

annotate the NGINX Ingress deployment to be scraped by Prometheus using prometheus.io/scrape: "true" and prometheus.io/port: "10254"

While Auto DevOps provides great defaults to get you started, you can customize almost everything to fit your needs; from custom buildpacks, to Dockerfiles, Helm charts, or even copying the complete CI/CD configuration into your project to enable staging and canary deployments, and more.

Auto DevOps uses Helm to deploy your application to Kubernetes.

make use of the HELM_UPGRADE_EXTRA_ARGS environment variable to override the default values in the values.yaml file in the default Helm chart.

specify the use of a custom Helm chart per environment by scoping the environment variable to the desired environment.

Your additions will be merged with the Auto DevOps template using the behaviour described for include

copy and paste the contents of the Auto DevOps template into your project and edit this as needed.

In order to support applications that require a database, PostgreSQL is provisioned by default.

Set up the replica variables using a project variable and scale your application by just redeploying it!

You should not scale your application using Kubernetes directly.

Some applications need to define secret variables that are accessible by the deployed application.

Auto DevOps pipelines will take your application secret variables to populate a Kubernetes secret.

Environment variables are generally considered immutable in a Kubernetes pod.

If STAGING_ENABLED is defined in your project (e.g., set STAGING_ENABLED to 1 as a CI/CD variable), then the application will be automatically deployed to a staging environment, and a production_manual job will be created for you when you’re ready to manually deploy to production.

If CANARY_ENABLED is defined in your project (e.g., set CANARY_ENABLED to 1 as a CI/CD variable) then two manual jobs will be created: canary which will deploy the application to the canary environment production_manual which is to be used by you when you’re ready to manually deploy to production.

If INCREMENTAL_ROLLOUT_MODE is set to manual in your project, then instead of the standard production job, 4 different manual jobs will be created: rollout 10% rollout 25% rollout 50% rollout 100%

To start a job, click on the play icon next to the job’s name.

not all buildpacks support Auto Test yet

When a project has been marked as private, GitLab’s Container Registry requires authentication when downloading containers.

Authentication credentials will be valid while the pipeline is running, allowing for a successful initial deployment.

We strongly advise using GitLab Container Registry with Auto DevOps in order to simplify configuration and prevent any unforeseen issues.

其實 Auto DevOps 就是一份官方寫好的 gitlab-ci.yml，在啟動 Auto DevOps 的專案裡，如果找不到 gitlab-ci.yml 檔，那就會直接用官方 gitlab-ci.yml 去跑 CI / CD 流程。

Pod 是 K8S 中可以被部署的最小元件，一個 Pod 是由一到多個 Container 組成，同個 Pod 的不同 Container 之間彼此共享網路資源。

Node 又分成 Worker Node 和 Master Node 兩種

Helm 透過參數 (parameter) 跟模板 (template) 的方式，讓我們可以在只修改參數的方式重複利用模板。

為了要有 CI CD 的功能我們會把 .gitlab-ci.yml 放在專案的根目錄裡， GitLab 會依造 .gitlab-ci.yml 的設定產生 CI/CD Pipeline，每個 Pipeline 裡面可能有多個 Job，這時候就會需要有 GitLab Runner 來執行這些 Job 並把執行的結果回傳給 GitLab 讓它知道這個 Job 是否有正常執行。

把專案打包成 Docker Image 這工作又或是 helm 的操作都會在 Container 內執行

CI/CD Pipeline 是由 stage 還有 job 組成的，stage 是有順序性的，前面的 stage 完成後才會開始下一個 stage。

每個 stage 裡面包含一到多個 Job

Auto Devops 裡也會大量用到這種在指定 Container 內運行的工作。

可以通過 health checks

開 private 的話還要注意使用 Container Registry 的權限問題

申請好的 wildcard 的 DNS

Auto Devops 也提供只要設定環境變數就能一定程度客製化的選項

特別注意 namespace 有沒有設定對，不然會找不到資料喔

Auto Devops，如果想要進一步的客製化，而且是改 GitLab 環境變數都無法實現的客製化，這時候還是得回到 .gitlab-ci.yml 設定檔

在 Docker in Docker 的環境用 Dockerfile 打包 Image

用 helm upgrade 把 chart 部署到 K8S 上

把專案打包成 Docker Image 首先需要在專案下新增一份 Dockerfile

在 Runner 的環境中是沒有 docker 指令可以用的，所以這邊啟動一個 Docker Container 在裡面執行就可以用 docker 指令了。

其中 $CI_COMMIT_SHA $CI_COMMIT_BEFORE_SHA 這兩個都是 GitLab 預設環境變數，代表這次 commit 還有上次 commit 的 SHA 值。

dind 則是直接啟動 docker daemon，此外 dind 還會自動產生 TLS certificates

為了在 Docker Container 內運行 Docker，會把 Host 上面的 Docker API 分享給 Container。

docker:stable 有執行 docker 需要的執行檔，他裡面也包含要啟動 docker 的程式(docker daemon)，但啟動 Container 的 entrypoint 是 sh

docker:dind 繼承自 docker:stable，而且它 entrypoint 就是啟動 docker 的腳本，此外還會做完 TLS certificates

Container 要去連 Host 上的 Docker API 。但現在連線失敗卻是找 http://docker:2375，現在的 dind 已經不是被當做 services 來用了，而是要直接在裡面跑 Docker，所以他應該是要 unix:///var/run/docker.sock 用這種連線，於是把環境變數 DOCKER_HOST 從 tcp://docker:2375 改成空字串，讓 docker daemon 走預設連線就能成功囉！

auto-deploy preparationhelm init 建立 helm 專案設定 tiller 在背景執行設定 cluster 的 namespace

auto-deploy deploy使用 helm upgrade 部署 chart 到 K8S 上透過 --set 來設定要注入 template 的參數

set -x，這樣就能在執行前，顯示指令內容。

用 helm repo list 看看現在有註冊哪些 Chart Repository

helm fetch gitlab/auto-deploy-app --untar

nohup 可以讓你在離線或登出系統後，還能夠讓工作繼續進行

在不特別設定 CI_APPLICATION_REPOSITORY 的情況下，image_repository 的值就是預設環境變數 CI_REGISTRY_IMAGE/CI_COMMIT_REF_SLUG

A:-B 的意思是如果有 A 就用它，沒有就用 B

研究 Auto Devops 難度最高的地方就是太多工具整合在一起，搞不清楚他們之間的關係，出錯也不知道從何查起

skips for all the scans, as a way of speeding up the build process while working on the CI configuration

The Auto Devops test job, which uses Herokuish for testing, does not rely on the Docker image that’s generated during the Build job

moving the Test job to the Build stage to speed things along

Literally any part of Auto Devops can be overridden in your own CI configuration.

you don’t need to set up the deployment upfront. Auto DevOps still builds and tests your application. You can define the deployment later.

ship your app first, then explore the customizations later.

Consistency

Auto DevOps works with any Kubernetes cluster.

To use Auto DevOps for individual projects, you can enable it in a project-by-project basis.

Only project Maintainers can enable or disable Auto DevOps at the project level.

We strongly advise you to use GitLab Container Registry with Auto DevOps to simplify configuration and prevent any unforeseen issues.

The GitLab integration with Helm does not support installing applications when behind a proxy.

A build system, git, and development headers for many popular libraries, so that the most popular Ruby, Python and Node.js native extensions can be compiled without problems.

Nginx 1.18. Disabled by default

production-grade features, such as process monitoring, administration and status inspection.

Redis 5.0. Not installed by default.

The image has an app user with UID 9999 and home directory /home/app. Your application is supposed to run as this user.

Passenger works like a mod_ruby, mod_nodejs, etc. It changes Nginx into an application server and runs your app from Nginx.

placing a .conf file in the directory /etc/nginx/sites-enabled

The best way to configure Nginx is by adding .conf files to /etc/nginx/main.d and /etc/nginx/conf.d

To preserve these variables, place an Nginx config file ending with *.conf in the directory /etc/nginx/main.d, in which you tell Nginx to preserve these variables.

By default, Phusion Passenger sets all of the following environment variables to the value production

PASSENGER_APP_ENV environment variable

passenger-docker autogenerates an Nginx configuration file (/etc/nginx/conf.d/00_app_env.conf) during container boot.

The configuration file is in /etc/redis/redis.conf. Modify it as you see fit, but make sure daemonize no is set.

You can add additional daemons to the image by creating runit entries.

the shell script must run the daemon without letting it daemonize/fork it.

We use RVM to install and to manage Ruby interpreters.

Don't need to disable Docker's iptables and let Docker to manage it's network.

The public network cannot access ports that published by Docker.

In a very convenient way to allow/deny public networks to access container ports without additional software and extra configurations

Enable Docker's iptables feature. Remove all changes like --iptables=false , including configuration file /etc/docker/daemon.json

Modify the UFW configuration file /etc/ufw/after.rules

There may be some unknown reasons cause the UFW rules will not take effect after restart UFW, please reboot servers.

If we publish a port by using option -p 8080:80, we should use the container port 80, not the host port 8080

allow the private networks to be able to visit each other.

The following rules block connection requests initiated by all public networks, but allow internal networks to access external networks.

Since the UDP protocol is stateless, it is not possible to block the handshake signal that initiates the connection request as TCP does.

For GNU/Linux we can find the local port range in the file /proc/sys/net/ipv4/ip_local_port_range. The default range is 32768 60999

It not only exposes ports of containers but also exposes ports of the host.

Cannot expose services running on hosts and containers at the same time by the same command.

the desire for an interactive approval step between plan and apply.

terraform init -input=false to initialize the working directory.

terraform plan -out=tfplan -input=false to create a plan and save it to the local file tfplan.

terraform apply -input=false tfplan to apply the plan stored in the file tfplan.

the environment variable TF_IN_AUTOMATION is set to any non-empty value, Terraform makes some minor adjustments to its output to de-emphasize specific commands to run.

it can be difficult or impossible to ensure that the plan and apply subcommands are run on the same machine, in the same directory, with all of the same files present.

to allow only one plan to be outstanding at a time.

forcing plans to be approved (or dismissed) in sequence

The -auto-approve option tells Terraform not to require interactive approval of the plan before applying it.

obtain the archive created in the previous step and extract it at the same absolute path. This re-creates everything that was present after plan, avoiding strange issues where local files were created during the plan step.

a "build artifact"

主从关系（Master-Slave），此时所有操作还是由主数据库完成，主数据库再同步到从数据库上，而从数据库只需要在主数据库挂掉之后代替其工作。

一般来说读写分离加上缓存已经可以应付绝大多数情况了，并且几乎不需要对业务层面进行修改。

基于 id 的区间分片，例如：将 id 为 1-2w 的数据存放在 A 数据库，2w-4w 的数据存放在 B 数据库。

基于 id 的 hash 分片，例如：将 id%2=0 的数据存放在 A 数据库，id%2=1 的数据放在 B 数据库。

基于时间的区间分片，大部分软件都会有一个特征：越新的数据被操作的几率越大，老数据几乎不会被操作。所以通过数据的插入时间进行分库（也称为冷热分离）

基于检索表分片，通过额外建立一张检索表保存 id 与所在数据库节点的对应关系，优点是逻辑简单，自由且不会有迁移问题，缺点是每次查询都需要额外查询检索表，所以一般会选择将检索表缓存到内存中。

基于地理位置分片

分库策略

在数据库表设计时，为了保证 id 唯一，大部分人都会将主键设为自增的 int 类型。但是由于 auto\_increment 是和表所绑定的，所以在分库后每个表的自增 id 也是独立的。这样就肯定会发生主键冲突

但是很多人都希望主键即使不是连续自增，也是一个有序的整数，这样在排序等情况下会有用。这时候就需要自己实现一个 id 生成算法了，一般都会使用 unix 时间戳保持有序，混入 Mac 地址等保证唯一。

When you run an image and generate a container, you add a new writable layer (the “container layer”) on top of the underlying layers.

Inadvertently including files that are not necessary for building an image results in a larger build context and larger image size.

To exclude files not relevant to the build (without restructuring your source repository) use a .dockerignore file. This file supports exclusion patterns similar to .gitignore files.

minimize image layers by leveraging build cache.

avoid installing extra or unnecessary packages just because they might be “nice to have.”

Decoupling applications into multiple containers makes it easier to scale horizontally and reuse containers

Limiting each container to one process is a good rule of thumb, but it is not a hard and fast rule.

do multi-stage builds and only copy the artifacts you need into the final image. This allows you to include tools and debug information in your intermediate build stages without increasing the size of the final image.

avoid duplication of packages and make the list much easier to update.

When building an image, Docker steps through the instructions in your Dockerfile, executing each in the order specified.

simply comparing the instruction in the Dockerfile with one of the child images is sufficient.

For the ADD and COPY instructions, the contents of the file(s) in the image are examined and a checksum is calculated for each file.

If anything has changed in the file(s), such as the contents and metadata, then the cache is invalidated.

In that case just the command string itself is used to find a match.

Whenever possible, use current official repositories as the basis for your images.

Using RUN apt-get update && apt-get install -y ensures your Dockerfile installs the latest package versions with no further coding or manual intervention.

cache busting

Docker executes these commands using the /bin/sh -c interpreter, which only evaluates the exit code of the last operation in the pipe to determine success.

set -o pipefail && to ensure that an unexpected error prevents the build from inadvertently succeeding.

The CMD instruction should be used to run the software contained by your image, along with any arguments.

CMD should rarely be used in the manner of CMD [“param”, “param”] in conjunction with ENTRYPOINT

The ENV instruction is also useful for providing required environment variables specific to services you wish to containerize,

COPY only supports the basic copying of local files into the container

the best use for ADD is local tar file auto-extraction into the image, as in ADD rootfs.tar.xz /

using ADD to fetch packages from remote URLs is strongly discouraged; you should use curl or wget instead

The best use for ENTRYPOINT is to set the image’s main command, allowing that image to be run as though it was that command (and then use CMD as the default flags).

the image name can double as a reference to the binary as shown in the command

The VOLUME instruction should be used to expose any database storage area, configuration storage, or files/folders created by your docker container.

use VOLUME for any mutable and/or user-serviceable parts of your image

If you absolutely need functionality similar to sudo, such as initializing the daemon as root but running it as non-root), consider using “gosu”.

always use absolute paths for your WORKDIR

An ONBUILD command executes after the current Dockerfile build completes.

Think of the ONBUILD command as an instruction the parent Dockerfile gives to the child Dockerfile

Be careful when putting ADD or COPY in ONBUILD. The “onbuild” image fails catastrophically if the new build’s context is missing the resource being added.

.tf files can accept values from input variables.

variable definitions can have default values assigned to them.

values are stored in separate files with the .tfvars extension.

looks through the working directory for a file named terraform.tfvars, or for files with the .auto.tfvars extension.

add the terraform.tfvars file to your .gitignore file and keep it out of version control.

include an example terraform.tfvars.example in your Git repository with all of the variable names recorded (but none of the values entered).

terraform apply -var-file=myvars.tfvars

Terraform allows you to keep input variable values in environment variables.

If Terraform does not find a default value for a defined variable; or a value from a .tfvars file, environment variable, or CLI flag; it will prompt you for a value before running an action

state file contains a JSON object that holds your managed infrastructure’s current state

state is a snapshot of the various attributes of your infrastructure at the time it was last modified

Some backends, like Consul, also allow for state locking. If one user is applying a state, another user will be unable to make any changes.

Terraform backends allow the user to securely store their state in a remote location, such as a key/value store like Consul, or an S3 compatible bucket storage like Minio.

at minimum the repository should be private.

A controller tracks at least one Kubernetes resource type.

The controller(s) for that resource are responsible for making the current state come closer to that desired state.

in Kubernetes, a controller will send messages to the API server that have useful side effects.

Built-in controllers manage state by interacting with the cluster API server.

By contrast with Job, some controllers need to make changes to things outside of your cluster.

As long as the controllers for your cluster are running and able to make useful changes, it doesn't matter if the overall state is stable or not.

Kubernetes uses lots of controllers that each manage a particular aspect of cluster state.

a particular control loop (controller) uses one kind of resource as its desired state, and has a different kind of resource that it manages to make that desired state happen.

There can be several controllers that create or update the same kind of object.