Skip to main content

Home/ Larvata/ Group items tagged devops

Rss Feed Group items tagged

張 旭

Auto DevOps | GitLab - 0 views

  • Auto DevOps provides pre-defined CI/CD configuration which allows you to automatically detect, build, test, deploy, and monitor your applications
  • Just push your code and GitLab takes care of everything else.
  • Auto DevOps will be automatically disabled on the first pipeline failure.
  • ...78 more annotations...
  • Your project will continue to use an alternative CI/CD configuration file if one is found
  • Auto DevOps works with any Kubernetes cluster;
  • using the Docker or Kubernetes executor, with privileged mode enabled.
  • Base domain (needed for Auto Review Apps and Auto Deploy)
  • Kubernetes (needed for Auto Review Apps, Auto Deploy, and Auto Monitoring)
  • Prometheus (needed for Auto Monitoring)
  • scrape your Kubernetes cluster.
  • project level as a variable: KUBE_INGRESS_BASE_DOMAIN
  • A wildcard DNS A record matching the base domain(s) is required
  • Once set up, all requests will hit the load balancer, which in turn will route them to the Kubernetes pods that run your application(s).
  • review/ (every environment starting with review/)
  • staging
  • production
  • need to define a separate KUBE_INGRESS_BASE_DOMAIN variable for all the above based on the environment.
  • Continuous deployment to production: Enables Auto Deploy with master branch directly deployed to production.
  • Continuous deployment to production using timed incremental rollout
  • Automatic deployment to staging, manual deployment to production
  • Auto Build creates a build of the application using an existing Dockerfile or Heroku buildpacks.
  • If a project’s repository contains a Dockerfile, Auto Build will use docker build to create a Docker image.
  • Each buildpack requires certain files to be in your project’s repository for Auto Build to successfully build your application.
  • Auto Test automatically runs the appropriate tests for your application using Herokuish and Heroku buildpacks by analyzing your project to detect the language and framework.
  • Auto Code Quality uses the Code Quality image to run static analysis and other code checks on the current code.
  • Static Application Security Testing (SAST) uses the SAST Docker image to run static analysis on the current code and checks for potential security issues.
  • Dependency Scanning uses the Dependency Scanning Docker image to run analysis on the project dependencies and checks for potential security issues.
  • License Management uses the License Management Docker image to search the project dependencies for their license.
  • Vulnerability Static Analysis for containers uses Clair to run static analysis on a Docker image and checks for potential security issues.
  • Review Apps are temporary application environments based on the branch’s code so developers, designers, QA, product managers, and other reviewers can actually see and interact with code changes as part of the review process. Auto Review Apps create a Review App for each branch. Auto Review Apps will deploy your app to your Kubernetes cluster only. When no cluster is available, no deployment will occur.
  • The Review App will have a unique URL based on the project ID, the branch or tag name, and a unique number, combined with the Auto DevOps base domain.
  • Review apps are deployed using the auto-deploy-app chart with Helm, which can be customized.
  • Your apps should not be manipulated outside of Helm (using Kubernetes directly).
  • Dynamic Application Security Testing (DAST) uses the popular open source tool OWASP ZAProxy to perform an analysis on the current code and checks for potential security issues.
  • Auto Browser Performance Testing utilizes the Sitespeed.io container to measure the performance of a web page.
  • add the paths to a file named .gitlab-urls.txt in the root directory, one per line.
  • After a branch or merge request is merged into the project’s default branch (usually master), Auto Deploy deploys the application to a production environment in the Kubernetes cluster, with a namespace based on the project name and unique project ID
  • Auto Deploy doesn’t include deployments to staging or canary by default, but the Auto DevOps template contains job definitions for these tasks if you want to enable them.
  • Apps are deployed using the auto-deploy-app chart with Helm.
  • For internal and private projects a GitLab Deploy Token will be automatically created, when Auto DevOps is enabled and the Auto DevOps settings are saved.
  • If the GitLab Deploy Token cannot be found, CI_REGISTRY_PASSWORD is used. Note that CI_REGISTRY_PASSWORD is only valid during deployment.
  • If present, DB_INITIALIZE will be run as a shell command within an application pod as a helm post-install hook.
  • a post-install hook means that if any deploy succeeds, DB_INITIALIZE will not be processed thereafter.
  • DB_MIGRATE will be run as a shell command within an application pod as a helm pre-upgrade hook.
    • 張 旭
       
      如果專案類型不同,就要去查 buildpacks 裡面如何叫用該指令,例如 laravel 的 migration
    • 張 旭
       
      如果是自己的 Dockerfile 建立起來的,看來就不用鳥 buildpacks 的作法
  • Once your application is deployed, Auto Monitoring makes it possible to monitor your application’s server and response metrics right out of the box.
  • annotate the NGINX Ingress deployment to be scraped by Prometheus using prometheus.io/scrape: "true" and prometheus.io/port: "10254"
  • If you are also using Auto Review Apps and Auto Deploy and choose to provide your own Dockerfile, make sure you expose your application to port 5000 as this is the port assumed by the default Helm chart.
  • While Auto DevOps provides great defaults to get you started, you can customize almost everything to fit your needs; from custom buildpacks, to Dockerfiles, Helm charts, or even copying the complete CI/CD configuration into your project to enable staging and canary deployments, and more.
  • If your project has a Dockerfile in the root of the project repo, Auto DevOps will build a Docker image based on the Dockerfile rather than using buildpacks.
  • Auto DevOps uses Helm to deploy your application to Kubernetes.
  • Bundled chart - If your project has a ./chart directory with a Chart.yaml file in it, Auto DevOps will detect the chart and use it instead of the default one.
  • Create a project variable AUTO_DEVOPS_CHART with the URL of a custom chart to use or create two project variables AUTO_DEVOPS_CHART_REPOSITORY with the URL of a custom chart repository and AUTO_DEVOPS_CHART with the path to the chart.
  • make use of the HELM_UPGRADE_EXTRA_ARGS environment variable to override the default values in the values.yaml file in the default Helm chart.
  • specify the use of a custom Helm chart per environment by scoping the environment variable to the desired environment.
    • 張 旭
       
      Auto DevOps 就是一套人家寫好好的傳便便的 .gitlab-ci.yml
  • Your additions will be merged with the Auto DevOps template using the behaviour described for include
  • copy and paste the contents of the Auto DevOps template into your project and edit this as needed.
  • In order to support applications that require a database, PostgreSQL is provisioned by default.
  • Set up the replica variables using a project variable and scale your application by just redeploying it!
  • You should not scale your application using Kubernetes directly.
  • Some applications need to define secret variables that are accessible by the deployed application.
  • Auto DevOps detects variables where the key starts with K8S_SECRET_ and make these prefixed variables available to the deployed application, as environment variables.
  • Auto DevOps pipelines will take your application secret variables to populate a Kubernetes secret.
  • Environment variables are generally considered immutable in a Kubernetes pod.
  • if you update an application secret without changing any code then manually create a new pipeline, you will find that any running application pods will not have the updated secrets.
  • Variables with multiline values are not currently supported
  • The normal behavior of Auto DevOps is to use Continuous Deployment, pushing automatically to the production environment every time a new pipeline is run on the default branch.
  • If STAGING_ENABLED is defined in your project (e.g., set STAGING_ENABLED to 1 as a CI/CD variable), then the application will be automatically deployed to a staging environment, and a production_manual job will be created for you when you’re ready to manually deploy to production.
  • If CANARY_ENABLED is defined in your project (e.g., set CANARY_ENABLED to 1 as a CI/CD variable) then two manual jobs will be created: canary which will deploy the application to the canary environment production_manual which is to be used by you when you’re ready to manually deploy to production.
  • If INCREMENTAL_ROLLOUT_MODE is set to manual in your project, then instead of the standard production job, 4 different manual jobs will be created: rollout 10% rollout 25% rollout 50% rollout 100%
  • The percentage is based on the REPLICAS variable and defines the number of pods you want to have for your deployment.
  • To start a job, click on the play icon next to the job’s name.
  • Once you get to 100%, you cannot scale down, and you’d have to roll back by redeploying the old version using the rollback button in the environment page.
  • With INCREMENTAL_ROLLOUT_MODE set to manual and with STAGING_ENABLED
  • not all buildpacks support Auto Test yet
  • When a project has been marked as private, GitLab’s Container Registry requires authentication when downloading containers.
  • Authentication credentials will be valid while the pipeline is running, allowing for a successful initial deployment.
  • After the pipeline completes, Kubernetes will no longer be able to access the Container Registry.
  • We strongly advise using GitLab Container Registry with Auto DevOps in order to simplify configuration and prevent any unforeseen issues.
張 旭

What is DevOps? | Atlassian - 0 views

  • DevOps is a set of practices that automates the processes between software development and IT teams, in order that they can build, test, and release software faster and more reliably.
  • increased trust, faster software releases, ability to solve critical issues quickly, and better manage unplanned work.
  • bringing together the best of software development and IT operations.
  • ...39 more annotations...
  • DevOps is a culture, a movement, a philosophy.
  • a firm handshake between development and operations
  • DevOps isn’t magic, and transformations don’t happen overnight.
  • Infrastructure as code
  • Culture is the #1 success factor in DevOps.
  • Building a culture of shared responsibility, transparency and faster feedback is the foundation of every high performing DevOps team.
  •  'not our problem' mentality
  • DevOps is that change in mindset of looking at the development process holistically and breaking down the barrier between Dev and Ops.
  • Speed is everything.
  • Lack of automated test and review cycles block the release to production and poor incident response time kills velocity and team confidence
  • Open communication helps Dev and Ops teams swarm on issues, fix incidents, and unblock the release pipeline faster.
  • Unplanned work is a reality that every team faces–a reality that most often impacts team productivity.
  • “cross-functional collaboration.”
  • All the tooling and automation in the world are useless if they aren’t accompanied by a genuine desire on the part of development and IT/Ops professionals to work together.
  • DevOps doesn’t solve tooling problems. It solves human problems.
  • Forming project- or product-oriented teams to replace function-based teams is a step in the right direction.
  • sharing a common goal and having a plan to reach it together
  • join sprint planning sessions, daily stand-ups, and sprint demos.
  • DevOps culture across every department
  • open channels of communication, and talk regularly
  • DevOps isn’t one team’s job. It’s everyone’s job.
  • automation eliminates repetitive manual work, yields repeatable processes, and creates reliable systems.
  • Build, test, deploy, and provisioning automation
  • continuous delivery: the practice of running each code change through a gauntlet of automated tests, often facilitated by cloud-based infrastructure, then packaging up successful builds and promoting them up toward production using automated deploys.
  • automated deploys alert IT/Ops to server “drift” between environments, which reduces or eliminates surprises when it’s time to release.
  • “configuration as code.”
  • when DevOps uses automated deploys to send thoroughly tested code to identically provisioned environments, “Works on my machine!” becomes irrelevant.
  • A DevOps mindset sees opportunities for continuous improvement everywhere.
  • regular retrospectives
  • A/B testing
  • failure is inevitable. So you might as well set up your team to absorb it, recover, and learn from it (some call this “being anti-fragile”).
  • Postmortems focus on where processes fell down and how to strengthen them – not on which team member f'ed up the code.
  • Our engineers are responsible for QA, writing, and running their own tests to get the software out to customers.
  • How long did it take to go from development to deployment? 
  • How long does it take to recover after a system failure?
  • service level agreements (SLAs)
  • Devops isn't any single person's job. It's everyone's job.
  • DevOps is big on the idea that the same people who build an application should be involved in shipping and running it.
  • developers and operators pair with each other in each phase of the application’s lifecycle.
張 旭

GitLab Auto DevOps 深入淺出,自動部署,連設定檔不用?! | 五倍紅寶石・專業程式教育 - 0 views

  • 一個 K8S 的 Cluster,Auto DevOps 將會把網站部署到這個 Cluster
  • 需要有一個 wildcard 的 DNS 讓部署在這個環境的網站有 Domain name
  • 一個可以跑 Docker 的 GitLab Runner,將會為由它來執行 CI / CD 的流程。
  • ...37 more annotations...
  • 其實 Auto DevOps 就是一份官方寫好的 gitlab-ci.yml,在啟動 Auto DevOps 的專案裡,如果找不到 gitlab-ci.yml 檔,那就會直接用官方 gitlab-ci.yml 去跑 CI / CD 流程。
  • Pod 是 K8S 中可以被部署的最小元件,一個 Pod 是由一到多個 Container 組成,同個 Pod 的不同 Container 之間彼此共享網路資源。
  • 每個 Pod 都會有它的 yaml 檔,用以描述 Pod 會使用的 Image 還有連接的 Port 等資訊。
  • Node 又分成 Worker Node 和 Master Node 兩種
  • Helm 透過參數 (parameter) 跟模板 (template) 的方式,讓我們可以在只修改參數的方式重複利用模板。
  • 為了要有 CI CD 的功能我們會把 .gitlab-ci.yml 放在專案的根目錄裡, GitLab 會依造 .gitlab-ci.yml 的設定產生 CI/CD Pipeline,每個 Pipeline 裡面可能有多個 Job,這時候就會需要有 GitLab Runner 來執行這些 Job 並把執行的結果回傳給 GitLab 讓它知道這個 Job 是否有正常執行。
  • 把專案打包成 Docker Image 這工作又或是 helm 的操作都會在 Container 內執行
  • CI/CD Pipeline 是由 stage 還有 job 組成的,stage 是有順序性的,前面的 stage 完成後才會開始下一個 stage。
  • 每個 stage 裡面包含一到多個 Job
  • Auto Devops 裡也會大量用到這種在指定 Container 內運行的工作。
  • 可以通過 health checks
  • 開 private 的話還要注意使用 Container Registry 的權限問題
  • 申請好的 wildcard 的 DNS
  • Auto Devops 也提供只要設定環境變數就能一定程度客製化的選項
  • 特別注意 namespace 有沒有設定對,不然會找不到資料喔
  • Auto Devops,如果想要進一步的客製化,而且是改 GitLab 環境變數都無法實現的客製化,這時候還是得回到 .gitlab-ci.yml 設定檔
  • 在 Docker in Docker 的環境用 Dockerfile 打包 Image
  • 用 helm upgrade 把 chart 部署到 K8S 上
  • GitLab CI 的環境變數主要有三個來源,優先度高到低依序為Settings > CI/CD 介面定義的變數gitlab_ci.yml 定義環境變數GitLab 預設環境變數
  • 把專案打包成 Docker Image 首先需要在專案下新增一份 Dockerfile
  • Auto Devops 裡面的做法,用 herokuish 提供的 Image 來打包專案
  • 在 Runner 的環境中是沒有 docker 指令可以用的,所以這邊啟動一個 Docker Container 在裡面執行就可以用 docker 指令了。
  • 其中 $CI_COMMIT_SHA $CI_COMMIT_BEFORE_SHA 這兩個都是 GitLab 預設環境變數,代表這次 commit 還有上次 commit 的 SHA 值。
  • dind 則是直接啟動 docker daemon,此外 dind 還會自動產生 TLS certificates
  • 為了在 Docker Container 內運行 Docker,會把 Host 上面的 Docker API 分享給 Container。
  • docker:stable 有執行 docker 需要的執行檔,他裡面也包含要啟動 docker 的程式(docker daemon),但啟動 Container 的 entrypoint 是 sh
  • docker:dind 繼承自 docker:stable,而且它 entrypoint 就是啟動 docker 的腳本,此外還會做完 TLS certificates
  • Container 要去連 Host 上的 Docker API 。但現在連線失敗卻是找 http://docker:2375,現在的 dind 已經不是被當做 services 來用了,而是要直接在裡面跑 Docker,所以他應該是要 unix:///var/run/docker.sock 用這種連線,於是把環境變數 DOCKER_HOST 從 tcp://docker:2375 改成空字串,讓 docker daemon 走預設連線就能成功囉!
  • auto-deploy preparationhelm init 建立 helm 專案設定 tiller 在背景執行設定 cluster 的 namespace
  • auto-deploy deploy使用 helm upgrade 部署 chart 到 K8S 上透過 --set 來設定要注入 template 的參數
  • set -x,這樣就能在執行前,顯示指令內容。
  • 用 helm repo list 看看現在有註冊哪些 Chart Repository
  • helm fetch gitlab/auto-deploy-app --untar
  • nohup 可以讓你在離線或登出系統後,還能夠讓工作繼續進行
  • 在不特別設定 CI_APPLICATION_REPOSITORY 的情況下,image_repository 的值就是預設環境變數 CI_REGISTRY_IMAGE/CI_COMMIT_REF_SLUG
  • A:-B 的意思是如果有 A 就用它,沒有就用 B
  • 研究 Auto Devops 難度最高的地方就是太多工具整合在一起,搞不清楚他們之間的關係,出錯也不知道從何查起
crazylion lee

The Pragmatic Bookshelf | DevOps in Practice - 0 views

  •  
    "Delivering production software can often be a painful task. Long test periods and the integration between operations and development can ruin or delay a promising delivery. That's what DevOps can fix. DevOps is a cultural change that aims to smoothly integrate development and operations procedures, breaking the barriers between them and focusing on automation, collaboration, and sharing of knowledge and tools. This book shows you how to implement DevOps and Continuous Delivery practices to raise your system's deployment frequency, increasing your production application's stability and robustness."
張 旭

Overriding Auto Devops - 0 views

  • most customers need to modify the devops pipeline to suit there needs
  • include Auto Devops and override it.
  • include all of Auto Devops, just as if the Auto Devops checkbox were checked for the project
  • ...4 more annotations...
  • skips for all the scans, as a way of speeding up the build process while working on the CI configuration
  • The Auto Devops test job, which uses Herokuish for testing, does not rely on the Docker image that’s generated during the Build job
  • moving the Test job to the Build stage to speed things along
  • Literally any part of Auto Devops can be overridden in your own CI configuration.
張 旭

Auto DevOps | GitLab - 0 views

  • Scan for vulnerabilities and security flaws.
  • Auto DevOps starts by building and testing your application.
  • preview your changes in a per-branch basis.
  • ...9 more annotations...
  • you don’t need to set up the deployment upfront. Auto DevOps still builds and tests your application. You can define the deployment later.
  • ship your app first, then explore the customizations later.
  • Consistency
  • Auto DevOps works with any Kubernetes cluster.
  • To use Auto DevOps for individual projects, you can enable it in a project-by-project basis.
  • Only project Maintainers can enable or disable Auto DevOps at the project level.
  • We strongly advise you to use GitLab Container Registry with Auto DevOps to simplify configuration and prevent any unforeseen issues.
  • The GitLab integration with Helm does not support installing applications when behind a proxy.
    • 張 旭
       
      已經廢棄了,不要用
    • 張 旭
       
      已經廢棄了,不要用
張 旭

DevOps - 0 views

  • 对于运维来说,知识的传承非常重要,非常有必要建立运维的知识库。一方面 有利于对事件的复盘回顾,另一方面也有助于日后参加运维的人员尽快熟悉与掌握系统的运维技能
  • 云平台主要从以下3个方面对DevOps提供支撑(括号内为承载此能力的软件工具): 1. 基于IaaS的自服务与环境编排能力(VMWare) 2. 基于PaaS的弹性伸缩能力(K8s) 3. 基于SaaS的软件服务能力
  • 考虑自建私有云,至少是混合云。
  • ...11 more annotations...
  • 内网建立所谓的私库,作为代理与外网的公共库同步。
  • 很难做到真正意义上的DevOps to Production
  • 可视化是为了实时展现持续交付流水线执行情况与单元测试的执行报告
  • 通过持续交付流水线串联自动化测试,在测试环境部署成功后触发自动化测试。
  • 测试阶段也需要测试报告的可视化与结果通知
  • 企业的持续交付流水线往往都打不通到生产环境
  • Service Desk不是某一款软件的名字,而是ITIL(信息技术基础架构库,可认为是ITSM的落地实现)里面承载变更管理与事件管理的工具统称。
  • 构建底层的云平台,是整个DevOps基础架构的基石
  • 架构不是一成不变的,而是应该随着实际需求变化而持续演化,能力也要跟着持续提升。
  • 并行测试的执行环境通过PaaS平台按需自动生成,测试执行完毕后自动销毁。
  • 即使是雷同的项目,在对编译构建上的一些细枝末节的差别也很可能导致它们的持续交付流水线设计非常不一样。
  •  
    "对于运维来说,知识的传承非常重要,非常有必要建立运维的知识库。一方面 有利于对事件的复盘回顾,另一方面也有助于日后参加运维的人员尽快熟悉与掌握系统的运维技能。"
張 旭

从《凤凰项目》谈一谈"业务IT一体化" - 知乎 - 0 views

  • IT能多大程度上参与到业务系统中去帮助到业务部门,甚至影响到业务部门,你的价值就有多大。这项工作列为紧急并且重要
  • “IT内部的项目”,有一些IT部门很热衷做这方面的项目,在我看来部分的原因是因为做这些东西相对来说是IT比较好玩或者擅长的。
  • 重要但不紧急的工作,例如认真地研究和建立devops的基础环境。
  • ...11 more annotations...
  • 但变更虽然不可避免,我个人觉得应该尽可能减少(至少做到可预测),并且将变更流程自动化。
  • 最容易被人忽视的是“计划外工作”,它偷走了我们的时间。这就等于我们经常在讲的那些紧急但不重要,或者那些不紧急也不重要的事情。
  • 约束管理( Theory Of Constrain )理论
  • 他太厉害,所以不屑于写文档;他太重要,所以可以随心所欲地改东西而不走流程;他太忙,所以很多事情都要排队等他的时间来做。
  • 需要布伦特这样的人才,只是说布伦特成为了团队的约束点,怎么利用好他成为工作成败的关键(建立一些好的机制,确保他们能够更好地工作,而不是在一些低价值的内容上),而怎么帮助他提高到一个新的水平(或者培养更多的布伦特)才是长治久安的方法。
  • “业务IT一体化”与传统的模式有一个根本的区别,就是大量地使用了自动化的技术,减少中间环节。
  • IT部门内部开发、测试、运维、安全等环节的信任
  • 第一步,从产品构想、设计、开发、测试、运维到客户,这个正向的工作流,一定要理顺
  • 第二步,从客户往回推,如何建立一个健康和高效的反馈流。这里我总结为快速试错和迭代
  • 第三步,我觉得是讲到点子上了—— 业务IT一体化当然是好啊,但流程再合理,工具再强大,领导再重视,如果没有一个所有员工都认同的企业文化来做支撑,都将流于形式。没有信任来谈创新,终究是扯淡。
  • 各部门只关注自己的小目标,以自己干了多少事为荣,而不管这些事到底对于整个公司的目标实现意味着什么。
張 旭

Introducing Infrastructure as Code | Linode - 0 views

  • Infrastructure as Code (IaC) is a technique for deploying and managing infrastructure using software, configuration files, and automated tools.
  • With the older methods, technicians must configure a device manually, perhaps with the aid of an interactive tool. Information is added to configuration files by hand or through the use of ad-hoc scripts. Configuration wizards and similar utilities are helpful, but they still require hands-on management. A small group of experts owns the expertise, the process is typically poorly defined, and errors are common.
  • The development of the continuous integration and continuous delivery (CI/CD) pipeline made the idea of treating infrastructure as software much more attractive.
  • ...20 more annotations...
  • Infrastructure as Code takes advantage of the software development process, making use of quality assurance and test automation techniques.
  • Consistency/Standardization
  • Each node in the network becomes what is known as a snowflake, with its own unique settings. This leads to a system state that cannot easily be reproduced and is difficult to debug.
  • With standard configuration files and software-based configuration, there is greater consistency between all equipment of the same type. A key IaC concept is idempotence.
  • Idempotence makes it easy to troubleshoot, test, stabilize, and upgrade all the equipment.
  • Infrastructure as Code is central to the culture of DevOps, which is a mix of development and operations
  • edits are always made to the source configuration files, never on the target.
  • A declarative approach describes the final state of a device, but does not mandate how it should get there. The specific IaC tool makes all the procedural decisions. The end state is typically defined through a configuration file, a JSON specification, or a similar encoding.
  • An imperative approach defines specific functions or procedures that must be used to configure the device. It focuses on what must happen, but does not necessarily describe the final state. Imperative techniques typically use scripts for the implementation.
  • With a push configuration, the central server pushes the configuration to the destination device.
  • If a device is mutable, its configuration can be changed while it is active
  • Immutable devices cannot be changed. They must be decommissioned or rebooted and then completely rebuilt.
  • an immutable approach ensures consistency and avoids drift. However, it usually takes more time to remove or rebuild a configuration than it does to change it.
  • System administrators should consider security issues as part of the development process.
  • Ansible is a very popular open source IaC application from Red Hat
  • Ansible is often used in conjunction with Kubernetes and Docker.
  • Linode offers a collection of several Ansible guides for a more comprehensive overview.
  • Pulumi permits the use of a variety of programming languages to deploy and manage infrastructure within a cloud environment.
  • Terraform allows users to provision data center infrastructure using either JSON or Terraform’s own declarative language.
  • Terraform manages resources through the use of providers, which are similar to APIs.
張 旭

LXC vs Docker: Why Docker is Better | UpGuard - 0 views

  • LXC (LinuX Containers) is a OS-level virtualization technology that allows creation and running of multiple isolated Linux virtual environments (VE) on a single control host.
  • Docker, previously called dotCloud, was started as a side project and only open-sourced in 2013. It is really an extension of LXC’s capabilities.
  • run processes in isolation.
  • ...35 more annotations...
  • Docker is developed in the Go language and utilizes LXC, cgroups, and the Linux kernel itself. Since it’s based on LXC, a Docker container does not include a separate operating system; instead it relies on the operating system’s own functionality as provided by the underlying infrastructure.
  • Docker acts as a portable container engine, packaging the application and all its dependencies in a virtual container that can run on any Linux server.
  • a VE there is no preloaded emulation manager software as in a VM.
  • In a VE, the application (or OS) is spawned in a container and runs with no added overhead, except for a usually minuscule VE initialization process.
  • LXC will boast bare metal performance characteristics because it only packages the needed applications.
  • the OS is also just another application that can be packaged too.
  • a VM, which packages the entire OS and machine setup, including hard drive, virtual processors and network interfaces. The resulting bloated mass usually takes a long time to boot and consumes a lot of CPU and RAM.
  • don’t offer some other neat features of VM’s such as IaaS setups and live migration.
  • LXC as supercharged chroot on Linux. It allows you to not only isolate applications, but even the entire OS.
  • Libvirt, which allows the use of containers through the LXC driver by connecting to 'lxc:///'.
  • 'LXC', is not compatible with libvirt, but is more flexible with more userspace tools.
  • Portable deployment across machines
  • Versioning: Docker includes git-like capabilities for tracking successive versions of a container
  • Component reuse: Docker allows building or stacking of already created packages.
  • Shared libraries: There is already a public registry (http://index.docker.io/ ) where thousands have already uploaded the useful containers they have created.
  • Docker taking the devops world by storm since its launch back in 2013.
  • LXC, while older, has not been as popular with developers as Docker has proven to be
  • LXC having a focus on sys admins that’s similar to what solutions like the Solaris operating system, with its Solaris Zones, Linux OpenVZ, and FreeBSD, with its BSD Jails virtualization system
  • it started out being built on top of LXC, Docker later moved beyond LXC containers to its own execution environment called libcontainer.
  • Unlike LXC, which launches an operating system init for each container, Docker provides one OS environment, supplied by the Docker Engine
  • LXC tooling sticks close to what system administrators running bare metal servers are used to
  • The LXC command line provides essential commands that cover routine management tasks, including the creation, launch, and deletion of LXC containers.
  • Docker containers aim to be even lighter weight in order to support the fast, highly scalable, deployment of applications with microservice architecture.
  • With backing from Canonical, LXC and LXD have an ecosystem tightly bound to the rest of the open source Linux community.
  • Docker Swarm
  • Docker Trusted Registry
  • Docker Compose
  • Docker Machine
  • Kubernetes facilitates the deployment of containers in your data center by representing a cluster of servers as a single system.
  • Swarm is Docker’s clustering, scheduling and orchestration tool for managing a cluster of Docker hosts. 
  • rkt is a security minded container engine that uses KVM for VM-based isolation and packs other enhanced security features. 
  • Apache Mesos can run different kinds of distributed jobs, including containers. 
  • Elastic Container Service is Amazon’s service for running and orchestrating containerized applications on AWS
  • LXC offers the advantages of a VE on Linux, mainly the ability to isolate your own private workloads from one another. It is a cheaper and faster solution to implement than a VM, but doing so requires a bit of extra learning and expertise.
  • Docker is a significant improvement of LXC’s capabilities.
張 旭

The differences between Docker, containerd, CRI-O and runc - Tutorial Works - 0 views

  • Docker isn’t the only container contender on the block.
  • Container Runtime Interface (CRI), which defines an API between Kubernetes and the container runtime
  • Open Container Initiative (OCI) which publishes specifications for images and containers.
  • ...20 more annotations...
  • for a lot of people, the name “Docker” itself is synonymous with the word “container”.
  • Docker created a very ergonomic (nice-to-use) tool for working with containers – also called docker.
  • docker is designed to be installed on a workstation or server and comes with a bunch of tools to make it easy to build and run containers as a developer, or DevOps person.
  • containerd: This is a daemon process that manages and runs containers.
  • runc: This is the low-level container runtime (the thing that actually creates and runs containers).
  • libcontainer, a native Go-based implementation for creating containers.
  • Kubernetes includes a component called dockershim, which allows it to support Docker.
  • Kubernetes prefers to run containers through any container runtime which supports its Container Runtime Interface (CRI).
  • Kubernetes will remove support for Docker directly, and prefer to use only container runtimes that implement its Container Runtime Interface.
  • Both containerd and CRI-O can run Docker-formatted (actually OCI-formatted) images, they just do it without having to use the docker command or the Docker daemon.
  • Docker images, are actually images packaged in the Open Container Initiative (OCI) format.
  • CRI is the API that Kubernetes uses to control the different runtimes that create and manage containers.
  • CRI makes it easier for Kubernetes to use different container runtimes
  • containerd is a high-level container runtime that came from Docker, and implements the CRI spec
  • containerd was separated out of the Docker project, to make Docker more modular.
  • CRI-O is another high-level container runtime which implements the Container Runtime Interface (CRI).
  • The idea behind the OCI is that you can choose between different runtimes which conform to the spec.
  • runc is an OCI-compatible container runtime.
  • A reference implementation is a piece of software that has implemented all the requirements of a specification or standard.
  • runc provides all of the low-level functionality for containers, interacting with existing low-level Linux features, like namespaces and control groups.
張 旭

Full Cycle Developers at Netflix - Operate What You Build - 1 views

  • Researching issues felt like bouncing a rubber ball between teams, hard to catch the root cause and harder yet to stop from bouncing between one another.
  • In the past, Edge Engineering had ops-focused teams and SRE specialists who owned the deploy+operate+support parts of the software life cycle
  • hearing about those problems second-hand
  • ...17 more annotations...
  • devs could push code themselves when needed, and also were responsible for off-hours production issues and support requests
  • What were we trying to accomplish and why weren’t we being successful?
  • These specialized roles create efficiencies within each segment while potentially creating inefficiencies across the entire life cycle.
  • Grouping differing specialists together into one team can reduce silos, but having different people do each role adds communication overhead, introduces bottlenecks, and inhibits the effectiveness of feedback loops.
  • devops principles
  • develops a system also be responsible for operating and supporting that system
  • Each development team owns deployment issues, performance bugs, capacity planning, alerting gaps, partner support, and so on.
  • Those centralized teams act as force multipliers by turning their specialized knowledge into reusable building blocks.
  • Communication and alignment are the keys to success.
  • Full cycle developers are expected to be knowledgeable and effective in all areas of the software life cycle.
  • ramping up on areas they haven’t focused on before
  • We run dev bootcamps and other forms of ongoing training to impart this knowledge and build up these skills
  • “how can I automate what is needed to operate this system?”
  • “what self-service tool will enable my partners to answer their questions without needing me to be involved?”
  • A full cycle developer thinks and acts like an SWE, SDET, and SRE. At times they create software that solves business problems, at other times they write test cases for that, and still other times they automate operational aspects of that system.
  • the need for continuous delivery pipelines, monitoring/observability, and so on.
  • Tooling and automation help to scale expertise, but no tool will solve every problem in the developer productivity and operations space
張 旭

How to Deploy Software - 0 views

  •  
    "Make your team's deploys as boring as hell and stop stressing about it."
crazylion lee

Deploy together. Fearlessly. - 0 views

shared by crazylion lee on 03 Apr 16 - No Cached
  •  
    "Dockbit turns your software deployments into repeatable, manageable workflows. Bring your team together and ship better."
張 旭

Containers Vs. Config Management - 0 views

  • With configuration management systems, you write code that describes how you want some component of your systems to be installed and configured, and when you execute the code on your server, it should end up in the desired state.
  • building a hosting platform that is capable of a lot of things that system administrators used to do manually
  • build modules on deployment via bundler or npm or similar, it can be incredibly slow to run, taking minutes or longer in some cases
  • ...10 more annotations...
  • pulling from git is slow.
  • deploying with configuration management tools is a pain in the ass and error prone.
  • Support for containers has existed in the Linux kernel since version 2.6.24 when cgroup support was added
  • All of the logic that used to live in your cookbooks/playbooks/manifests/etc now lives in a Dockerfile that resides directly in the repository for the application it is designed to build
  • All of the dependencies of the application are bundled with the container which means no need to build on the fly on every server during deployment.
  • Containers bring standardization which allows for systems like centralized logging, monitoring, and metrics to easily snap into place no matter what is running in the container.
  • Dockerfiles do not give you the same level of control over configuration as your application transitions between environments, like dev, staging, and production.
  • You may even need to have different Dockerfile’s for each environment in certain cases.
  • configuration management systems now have hooks for docker integration.
  • Config management will only be used to install Docker, an orchestration system, configure PAM/SSH auth, and tune OS sysctl values.
  •  
    "With configuration management systems, you write code that describes how you want some component of your systems to be installed and configured, and when you execute the code on your server, it should end up in the desired state."
張 旭

一位开发者的 Linux 容器之旅 - 51CTO.COM - 0 views

  • 容器是一个 Linux 进程,Linux 认为它只是一个运行中的进程。该进程只知道它被告知的东西。
  • 容器进程也分配了它自己的 IP 地址
  • 和典型虚拟机的静态方式不同。所有这些资源的共享都由容器管理器来管理。
  • ...26 more annotations...
  • 可以在容器管理器上运行命令,使容器 IP 映射到主机中能访问公网的 IP 地址。建立了该映射,无论出于什么意图和目的,容器就是网络上一个可访问的独立机器,从概念上类似于虚拟机。
  • 容器是拥有不同 IP 地址从而使其成为网络上可识别的独立 Linux 进程
  • 容器/进程以动态、合作的方式共享主机上的资源。
  • 容器能非常快速地启动
  • 操作系统被所有容器所共享,减少了容器足迹的重复和冗余。每个容器只包括该容器特有的部分
  • 获得了虚拟机独立和封装的好处,而抛弃了静态资源专有的缺陷
  • 托管容器的计算机运行着被剥离的只剩下主要部分的某个 Linux 版本
  • Ubuntu Snappy
  • Red Hat Atomic Host
  • CoreOS
  • 在容器化方面,容器进程有它自己的 IP 地址。一旦给予了一个 IP 地址,该进程就是宿主网络中可识别的资源
  • 一个容器组件被称为层(layer)
  • 层是一个容器镜像
  • 容器管理器只提供你所要的操作系统在宿主操作系统中不存在的部分
  • 在容器配置文件中重新定义层
  • 容器的各种功能都由一个称为容器管理器(container manager)的软件控制
  • Docker
  • Rocket
  • 镜像存储在注册库(registry)中,注册库通过网络访问
  • 注册库类似于一个使用 Java 的人眼中的 Maven 仓库、使用 .NET 的人眼中的 NuGet 服务器。
  • 容器管理器会封装你应用程序的所有东西为一个独立容器,该容器将会在容器管理器的管理下运行在宿主计算机上。
  • 每个容器有一个独立的 IP 地址
  • 在一个负载均衡容器后运行容器集群以获得更高的性能和高可用计算
  • Deis 的容器配置技术
  • 每次添加实例到环境中时,你不需要手动配置负载均衡器以便接受你的容器镜像。
  • 使用服务发现技术让容器告知均衡器它可用
張 旭

一位开发者的 Linux 容器之旅-技术 ◆ 学习|Linux.中国-开源社区 - 1 views

  • 容器是一个 Linux 进程,Linux 认为它只是一个运行中的进程。该进程只知道它被告知的东西。
  • 容器进程也分配了它自己的 IP 地址。
  • 在容器化方面,容器进程有它自己的 IP 地址。一旦给予了一个 IP 地址,该进程就是宿主网络中可识别的资源
  • ...20 more annotations...
  • 使容器 IP 映射到主机中能访问公网的 IP 地址。建立了该映射,无论出于什么意图和目的,容器就是网络上一个可访问的独立机器,从概念上类似于虚拟机。
  • 容器是拥有不同 IP 地址从而使其成为网络上可识别的独立 Linux 进程
  • CPU、内存和存储空间的分配是动态的,和典型虚拟机的静态方式不同。所有这些资源的共享都由容器管理器来管理。
  • 容器能非常快速地启动
  • 托管容器的计算机运行着被剥离的只剩下主要部分的某个 Linux 版本。
  • 操作系统被所有容器所共享,减少了容器足迹的重复和冗余。每个容器只包括该容器特有的部分
  • 层是一个容器镜像
  • 一个容器组件被称为层(layer)
  • 容器的各种功能都由一个称为容器管理器(container manager)的软件控制
  • 流行的容器管理器是 Docker 和 Rocket
  • 镜像存储在注册库(registry)中,注册库通过网络访问
  • 镜像代表了你的容器需要完成其工作的容器模板
  • 应用程序所需镜像的容器配置文件
  • 每个容器有一个独立的 IP 地址。因此,能把它放到负载均衡器后面。将容器放到负载均衡器后面,这就上升了一个层面。
  • Deis 的容器配置技术
  • 可以部署一个或多个容器镜像到主机上的负载均衡器下
  • 每次添加实例到环境中时,你不需要手动配置负载均衡器以便接受你的容器镜像。你可以使用服务发现技术让容器告知均衡器它可用。
  • 类似 CoreOS、RHEL Atomic、和 Ubuntu 的 Snappy 宿主操作系统
  • 类似 Docker 和 Rocket 的容器管理技术结合起来
  • 类似 Deis 这样的配置技术使容器创建和部署变得更加简单
張 旭

跳出运维,才能做好运维(凤凰项目)书评 - 0 views

  • 1. 业务项目:这些通常是公司业务部门,比如产品研发部门或销售部门所提出的需求,比如新产品发布上线、为客户做实施、双十一这种大促活动的规划等等。这些工作通常具有一定的系统性,需要部门间通力合作。2. 内部项目:运维部门内部围绕业务项目所实施的一些列基础设施研发,部署自动化、多环境构建、持续交付、监控报警等等。3. 变更:根据其它部门申请,对运维组件进行变更操作,相对于业务项目,变更通常都会比较零散,比如加权限、开端口、开机器等等。对于变更,我们要维护好操作记录,做到有迹可寻。4. 计划外的工作:无法预料的问题处理,即“救火”。
  • 计划外的工作越多,就会占用其它三种工作的时间,导致其它三种工作大量积累。
  • 运维内部的基础建设跟不上,无法从根上去思考和解决计划外的工作,就会造成一种恶性循环,做不完的工作越堆越多(书中喜欢叫这玩意儿"半成品")直至让人崩溃。
  • ...4 more annotations...
  • 一些周期性的工作对于运维其实也是非常重要的,比如对组件进行周期性的巡检、压测、像ChaosMonkey那样对分布式系统进行随机破坏、灾难演习、备份演习等等。不过这些也可以归结在内部项目之中
  • 流水线可视化了后,除了解决掉瓶颈节点,其它的诸如任务优先级、任务的依赖关系、每批次解决的任务数量,也都会显示出来并得到解决。
  • 反馈和朔源。我们需要知道每种任务流水线的每个环节的执行情况,效率、质量,然后与该环节相关的部门一起去探讨和优化改进的措施,比如部署时间过长,那么就要与开发团队一起优化打包流程和部署所依赖的中间件架构;线上故障太多,那么就需要联合开发和QA部门一道,去审视这些故障所爆发的源头,定制解决方案,等等
  • 第三步,就需要建立一种文化机制,努力把这种工作方式延续下去,做到持续改进,越来越好!
張 旭

Variables - Ansible Documentation - 0 views

  • with the last listed variables winning prioritization
  • anything that goes into “role defaults” (the defaults folder inside the role) is the most malleable and easily overridden.
  • Anything in the vars directory of the role overrides previous versions of that variable in namespace.
  • ...1 more annotation...
  • with command line -e extra vars always winning
1 - 20 of 85 Next › Last »
Showing 20 items per page