Skip to main content

Home/ Larvata/ Group items tagged virtual

Rss Feed Group items tagged

張 旭

你到底知不知道什麼是 Kubernetes? | Hwchiu Learning Note - 0 views

  • Storage(儲存) 實際上一直都不是一個簡單處理的問題,從軟體面來看實際上牽扯到非常多的層級,譬如 Linux Kernel, FileSystem, Block/File-Level, Cache, Snapshot, Object Storage 等各式各樣的議題可以討論。
  • DRBD
  • 異地備援,容錯機制,快照,重複資料刪除等超多相關的議題基本上從來沒有一個完美的解法能夠滿足所有使用情境。
  • ...20 more annotations...
  • 管理者可能會直接在 NFS Server 上進行 MDADM 來設定相關的 Block Device 並且基於上面提供 Export 供 NFS 使用,甚至底層套用不同的檔案系統 (EXT4/BTF4) 來獲取不同的功能與效能。
  • Kubernetes 就只是 NFS Client 的角色
  • CSI(Container Storage Interface)。CSI 本身作為 Kubernetes 與 Storage Solution 的中介層。
  • 基本上 Pod 裡面每個 Container 會使用 Volume 這個物件來代表容器內的掛載點,而在外部實際上會透過 PVC 以及 PV 的方式來描述這個 Volume 背後的儲存方案伺服器的資訊。
  • 整體會透過 CSI 的元件們與最外面實際上的儲存設備連接,所有儲存相關的功能是否有實現,有支援全部都要仰賴最後面的實際提供者, kubernetes 只透過 CSI 的標準去執行。
  • 在網路部分也有與之對應的 CNI(Container Network Interface). kubernetes 透過 CNI 這個介面來與後方的 網路解決方案 溝通
  • CNI 最基本的要求就是在在對應的階段為對應的容器提供網路能力
  • 目前最常見也是 IPv4 + TCP/UDP 的傳輸方式,因此才會看到大部分的 CNI 都在講這些。
  • 希望所有容器彼此之間可以透過 IPv4 來互相存取彼此,不論是同節點或是跨節點的容器們都要可以滿足這個需求。
  • 容器間到底怎麼傳輸的,需不需要封裝,透過什麼網卡,要不要透過 NAT 處理? 這一切都是 CNI 介面背後的實現
  • 外部網路存取容器服務 (Service/Ingress)
  • kubernetes 在 Service/Ingress 中間自行實現了一個模組,大抵上稱為 kube-proxy, 其底層可以使用 iptables, IPVS, user-space software 等不同的實現方法,這部分是跟 CNI 完全無關。
  • CNI 跟 Service/Ingress 是會衝突的,也有可能彼此沒有配合,這中間沒有絕對的穩定整合。
  • CNI 一般會處理的部份,包含了容器內的 網卡數量,網卡名稱,網卡IP, 以及容器與外部節點的連接能力等
  • CRI (Container Runtime Interface) 或是 Device Plugin
  • 對於 kubernetes 來說,其實本身並不在意到底底下的容器化技術實際上是怎麼實現的,你要用 Docker, rkt, CRI-O 都無所謂,甚至背後是一個偽裝成 Container 的 Virtaul Machine virtlet 都可以。
  • 去思考到底為什麼自己本身的服務需要容器化,容器化可以帶來什麼優點
  • 太多太多的人都認為只要寫一個 Dockerfile 將原先的應用程式們全部包裝起來放在一起就是一個很好的容器 來使用了。
  • 最後就會發現根本把 Container 當作 Virtual Machine 來使用,然後再補一句 Contaienr 根本不好用啊
  • 容器化 不是把直接 Virtual Machine 的使用習慣換個環境使用就叫做 容器化,而是要從概念上去暸解與使用
張 旭

jwilder/nginx-proxy: Automated nginx proxy for Docker containers using docker-gen - 0 views

  • docker-gen generates reverse proxy configs for nginx and reloads nginx when containers are started and stopped.
  • /var/run/docker.sock:/tmp/docker.sock:ro
  • Use this image to fully support HTTP/2 (including ALPN required by recent Chrome versions).
  • ...10 more annotations...
  • support multiple virtual hosts for a container
  • to connect to your backend using HTTPS instead of HTTP, set VIRTUAL_PROTO=https on the backend container.
  • The contents of /path/to/certs should contain the certificates and private keys for any virtual hosts in use.
  • to replace the default proxy settings for the nginx container, add a configuration file at /etc/nginx/proxy.conf
  • The default configuration blocks the Proxy HTTP request header from being sent to downstream servers
  • add your configuration file under /etc/nginx/conf.d using a name ending in .conf
  • If your container exposes multiple ports, nginx-proxy will default to the service running on port 80. If you need to specify a different port, you can set a VIRTUAL_PORT env var to select a different one.
  • To add settings on a per-VIRTUAL_HOST basis, add your configuration file under /etc/nginx/vhost.d
  • SNI
  • The default behavior for the proxy when port 80 and 443 are exposed is as follows: If a container has a usable cert, port 80 will redirect to 443 for that container so that HTTPS is always preferred when available. If the container does not have a usable cert, a 503 will be returned.
張 旭

vSphere Cloud Provider | vSphere Storage for Kubernetes - 0 views

  • Containers are stateless and ephemeral but applications are stateful and need persistent storage.
  • Cloud Provider
  • Kubernetes cloud providers are an interface to integrate various node (i.e. hosts), load balancers and networking routes
  • ...8 more annotations...
  • VMware offers a Cloud Provider known as the vSphere Cloud Provider (VCP) for Kubernetes which allows Pods to use enterprise grade persistent storage.
  • A vSphere datastore is an abstraction which hides storage details (such as LUNs) and provides a uniform interface for storing persistent data.
  • the datastores can be of the type vSAN, VMFS, NFS & VVol.
  • VMFS (Virtual Machine File System) is a cluster file system that allows virtualization to scale beyond a single node for multiple VMware ESX servers.
  • NFS (Network File System) is a distributed file protocol to access storage over network like local storage.
  • vSphere Cloud Provider supports every storage primitive exposed by Kubernetes
  • Kubernetes PVs are defined in Pod specifications.
  • PVCs when using Dynamic Provisioning (preferred).
crazylion lee

Packer - 0 views

  •  
    Packer is a tool for creating identical machine images for multiple platforms from a single source configuration.
張 旭

Getting Started with Docker - Servers for Hackers - 0 views

  • Docker is an isolated portion of the host computer, sharing the host kernel (OS) and even its bin/libraries if appropriate.
  • the Docker Container contains the parts that make Ubuntu different from CoreOS.
  • A Docker container only stays alive as long as there is an active process being run in it.
  • ...10 more annotations...
  • Allocate a (pseudo) tty
  • Keep stdin open (so we can interact with it)
  • Docker allows us make changes to an image, commit those changes, and then push those changes out somehwere.
  • Docker tracks any changes we make to a container
  • The Dockerfile provides a set of instructions for Docker to run on a container.
  • what image (and tag in this case) to base this off of
  • run the given command (as user "root")
  • copy a file from the host machine into the container
  • expose a port to the host machine. You can expose multiple ports
  • run a command
張 旭

How To Install and Use Docker: Getting Started | DigitalOcean - 0 views

  • docker as a project offers you the complete set of higher-level tools to carry everything that forms an application across systems and machines - virtual or physical - and brings along loads more of great benefits with it
  • docker daemon: used to manage docker (LXC) containers on the host it runs
  • docker CLI: used to command and communicate with the docker daemon
  • ...20 more annotations...
  • containers: directories containing everything-your-application
  • images: snapshots of containers or base OS (e.g. Ubuntu) images
  • Dockerfiles: scripts automating the building process of images
  • Docker containers are basically directories which can be packed (e.g. tar-archived) like any other, then shared and run across various different machines and platforms (hosts).
  • Linux Containers can be defined as a combination various kernel-level features (i.e. things that Linux-kernel can do) which allow management of applications (and resources they use) contained within their own environment
  • Each container is layered like an onion and each action taken within a container consists of putting another block (which actually translates to a simple change within the file system) on top of the previous one.
  • Each docker container starts from a docker image which forms the base for other applications and layers to come.
  • Docker images constitute the base of docker containers from which everything starts to form
  • a solid, consistent and dependable base with everything that is needed to run the applications
  • As more layers (tools, applications etc.) are added on top of the base, new images can be formed by committing these changes.
  • a Dockerfile for automated image building
  • Dockerfiles are scripts containing a successive series of instructions, directions, and commands which are to be executed to form a new docker image.
  • As you work with a container and continue to perform actions on it (e.g. download and install software, configure files etc.), to have it keep its state, you need to “commit”.
  • Please remember to “commit” all your changes.
  • When you "run" any process using an image, in return, you will have a container.
  • When the process is not actively running, this container will be a non-running container. Nonetheless, all of them will reside on your system until you remove them via rm command.
  • To create a new container, you need to use a base image and specify a command to run.
  • you can not change the command you run after having created a container (hence specifying one during "creation")
  • If you would like to save the progress and changes you made with a container, you can use “commit”
  • turns your container to an image
張 旭

How services work | Docker Documentation - 0 views

  • a service is the image for a microservice within the context of some larger application.
  • When you create a service, you specify which container image to use and which commands to execute inside running containers.
  • an overlay network for the service to connect to other services in the swarm
  • ...13 more annotations...
  • In the swarm mode model, each task invokes exactly one container
  • A task is analogous to a “slot” where the scheduler places a container.
  • A task is the atomic unit of scheduling within a swarm.
  • A task is a one-directional mechanism. It progresses monotonically through a series of states: assigned, prepared, running, etc.
  • Docker swarm mode is a general purpose scheduler and orchestrator.
  • Hypothetically, you could implement other types of tasks such as virtual machine tasks or non-containerized process tasks.
  • If all nodes are paused or drained, and you create a service, it is pending until a node becomes available.
  • reserve a specific amount of memory for a service.
  • impose placement constraints on the service
  • As the administrator of a swarm, you declare the desired state of your swarm, and the manager works with the nodes in the swarm to create that state.
  • two types of service deployments, replicated and global.
  • A global service is a service that runs one task on every node.
  • Good candidates for global services are monitoring agents, an anti-virus scanners or other types of containers that you want to run on every node in the swarm.
張 旭

Internet Gateways - Amazon Virtual Private Cloud - 0 views

  • to provide a target in your VPC route tables for internet-routable traffic
  • to perform network address translation (NAT) for instances that have been assigned public IPv4 addresses
  • Ensure that instances in your subnet have a globally unique IP address (public IPv4 address, Elastic IP address, or IPv6 address)
  • ...10 more annotations...
  • To use an internet gateway, your subnet's route table must contain a route that directs internet-bound traffic to the internet gateway.
  • If your subnet is associated with a route table that has a route to an internet gateway, it's known as a public subnet.
  • To enable communication over the internet for IPv4, your instance must have a public IPv4 address or an Elastic IP address that's associated with a private IPv4 address on your instance.
  • Your instance is only aware of the private (internal) IP address space defined within the VPC and subnet
  • internet gateway logically provides the one-to-one NAT on behalf of your instance
  • To enable communication over the internet for IPv6, your VPC and subnet must have an associated IPv6 CIDR block, and your instance must be assigned an IPv6 address from the range of the subnet.
  • When you create a subnet, we automatically associate it with the main route table for the VPC.
  • the main route table doesn't contain a route to an internet gateway
  • Each instance that you launch into a VPC is automatically associated with its default security group.
  • a default security group allow no inbound traffic from the internet and allow all outbound traffic to the internet.
張 旭

NAT Gateways - Amazon Virtual Private Cloud - 0 views

  • a network address translation (NAT) gateway to enable instances in a private subnet to connect to the internet or other AWS services
  • but prevent the internet from initiating a connection with those instances
  • NAT gateways are not supported for IPv6 traffic
  • ...11 more annotations...
  • must specify the public subnet in which the NAT gateway should reside
  • update the route table associated with one or more of your private subnets to point Internet-bound traffic to the NAT gateway.
  • NAT gateway is created in a specific Availability Zone and implemented with redundancy in that zone.
  • ensure that resources use the NAT gateway in the same Availability Zone
  • The main route table sends internet traffic from the instances in the private subnet to the NAT gateway. The NAT gateway sends the traffic to the internet gateway using the NAT gateway’s Elastic IP address as the source IP address
  • A NAT gateway supports 5 Gbps of bandwidth and automatically scales up to 45 Gbps
  • You can associate exactly one Elastic IP address with a NAT gateway
  • A NAT gateway supports the following protocols: TCP, UDP, and ICMP
  • cannot associate a security group with a NAT gateway.
  • create a NAT gateway in the same subnet as your NAT instance, and then replace the existing route in your route table that points to the NAT instance with a route that points to the NAT gateway
  • A NAT gateway cannot send traffic over VPC endpoints, VPN connections, AWS Direct Connect, or VPC peering connections.
張 旭

Using Infrastructure as Code to Automate VMware Deployments - 1 views

  • Infrastructure as code is at the heart of provisioning for cloud infrastructure marking a significant shift away from monolithic point-and-click management tools.
  • infrastructure as code enables operators to take a programmatic approach to provisioning.
  • provides a single workflow to provision and maintain infrastructure and services from all of your vendors, making it not only easier to switch providers
  • ...5 more annotations...
  • A Terraform Provider is responsible for understanding API interactions between and exposing the resources from a given Infrastructure, Platform, or SaaS offering to Terraform.
  • write a Terraform file that describes the Virtual Machine that you want, apply that file with Terraform and create that VM as you described without ever needing to log into the vSphere dashboard.
  • HashiCorp Configuration Language (HCL)
  • the provider credentials are passed in at the top of the script to connect to the vSphere account.
  • modules— a way to encapsulate infrastructure resources into a reusable format.
  •  
    "revolutionizing"
張 旭

Introduction - Packer by HashiCorp - 0 views

  • A machine image is a single static unit that contains a pre-configured operating system and installed software which is used to quickly create new running machines.
  •  
    "A machine image is a single static unit that contains a pre-configured operating system and installed software which is used to quickly create new running machines."
張 旭

User Variables - Templates - Packer by HashiCorp - 0 views

  • User variables allow your templates to be further configured with variables from the command-line, environment variables, Vault, or files.
  • define it either within the variables section within your template, or using the command-line -var or -var-file flags.
  • If the default value is null, then the user variable will be required.
  • ...7 more annotations...
  • User variables are available globally within the rest of the template.
  • The env function is available only within the default value of a user variable, allowing you to default a user variable to an environment variable.
  • As Packer doesn't run inside a shell, it won't expand ~
  • To set user variables from the command line, the -var flag is used as a parameter to packer build (and some other commands).
  • Variables can also be set from an external JSON file. The -var-file flag reads a file containing a key/value mapping of variables to values and sets those variables.
  • -var-file=
  • sensitive variables won't get printed to the logs by adding them to the "sensitive-variables" list within the Packer template
張 旭

Boosting your kubectl productivity ♦︎ Learnk8s - 0 views

  • kubectl is your cockpit to control Kubernetes.
  • kubectl is a client for the Kubernetes API
  • Kubernetes API is an HTTP REST API.
  • ...75 more annotations...
  • This API is the real Kubernetes user interface.
  • Kubernetes is fully controlled through this API
  • every Kubernetes operation is exposed as an API endpoint and can be executed by an HTTP request to this endpoint.
  • the main job of kubectl is to carry out HTTP requests to the Kubernetes API
  • Kubernetes maintains an internal state of resources, and all Kubernetes operations are CRUD operations on these resources.
  • Kubernetes is a fully resource-centred system
  • Kubernetes API reference is organised as a list of resource types with their associated operations.
  • This is how kubectl works for all commands that interact with the Kubernetes cluster.
  • kubectl simply makes HTTP requests to the appropriate Kubernetes API endpoints.
  • it's totally possible to control Kubernetes with a tool like curl by manually issuing HTTP requests to the Kubernetes API.
  • Kubernetes consists of a set of independent components that run as separate processes on the nodes of a cluster.
  • components on the master nodes
  • Storage backend: stores resource definitions (usually etcd is used)
  • API server: provides Kubernetes API and manages storage backend
  • Controller manager: ensures resource statuses match specifications
  • Scheduler: schedules Pods to worker nodes
  • component on the worker nodes
  • Kubelet: manages execution of containers on a worker node
  • triggers the ReplicaSet controller, which is a sub-process of the controller manager.
  • the scheduler, who watches for Pod definitions that are not yet scheduled to a worker node.
  • creating and updating resources in the storage backend on the master node.
  • The kubelet of the worker node your ReplicaSet Pods have been scheduled to instructs the configured container runtime (which may be Docker) to download the required container images and run the containers.
  • Kubernetes components (except the API server and the storage backend) work by watching for resource changes in the storage backend and manipulating resources in the storage backend.
  • However, these components do not access the storage backend directly, but only through the Kubernetes API.
    • 張 旭
       
      很精彩,相互之間都是使用 API call 溝通,良好的微服務行為。
  • double usage of the Kubernetes API for internal components as well as for external users is a fundamental design concept of Kubernetes.
  • All other Kubernetes components and users read, watch, and manipulate the state (i.e. resources) of Kubernetes through the Kubernetes API
  • The storage backend stores the state (i.e. resources) of Kubernetes.
  • command completion is a shell feature that works by the means of a completion script.
  • A completion script is a shell script that defines the completion behaviour for a specific command. Sourcing a completion script enables completion for the corresponding command.
  • kubectl completion zsh
  • /etc/bash_completion.d directory (create it, if it doesn't exist)
  • source <(kubectl completion bash)
  • source <(kubectl completion zsh)
  • autoload -Uz compinit compinit
  • the API reference, which contains the full specifications of all resources.
  • kubectl api-resources
  • displays the resource names in their plural form (e.g. deployments instead of deployment). It also displays the shortname (e.g. deploy) for those resources that have one. Don't worry about these differences. All of these name variants are equivalent for kubectl.
  • .spec
  • custom columns output format comes in. It lets you freely define the columns and the data to display in them. You can choose any field of a resource to be displayed as a separate column in the output
  • kubectl get pods -o custom-columns='NAME:metadata.name,NODE:spec.nodeName'
  • kubectl explain pod.spec.
  • kubectl explain pod.metadata.
  • browse the resource specifications and try it out with any fields you like!
  • JSONPath is a language to extract data from JSON documents (it is similar to XPath for XML).
  • with kubectl explain, only a subset of the JSONPath capabilities is supported
  • Many fields of Kubernetes resources are lists, and this operator allows you to select items of these lists. It is often used with a wildcard as [*] to select all items of the list.
  • kubectl get pods -o custom-columns='NAME:metadata.name,IMAGES:spec.containers[*].image'
  • a Pod may contain more than one container.
  • The availability zones for each node are obtained through the special failure-domain.beta.kubernetes.io/zone label.
  • kubectl get nodes -o yaml kubectl get nodes -o json
  • The default kubeconfig file is ~/.kube/config
  • with multiple clusters, then you have connection parameters for multiple clusters configured in your kubeconfig file.
  • Within a cluster, you can set up multiple namespaces (a namespace is kind of "virtual" clusters within a physical cluster)
  • overwrite the default kubeconfig file with the --kubeconfig option for every kubectl command.
  • Namespace: the namespace to use when connecting to the cluster
  • a one-to-one mapping between clusters and contexts.
  • When kubectl reads a kubeconfig file, it always uses the information from the current context.
  • just change the current context in the kubeconfig file
  • to switch to another namespace in the same cluster, you can change the value of the namespace element of the current context
  • kubectl also provides the --cluster, --user, --namespace, and --context options that allow you to overwrite individual elements and the current context itself, regardless of what is set in the kubeconfig file.
  • for switching between clusters and namespaces is kubectx.
  • kubectl config get-contexts
  • just have to download the shell scripts named kubectl-ctx and kubectl-ns to any directory in your PATH and make them executable (for example, with chmod +x)
  • kubectl proxy
  • kubectl get roles
  • kubectl get pod
  • Kubectl plugins are distributed as simple executable files with a name of the form kubectl-x. The prefix kubectl- is mandatory,
  • To install a plugin, you just have to copy the kubectl-x file to any directory in your PATH and make it executable (for example, with chmod +x)
  • krew itself is a kubectl plugin
  • check out the kubectl-plugins GitHub topic
  • The executable can be of any type, a Bash script, a compiled Go program, a Python script, it really doesn't matter. The only requirement is that it can be directly executed by the operating system.
  • kubectl plugins can be written in any programming or scripting language.
  • you can write more sophisticated plugins with real programming languages, for example, using a Kubernetes client library. If you use Go, you can also use the cli-runtime library, which exists specifically for writing kubectl plugins.
  • a kubeconfig file consists of a set of contexts
  • changing the current context means changing the cluster, if you have only a single context per cluster.
張 旭

Build an Image - Getting Started - Packer by HashiCorp - 0 views

  • The configuration file used to define what image we want built and how is called a template in Packer terminology.
  • JSON struck the best balance between human-editable and machine-editable, allowing both hand-made templates as well as machine generated templates to easily be made.
  • keeping your secret keys out of the template
  • ...3 more annotations...
  • validate the template by running packer validate example.json. This command checks the syntax as well as the configuration values to verify they look valid.
  • At the end of running packer build, Packer outputs the artifacts that were created as part of the build.
  • Packer only builds images. It does not attempt to manage them in any way.
張 旭

Template Engine - Templates - Packer by HashiCorp - 0 views

  • All strings within templates are processed by a common Packer templating engine, where variables and functions can be used to modify the value of a configuration parameter at runtime.
  • Anything template related happens within double-braces: {{ }}.
  • Functions are specified directly within the braces, such as {{timestamp}}
  • ...8 more annotations...
  • Template variables are prefixed with a period and capitalized, such as {{.Variable}}.
  • Functions perform operations on and within strings
  • the {{timestamp}} function can be used in any string to generate the current timestamp.
  • pwd - The working directory while executing Packer.
  • template_dir - The directory to the template for the build.
  • uuid - Returns a random UUID.
  • user - Specifies a user variable.
  • Template variables are special variables automatically set by Packer at build time.
張 旭

The Twelve-Factor App - 0 views

  • Keep development, staging, and production as similar as possible
  • Developers write code, ops engineers deploy it.
  • The twelve-factor app is designed for continuous deployment by keeping the gap between development and production small
  • ...4 more annotations...
  • Backing services, such as the app’s database, queueing system, or cache, is one area where dev/prod parity is important
  • The twelve-factor developer resists the urge to use different backing services between development and production, even when adapters theoretically abstract away any differences in backing services.
  • declarative provisioning tools such as Chef and Puppet combined with light-weight virtual environments such as Docker and Vagrant allow developers to run local environments which closely approximate production environments.
  • all deploys of the app (developer environments, staging, production) should be using the same type and version of each of the backing services.
  •  
    "as similar as possible "
‹ Previous 21 - 40 of 52 Next ›
Showing 20 items per page