Skip to main content

Home/ Larvata/ Group items tagged layer

Rss Feed Group items tagged

張 旭

Docker image building on GitLab CI | $AYMDEV() - 0 views

  • Continuous Integration (or CI) is a practice where you continously test an application to detect errors as soon as possible.
  • Docker is a container technology, many CI tools execute jobs (the tasks of a pipeline) in container to have an isolated environment.
  • Docker in Docker (« DinD » in short) means executing Docker in a Docker container.
  • ...11 more annotations...
  • images are saved in the host registry, we can benefit from Docker layer caching
  • All jobs will share the same environment, if many of them run simultaneously they might get into conflicts.
  • storage management (accumulating images)
  • The Docker socket binding technique means making a volume of /var/run/docker.sock between host and containers.
  • all containers would share the same Docker daemon.
  • Add privileged = true in the [runners.docker] section, the privileged mode is mandatory to use DinD.
  • To avoid that the runner only run one job at a time, change the concurrent value on the first line.
  • To avoid building a Docker image at each job, it can be built in a first job, pushed to the image registry provided by GitLab, and pulled in the next jobs.
  • functional tests depending on a database.
  • Docker Compose allows you to easily start multiple containers, but it has no more feature than Docker itself
  • Docker in Docker works well, but has its drawbacks, like Docker layer caching which needs some more commands to be used.
張 旭

Cluster Networking - Kubernetes - 0 views

  • Networking is a central part of Kubernetes, but it can be challenging to understand exactly how it is expected to work
  • Highly-coupled container-to-container communications
  • Pod-to-Pod communications
  • ...57 more annotations...
  • this is the primary focus of this document
    • 張 旭
       
      Cluster Networking 所關注處理的是: Pod 到 Pod 之間的連線
  • Pod-to-Service communications
  • External-to-Service communications
  • Kubernetes is all about sharing machines between applications.
  • sharing machines requires ensuring that two applications do not try to use the same ports.
  • Dynamic port allocation brings a lot of complications to the system
  • Every Pod gets its own IP address
  • do not need to explicitly create links between Pods
  • almost never need to deal with mapping container ports to host ports.
  • Pods can be treated much like VMs or physical hosts from the perspectives of port allocation, naming, service discovery, load balancing, application configuration, and migration.
  • pods on a node can communicate with all pods on all nodes without NAT
  • agents on a node (e.g. system daemons, kubelet) can communicate with all pods on that node
  • pods in the host network of a node can communicate with all pods on all nodes without NAT
  • If your job previously ran in a VM, your VM had an IP and could talk to other VMs in your project. This is the same basic model.
  • containers within a Pod share their network namespaces - including their IP address
  • containers within a Pod can all reach each other’s ports on localhost
  • containers within a Pod must coordinate port usage
  • “IP-per-pod” model.
  • request ports on the Node itself which forward to your Pod (called host ports), but this is a very niche operation
  • The Pod itself is blind to the existence or non-existence of host ports.
  • AOS is an Intent-Based Networking system that creates and manages complex datacenter environments from a simple integrated platform.
  • Cisco Application Centric Infrastructure offers an integrated overlay and underlay SDN solution that supports containers, virtual machines, and bare metal servers.
  • AOS Reference Design currently supports Layer-3 connected hosts that eliminate legacy Layer-2 switching problems.
  • The AWS VPC CNI offers integrated AWS Virtual Private Cloud (VPC) networking for Kubernetes clusters.
  • users can apply existing AWS VPC networking and security best practices for building Kubernetes clusters.
  • Using this CNI plugin allows Kubernetes pods to have the same IP address inside the pod as they do on the VPC network.
  • The CNI allocates AWS Elastic Networking Interfaces (ENIs) to each Kubernetes node and using the secondary IP range from each ENI for pods on the node.
  • Big Cloud Fabric is a cloud native networking architecture, designed to run Kubernetes in private cloud/on-premises environments.
  • Cilium is L7/HTTP aware and can enforce network policies on L3-L7 using an identity based security model that is decoupled from network addressing.
  • CNI-Genie is a CNI plugin that enables Kubernetes to simultaneously have access to different implementations of the Kubernetes network model in runtime.
  • CNI-Genie also supports assigning multiple IP addresses to a pod, each from a different CNI plugin.
  • cni-ipvlan-vpc-k8s contains a set of CNI and IPAM plugins to provide a simple, host-local, low latency, high throughput, and compliant networking stack for Kubernetes within Amazon Virtual Private Cloud (VPC) environments by making use of Amazon Elastic Network Interfaces (ENI) and binding AWS-managed IPs into Pods using the Linux kernel’s IPvlan driver in L2 mode.
  • to be straightforward to configure and deploy within a VPC
  • Contiv provides configurable networking
  • Contrail, based on Tungsten Fabric, is a truly open, multi-cloud network virtualization and policy management platform.
  • DANM is a networking solution for telco workloads running in a Kubernetes cluster.
  • Flannel is a very simple overlay network that satisfies the Kubernetes requirements.
  • Any traffic bound for that subnet will be routed directly to the VM by the GCE network fabric.
  • sysctl net.ipv4.ip_forward=1
  • Jaguar provides overlay network using vxlan and Jaguar CNIPlugin provides one IP address per pod.
  • Knitter is a network solution which supports multiple networking in Kubernetes.
  • Kube-OVN is an OVN-based kubernetes network fabric for enterprises.
  • Kube-router provides a Linux LVS/IPVS-based service proxy, a Linux kernel forwarding-based pod-to-pod networking solution with no overlays, and iptables/ipset-based network policy enforcer.
  • If you have a “dumb” L2 network, such as a simple switch in a “bare-metal” environment, you should be able to do something similar to the above GCE setup.
  • Multus is a Multi CNI plugin to support the Multi Networking feature in Kubernetes using CRD based network objects in Kubernetes.
  • NSX-T can provide network virtualization for a multi-cloud and multi-hypervisor environment and is focused on emerging application frameworks and architectures that have heterogeneous endpoints and technology stacks.
  • NSX-T Container Plug-in (NCP) provides integration between NSX-T and container orchestrators such as Kubernetes
  • Nuage uses the open source Open vSwitch for the data plane along with a feature rich SDN Controller built on open standards.
  • OpenVSwitch is a somewhat more mature but also complicated way to build an overlay network
  • OVN is an opensource network virtualization solution developed by the Open vSwitch community.
  • Project Calico is an open source container networking provider and network policy engine.
  • Calico provides a highly scalable networking and network policy solution for connecting Kubernetes pods based on the same IP networking principles as the internet
  • Calico can be deployed without encapsulation or overlays to provide high-performance, high-scale data center networking.
  • Calico can also be run in policy enforcement mode in conjunction with other networking solutions such as Flannel, aka canal, or native GCE, AWS or Azure networking.
  • Romana is an open source network and security automation solution that lets you deploy Kubernetes without an overlay network
  • Weave Net runs as a CNI plug-in or stand-alone. In either version, it doesn’t require any configuration or extra code to run, and in both cases, the network provides one IP address per pod - as is standard for Kubernetes.
  • The network model is implemented by the container runtime on each node.
張 旭

Installing Addons | Kubernetes - 0 views

  • Calico is a networking and network policy provider. Calico supports a flexible set of networking options so you can choose the most efficient option for your situation, including non-overlay and overlay networks, with or without BGP. Calico uses the same engine to enforce network policy for hosts, pods, and (if using Istio & Envoy) applications at the service mesh layer.
  • Cilium is a networking, observability, and security solution with an eBPF-based data plane. Cilium provides a simple flat Layer 3 network with the ability to span multiple clusters in either a native routing or overlay/encapsulation mode, and can enforce network policies on L3-L7 using an identity-based security model that is decoupled from network addressing. Cilium can act as a replacement for kube-proxy; it also offers additional, opt-in observability and security features.
  • CoreDNS is a flexible, extensible DNS server which can be installed as the in-cluster DNS for pods.
  • ...1 more annotation...
  • The node problem detector runs on Linux nodes and reports system issues as either Events or Node conditions.
張 旭

Rails Database Best Practices - 0 views

  • Databases are extremely feature rich and are really freakin fast when used properly
  • create succinct helpers for accessing subsets of data that are relevant in specific situations
  • Relations are chainable
  • ...24 more annotations...
  • Return an ActiveRecord::Relation
  • Filtering in Ruby is slower
  • Please don't do this
  • trigger the query and therefore, we lose our Relation
  • leaving trivial ordering out of scopes all together.
  • where
  • where
  • .merge() makes it easy to use scopes from other models that have been joined into the query, reducing potential duplication.
  • ActiveRecord provides an easy API for doing many things with our database, but it also makes it pretty easy to do things inefficiently. The layer of abstraction hides what’s really happening.
  • first pure SQL, then ActiveRecord
  • Databases can only do fast lookups for columns with indexes, otherwise it’s doing a sequential scan
  • Add an index on every id column as well as any column that is used in a where clause.
  • use a Query class to encapsulate the potentially gnarly query.
  • subqueries
  • this Query returns an ActiveRecord::Relation
  • where
  • where
  • Single Responsibility Principle
  • Avoid ad-hoc queries outside of Scopes and Query Objects
  • encapsulate data access into scopes and Query objects
  • An ad-hoc query embedded in a controller (or view, task, etc) is harder to test in isolation and cannot be reused
  • to scopes and Query objects
    • 張 旭
       
      將查詢方式都封裝成 scope 或 query 物件。
  • Every databases provides more datatypes than your ORM might have you believe
  • Both Postgres and MySQL have full-text search capabilities
張 旭

Blog Tutorial - Adding a layer - CakePHP Cookbook v2.x documentation - 1 views

  • Cake views are just presentation-flavored fragments that fit inside an application’s layout. For most applications they’re HTML mixed with PHP, but they may end up as XML, CSV, or even binary data.
  • Layouts are presentation code that is wrapped around a view, and can be defined and switched between
  • if the HTTP method of the request was POST, try to save the data using the Post model.
  • ...6 more annotations...
  • You can also specify URLs relative to the base of the application in the form of /controller/action/param1/param2.
  • Every CakePHP request includes a CakeRequest object which is accessible using $this->request.
  • When a user uses a form to POST data to your application, that information is available in $this->request->data.
  • The $this->Form->input() method is used to create form elements of the same name.
  • postLink() will create a link that uses Javascript to do a POST request deleting our post.
  • Allowing content to be deleted using GET requests is dangerous
張 旭

javascript - How do I "think in AngularJS" if I have a jQuery background? - Stack Overflow - 0 views

  • in AngularJS, we have a separate model layer that we can manage in any way we want, completely independently from the view.
  • keep your concerns separate
  • do DOM manipulation and augment your view with directives
  • ...34 more annotations...
  • DI means that you can declare components very freely and then from any other component, just ask for an instance of it and it will be granted
  • do test-driven development iteratively in AngularJS!
  • only do DOM manipulation in a directive
  • with ngClass we can dynamically update the class;
  • ngBind allows two-way data binding;
  • ngShow and ngHide programmatically show or hide an element;
  • The less DOM manipulation, the easier directives are to test, the easier they are to style, the easier they are to change in the future, and the more re-usable and distributable they are.
  • still wrong.
  • Before doing DOM manipulation anywhere in your application, ask yourself if you really need to.
  • a few things wrong with this
  • jQuery was never necessary
  • use angular.element and our component will still work when dropped into a project that doesn't have jQuery.
  • just use angular.element
  • the element that is passed to the link function would already be a jQuery element!
  • directives aren't just collections of jQuery-like functions
  • Directives are actually extensions of HTML
  • If HTML doesn't do something you need it to do, you write a directive to do it for you, and then use it just as if it was part of HTML.
  • think how the team would accomplish it to fit right in with ngClick, ngClass, et al.
  • Don't even use jQuery. Don't even include it.
  • ry to think about how to do it within the confines the AngularJS.
  • In jQuery, selectors are used to find DOM elements and then bind/register event handlers to them.
  • Views are (declarative) HTML that contain AngularJS directives
  • Directives set up the event handlers behind the scenes for us and give us dynamic databinding.
  • Views are tied to models (via scopes). Views are a projection of the model
  • In AngularJS, think about models, rather than jQuery-selected DOM elements that hold your data.
  • AngularJS uses controllers and directives (each of which can have their own controller, and/or compile and linking functions) to remove behavior from the view/structure (HTML). Angular also has services and filters to help separate/organize your application.
  • Think about your models
  • Think about how you want to present your models -- your views.
  • using the necessary directives to get dynamic databinding.
  • Attach a controller to each view (using ng-view and routing, or ng-controller)
  • Make controllers as thin as possible.
  • You can do a lot with jQuery without knowing about how JavaScript prototypal inheritance works.
  • jQuery is a library
  • AngularJS is a beautiful client-side framework
張 旭

Docker Explained: Using Dockerfiles to Automate Building of Images | DigitalOcean - 0 views

  • CMD would be running an application upon creation of a container which is already installed using RUN (e.g. RUN apt-get install …) inside the image
  • ENTRYPOINT argument sets the concrete default application that is used every time a container is created using the image.
  • ENV command is used to set the environment variables (one or more).
  • ...6 more annotations...
  • EXPOSE command is used to associate a specified port to enable networking between the running process inside the container and the outside world
  • defines the base image to use to start the build process
  • Unlike CMD, it actually is used to build the image (forming another layer on top of the previous one which is committed).
  • VOLUME command is used to enable access from your container to a directory on the host machine
  • set where the command defined with CMD is to be executed
  • To detach yourself from the container, use the escape sequence CTRL+P followed by CTRL+Q
張 旭

MonolithFirst - 0 views

  • Microservices are a useful architecture, but even their advocates say that using them incurs a significant MicroservicePremium, which means they are only useful with more complex systems.
  • you should build a new application as a monolith initially, even if you think it's likely that it will benefit from a microservices architecture later on.
  • Any refactoring of functionality between services is much harder than it is in a monolith.
  • ...4 more annotations...
  • By building a monolith first, you can figure out what the right boundaries are, before a microservices design brushes a layer of treacle over them.
  • The logical way is to design a monolith carefully, paying attention to modularity within the software, both at the API boundaries and how the data is stored.
  • start with a monolith and gradually peel off microservices at the edges
  • Don't be afraid of building a monolith that you will discard, particularly if a monolith can get you to market quickly
張 旭

一位开发者的 Linux 容器之旅 - 51CTO.COM - 0 views

  • 容器是一个 Linux 进程,Linux 认为它只是一个运行中的进程。该进程只知道它被告知的东西。
  • 容器进程也分配了它自己的 IP 地址
  • 和典型虚拟机的静态方式不同。所有这些资源的共享都由容器管理器来管理。
  • ...26 more annotations...
  • 可以在容器管理器上运行命令,使容器 IP 映射到主机中能访问公网的 IP 地址。建立了该映射,无论出于什么意图和目的,容器就是网络上一个可访问的独立机器,从概念上类似于虚拟机。
  • 容器是拥有不同 IP 地址从而使其成为网络上可识别的独立 Linux 进程
  • 容器/进程以动态、合作的方式共享主机上的资源。
  • 容器能非常快速地启动
  • 操作系统被所有容器所共享,减少了容器足迹的重复和冗余。每个容器只包括该容器特有的部分
  • 获得了虚拟机独立和封装的好处,而抛弃了静态资源专有的缺陷
  • 托管容器的计算机运行着被剥离的只剩下主要部分的某个 Linux 版本
  • Ubuntu Snappy
  • Red Hat Atomic Host
  • CoreOS
  • 在容器化方面,容器进程有它自己的 IP 地址。一旦给予了一个 IP 地址,该进程就是宿主网络中可识别的资源
  • 一个容器组件被称为层(layer)
  • 层是一个容器镜像
  • 容器管理器只提供你所要的操作系统在宿主操作系统中不存在的部分
  • 在容器配置文件中重新定义层
  • 容器的各种功能都由一个称为容器管理器(container manager)的软件控制
  • Docker
  • Rocket
  • 镜像存储在注册库(registry)中,注册库通过网络访问
  • 注册库类似于一个使用 Java 的人眼中的 Maven 仓库、使用 .NET 的人眼中的 NuGet 服务器。
  • 容器管理器会封装你应用程序的所有东西为一个独立容器,该容器将会在容器管理器的管理下运行在宿主计算机上。
  • 每个容器有一个独立的 IP 地址
  • 在一个负载均衡容器后运行容器集群以获得更高的性能和高可用计算
  • Deis 的容器配置技术
  • 每次添加实例到环境中时,你不需要手动配置负载均衡器以便接受你的容器镜像。
  • 使用服务发现技术让容器告知均衡器它可用
張 旭

一位开发者的 Linux 容器之旅-技术 ◆ 学习|Linux.中国-开源社区 - 1 views

  • 容器是一个 Linux 进程,Linux 认为它只是一个运行中的进程。该进程只知道它被告知的东西。
  • 容器进程也分配了它自己的 IP 地址。
  • 在容器化方面,容器进程有它自己的 IP 地址。一旦给予了一个 IP 地址,该进程就是宿主网络中可识别的资源
  • ...20 more annotations...
  • 使容器 IP 映射到主机中能访问公网的 IP 地址。建立了该映射,无论出于什么意图和目的,容器就是网络上一个可访问的独立机器,从概念上类似于虚拟机。
  • 容器是拥有不同 IP 地址从而使其成为网络上可识别的独立 Linux 进程
  • CPU、内存和存储空间的分配是动态的,和典型虚拟机的静态方式不同。所有这些资源的共享都由容器管理器来管理。
  • 容器能非常快速地启动
  • 托管容器的计算机运行着被剥离的只剩下主要部分的某个 Linux 版本。
  • 操作系统被所有容器所共享,减少了容器足迹的重复和冗余。每个容器只包括该容器特有的部分
  • 层是一个容器镜像
  • 一个容器组件被称为层(layer)
  • 容器的各种功能都由一个称为容器管理器(container manager)的软件控制
  • 流行的容器管理器是 Docker 和 Rocket
  • 镜像存储在注册库(registry)中,注册库通过网络访问
  • 镜像代表了你的容器需要完成其工作的容器模板
  • 应用程序所需镜像的容器配置文件
  • 每个容器有一个独立的 IP 地址。因此,能把它放到负载均衡器后面。将容器放到负载均衡器后面,这就上升了一个层面。
  • Deis 的容器配置技术
  • 可以部署一个或多个容器镜像到主机上的负载均衡器下
  • 每次添加实例到环境中时,你不需要手动配置负载均衡器以便接受你的容器镜像。你可以使用服务发现技术让容器告知均衡器它可用。
  • 类似 CoreOS、RHEL Atomic、和 Ubuntu 的 Snappy 宿主操作系统
  • 类似 Docker 和 Rocket 的容器管理技术结合起来
  • 类似 Deis 这样的配置技术使容器创建和部署变得更加简单
張 旭

Active Record Basics - Ruby on Rails Guides - 0 views

  • the model - which is the layer of the system responsible for representing business data and logic.
  • Active Record facilitates the creation and use of business objects whose data requires persistent storage to a database
  • Rails will pluralize your class names to find the respective database table.
  • ...33 more annotations...
  • objects carry both persistent data and behavior which operates on that data
  • Object-Relational Mapping, commonly referred to as its abbreviation ORM, is a technique that connects the rich objects of an application to tables in a relational database management system
  • Represent associations between these models
  • Validate models before they get persisted to the database
  • The idea is that if you configure your applications in the very same way most of the times then this should be the default way.
  • Database Table - Plural with underscores separating words
  • use the ActiveRecord::Base.table_name= method to specify the table name
  • Model Class - Singular with the first letter of each word capitalized
  • Foreign keys - These fields should be named following the pattern singularized_table_name_id
  • Primary keys - By default, Active Record will use an integer column named id as the table's primary key
  • created_at
  • updated_at
  • (table_name)_count - Used to cache the number of belonging objects on associations.
  • Single Table Inheritance (STI)
  • Object Relational Mapping
  • class_name.yml
  • ActiveRecord::Base.primary_key=
  • CRUD is an acronym for the four verbs we use to operate on data: Create, Read, Update and Delete.
  • new method will return a new object
  • create will return the object and save it to the database.
  • Using the new method, an object can be instantiated without being saved
  • user.save will commit the record to the database
  • update_all class method
  • an Active Record object can be destroyed which removes it from the database
  • Validation is a very important issue to consider when persisting to database, so the methods create, save and update take it into account when running: they return false when validation fails and they didn't actually perform any operation on database.
  • a bang counterpart
  • Active Record callbacks allow you to attach code to certain events in the life-cycle of your models
  • Rails keeps track of which files have been committed to the database and provides rollback features
  • rake db:migrate
  • rake db:rollback
  • Convention over Configuration
    • 張 旭
       
      Model 是單數,Table 是複數。想象一下,處理 Object 的時候是逐一處理,但是存放的地方是放了一堆 Objects。
    • 張 旭
       
      外鍵是單數的形式,這也很好理解:因為關聯到的是一個外部的 Object
張 旭

Using Workflows to Schedule Jobs - CircleCI - 1 views

  • A workflow is a set of rules for defining a collection of jobs and their run order.
  • Schedule workflows for jobs that should only run periodically.
  • run multiple jobs in parallel
  • ...37 more annotations...
  • rerun just the failed job
  • Builds without workflows require a build job.
  • Refer the YAML Anchors/Aliases documentation for information about how to alias and reuse syntax to keep your .circleci/config.yml file small.
  • workflow orchestration with two parallel jobs
  • jobs run according to configured requirements, each job waiting to start until the required job finishes successfully
  • requires: key
  • fans-out to run a set of acceptance test jobs in parallel, and finally fans-in to run a common deploy job.
  • Holding a Workflow for a Manual Approval
  • Workflows can be configured to wait for manual approval of a job before continuing to the next job
  • add a job to the jobs list with the key type: approval
  • approval is a special job type that is only available to jobs under the workflow key
  • The name of the job to hold is arbitrary - it could be wait or pause, for example, as long as the job has a type: approval key in it.
  • schedule a workflow to run at a certain time for specific branches.
  • The triggers key is only added under your workflows key
  • using cron syntax to represent Coordinated Universal Time (UTC) for specified branches.
  • By default, a workflow is triggered on every git push
  • the commit workflow has no triggers key and will run on every git push
  • The nightly workflow has a triggers key and will run on the specified schedule
  • Cron step syntax (for example, */1, */20) is not supported.
  • use a context to share environment variables
  • use the same shared environment variables when initiated by a user who is part of the organization.
  • CircleCI does not run workflows for tags unless you explicitly specify tag filters.
  • CircleCI branch and tag filters support the Java variant of regex pattern matching.
  • Each workflow has an associated workspace which can be used to transfer files to downstream jobs as the workflow progresses.
  • The workspace is an additive-only store of data.
  • Jobs can persist data to the workspace
  • Downstream jobs can attach the workspace to their container filesystem.
  • Attaching the workspace downloads and unpacks each layer based on the ordering of the upstream jobs in the workflow graph.
  • Workflows that include jobs running on multiple branches may require data to be shared using workspaces
  • To persist data from a job and make it available to other jobs, configure the job to use the persist_to_workspace key.
  • Files and directories named in the paths: property of persist_to_workspace will be uploaded to the workflow’s temporary workspace relative to the directory specified with the root key.
  • Configure a job to get saved data by configuring the attach_workspace key.
  • persist_to_workspace
  • attach_workspace
  • To rerun only a workflow’s failed jobs, click the Workflows icon in the app and select a workflow to see the status of each job, then click the Rerun button and select Rerun from failed.
  • if you do not see your workflows triggering, a configuration error is preventing the workflow from starting.
  • check your Workflows page of the CircleCI app (not the Job page)
  •  
    "A workflow is a set of rules for defining a collection of jobs and their run order."
張 旭

Setup ProxySQL for High Availability (not a Single Point of Failure) - Percona Database... - 0 views

  • ProxySQL doesn’t natively support any high availability solution
  • most common solution is setting up ProxySQL as part of a tile architecture, where Application/ProxySQL are deployed together.
    • 張 旭
       
      直接把 ProxySQL 跟 App 捆綁發佈
  • If we have 400 instances of ProxySQL, we end up keeping our databases busy just performing the checks.
  • ...5 more annotations...
  • Another possible approach is to have two layers of ProxySQL, one close to the application and another in the middle to connect to the database.
  • creates additional complexity in the management of the platform, and it adds network hops.
  • combining existing solutions and existing blocks: KeepAlived + ProxySQl + MySQL.
  • Keepalived implements a set of checkers to dynamically and adaptively maintain and manage load-balanced server pool according to their health.
  • Keepalived implements a set of hooks to the VRRP finite state machine providing low-level and high-speed protocol interactions.
張 旭

What Is a Service Mesh? - NGINX - 0 views

  • The mesh provides service discovery, load balancing, encryption, authentication and authorization, support for the circuit breaker pattern, and other capabilities.
  • The service mesh is usually implemented by providing a proxy instance, called a sidecar, for each service instance
  • the term service is often used for both the instance definitions and the instances themselves.
  • ...5 more annotations...
  • A sidecar proxy is a proxy instance that’s dedicated to a specific service instance.
  • The container management framework keeps a list of instances that are ready to receive requests.
  • The service mesh can encrypt and decrypt requests and responses
  • a service mesh application also includes a monitoring and management layer, called the control plane.
  • Service mesh architectures are not ever likely to be the answer to all application development and delivery problems
張 旭

What Is Amazon VPC? - Amazon Virtual Private Cloud - 0 views

  • to allow an instance in your VPC to initiate outbound connections to the internet but prevent unsolicited inbound connections from the internet, you can use a network address translation (NAT) device for IPv4 traffic
  • A NAT device has an Elastic IP address and is connected to the internet through an internet gateway.
  • By default, each instance that you launch into a nondefault subnet has a private IPv4 address, but no public IPv4 address, unless you specifically assign one at launch, or you modify the subnet's public IP address attribute.
  • ...11 more annotations...
  • Amazon VPC is the networking layer for Amazon EC2.
  • A virtual private cloud (VPC) is a virtual network dedicated to your AWS account. It is logically isolated from other virtual networks in the AWS Cloud.
  • Instances can connect to the internet over IPv6 through an internet gateway
  • IPv6 traffic is separate from IPv4 traffic; your route tables must include separate routes for IPv6 traffic.
  • You can optionally connect your VPC to your own corporate data center using an IPsec AWS managed VPN connection, making the AWS Cloud an extension of your data center.
  • A VPN connection consists of a virtual private gateway attached to your VPC and a customer gateway located in your data center.
  • A virtual private gateway is the VPN concentrator on the Amazon side of the VPN connection. A customer gateway is a physical device or software appliance on your side of the VPN connection.
  • AWS PrivateLink is a highly available, scalable technology that enables you to privately connect your VPC to supported AWS services, services hosted by other AWS accounts (VPC endpoint services)
  • Traffic between your VPC and the service does not leave the Amazon network
  • To use AWS PrivateLink, create an interface VPC endpoint for a service in your VPC. This creates an elastic network interface in your subnet with a private IP address that serves as an entry point for traffic destined to the service.
  • create your own AWS PrivateLink-powered service (endpoint service) and enable other AWS customers to access your service.
張 旭

What are Docker : images? - Project Atomic - 0 views

  • Now we understand what these <none>:<none> images stand for. They stand for intermediate images and can be seen using docker images -a
  • They don’t result into a disk space problem but it is definitely a screen real estate problem
  • dangling images
  • ...7 more annotations...
  • Another style of <none>:<none> images are the dangling images which can cause disk space problems.
  • In programming languages like Java or Golang a dangling block of memory is a block that is not referenced by any piece of code.
  • a dangling file system layer in Docker is something that is unused and is not being referenced by any images.
  • intermediate images
  • do docker images and see <none>:<none> images in the list, these are dangling images and needs to be pruned.
  • These dangling images are produced as a result of docker build or pull command.
  • docker rmi $(docker images -f "dangling=true" -q)
張 旭

Let's Encrypt & Docker - Træfik - 0 views

  • automatically discover any services on the Docker host and let Træfik reconfigure itself automatically when containers get created (or shut down) so HTTP traffic can be routed accordingly.
  • use Træfik as a layer-7 load balancer with SSL termination for a set of micro-services used to run a web application.
  • Docker containers can only communicate with each other over TCP when they share at least one network.
  • ...15 more annotations...
  • Docker under the hood creates IPTable rules so containers can't reach other containers unless you'd want to
  • Træfik can listen to Docker events and reconfigure its own internal configuration when containers are created (or shut down).
  • Enable the Docker provider and listen for container events on the Docker unix socket we've mounted earlier.
  • Enable automatic request and configuration of SSL certificates using Let's Encrypt. These certificates will be stored in the acme.json file, which you can back-up yourself and store off-premises.
  • there isn't a single container that has any published ports to the host -- everything is routed through Docker networks.
  • Thanks to Docker labels, we can tell Træfik how to create its internal routing configuration.
  • container labels and service labels
  • With the traefik.enable label, we tell Træfik to include this container in its internal configuration.
  • tell Træfik to use the web network to route HTTP traffic to this container.
  • Service labels allow managing many routes for the same container.
  • When both container labels and service labels are defined, container labels are just used as default values for missing service labels but no frontend/backend are going to be defined only with these labels.
  • In the example, two service names are defined : basic and admin. They allow creating two frontends and two backends.
  • Always specify the correct port where the container expects HTTP traffic using traefik.port label.
  • all containers that are placed in the same network as Træfik will automatically be reachable from the outside world
  • With the traefik.frontend.auth.basic label, it's possible for Træfik to provide a HTTP basic-auth challenge for the endpoints you provide the label for.
張 旭

The Asset Pipeline - Ruby on Rails Guides - 0 views

  • provides a framework to concatenate and minify or compress JavaScript and CSS assets
  • adds the ability to write these assets in other languages and pre-processors such as CoffeeScript, Sass and ERB
  • invalidate the cache by altering this fingerprint
  • ...80 more annotations...
  • Rails 4 automatically adds the sass-rails, coffee-rails and uglifier gems to your Gemfile
  • reduce the number of requests that a browser makes to render a web page
  • Starting with version 3.1, Rails defaults to concatenating all JavaScript files into one master .js file and all CSS files into one master .css file
  • In production, Rails inserts an MD5 fingerprint into each filename so that the file is cached by the web browser
  • The technique sprockets uses for fingerprinting is to insert a hash of the content into the name, usually at the end.
  • asset minification or compression
  • The sass-rails gem is automatically used for CSS compression if included in Gemfile and no config.assets.css_compressor option is set.
  • Supported languages include Sass for CSS, CoffeeScript for JavaScript, and ERB for both by default.
  • When a filename is unique and based on its content, HTTP headers can be set to encourage caches everywhere (whether at CDNs, at ISPs, in networking equipment, or in web browsers) to keep their own copy of the content
  • asset pipeline is technically no longer a core feature of Rails 4
  • Rails uses for fingerprinting is to insert a hash of the content into the name, usually at the end
  • With the asset pipeline, the preferred location for these assets is now the app/assets directory.
  • Fingerprinting is enabled by default for production and disabled for all other environments
  • The files in app/assets are never served directly in production.
  • Paths are traversed in the order that they occur in the search path
  • You should use app/assets for files that must undergo some pre-processing before they are served.
  • By default .coffee and .scss files will not be precompiled on their own
  • app/assets is for assets that are owned by the application, such as custom images, JavaScript files or stylesheets.
  • lib/assets is for your own libraries' code that doesn't really fit into the scope of the application or those libraries which are shared across applications.
  • vendor/assets is for assets that are owned by outside entities, such as code for JavaScript plugins and CSS frameworks.
  • Any path under assets/* will be searched
  • By default these files will be ready to use by your application immediately using the require_tree directive.
  • By default, this means the files in app/assets take precedence, and will mask corresponding paths in lib and vendor
  • Sprockets uses files named index (with the relevant extensions) for a special purpose
  • Rails.application.config.assets.paths
  • causes turbolinks to check if an asset has been updated and if so loads it into the page
  • if you add an erb extension to a CSS asset (for example, application.css.erb), then helpers like asset_path are available in your CSS rules
  • If you add an erb extension to a JavaScript asset, making it something such as application.js.erb, then you can use the asset_path helper in your JavaScript code
  • The asset pipeline automatically evaluates ERB
  • data URI — a method of embedding the image data directly into the CSS file — you can use the asset_data_uri helper.
  • Sprockets will also look through the paths specified in config.assets.paths, which includes the standard application paths and any paths added by Rails engines.
  • image_tag
  • the closing tag cannot be of the style -%>
  • asset_data_uri
  • app/assets/javascripts/application.js
  • sass-rails provides -url and -path helpers (hyphenated in Sass, underscored in Ruby) for the following asset classes: image, font, video, audio, JavaScript and stylesheet.
  • Rails.application.config.assets.compress
  • In JavaScript files, the directives begin with //=
  • The require_tree directive tells Sprockets to recursively include all JavaScript files in the specified directory into the output.
  • manifest files contain directives — instructions that tell Sprockets which files to require in order to build a single CSS or JavaScript file.
  • You should not rely on any particular order among those
  • Sprockets uses manifest files to determine which assets to include and serve.
  • the family of require directives prevents files from being included twice in the output
  • which files to require in order to build a single CSS or JavaScript file
  • Directives are processed top to bottom, but the order in which files are included by require_tree is unspecified.
  • In JavaScript files, Sprockets directives begin with //=
  • If require_self is called more than once, only the last call is respected.
  • require directive is used to tell Sprockets the files you wish to require.
  • You need not supply the extensions explicitly. Sprockets assumes you are requiring a .js file when done from within a .js file
  • paths must be specified relative to the manifest file
  • require_directory
  • Rails 4 creates both app/assets/javascripts/application.js and app/assets/stylesheets/application.css regardless of whether the --skip-sprockets option is used when creating a new rails application.
  • The file extensions used on an asset determine what preprocessing is applied.
  • app/assets/stylesheets/application.css
  • Additional layers of preprocessing can be requested by adding other extensions, where each extension is processed in a right-to-left manner
  • require_self
  • use the Sass @import rule instead of these Sprockets directives.
  • Keep in mind that the order of these preprocessors is important
  • In development mode, assets are served as separate files in the order they are specified in the manifest file.
  • when these files are requested they are processed by the processors provided by the coffee-script and sass gems and then sent back to the browser as JavaScript and CSS respectively.
  • css.scss.erb
  • js.coffee.erb
  • Keep in mind the order of these preprocessors is important.
  • By default Rails assumes that assets have been precompiled and will be served as static assets by your web server
  • with the Asset Pipeline the :cache and :concat options aren't used anymore
  • Assets are compiled and cached on the first request after the server is started
  • RAILS_ENV=production bundle exec rake assets:precompile
  • Debug mode can also be enabled in Rails helper methods
  • If you set config.assets.initialize_on_precompile to false, be sure to test rake assets:precompile locally before deploying
  • By default Rails assumes assets have been precompiled and will be served as static assets by your web server.
  • a rake task to compile the asset manifests and other files in the pipeline
  • RAILS_ENV=production bin/rake assets:precompile
  • a recipe to handle this in deployment
  • links the folder specified in config.assets.prefix to shared/assets
  • config/initializers/assets.rb
  • The initialize_on_precompile change tells the precompile task to run without invoking Rails
  • The X-Sendfile header is a directive to the web server to ignore the response from the application, and instead serve a specified file from disk
  • the jquery-rails gem which comes with Rails as the standard JavaScript library gem.
  • Possible options for JavaScript compression are :closure, :uglifier and :yui
  • concatenate assets
張 旭

Helm | - 0 views

  • Helm will figure out where to install Tiller by reading your Kubernetes configuration file (usually $HOME/.kube/config). This is the same file that kubectl uses.
  • kubectl cluster-info
  • Role-Based Access Control (RBAC) enabled
  • ...133 more annotations...
  • initialize the local CLI
  • install Tiller into your Kubernetes cluster
  • helm install
  • helm init --upgrade
  • By default, when Tiller is installed, it does not have authentication enabled.
  • helm repo update
  • Without a max history set the history is kept indefinitely, leaving a large number of records for helm and tiller to maintain.
  • helm init --upgrade
  • Whenever you install a chart, a new release is created.
  • one chart can be installed multiple times into the same cluster. And each can be independently managed and upgraded.
  • helm list function will show you a list of all deployed releases.
  • helm delete
  • helm status
  • you can audit a cluster’s history, and even undelete a release (with helm rollback).
  • the Helm server (Tiller).
  • The Helm client (helm)
  • brew install kubernetes-helm
  • Tiller, the server portion of Helm, typically runs inside of your Kubernetes cluster.
  • it can also be run locally, and configured to talk to a remote Kubernetes cluster.
  • Role-Based Access Control - RBAC for short
  • create a service account for Tiller with the right roles and permissions to access resources.
  • run Tiller in an RBAC-enabled Kubernetes cluster.
  • run kubectl get pods --namespace kube-system and see Tiller running.
  • helm inspect
  • Helm will look for Tiller in the kube-system namespace unless --tiller-namespace or TILLER_NAMESPACE is set.
  • For development, it is sometimes easier to work on Tiller locally, and configure it to connect to a remote Kubernetes cluster.
  • even when running locally, Tiller will store release configuration in ConfigMaps inside of Kubernetes.
  • helm version should show you both the client and server version.
  • Tiller stores its data in Kubernetes ConfigMaps, you can safely delete and re-install Tiller without worrying about losing any data.
  • helm reset
  • The --node-selectors flag allows us to specify the node labels required for scheduling the Tiller pod.
  • --override allows you to specify properties of Tiller’s deployment manifest.
  • helm init --override manipulates the specified properties of the final manifest (there is no “values” file).
  • The --output flag allows us skip the installation of Tiller’s deployment manifest and simply output the deployment manifest to stdout in either JSON or YAML format.
  • By default, tiller stores release information in ConfigMaps in the namespace where it is running.
  • switch from the default backend to the secrets backend, you’ll have to do the migration for this on your own.
  • a beta SQL storage backend that stores release information in an SQL database (only postgres has been tested so far).
  • Once you have the Helm Client and Tiller successfully installed, you can move on to using Helm to manage charts.
  • Helm requires that kubelet have access to a copy of the socat program to proxy connections to the Tiller API.
  • A Release is an instance of a chart running in a Kubernetes cluster. One chart can often be installed many times into the same cluster.
  • helm init --client-only
  • helm init --dry-run --debug
  • A panic in Tiller is almost always the result of a failure to negotiate with the Kubernetes API server
  • Tiller and Helm have to negotiate a common version to make sure that they can safely communicate without breaking API assumptions
  • helm delete --purge
  • Helm stores some files in $HELM_HOME, which is located by default in ~/.helm
  • A Chart is a Helm package. It contains all of the resource definitions necessary to run an application, tool, or service inside of a Kubernetes cluster.
  • it like the Kubernetes equivalent of a Homebrew formula, an Apt dpkg, or a Yum RPM file.
  • A Repository is the place where charts can be collected and shared.
  • Set the $HELM_HOME environment variable
  • each time it is installed, a new release is created.
  • Helm installs charts into Kubernetes, creating a new release for each installation. And to find new charts, you can search Helm chart repositories.
  • chart repository is named stable by default
  • helm search shows you all of the available charts
  • helm inspect
  • To install a new package, use the helm install command. At its simplest, it takes only one argument: The name of the chart.
  • If you want to use your own release name, simply use the --name flag on helm install
  • additional configuration steps you can or should take.
  • Helm does not wait until all of the resources are running before it exits. Many charts require Docker images that are over 600M in size, and may take a long time to install into the cluster.
  • helm status
  • helm inspect values
  • helm inspect values stable/mariadb
  • override any of these settings in a YAML formatted file, and then pass that file during installation.
  • helm install -f config.yaml stable/mariadb
  • --values (or -f): Specify a YAML file with overrides.
  • --set (and its variants --set-string and --set-file): Specify overrides on the command line.
  • Values that have been --set can be cleared by running helm upgrade with --reset-values specified.
  • Chart designers are encouraged to consider the --set usage when designing the format of a values.yaml file.
  • --set-file key=filepath is another variant of --set. It reads the file and use its content as a value.
  • inject a multi-line text into values without dealing with indentation in YAML.
  • An unpacked chart directory
  • When a new version of a chart is released, or when you want to change the configuration of your release, you can use the helm upgrade command.
  • Kubernetes charts can be large and complex, Helm tries to perform the least invasive upgrade.
  • It will only update things that have changed since the last release
  • $ helm upgrade -f panda.yaml happy-panda stable/mariadb
  • deployment
  • If both are used, --set values are merged into --values with higher precedence.
  • The helm get command is a useful tool for looking at a release in the cluster.
  • helm rollback
  • A release version is an incremental revision. Every time an install, upgrade, or rollback happens, the revision number is incremented by 1.
  • helm history
  • a release name cannot be re-used.
  • you can rollback a deleted resource, and have it re-activate.
  • helm repo list
  • helm repo add
  • helm repo update
  • The Chart Development Guide explains how to develop your own charts.
  • helm create
  • helm lint
  • helm package
  • Charts that are archived can be loaded into chart repositories.
  • chart repository server
  • Tiller can be installed into any namespace.
  • Limiting Tiller to only be able to install into specific namespaces and/or resource types is controlled by Kubernetes RBAC roles and rolebindings
  • Release names are unique PER TILLER INSTANCE
  • Charts should only contain resources that exist in a single namespace.
  • not recommended to have multiple Tillers configured to manage resources in the same namespace.
  • a client-side Helm plugin. A plugin is a tool that can be accessed through the helm CLI, but which is not part of the built-in Helm codebase.
  • Helm plugins are add-on tools that integrate seamlessly with Helm. They provide a way to extend the core feature set of Helm, but without requiring every new feature to be written in Go and added to the core tool.
  • Helm plugins live in $(helm home)/plugins
  • The Helm plugin model is partially modeled on Git’s plugin model
  • helm referred to as the porcelain layer, with plugins being the plumbing.
  • helm plugin install https://github.com/technosophos/helm-template
  • command is the command that this plugin will execute when it is called.
  • Environment variables are interpolated before the plugin is executed.
  • The command itself is not executed in a shell. So you can’t oneline a shell script.
  • Helm is able to fetch Charts using HTTP/S
  • Variables like KUBECONFIG are set for the plugin if they are set in the outer environment.
  • In Kubernetes, granting a role to an application-specific service account is a best practice to ensure that your application is operating in the scope that you have specified.
  • restrict Tiller’s capabilities to install resources to certain namespaces, or to grant a Helm client running access to a Tiller instance.
  • Service account with cluster-admin role
  • The cluster-admin role is created by default in a Kubernetes cluster
  • Deploy Tiller in a namespace, restricted to deploying resources only in that namespace
  • Deploy Tiller in a namespace, restricted to deploying resources in another namespace
  • When running a Helm client in a pod, in order for the Helm client to talk to a Tiller instance, it will need certain privileges to be granted.
  • SSL Between Helm and Tiller
  • The Tiller authentication model uses client-side SSL certificates.
  • creating an internal CA, and using both the cryptographic and identity functions of SSL.
  • Helm is a powerful and flexible package-management and operations tool for Kubernetes.
  • default installation applies no security configurations
  • with a cluster that is well-secured in a private network with no data-sharing or no other users or teams.
  • With great power comes great responsibility.
  • Choose the Best Practices you should apply to your helm installation
  • Role-based access control, or RBAC
  • Tiller’s gRPC endpoint and its usage by Helm
  • Kubernetes employ a role-based access control (or RBAC) system (as do modern operating systems) to help mitigate the damage that can be done if credentials are misused or bugs exist.
  • In the default installation the gRPC endpoint that Tiller offers is available inside the cluster (not external to the cluster) without authentication configuration applied.
  • Tiller stores its release information in ConfigMaps. We suggest changing the default to Secrets.
  • release information
  • charts
  • charts are a kind of package that not only installs containers you may or may not have validated yourself, but it may also install into more than one namespace.
  • As with all shared software, in a controlled or shared environment you must validate all software you install yourself before you install it.
  • Helm’s provenance tools to ensure the provenance and integrity of charts
  •  
    "Helm will figure out where to install Tiller by reading your Kubernetes configuration file (usually $HOME/.kube/config). This is the same file that kubectl uses."
張 旭

MySQL :: MySQL 5.7 Reference Manual :: 19.1.1.2 Group Replication - 0 views

  • The replication group is a set of servers that interact with each other through message passing.
  • The communication layer provides a set of guarantees such as atomic message and total order message delivery.
  • a multi-master update everywhere replication protocol
  • ...8 more annotations...
  • a replication group is formed by multiple servers and each server in the group may execute transactions independently
  • Read-only (RO) transactions need no coordination within the group and thus commit immediately
  • any RW transaction the group needs to decide whether it commits or not, thus the commit operation is not a unilateral decision from the originating server
  • when a transaction is ready to commit at the originating server, the server atomically broadcasts the write values (rows changed) and the correspondent write set (unique identifiers of the rows that were updated). Then a global total order is established for that transaction.
  • all servers receive the same set of transactions in the same order
  • The resolution procedure states that the transaction that was ordered first commits on all servers, whereas the transaction ordered second aborts, and thus is rolled back on the originating server and dropped by the other servers in the group. This is in fact a distributed first commit wins rule
  • Group Replication is a shared-nothing replication scheme where each server has its own entire copy of the data
  • MySQL Group Replication protocol
‹ Previous 21 - 40 of 48 Next ›
Showing 20 items per page