Skip to main content

Home/ Larvata/ Group items tagged system

Rss Feed Group items tagged

Filter: All | Bookmarks | Topics Simple Middle
張 旭

Kubernetes 基本概念 · Kubernetes指南 - 0 views

feisky.gitbooks.io/...concepts.html

kubernetes volume namespace system devops

shared by 張 旭 on 30 May 19 - No Cached
  • Container（容器）是一种便携式、轻量级的操作系统级虚拟化技术。它使用 namespace 隔离不同的软件运行环境，并通过镜像自包含软件的运行环境，从而使得容器可以很方便的在任何地方运行。
  • 每个应用程序用容器封装，管理容器部署就等同于管理应用程序部署。+
  • Pod 是一组紧密关联的容器集合，它们共享 PID、IPC、Network 和 UTS namespace，是 Kubernetes 调度的基本单位。
  • ...9 more annotations...
  • 进程间通信和文件共享
  • 在 Kubernetes 中，所有对象都使用 manifest（yaml 或 json）来定义
  • Node 是 Pod 真正运行的主机，可以是物理机，也可以是虚拟机。
  • 每个 Node 节点上至少要运行 container runtime（比如 docker 或者 rkt）、kubelet 和 kube-proxy 服务。
  • 常见的 pods, services, replication controllers 和 deployments 等都是属于某一个 namespace 的（默认是 default）
  • node, persistentVolumes 等则不属于任何 namespace
  • Service 是应用服务的抽象，通过 labels 为应用提供负载均衡和服务发现。
  • 匹配 labels 的 Pod IP 和端口列表组成 endpoints，由 kube-proxy 负责将服务 IP 负载均衡到这些 endpoints 上。
  • 每个 Service 都会自动分配一个 cluster IP（仅在集群内部可访问的虚拟地址）和 DNS 名
  • 張 旭 on 30 May 19
     
    "常见的 pods, services, replication controllers 和 deployments 等都是属于某一个 namespace 的（默认是 default），而 node, persistentVolumes 等则不属于任何 namespace。"
張 旭

podman/rootless.md at master · containers/podman - 0 views

github.com/...rootless.md

docker kubernetes

shared by 張 旭 on 03 Aug 20 - No Cached
  • Podman can not create containers that bind to ports < 1024
  • If /etc/subuid and /etc/subgid are not setup for a user, then podman commands can easily fail
  • Fedora 31 defaults to cgroup V2, which has full support of rootless cgroup management.
  • ...6 more annotations...
  • Some system unit configuration options do not work in the rootless container
  • it's better to create an override.conf drop-in that sets PrivateNetwork=no
  • Difficult to use additional stores for sharing content
  • Can not use overlayfs driver, but does support fuse-overlayfs
  • No CNI Support
  • Making device nodes within a container fails, even when running --privileged.
張 旭

Running rootless Podman as a non-root user | Enable Sysadmin - 0 views

www.redhat.com/...rootless-podman-makes-sense

docker podman linux system

shared by 張 旭 on 04 Aug 20 - No Cached
  • By default, rootless Podman runs as root within the container.
  • the processes in the container have the default list of namespaced capabilities which allow the processes to act like root inside of the user namespace
  • the directory is owned by UID 26, but UID 26 is not mapped into the container and is not the same UID that Postgres runs with while in the container.
  • ...8 more annotations...
  • Podman launches a container inside of the user namespace, which is mapped with the range of UIDs defined for the user in /etc/subuid and /etc/subgid
  • The easy solution to this problem is to chown the html directory to match the UID that Postgresql runs with inside of the container.
  • use the podman unshare command, which drops you into the same user namespace that rootless Podman uses
  • This setup also means that the processes inside of the container are running as the user’s UID. If the container process escaped the container, the process would have full access to files in your home directory based on UID separation.
  • SELinux would still block the access, but I have heard that some people disable SELinux.
  • If you run the processes within the container as a different non-root UID, however, then those processes will run as that UID. If they escape the container, they would only have world access to content in your home directory.
  • run a podman unshare command, or set up the directories' group ownership as owned by your UID (root inside of the container).
  • running containers as non-root should always be your top priority for security reasons.
張 旭

Deploy a Replica Set - MongoDB Manual - 0 views

docs.mongodb.com/...deploy-replica-set

database mongodb

shared by 張 旭 on 13 Oct 20 - No Cached
  • Three member replica sets provide enough redundancy to survive most network partitions and other system failures.
張 旭

Deploy Replica Set With Keyfile Authentication - MongoDB Manual - 0 views

docs.mongodb.com/...et-with-keyfile-access-control

database mongodb

shared by 張 旭 on 13 Oct 20 - No Cached
  • Keyfiles are bare-minimum forms of security and are best suited for testing or development environments.
  • With keyfile authentication, each mongod instances in the replica set uses the contents of the keyfile as the shared password for authenticating other members in the deployment.
  • On UNIX systems, the keyfile must not have group or world permissions.
  • ...3 more annotations...
  • Copy the keyfile to each server hosting the replica set members.
  • the user running the mongod instances is the owner of the file and can access the keyfile.
  • For each member in the replica set, start the mongod with either the security.keyFile configuration file setting or the --keyFile command-line option.
張 旭

Helm | Getting Started - 0 views

helm.sh/...getting_started

helm

shared by 張 旭 on 02 Oct 21 - No Cached
  • The templates/ directory is for template files. When Helm evaluates a chart, it will send all of the files in the templates/ directory through the template rendering engine. It then collects the results of those templates and sends them on to Kubernetes.
  • The charts/ directory may contain other charts (which we call subcharts).
  • we recommend using the suffix .yaml for YAML files and .tpl for helpers.
  • ...8 more annotations...
  • The helm get manifest command takes a release name (full-coral) and prints out all of the Kubernetes resources that were uploaded to the server.
  • Each file begins with --- to indicate the start of a YAML document, and then is followed by an automatically generated comment line that tells us what template file generated this YAML document.
  • name: field is limited to 63 characters because of limitations to the DNS system.
  • The template directive {{ .Release.Name }} injects the release name into the template. The values that are passed into a template can be thought of as namespaced objects, where a dot (.) separates each namespaced element.
  • The leading dot before Release indicates that we start with the top-most namespace for this scope
  • helm install --debug --dry-run goodly-guppy ./mychart. This will render the templates. But instead of installing the chart, it will return the rendered template to you
  • Using --dry-run will make it easier to test your code, but it won't ensure that Kubernetes itself will accept the templates you generate.
  • It's best not to assume that your chart will install just because --dry-run works.
張 旭

Introducing CNAME Flattening: RFC-Compliant CNAMEs at a Domain's Root - 0 views

blog.cloudflare.com/liant-cnames-at-a-domains-root

dns networking

shared by 張 旭 on 15 Mar 22 - No Cached
  • you can now safely use a CNAME record, as opposed to an A record that points to a fixed IP address, as your root record in CloudFlare DNS without triggering a number of edge case error conditions because you’re violating the DNS spec.
  • CNAME Flattening allowed us to use a root domain while still maintaining DNS fault-tolerance across multiple IP addresses.
  • Traditionally, the root record of a domain needed to point to an IP address (known as an A -- for "address" -- Record).
  • ...13 more annotations...
  • WordPlumblr allows its users to use custom domains that point to the WordPlumblr infrastructure
  • A CNAME is an alias. It allows one domain to point to another domain which, eventually if you follow the CNAME chain, will resolve to an A record and IP address.
  • For example, WordPlumblr might have assigned the CNAME 6equj5.wordplumblr.com for Foo.com. Foo.com and the other customers may have all initially resolved, at the end of the CNAME chain, to the same IP address.
  • you usually don't want to address memory directly but, instead, you set up a pointer to a block of memory where you're going to store something. If the operating system needs to move the memory around then it just updates the pointer to point to wherever the chunk of memory has been moved to.
  • CNAMEs work great for subdomains like www.foo.com or blog.foo.com. Unfortunately, they don't work for a naked domain like foo.com itself.
  • the DNS spec enshrined that the root record -- the naked domain without any subdomain -- could not be a CNAME.
  • Technically, the root could be a CNAME but the RFCs state that once a record has a CNAME it can't have any other entries associated with it
  • a way to support a CNAME at the root, but still follow the RFC and return an IP address for any query for the root record.
  • extended our authoritative DNS infrastructure to, in certain cases, act as a kind of DNS resolver.
  • if there's a CNAME at the root, rather than returning that record directly we recurse through the CNAME chain ourselves until we find an A Record.
  • allows the flexibility of having CNAMEs at the root without breaking the DNS specification.
  • We cache the CNAME responses -- respecting the DNS TTLs, just like a recursor should -- which means often we have the answer without having to traverse the chain.
  • CNAME flattening solved email resolution errors for us which was very key.
張 旭

Docker can now run within Docker - Docker Blog - 0 views

www.docker.com/...cker-can-now-run-within-docker

devops docker

shared by 張 旭 on 11 Mar 22 - No Cached
  • Docker 0.6 is the new “privileged” mode for containers. It allows you to run some containers with (almost) all the capabilities of their host machine, regarding kernel features and device access.
  • Among the (many!) possibilities of the “privileged” mode, you can now run Docker within Docker itself.
  • in the new privileged mode.
  • ...8 more annotations...
  • that /var/lib/docker should be a volume. This is important, because the filesystem of a container is an AUFS mountpoint, composed of multiple branches; and those branches have to be “normal” filesystems (i.e. not AUFS mountpoints).
  • /var/lib/docker, the place where Docker stores its containers, cannot be an AUFS filesystem.
  • we use them as a pass-through to the “normal” filesystem of the host machine.
  • The /var/lib/docker directory of the nested Docker will live somewhere in /var/lib/docker/volumes on the host system.
  • since the private Docker instances run in privileged mode, they can easily escalate to the host, and you probably don’t want this! If you really want to run something like this and expose it to the public, you will have to fine-tune the LXC template file, to restrict the capabilities and devices available to the Docker instances.
  • When you are inside a privileged container, you can always nest one more level
  • the LXC tools cannot start nested containers if the devices control group is not in its own hierarchy.
  • if you use AppArmor, you need a special policy to support nested containers.
張 旭

Creating Highly Available clusters with kubeadm | Kubernetes - 0 views

kubernetes.io/...high-availability

kubernetes system cluster

shared by 張 旭 on 17 May 21 - No Cached
  • If instead, you prefer to copy certs across control-plane nodes manually or using automation tools, please remove this flag and refer to Manual certificate distribution section below.
  • if you are using a kubeadm configuration file set the podSubnet field under the networking object of ClusterConfiguration.
  • manually copy the certificates from the primary control plane node to the joining control plane nodes.
  • ...1 more annotation...
  • Copy only the certificates in the above list. kubeadm will take care of generating the rest of the certificates with the required SANs for the joining control-plane instances.
張 旭

Production environment | Kubernetes - 0 views

kubernetes.io/...production-environment

devops kubernetes

shared by 張 旭 on 25 Jan 23 - No Cached
  • to promote an existing cluster for production use
  • Separating the control plane from the worker nodes.
  • Having enough worker nodes available
  • ...22 more annotations...
  • You can use role-based access control (RBAC) and other security mechanisms to make sure that users and workloads can get access to the resources they need, while keeping workloads, and the cluster itself, secure. You can set limits on the resources that users and workloads can access by managing policies and container resources.
  • you need to plan how to scale to relieve increased pressure from more requests to the control plane and worker nodes or scale down to reduce unused resources.
  • Managed control plane: Let the provider manage the scale and availability of the cluster's control plane, as well as handle patches and upgrades.
  • The simplest Kubernetes cluster has the entire control plane and worker node services running on the same machine.
  • You can deploy a control plane using tools such as kubeadm, kops, and kubespray.
  • Secure communications between control plane services are implemented using certificates.
  • Certificates are automatically generated during deployment or you can generate them using your own certificate authority.
  • Separate and backup etcd service: The etcd services can either run on the same machines as other control plane services or run on separate machines
  • Create multiple control plane systems: For high availability, the control plane should not be limited to a single machine
  • Some deployment tools set up Raft consensus algorithm to do leader election of Kubernetes services. If the primary goes away, another service elects itself and take over.
  • Groups of zones are referred to as regions.
  • if you installed with kubeadm, there are instructions to help you with Certificate Management and Upgrading kubeadm clusters.
  • Production-quality workloads need to be resilient and anything they rely on needs to be resilient (such as CoreDNS).
  • Add nodes to the cluster: If you are managing your own cluster you can add nodes by setting up your own machines and either adding them manually or having them register themselves to the cluster’s apiserver.
  • Set up node health checks: For important workloads, you want to make sure that the nodes and pods running on those nodes are healthy.
  • Authentication: The apiserver can authenticate users using client certificates, bearer tokens, an authenticating proxy, or HTTP basic auth.
  • Authorization: When you set out to authorize your regular users, you will probably choose between RBAC and ABAC authorization.
  • Role-based access control (RBAC): Lets you assign access to your cluster by allowing specific sets of permissions to authenticated users. Permissions can be assigned for a specific namespace (Role) or across the entire cluster (ClusterRole).
  • Attribute-based access control (ABAC): Lets you create policies based on resource attributes in the cluster and will allow or deny access based on those attributes.
  • Set limits on workload resources
  • Set namespace limits: Set per-namespace quotas on things like memory and CPU
  • Prepare for DNS demand: If you expect workloads to massively scale up, your DNS service must be ready to scale up as well.
張 旭

Installing Addons | Kubernetes - 0 views

kubernetes.io/...addons

kubernetes network networking

shared by 張 旭 on 25 Jan 23 - No Cached
  • Calico is a networking and network policy provider. Calico supports a flexible set of networking options so you can choose the most efficient option for your situation, including non-overlay and overlay networks, with or without BGP. Calico uses the same engine to enforce network policy for hosts, pods, and (if using Istio & Envoy) applications at the service mesh layer.
  • Cilium is a networking, observability, and security solution with an eBPF-based data plane. Cilium provides a simple flat Layer 3 network with the ability to span multiple clusters in either a native routing or overlay/encapsulation mode, and can enforce network policies on L3-L7 using an identity-based security model that is decoupled from network addressing. Cilium can act as a replacement for kube-proxy; it also offers additional, opt-in observability and security features.
  • CoreDNS is a flexible, extensible DNS server which can be installed as the in-cluster DNS for pods.
  • ...1 more annotation...
  • The node problem detector runs on Linux nodes and reports system issues as either Events or Node conditions.
張 旭

Share Process Namespace between Containers in a Pod | Kubernetes - 0 views

kubernetes.io/...share-process-namespace

kubernetes pod namespace

shared by 張 旭 on 19 Aug 22 - No Cached
  • When process namespace sharing is enabled, processes in a container are visible to all other containers in the same pod.
  • It's even possible to access the file system of another container using the /proc/$pid/root link.
  • Pods share many resources so it makes sense they would also share a process namespace.
  • ...2 more annotations...
  • Processes are visible to other containers in the pod. This includes all information visible in /proc, such as passwords that were passed as arguments or environment variables. These are protected only by regular Unix permissions.
  • Container filesystems are visible to other containers in the pod through the /proc/$pid/root link. This makes debugging easier, but it also means that filesystem secrets are protected only by filesystem permissions.
  • 張 旭 on 19 Aug 22
     
    "When process namespace sharing is enabled, processes in a container are visible to all other containers in the same pod. "
張 旭

我做系统架构的一些原则 | 酷 壳 - CoolShell - 0 views

coolshell.cn/21672.html

system architecture

shared by 張 旭 on 03 Jan 22 - No Cached
  • 如果不说收益，只是为了技术而技术，而没有任何意义。
  • 有计划和无计划的停机做相应的解决方案
  • 经常不断的 human error
  • ...35 more annotations...
  • 运维又会分成基础运维和应用运维，开发则会分成基础核心开发和业务开发。
  • 基础运维和开发的同学更多的只是关注资源的利用率和性能，而应用运维和业务开发则更多关注的是应用和服务上的东西。
  • 有一些系统已经说不清楚是基础层的还是应用层的了，比如像服务治理上的东西，里面即有底层基础技术，也需要业务的同学来配合，包括 k8s 也样，里面即有底层的如网络这样的技术，也有需要业务配合的 readniess和 liveness 这样的健康检查，以及业务应用需要 configMap 等等 ……
  • 试想一下城市交通的优化，当城市规模到一定程度的时候，整体的性能你是无法通过优化几条路或是几条街区来完成的，你需要对整个城市做整体的功能体的规划才可能达到整体效率的提升
  • 当系统越来越复杂的时候，用户把他们的  PHP，Python, .NET，或 Node.js 的架构完全都迁移到 Java + Go 的架构上来的案例不断的发生。
  • 更为工业化的技术
  • 使用更为成熟更为工业化的技术栈，而不是自己熟悉的技术栈
  • 不要自己发明轮子，更不要魔改
  • 完全没有必要。不重新发明轮子，不魔改，不是因为自己技术不能，而是因为，这个世界早已不是自己干所有事的年代了
  • 好些公司的架构都被技术负责人个人的喜好、擅长和个人经验给绑架了，完全不是从一个客观的角度来进行技术选型
  • 全中国所有的电商平台，几百家银行，三大电信运营商，所有的保险公司，劵商的系统，医院里的系统，电子政府系统，等等，基本都是用 Java 开发的，包括 AWS 的主流语言也是 Java
  • NoSQL 的数据库在 Join 上都表现的太差
  • 为了不做 Join 就开始冗余数据，然而自己又维护不好冗余数据后带来的数据一致性的问题，导致数据上的各种错乱丢失。
  • 永远使用完备支持 ACID 的关系型数据库
  • 性能上的事，总是有解的，手段也是最多的，这个比起架构的完备性和扩展性来说真的不必太过担心。
  • 很多公司的系统既没有服从业界标准，也没有形成自己公司的标准，感觉就像一群乌合之众一样。
  • 最典型的例子就是 HTTP 调用的状态返回码。业内给你的标准是 200表示成功，3xx 跳转，4xx 表示调用端出错，5xx 表示服务端出错，我实在是不明白为什么无论成功和失败大家都喜欢返回 200，然后在 body 里指出是否error
  • Restful API 的规范。我觉得是非常重要的，这里给两个我觉得写得最好的参考：Paypal 和 Microsoft 。
  • 监控系统宁可自己死了也不能干扰实际应用。
  • 一个公司至少一年要有一次软件版本升级的review，然后形成软件版本的统一和一致
  • 架构和软件不是写好就完的，是需要不断修改不断维护的，80%的软件成本都是在维护上。
  • 通过服务发现或服务网关来降低服务依赖所带来的运维复杂度
  • 一定要使用各种软件设计的原则。比如：像SOLID这样的原则（参看《一些软件设计的原则》），IoC/DIP，SOA 或 Spring Cloud 等 架构的最佳实践（参看《SteveY对Amazon和Google平台的吐槽》中的 Service Interface 的那几条军规），分布式系统架构的相关实践（参看：《分布式系统的事务处理》，或微软件的 《Cloud Design Patterns》）……等等
  • 没有自动化测试，没有好的软件文档，没有质量好的代码，没有标准和规范
  • 以前欠下的技术债，都得要还，没打好的地基要重新打，没建配套设施都要建。这些基础设施如果不按照正确科学的方式建立的话，你是不可能有一个好的的系统
  • 与其花大力气迁就技术债务，不如直接还技术债
  • 建设没有技术债的“新城区”，并通过“防腐层 ”的架构模型，不要让技术债侵入“新城区”。
  • 如果有一天你在做技术决定的时候，开始凭自己以往的经验，那么你就已经不可能再成长了。
  • 做任何决定之前，最好花上一点时间，上网查一下相关的资料，技术博客，文章，论文等 ，同时，也看看各个公司，或是各个开源软件他们是怎么做的？然后，比较多种方案的 Pros/Cons，最终形成自己的决定
  • 对于 X-Y 问题，也就是说，用户为了解决 X问题，他觉得用 Y 可以解，于是问我 Y 怎么搞，结果搞到最后，发现原来要解决的 X 问题，这个时候最好的解决方案不是 Y，而是 Z。
  • 我很喜欢追问为什么 ，这种追问，会让客户也跟着来一起重新思考。
  • 激进并不是瞎搞，也不是见新技术就上，而是积极拥抱会改变未来的新技术
  • 不是不喜欢的就不学了，我对区块链和 Rust 我一样学习，我也知道这些技术的优势，但我不会大规模使用它们。
  • 进步永远来自于探索，探索是要付出代价的，但是收益更大。
  • 不敢冒险才是最大的冒险，不敢犯错才是最大的错误，害怕失去会让你失去的更多
張 旭

What ChatOps Solutions Should You Use Today? | PäksTech - 0 views

pakstech.com/chatops

devops chat

shared by 張 旭 on 16 Feb 22 - No Cached
  • The big elephant in the room is of course Hubot, which now hasn’t seen new commits in over three years.
  • Botkit bots are written in JavaScript and they run on Node.js
  • Errbot is a chatbot written in Python, it comes with a ton of features, and it is extendable with custom plugins.
  • ...8 more annotations...
  • by default they react to !commands in your chatroom. Commands can also trigger on regular expression matches, with or without a bot prefix.
  • Errbot also supports Markdown responses with Jinja2 templating.
  • Errbot supports webhooks; It has a small web server that can translate endpoints to your custom plugins.
  • It’s recommended that you configure this behind a web server such as nginx or Apache.
  • It works with the If This Then That (IFTTT) principle, meaning that you define a set of rules that the system then uses to take action.
  • Lita is a chat bot written in Ruby. Like the other bots I’ve mentioned, it is also open source and supports different chat platforms via plugins.
  • Gort is a newer entrant to the ChatOps space. As the name suggests it has been written in Go, and it is still under active development.
  • can persist information in databases, supports advanced parsers, and is extendable with custom skills.
« First ‹ Previous 241 - 257 of 257
Showing 20▼ items per page