Skip to main content

Home/ Larvata/ Contents contributed and discussions participated by 張 旭

Contents contributed and discussions participated by 張 旭

Simple Middle Filter: All | Bookmarks | Topics
張 旭

Flynn: first preview release | Hacker News - 0 views

news.ycombinator.com/item

docker consul system service discovery dns

shared by 張 旭 on 25 Nov 15 - No Cached
  • Etcd and Zookeeper provide essentially the same functionality. They are both a strongly consistent key/value stores that support notifications to clients of changes. These two projects are limited to service discovery
  • So lets say you had a client application that would talk to a node application that could be on any number of servers. What you could do is hard code that list into your application and randomly select one, in order to "fake" load balancing. However every time a machine went up or down you would have to update that list.
  • What Consul provides is you just tell your app to connect to "mynodeapp.consul" and then consul will give you the proper address of one of your node apps.
  • ...9 more annotations...
  • Consul and Skydock are both applications that build on top of a tool like Zookeeper and Etcd.
  • What a developer ideally wants to do is just push code and not have to worry about what servers are running what, and worry about failover and the like
  • What Flynn provides (if I get it), is a diy Heroku like platform
  • Raft is a consensus algorithm for keeping a set of distributed state machines in a consistent state.
  • a self hosted Heroku
  • Google Omega is Google's answer to Apache Mesos
  • Omega would need a service like Raft to understand what services are currently available
  • Another project that I believe may be similar to Flynn is Apache Mesos.
  • I want to use Docker, but it has no easy way to say "take this file that contains instructions and make everything". You can write Dockerfiles, but you can only use one part of the stack in them, otherwise you run into trouble.
  • 張 旭 on 25 Nov 15
     
    " So lets say you had a client application that would talk to a node application that could be on any number of servers. What you could do is hard code that list into your application and randomly select one, in order to "fake" load balancing. However every time a machine went up or down you would have to update that list. What Consul provides is you just tell your app to connect to "mynodeapp.consul" and then consul will give you the proper address of one of your node apps."
張 旭

一位开发者的 Linux 容器之旅-技术 ◆ 学习|Linux.中国-开源社区 - 1 views

linux.cn/article-6594-1.html

linux docker container system development devops

shared by 張 旭 on 22 Nov 15 - No Cached
  • 容器是一个 Linux 进程，Linux 认为它只是一个运行中的进程。该进程只知道它被告知的东西。
  • 容器进程也分配了它自己的 IP 地址。
  • 在容器化方面，容器进程有它自己的 IP 地址。一旦给予了一个 IP 地址，该进程就是宿主网络中可识别的资源
  • ...20 more annotations...
  • 使容器 IP 映射到主机中能访问公网的 IP 地址。建立了该映射，无论出于什么意图和目的，容器就是网络上一个可访问的独立机器，从概念上类似于虚拟机。
  • 容器是拥有不同 IP 地址从而使其成为网络上可识别的独立 Linux 进程
  • CPU、内存和存储空间的分配是动态的，和典型虚拟机的静态方式不同。所有这些资源的共享都由容器管理器来管理。
  • 容器能非常快速地启动
  • 托管容器的计算机运行着被剥离的只剩下主要部分的某个 Linux 版本。
  • 操作系统被所有容器所共享，减少了容器足迹的重复和冗余。每个容器只包括该容器特有的部分
  • 层是一个容器镜像
  • 一个容器组件被称为层（layer）
  • 容器的各种功能都由一个称为容器管理器（container manager）的软件控制
  • 流行的容器管理器是 Docker 和 Rocket
  • 镜像存储在注册库（registry）中，注册库通过网络访问
  • 镜像代表了你的容器需要完成其工作的容器模板
  • 应用程序所需镜像的容器配置文件
  • 每个容器有一个独立的 IP 地址。因此，能把它放到负载均衡器后面。将容器放到负载均衡器后面，这就上升了一个层面。
  • Deis 的容器配置技术
  • 可以部署一个或多个容器镜像到主机上的负载均衡器下
  • 每次添加实例到环境中时，你不需要手动配置负载均衡器以便接受你的容器镜像。你可以使用服务发现技术让容器告知均衡器它可用。
  • 类似 CoreOS、RHEL Atomic、和 Ubuntu 的 Snappy 宿主操作系统
  • 类似 Docker 和 Rocket 的容器管理技术结合起来
  • 类似 Deis 这样的配置技术使容器创建和部署变得更加简单
張 旭

一位开发者的 Linux 容器之旅 - 51CTO.COM - 0 views

developer.51cto.com/...497798.htm

docker linux container system development devops

shared by 張 旭 on 22 Nov 15 - No Cached
  • 容器是一个 Linux 进程，Linux 认为它只是一个运行中的进程。该进程只知道它被告知的东西。
  • 容器进程也分配了它自己的 IP 地址
  • 操作系统被所有容器所共享，减少了容器足迹的重复和冗余。每个容器只包括该容器特有的部分
  • ...26 more annotations...
  • 可以在容器管理器上运行命令，使容器 IP 映射到主机中能访问公网的 IP 地址。建立了该映射，无论出于什么意图和目的，容器就是网络上一个可访问的独立机器，从概念上类似于虚拟机。
  • 容器是拥有不同 IP 地址从而使其成为网络上可识别的独立 Linux 进程
  • 容器/进程以动态、合作的方式共享主机上的资源。
  • 容器能非常快速地启动
  • 和典型虚拟机的静态方式不同。所有这些资源的共享都由容器管理器来管理。
  • 获得了虚拟机独立和封装的好处，而抛弃了静态资源专有的缺陷
  • 托管容器的计算机运行着被剥离的只剩下主要部分的某个 Linux 版本
  • Ubuntu Snappy
  • Red Hat Atomic Host
  • CoreOS
  • 在容器化方面，容器进程有它自己的 IP 地址。一旦给予了一个 IP 地址，该进程就是宿主网络中可识别的资源
  • 一个容器组件被称为层（layer）
  • 层是一个容器镜像
  • 容器管理器只提供你所要的操作系统在宿主操作系统中不存在的部分
  • 在容器配置文件中重新定义层
  • 容器的各种功能都由一个称为容器管理器（container manager）的软件控制
  • Docker
  • Rocket
  • 镜像存储在注册库（registry）中，注册库通过网络访问
  • 注册库类似于一个使用 Java 的人眼中的 Maven 仓库、使用 .NET 的人眼中的 NuGet 服务器。
  • 容器管理器会封装你应用程序的所有东西为一个独立容器，该容器将会在容器管理器的管理下运行在宿主计算机上。
  • 每个容器有一个独立的 IP 地址
  • 在一个负载均衡容器后运行容器集群以获得更高的性能和高可用计算
  • Deis 的容器配置技术
  • 每次添加实例到环境中时，你不需要手动配置负载均衡器以便接受你的容器镜像。
  • 使用服务发现技术让容器告知均衡器它可用
張 旭

单表60亿记录等大数据场景的MySQL优化和运维之道 - 快课网 - 0 views

www.cricode.com/3981.html

mysql system performance HA database

shared by 張 旭 on 18 Nov 15 - No Cached
  • 存储引擎使用InnoDB
  • 变长字符串尽量使用varchar varbinary
  • 不在数据库中存储图片、文件等
  • ...34 more annotations...
  • 库名、表名、字段名、索引名使用小写字母，以下划线分割 ，需要见名知意
  • 所有字段均定义为NOT NULL ，除非你真的想存Null
  • 使用TIMESTAMP存储时间
  • 使用DECIMAL存储精确浮点数，用float有的时候会有问题
  • 单个索引字段数不超过5，单表索引数量不超过5，索引设计遵循B+ Tree索引最左前缀匹配原则
  • 建立的索引能覆盖80%主要的查询，不求全，解决问题的主要矛盾
  • 避免冗余索引
  • 索引这个东西是一把双刃剑，在加速读的同时也引入了很多额外的写入和锁，降低写入能力
  • 字段定义为varchar，但传入的值是个int，就会导致全表扫描，要求程序端要做好类型检查
  • 避免使用大表的JOIN，MySQL优化器对join优化策略过于简单
  • UPDATE、DELETE语句不使用LIMIT ，容易造成主从不一致
  • 高危操作检查，Drop前做好数据备份
  • 日志分析，主要是指的MySQL慢日志和错误日志
  • Percona公司根据Facebook OSC思路，用perl重写了一版，就是我们现在用得很多的pt-online-schema-change，软件本身非常成熟，支持目前主流版本
  • 原生主从同步肯定存在着性能和安全性问题
  • Sharding is very complex, so itʼs best not to shard until itʼs obvious that you will actually need to!
  • 有中间层控制拆分逻辑最好，否则拆分过细管理成本会很高
  • 全量binlog备份
  • xtrabackup热备
  • 采用分布式文件系统存储备份
  • 基于库级别的复制，所以如果你只有一个库，使用这个意义不大
  • 半同步复制，从5.5开始支持
  • 半同步通过从库返回ACK这种方式确认从库收到数据
  • Secondsbehindmaster来判断延时不可靠，在网络抖动或者一些特殊参数配置情况下，会造成这个值是0但其实延时很大了。通过heartbeat表插入时间戳这种机制判断延时是更靠谱的
  • Binlog格式，建议都采用row格式，数据一致性更好
  • 成熟开源事务存储引擎，支持ACID，支持事务四个隔离级别，更好的数据安全性，高性能高并发，MVCC，细粒度锁，支持O_DIRECT
  • 数据安全性至关重要，InnoDB完胜
  • 主流使用TokuDB主要是看中了它的高压缩比
  • TokuDB在测试过程中写入稳定性是非常好的
  • 单表容量在InnoDB下1TB+，使用Tokudb的lzma压缩到80GB
  • 独立写程序好一些，与程序解耦方便后期维护
  • 追踪字段值变化可以通过分析row格式binlog好一些
  • 解决了单表过大恢复时间问题，也支持online DDL
  • 物理备份采用xtrabackup热备方案比较好
張 旭

Automated Docker-based Rails deployments - 0 views

cocoahunter.com/...docker-3

docker deploy system rails development ruby web

shared by 張 旭 on 23 Oct 15 - No Cached
  • how to automate the whole deployment process with a real world
  • use Unicorn as our webserver
  • 張 旭 on 23 Oct 15
     
    "This is the third post in a series of 3 on how my company moved its infrastructure from PaaS to Docker based deployment."
張 旭

phusion/baseimage-docker - 1 views

github.com/...baseimage-docker

docker ubuntu base system

shared by 張 旭 on 24 Jun 15 - No Cached
    • 張 旭
      張 旭 on 24 Jun 15
       
      原始的 docker 在執行命令時，預設就是將傳入的 COMMAND 當成 PID 1 的程序，執行完畢就結束這個  docker，其他的 daemons 並不會執行，而 baseimage 解決了這個問題。
  • docker exec
  • Through SSH
  • ...57 more annotations...
  • docker exec -t -i YOUR-CONTAINER-ID bash -l
  • Login to the container
  • Baseimage-docker only advocates running multiple OS processes inside a single container.
  • Password and challenge-response authentication are disabled by default. Only key authentication is allowed.
  • A tool for running a command as another user
  • The Docker developers advocate the philosophy of running a single logical service per container. A logical service can consist of multiple OS processes.
  • All syslog messages are forwarded to "docker logs".
  • Baseimage-docker advocates running multiple OS processes inside a single container, and a single logical service can consist of multiple OS processes.
  • Baseimage-docker provides tools to encourage running processes as different users
  • sometimes it makes sense to run multiple services in a single container, and sometimes it doesn't.
  • Splitting your logical service into multiple OS processes also makes sense from a security standpoint.
  • using environment variables to pass parameters to containers is very much the "Docker way"
  • Baseimage-docker provides a facility to run a single one-shot command, while solving all of the aforementioned problems
  • the shell script must run the daemon without letting it daemonize/fork it.
  • All executable scripts in /etc/my_init.d, if this directory exists. The scripts are run in lexicographic order.
  • variables will also be passed to all child processes
  • Environment variables on Unix are inherited on a per-process basis
  • there is no good central place for defining environment variables for all applications and services
  • centrally defining environment variables
  • One of the ideas behind Docker is that containers should be stateless, easily restartable, and behave like a black box.
  • a one-shot command in a new container
  • immediately exit after the command exits,
  • However the downside of this approach is that the init system is not started. That is, while invoking COMMAND, important daemons such as cron and syslog are not running. Also, orphaned child processes are not properly reaped, because COMMAND is PID 1.
  • add additional daemons (e.g. your own app) to the image by creating runit entries.
  • Nginx is one such example: it removes all environment variables unless you explicitly instruct it to retain them through the env configuration option.
  • Mechanisms for easily running multiple processes, without violating the Docker philosophy
  • Ubuntu is not designed to be run inside Docker
  • According to the Unix process model, the init process -- PID 1 -- inherits all orphaned child processes and must reap them
  • Syslog-ng seems to be much more stable
  • cron daemon
  • Rotates and compresses logs
  • /sbin/setuser
  • A tool for installing apt packages that automatically cleans up after itself.
  • a single logical service inside a single container
  • A daemon is a program which runs in the background of its system, such as a web server.
  • The shell script must be called run, must be executable, and is to be placed in the directory /etc/service/<NAME>. runsv will switch to the directory and invoke ./run after your container starts.
  • If any script exits with a non-zero exit code, the booting will fail.
  • If your process is started with a shell script, make sure you exec the actual process, otherwise the shell will receive the signal and not your process.
  • any environment variables set with docker run --env or with the ENV command in the Dockerfile, will be picked up by my_init
  • not possible for a child process to change the environment variables of other processes
  • they will not see the environment variables that were originally passed by Docker.
  • We ignore HOME, SHELL, USER and a bunch of other environment variables on purpose, because not ignoring them will break multi-user containers.
  • my_init imports environment variables from the directory /etc/container_environment
  • /etc/container_environment.sh - a dump of the environment variables in Bash format.
  • modify the environment variables in my_init (and therefore the environment variables in all child processes that are spawned after that point in time), by altering the files in /etc/container_environment
  • my_init only activates changes in /etc/container_environment when running startup scripts
  • environment variables don't contain sensitive data, then you can also relax the permissions
  • Syslog messages are forwarded to the console
  • syslog-ng is started separately before the runit supervisor process, and shutdown after runit exits.
  • RUN apt-get update && apt-get upgrade -y -o Dpkg::Options::="--force-confold"
  • /sbin/my_init --skip-startup-files --quiet --
  • By default, no keys are installed, so nobody can login
  • provide a pregenerated, insecure key (PuTTY format)
  • RUN /usr/sbin/enable_insecure_key
  • docker run YOUR_IMAGE /sbin/my_init --enable-insecure-key
  • RUN cat /tmp/your_key.pub >> /root/.ssh/authorized_keys && rm -f /tmp/your_key.pub
  • The default baseimage-docker installs syslog-ng, cron and sshd services during the build process
張 旭

The Rubyist's Guide to Environment Variables - Honeybadger.io Blog - 0 views

blog.honeybadger.io/by-guide-environment-variables

ruby environment development shell

shared by 張 旭 on 12 Jun 15 - No Cached
  • If you want to be able to effectively manage web apps in development and in production, you have to understand environment variables.
  • Every program you run on your server has at least one process. That process gets its own set of environment variables.
  • Once it has them, nothing outside of that process can change them.
  • ...9 more annotations...
  • Every program you run on your server gets its own set of environment variables at the moment you launch it.
  • Environment variables die with their process
  • Every process has a parent. That’s because every program has to be started by some other program.
  • Child processes inherit env vars from their parent
  • By default a child will get copies of every environment variable that its parent has. But the parent has control over this.
  • Changes to the environment don’t sync between running processes
  • shells do provide their own “local” shell variable systems
  • Environment variables are NOT the same as shell variables
  • use the export command to convert the local variable into an environment variable.
張 旭

MonolithFirst - 0 views

martinfowler.com/...MonolithFirst.html

system architecture deploy

shared by 張 旭 on 05 Jun 15 - No Cached
  • Microservices are a useful architecture, but even their advocates say that using them incurs a significant MicroservicePremium, which means they are only useful with more complex systems.
  • you should build a new application as a monolith initially, even if you think it's likely that it will benefit from a microservices architecture later on.
  • Any refactoring of functionality between services is much harder than it is in a monolith.
  • ...4 more annotations...
  • By building a monolith first, you can figure out what the right boundaries are, before a microservices design brushes a layer of treacle over them.
  • The logical way is to design a monolith carefully, paying attention to modularity within the software, both at the API boundaries and how the data is stored.
  • start with a monolith and gradually peel off microservices at the edges
  • Don't be afraid of building a monolith that you will discard, particularly if a monolith can get you to market quickly
張 旭

Docker Explained: Using Dockerfiles to Automate Building of Images | DigitalOcean - 0 views

www.digitalocean.com/...to-automate-building-of-images

docker tutorial

shared by 張 旭 on 01 Jun 15 - No Cached
  • CMD would be running an application upon creation of a container which is already installed using RUN (e.g. RUN apt-get install …) inside the image
  • ENTRYPOINT argument sets the concrete default application that is used every time a container is created using the image.
  • ENV command is used to set the environment variables (one or more).
  • ...6 more annotations...
  • EXPOSE command is used to associate a specified port to enable networking between the running process inside the container and the outside world
  • defines the base image to use to start the build process
  • Unlike CMD, it actually is used to build the image (forming another layer on top of the previous one which is committed).
  • VOLUME command is used to enable access from your container to a directory on the host machine
  • set where the command defined with CMD is to be executed
  • To detach yourself from the container, use the escape sequence CTRL+P followed by CTRL+Q
張 旭

Best practices for writing Dockerfiles - Docker Documentation - 0 views

docs.docker.com/...dockerfile_best-practices

docker tutorial deploy

shared by 張 旭 on 29 May 15 - No Cached
  • Run only one process per container
  • use current Official Repositories as the basis for your image
  • put long or complex RUN statements on multiple lines separated with backslashes.
  • ...16 more annotations...
  • CMD instruction should be used to run the software contained by your image, along with any arguments
  • CMD should be given an interactive shell (bash, python, perl, etc)
  • COPY them individually, rather than all at once
  • COPY is preferred
  • using ADD to fetch packages from remote URLs is strongly discouraged
  • always use COPY
  • The best use for ENTRYPOINT is to set the image's main command, allowing that image to be run as though it was that command (and then use CMD as the default flags).
  • the image name can double as a reference to the binary as shown in the command above
  • ENTRYPOINT instruction can also be used in combination with a helper script
  • The VOLUME instruction should be used to expose any database storage area, configuration storage, or files/folders created by your docker container.
  • use USER to change to a non-root user
  • avoid installing or using sudo
  • avoid switching USER back and forth frequently.
  • always use absolute paths for your WORKDIR
  • ONBUILD is only useful for images that are going to be built FROM a given image
  • The “onbuild” image will fail catastrophically if the new build's context is missing the resource being added.
張 旭

#JavaScript：this用法整理 - 翼世界夢想領域 - 0 views

knightzone.org/?p=106

javascript this

shared by 張 旭 on 15 May 15 - No Cached
  • JavaScript裡的this看的是究竟是誰調用該函式，而不是看該函式被定義在哪個物件內
  • call與apply都是呼叫該函式並讓該函式的this指向給予call或apply的第一個參數。
  • call和apply的差別則是在於其後面給予被調用之函式的參數放入的方法不同
  • ...2 more annotations...
  • 為什麼常常別人實作的callback函式可讓this指向於調用放入該callback的函式之this所指向之物件呢？
  • 實作上會遵守一個規則，會將自己的this或是直觀上合理的物件傳給callback當作它的this來用！
« First ‹ Previous 521 - 540 of 596 Next › Last »
Showing 20▼ items per page