Skip to main content

Home/ Larvata/ Group items tagged unix

Rss Feed Group items tagged

Filter: All | Bookmarks | Topics Simple Middle
張 旭

鳥哥的 Linux 私房菜 -- 第一章、Linux是什麼與如何學習 - 0 views

linux.vbird.org/...0110whatislinux.php

system linux

shared by 張 旭 on 22 Jun 21 - Cached
  • Linux就是核心與系統呼叫介面那兩層
  • 核心與硬體的關係非常的強烈
  • Linux提供了一個完整的作業系統當中最底層的硬體控制與資源管理的完整架構， 這個架構是沿襲Unix良好的傳統來的，所以相當的穩定而功能強大
  • ...31 more annotations...
  • Linux的核心是由Linus Torvalds在1991年的時候給他開發出來的， 並且丟到網路上提供大家下載，後來大家覺得這個小東西(Linux Kernel)相當的小而精巧， 所以慢慢的就有相當多的朋友投入這個小東西的研究領域裡面去
  • 1960年代初期麻省理工學院(MIT)發展了所謂的： 『相容分時系統(Compatible Time-Sharing System, CTSS)』， 它可以讓大型主機透過提供數個終端機(terminal)以連線進入主機，來利用主機的資源進行運算工作
  • 為了更加強化大型主機的功能，以讓主機的資源可以提供更多使用者來利用，所以在1965年前後， 由貝爾實驗室(Bell)、麻省理工學院(MIT)及奇異公司(GE, 或稱為通用電器)共同發起了Multics的計畫
  • 以組合語言(Assembler)寫出了一組核心程式，同時包括一些核心工具程式， 以及一個小小的檔案系統。那個系統就是Unix的原型！ 當時Thompson將Multics龐大的複雜系統簡化了不少，於是同實驗室的朋友都戲稱這個系統為：Unics。(當時尚未有Unix的名稱)
  • 所有的程式或系統裝置都是檔案
  • 不管建構編輯器還是附屬檔案，所寫的程式只有一個目的，且要有效的完成目標。
  • Dennis Ritchie (註3) 將B語言重新改寫成C語言，再以C語言重新改寫與編譯Unics的核心， 最後正名與發行出Unix的正式版本！
  • 由於Unix是以較高階的C語言寫的，相對於組合語言需要與硬體有密切的配合， 高階的C語言與硬體的相關性就沒有這麼大了！所以，這個改變也使得Unix很容易被移植到不同的機器上面喔！
  • AT&T此時對於Unix是採取較開放的態度，此外，Unix是以高階的C語言寫成的， 理論上是具有可移植性的！亦即只要取得Unix的原始碼，並且針對大型主機的特性加以修訂原有的原始碼(Source Code)， 就可能將Unix移植到另一部不同的主機上頭了。
  • 柏克萊大學的Bill Joy (註4)在取得了Unix的核心原始碼後，著手修改成適合自己機器的版本， 並且同時增加了很多工具軟體與編譯程式，最終將它命名為Berkeley Software Distribution (BSD)。
  • 每一家公司自己出的Unix雖然在架構上面大同小異，但是卻真的僅能支援自身的硬體， 所以囉，早先的Unix只能與伺服器(Server)或者是大型工作站(Workstation)劃上等號！
  • AT&T在1979年發行的第七版Unix中，特別提到了 『不可對學生提供原始碼』的嚴格限制！
  • 純種的Unix指的就是System V以及BSD
  • AT&T自家的System V
  • 既然1979年的Unix第七版可以在Intel的x86架構上面進行移植， 那麼是否意味著可以將Unix改寫並移植到x86上面了呢？在這個想法上， 譚寧邦教授於是乎自己動手寫了Minix這個Unix Like的核心程式！
  • 『既然作業系統太複雜，我就先寫可以在Unix上面運行的小程式，這總可以了吧？』
  • 如果能夠寫出一個不錯的編譯器，那不就是大家都需要的軟體了嗎？ 因此他便開始撰寫C語言的編譯器，那就是現在相當有名的GNU C Compiler(gcc)！
  • 他還撰寫了更多可以被呼叫的C函式庫(GNU C library)，以及可以被使用來操作作業系統的基本介面BASH shell！ 這些都在1990年左右完成了！
  • 有鑑於圖形使用者介面(Graphical User Interface, GUI) 的需求日益加重，在1984年由MIT與其他協力廠商首次發表了X Window System ，並且更在1988年成立了非營利性質的XFree86這個組織。所謂的XFree86其實是 X Window System + Free + x86的整合名稱呢！
  • 譚寧邦教授為了教育需要而撰寫的Minix系統！ 他在購買了最新的Intel 386的個人電腦後，就立即安裝了Minix這個作業系統。 另外，上個小節當中也談到，Minix這個作業系統是有附上原始碼的， 所以托瓦茲也經由這個原始碼學習到了很多的核心程式設計的設計概念喔！
  • 托瓦茲自己也說：『我始終是個性能癖』^_^。 為了徹底發揮386的效能，於是托瓦茲花了不少時間在測試386機器上！ 他的重要測試就是在測試386的多功性能。首先，他寫了三個小程式，一個程式會持續輸出A、一個會持續輸出B， 最後一個會將兩個程式進行切換。他將三個程式同時執行，結果，他看到螢幕上很順利的一直出現ABABAB...... 他知道，他成功了！ ^_^
  • 為了讓所有的軟體都可以在Linux上執行，於是托瓦茲開始參考標準的POSIX規範。
  • POSIX是可攜式作業系統介面(Portable Operating System Interface)的縮寫，重點在規範核心與應用程式之間的介面， 這是由美國電器與電子工程師學會(IEEE)所發佈的一項標準喔
  • 因為托瓦茲放置核心的那個FTP網站的目錄為：Linux， 從此，大家便稱這個核心為Linux了。(請注意，此時的Linux就是那個kernel喔！ 另外，托瓦茲所丟到該目錄下的第一個核心版本為0.02呢！)
  • Linux其實就是一個作業系統最底層的核心及其提供的核心工具。 他是GNU GPL授權模式，所以，任何人均可取得原始碼與可執行這個核心程式，並且可以修改。
  • Linux參考POSIX設計規範，於是相容於Unix作業系統，故亦可稱之為Unix Like的一種
  • 為了讓使用者能夠接觸到Linux，於是很多的商業公司或非營利團體， 就將Linux Kernel(含tools)與可運行的軟體整合起來，加上自己具有創意的工具程式， 這個工具程式可以讓使用者以光碟/DVD或者透過網路直接安裝/管理Linux系統。 這個『Kernel + Softwares + Tools + 可完整安裝程序』的咚咚，我們稱之為Linux distribution， 一般中文翻譯成可完整安裝套件，或者Linux發佈商套件等。
  • 在1994年終於完成的Linux的核心正式版！version 1.0。 這一版同時還加入了X Window System的支援呢！且於1996年完成了2.0版、2011 年釋出 3.0 版，更於 2015 年 4 月釋出了 4.0 版哩！ 發展相當迅速喔！此外，托瓦茲指明了企鵝為Linux的吉祥物。
  • Linux本身就是個最陽春的作業系統，其開發網站設立在http://www.kernel.org，我們亦稱Linux作業系統最底層的資料為『核心(Kernel)』。
  • 常見的 Linux distributions 分類有『商業、社群』分類法，或『RPM、DPKG』分類法
  • 事實上鳥哥認為distributions主要分為兩大系統，一種是使用RPM方式安裝軟體的系統，包括Red Hat, Fedora, SuSE等都是這類； 一種則是使用Debian的dpkg方式安裝軟體的系統，包括Debian, Ubuntu, B2D等等。
crazylion lee

dspinellis/unix-history-repo: Continuous Unix commit history from 1970 until today - 1 views

github.com/...unix-history-repo

unix history commit

shared by crazylion lee on 30 Nov 16 - No Cached
  • crazylion lee on 30 Nov 16
     
    "Continuous Unix commit history from 1970 until today"
crazylion lee

Supervisor: A Process Control System - supervisor 3.1a1-dev documentation - 1 views

supervisord.org

monitoring sysadmin python linux unix supervisor

shared by crazylion lee on 13 Feb 14 - Cached
  • crazylion lee on 13 Feb 14
     
    Supervisor is a client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems. It shares some of the same goals of programs like launchd, daemontools, and runit. Unlike some of these programs, it is not meant to be run as a substitute for init as "process id 1". Instead it is meant to be used to control processes related to a project or a customer, and is meant to start like any other program at boot time.
張 旭

第 06 章 - 計算機概論 - 作業系統概論 - 0 views

dic.vbird.tw/...2020unit06.php

system

shared by 張 旭 on 22 Jun 21 - No Cached
  • 自行參考電腦硬體來設計出運算的軟體，當時的系統並沒有『作業系統』的概念，因為應用程式與作業系統是同時設計的。
  • 電腦裡面有儲存設備 (不論是硬碟還是記憶體)， 所以電腦硬體裡面會執行一隻監督程式 (monitor)，使用者可以預先將自己的程式讀進系統，系統先儲存該程式到佇列 (queue)，等到輪到該程式運作後， 就將該程式讀入讓 CPU 開始運作，直到運作結束輸出到印表機之後，將該工作丟棄，然後開始讀入在 queue 裡面的新的程式，依序執行。
  • 將 CPU 與 I/O 分離開
  • ...38 more annotations...
  • 透過卡片與讀卡機，將程式碼一次性的讀進大機器，然後就是等待大機器的運作， 結果再交由印表機印出。如果打卡紙打洞錯誤呢？只好重新打洞，重新排隊去運作程式了。
  • 允許兩個以上的程序在記憶體中等待被 CPU 執行，當 CPU 執行完其中一隻程式後， 第二隻程式就可以立刻被執行，因此效能會比較好。
  • 程序的狀態進入中斷狀態，CPU 不會理會該程序
  • CPU 的排程 (cpu scheduling)
  • 早期單核 CPU 的運作中，CPU 一次只能運作一個工作，因此，若有多個工作要同時進行， 那麼 CPU 就得要安排一個 CPU 運作時間給所有的工作，當該程序達到最大工作時間後，CPU 就會將該工作排回佇列，讓下一隻程序接著運作。
  • 你會覺得 CPU 是同時運作所有的程序，其實不是的！而是 CPU 在各個程序之間切換工作而已。
  • 分時系統其實與多元程式處理系統有點類似， 只是工作的輸入改為透過終端機操作輸入，CPU 可以在各個用戶操作間切換工作，於是每個使用者感覺似乎都是在同步操作電腦系統一般， 這就是分時系統。
  • 早期的程式設計師要設計程式是件苦差事，因為得要了解電腦硬體，並根據該電腦硬體來選擇程式語言，然後根據程式語言來設計運算工作、記憶體讀寫工作、 磁碟與影像輸入輸出工作、檔案存取工作等。等於從硬體、軟體、輸入輸出行為都得要在自己的程式碼裡面一口氣完成才行。
  • 在 1971 年開始的 unix 系統開發後，後續的系統大多使用 unix 的概念
  • 將硬體管理的工作統一交給一組程式碼去進行，而且這組程式碼還提供了一個開發界面
  • 軟體工程師只要依據這組程式碼規範的開發界面後，該軟體開發完成就能夠在這組程式碼上面運作了
  • 程式的執行
  • 作業系統需要將使用者交付的軟體程序分配到記憶體中， 然後透過 CPU 排程持續的交錯的完成各項任務才行。
  • CPU 中斷 (interrupt) 的功能
  • CPU 根據硬體擁有許多與週邊硬體的中斷通道， 當接收到中斷訊號時，CPU 就會嘗試將該程序列入等待的狀態下，讓該硬體自行完成相關的任務後，然後再接管系統。
  • 記憶體管理模組
  • 舊的環境底下，程式設計師需要自己判斷自己的程式會用到多少記憶體，然後自行指定記憶體使用位址的任務。
  • 系統會自動去偵測與管理主記憶體的使用狀態，避免同一個記憶體位址同時被兩個程序所使用而讓程序工作損毀
  • 作業系統核心也在記憶體中， 因此核心也會被這個子系統放入受保護的記憶體區段，一般用戶是無法直接操作該受保護的記憶區段的。
  • 虛擬記憶體 (virtual memory)
  • 主記憶體當中的資料並不是連續的，主記憶體的資料就像磁碟一樣，重複讀、刪、寫之後， 記憶區段是不會連續的
  • CPU 主要讀出虛擬記憶體，記憶體管理模組就會主動讀出資料
  • 一隻程序的資料是連續的 (左側)，但是實際上對應的是在主記憶體或其他位置上
  • CPU 排程
  • 作業系統好不好的重要指標之一！如何讓 CPU 在多工的情況下以最快速的方式將所有的工作完成，這方面的演算法是目前各主要作業系統持續在進步的部份。
  • 磁碟存取與檔案系統
  • 作業系統則需要驅動磁碟(不論是傳統硬碟還是 SSD)，然後也需要了解該磁碟內的檔案系統格式， 之後透過檔案系統這個子系統來進行資料的處理。
  • 裝置的驅動程式
  • 作業系統必須要能夠接受硬體裝置的驅動，所以硬體製造商可以推出給各個不同作業系統使用的驅動程式 (dirver / modules)， 這樣作業系統直接將該驅動程式載入後，即可開始使用該硬體，而不需要重新編譯作業系統。
  • 網路子系統
  • 使用者界面
  • 現代 CPU 設計的主要思考依據，讓一個 CPU 封裝 (單一一顆 CPU 硬體) 裡面，整合多個 CPU 核心，也就是多核心 CPU 製造的思考方向。
  • 對於單執行緒的程式來說， 多核心的 CPU 不見得會跑得比單核的快！這是因為單執行緒只有一個程序在進行，所以 CPU 時脈越高，代表會越快執行完畢。
  • 軟體會將單一工作拆分成數個小工作，分別交給不同的核心去執行，這樣每個核心只要負責一小段任務， 當然 CPU 時脈不用高，只要數量夠大，效能就會提昇很明顯
  • 由於 CPU 是由作業系統控制的，因此，你要使用到多核心的硬體系統，你的作業系統、應用程式都需要設計程可以支援多核心才行！
  • 所謂的平行處理功能，讓一件工作可以拆分成數個部份，讓這些不同的部份丟給不同的 CPU 去運算， 然後再透過一支監控程式，將各別的計算在一定的時間內收回統整後，再次的細分小工作發派出去，持續這些動作後，直到程式執行完畢為止。
  • 對於 Linux 來說，大部分都可以支援到 4096 個 CPU 核心數。
  • 銀行商用大型主機 Unix 系統
crazylion lee

The memory models that underlie programming languages - 0 views

canonical.org/...memory-models

ai memory

shared by crazylion lee on 30 Jan 17 - No Cached
  • crazylion lee on 30 Jan 17
     
    " There are about six major conceptualizations of memory, which I'm calling "memory models"², that dominate today's programming. Three of them derive from the three most historically important programming languages of the 1950s - COBOL, LISP, and FORTRAN - and the other three derive from the three historically important data storage systems: magnetic tape, Unix-style hierarchical filesystems, and relational databases."
crazylion lee

Nix: The Purely Functional Package Manager - 0 views

nixos.org/nix

nix homebrew package management

shared by crazylion lee on 28 May 16 - No Cached
  • crazylion lee on 28 May 16
     
    "Nix The Purely Functional Package Manager Nix is a powerful package manager for Linux and other Unix systems that makes package management reliable and reproducible. It provides atomic upgrades and rollbacks, side-by-side installation of multiple versions of a package, multi-user package management and easy setup of build environments. Read more…"
crazylion lee

README - md2man - 0 views

sunaku.github.io/...README.html

readme man markdown

shared by crazylion lee on 26 Mar 16 - No Cached
  • crazylion lee on 26 Mar 16
     
    "md2man is a Ruby library and a set of command-line programs that convert Markdown into UNIX manpages as well as HTML webpages using Redcarpet."
張 旭

php - CakePHP: No such file or directory (trying to connect via unix:///var/mysql/mysql... - 0 views

stackoverflow.com/...onnect-via-unix-var-mysql-mysq

cakephp config php MVC framework development mac mysql

shared by 張 旭 on 08 Sep 13 - No Cached
  • 張 旭 on 08 Sep 13
     
    'port' => '/path/to/mysql.sock'
  • 張 旭 on 08 Sep 13
     
    'port' => '/private/tmp/mysql.sock'
crazylion lee

ledger, a powerful command-line accounting system - 0 views

www.ledger-cli.org/index.html

accounting system linux cmd

shared by crazylion lee on 08 Nov 15 - No Cached
  • crazylion lee on 08 Nov 15
     
    "Ledger is a powerful, double-entry accounting system that is accessed from the UNIX command-line. Ledger, begun in 2003, is written by John Wiegley and released under the BSD license. It has also inspired several ports to other languages."
crazylion lee

Redox - Your Next(Gen) OS - 1 views

www.redox-os.org

rust os

shared by crazylion lee on 20 Mar 16 - No Cached
  • crazylion lee on 20 Mar 16
     
    "Redox is a Unix-like Operating System written in Rust, aiming to bring the innovations of Rust to a modern microkernel and full set of applications."
張 旭

Makefiles - Best practices and suggestions | MDN - 0 views

developer.mozilla.org/...Best_practices_and_suggestions

linux makefile programming development

shared by 張 旭 on 24 Dec 18 - No Cached
  • hardcoded values - avoid them like the plague
  • For classes of hardware (unix/windows) place your makefile in a subdirectory, unix/Makefile.in
  • Initial make call should always be the workhorse: build, generate, deploy, install, etc.
  • ...6 more annotations...
  • All subsequent make calls must become a NOP unless sources or dependencies change or have been removed.
    • 張 旭
      張 旭 on 24 Dec 18
       
      No Operation 或 No Operation Performed 的縮寫，意為無操作
    • 張 旭
      張 旭 on 24 Dec 18
  • Do not use directories as a dependency for generated targets, ever.
  • Parallel make: add an explicit timestamp dependency (.done) that make can synchronize threaded calls on to avoid a race condition.
  • Maintain clean targets - makefiles should be able to remove all content that is generated so "make clean" will return the sandbox/directory back to a clean state.
  • Wrapper check/unit tests with a ENABLE_TESTS conditional
張 旭

phusion/baseimage-docker - 1 views

github.com/...baseimage-docker

docker ubuntu base system

shared by 張 旭 on 24 Jun 15 - No Cached
    • 張 旭
      張 旭 on 24 Jun 15
       
      原始的 docker 在執行命令時，預設就是將傳入的 COMMAND 當成 PID 1 的程序，執行完畢就結束這個  docker，其他的 daemons 並不會執行，而 baseimage 解決了這個問題。
    • crazylion lee
      crazylion lee on 17 Dec 15
       
      好棒棒
  • docker exec
  • Through SSH
  • ...57 more annotations...
  • docker exec -t -i YOUR-CONTAINER-ID bash -l
  • Login to the container
  • Baseimage-docker only advocates running multiple OS processes inside a single container.
  • Password and challenge-response authentication are disabled by default. Only key authentication is allowed.
  • A tool for running a command as another user
  • The Docker developers advocate the philosophy of running a single logical service per container. A logical service can consist of multiple OS processes.
  • All syslog messages are forwarded to "docker logs".
  • Splitting your logical service into multiple OS processes also makes sense from a security standpoint.
  • Baseimage-docker provides tools to encourage running processes as different users
  • sometimes it makes sense to run multiple services in a single container, and sometimes it doesn't.
  • Baseimage-docker advocates running multiple OS processes inside a single container, and a single logical service can consist of multiple OS processes.
  • using environment variables to pass parameters to containers is very much the "Docker way"
  • add additional daemons (e.g. your own app) to the image by creating runit entries.
  • the shell script must run the daemon without letting it daemonize/fork it.
  • All executable scripts in /etc/my_init.d, if this directory exists. The scripts are run in lexicographic order.
  • variables will also be passed to all child processes
  • Environment variables on Unix are inherited on a per-process basis
  • there is no good central place for defining environment variables for all applications and services
  • centrally defining environment variables
  • One of the ideas behind Docker is that containers should be stateless, easily restartable, and behave like a black box.
  • a one-shot command in a new container
  • immediately exit after the command exits,
  • However the downside of this approach is that the init system is not started. That is, while invoking COMMAND, important daemons such as cron and syslog are not running. Also, orphaned child processes are not properly reaped, because COMMAND is PID 1.
  • Baseimage-docker provides a facility to run a single one-shot command, while solving all of the aforementioned problems
  • Nginx is one such example: it removes all environment variables unless you explicitly instruct it to retain them through the env configuration option.
  • Mechanisms for easily running multiple processes, without violating the Docker philosophy
  • Ubuntu is not designed to be run inside Docker
  • According to the Unix process model, the init process -- PID 1 -- inherits all orphaned child processes and must reap them
  • Syslog-ng seems to be much more stable
  • cron daemon
  • Rotates and compresses logs
  • /sbin/setuser
  • A tool for installing apt packages that automatically cleans up after itself.
  • a single logical service inside a single container
  • A daemon is a program which runs in the background of its system, such as a web server.
  • The shell script must be called run, must be executable, and is to be placed in the directory /etc/service/<NAME>. runsv will switch to the directory and invoke ./run after your container starts.
  • If any script exits with a non-zero exit code, the booting will fail.
  • If your process is started with a shell script, make sure you exec the actual process, otherwise the shell will receive the signal and not your process.
  • any environment variables set with docker run --env or with the ENV command in the Dockerfile, will be picked up by my_init
  • not possible for a child process to change the environment variables of other processes
  • they will not see the environment variables that were originally passed by Docker.
  • We ignore HOME, SHELL, USER and a bunch of other environment variables on purpose, because not ignoring them will break multi-user containers.
  • my_init imports environment variables from the directory /etc/container_environment
  • /etc/container_environment.sh - a dump of the environment variables in Bash format.
  • modify the environment variables in my_init (and therefore the environment variables in all child processes that are spawned after that point in time), by altering the files in /etc/container_environment
  • my_init only activates changes in /etc/container_environment when running startup scripts
  • environment variables don't contain sensitive data, then you can also relax the permissions
  • Syslog messages are forwarded to the console
  • syslog-ng is started separately before the runit supervisor process, and shutdown after runit exits.
  • RUN apt-get update && apt-get upgrade -y -o Dpkg::Options::="--force-confold"
  • /sbin/my_init --skip-startup-files --quiet --
  • By default, no keys are installed, so nobody can login
  • provide a pregenerated, insecure key (PuTTY format)
  • RUN /usr/sbin/enable_insecure_key
  • docker run YOUR_IMAGE /sbin/my_init --enable-insecure-key
  • RUN cat /tmp/your_key.pub >> /root/.ssh/authorized_keys && rm -f /tmp/your_key.pub
  • The default baseimage-docker installs syslog-ng, cron and sshd services during the build process
張 旭

Probably Done Before: Visualizing Docker Containers and Images - 0 views

merrigrove.blogspot.tw/...ker-containers-and-images.html

docker technology system

shared by 張 旭 on 07 Apr 17 - No Cached
  •  In my opinion, understanding how a technology works under the hood is the best way to achieve learning speed and to build confidence that you are using the tool in the correct way.
  • union view
    • 張 旭
      張 旭 on 07 Apr 17
       
      把多層 image layer 串接起來，看上去就像是在讀一個 image 檔案而已。
  • The top-level layer may be read by a union-ing file system (AUFS on my docker implementation) to present a single cohesive view of all the changes as one read-only file system
  • ...36 more annotations...
  • it is nearly the same thing as an image, except that the top layer is read-write
  • A container is defined only as a read-write layer atop an image (of read-only layers itself).  It does not have to be running.
  • a running container
    • 張 旭
      張 旭 on 07 Apr 17
       
      之前一直搞錯了！不是 run 起來的才會叫 container，只要有 read-write layer 就是了！
  • the the isolated process-space and processes within
  • A running container is defined as a read-write "union view" and
  • kernel-level technologies like cgroups, namespaces
  • The processes within this process-space may change, delete or create files within the "union view" file that will be captured in the read-write layer
  • there is no longer a running container
    • 張 旭
      張 旭 on 07 Apr 17
       
      這行指令執行結束之後，running container 就停掉了，但是該 container 還在！
  • each layer contains a pointer to a parent layer using the Id
  • The 'docker create' command adds a read-write layer to the top stack based on the image id.  It does not run this container.
  • The command 'docker start' creates a process space around the union view of the container's layers.
  • can only be one process space per container.
  • the docker run command starts with an image, creates a container, and starts the container
  • 'git pull' (which is a combination of 'git fetch' and 'git merge')
  • 'docker ps' lists out the inventory of running containers on your system
  • 'docker ps -a' where the 'a' is short for 'all' lists out all the containers on your system, whether stopped or running.
  • Only those images that have containers attached to them or that have been pulled are considered top-level.
  • 'docker stop' issues a SIGTERM to a running container which politely stops all the processes in that process-space.
  • results is a normal, but non-running, container
  • 'docker kill' issues a non-polite SIGKILL command to all the processes in a running container.
  • 'docker stop' and 'docker kill' which send actual UNIX signals to a running process
  • 'docker pause' uses a special cgroups feature to freeze/pause a running process-space
  • 'docker rm' removes the read-write layer that defines a container from your host system
  • It effectively deletes files
  • 'docker rmi' removes the read-layer that defines a "union view" of an image.
  • 'docker commit' takes a container's top-level read-write layer and burns it into a read-only layer.
  • turns a container (whether running or stopped) into an immutable image
  • uses the FROM directive in the Dockerfile file as the starting image and iteratively 1) runs (create and start) 2) modifies and 3) commits.
  • At each step in the iteration a new layer is created.
  • 'docker exec' command runs on a running container and executes a process in that running container's process space
  • 'docker inspect' fetches the metadata that has been associated with the top-layer of the container or image
  • 'docker save' creates a single tar file that can be used to import on a different host system
  • only be run on an image
  • 'docker export' command creates a tar file of the contents of the "union view" and flattens it for consumption for non-Docker usages
  • This command removes the metadata and the layers.  This command can only be run on containers.
  • 'docker history' command takes an image-id and recursively prints out the read-only layers
張 旭

数据库水平分片心得 · ScienJus's Blog - 0 views

www.scienjus.com/database-sharding-review

database mysql system

shared by 張 旭 on 29 Oct 18 - No Cached
  • 水平分片（也叫水平分库）指的是将整体存储在单个数据库中的数据，通过某种策略分摊到多个表结构与其相同的数据库中，这样每个数据库中的数据量就会相对减少很多，并且可以部署在不同物理服务器上，理论上能够实现数据库的无限横向拓展。
  • 当遇到第一次数据库性能问题时，最先想到的方案应该是读写分离，将所有写操作都放在主数据库上，所有读操作都放在从数据库上
  • 配置一主多从
  • ...16 more annotations...
  • 主从关系（Master-Slave），此时所有操作还是由主数据库完成，主数据库再同步到从数据库上，而从数据库只需要在主数据库挂掉之后代替其工作。
  • 一般来说读写分离加上缓存已经可以应付绝大多数情况了，并且几乎不需要对业务层面进行修改。
  • 对数据库进行垂直分库，将业务彼此无关的表放在单独的数据库中，分库后不同库中的表无法进行联合查询等操作，但是可以平摊压力，并且独立做读写分离。
  • 对数据库进行水平分表，建立多个结构相同的表分摊数据，使得每个表的数据量减少从而提升速度。
  • 分表却只能在单台机器的单个数据库上，如果是服务器本身的性能达到瓶颈，则分表不会有明显作用。
  • 分表后各个子表还是可以通过 union 等命令联合查询，分库后则不行
  • 基于 id 的区间分片，例如：将 id 为 1-2w 的数据存放在 A 数据库，2w-4w 的数据存放在 B 数据库。
  • 基于 id 的 hash 分片，例如：将 id%2=0 的数据存放在 A 数据库，id%2=1 的数据放在 B 数据库。
  • 基于时间的区间分片，大部分软件都会有一个特征：越新的数据被操作的几率越大，老数据几乎不会被操作。所以通过数据的插入时间进行分库（也称为冷热分离）
  • 基于检索表分片，通过额外建立一张检索表保存 id 与所在数据库节点的对应关系，优点是逻辑简单，自由且不会有迁移问题，缺点是每次查询都需要额外查询检索表，所以一般会选择将检索表缓存到内存中。
  • 基于地理位置分片
  • 分库策略
  • 在数据库表设计时，为了保证 id 唯一，大部分人都会将主键设为自增的 int 类型。但是由于 auto\_increment 是和表所绑定的，所以在分库后每个表的自增 id 也是独立的。这样就肯定会发生主键冲突
  • 但是很多人都希望主键即使不是连续自增，也是一个有序的整数，这样在排序等情况下会有用。这时候就需要自己实现一个 id 生成算法了，一般都会使用 unix 时间戳保持有序，混入 Mac 地址等保证唯一。
  • 在分库情况下，需要将大部分联合查询都替换为至少两次查询，先从关联的表中查询出符合条件的 id，在根据 id 去对应的数据库里查询主体信息。
  • 数据库水平分片作为数据库性能瓶颈的最终解决方案，确实可以有效的解决这个问题。但是它将业务逻辑变得非常复杂（主要是关联表冗余和字段冗余，以及这些数据的更新），并且有分布式事务这个难题。所以不到必要时刻，尽量不要轻易尝试数据库分片。
張 旭

Let's Encrypt & Docker - Træfik - 0 views

docs.traefik.io/...docker-and-lets-encrypt

traefik docker networking proxy

shared by 張 旭 on 19 Sep 18 - No Cached
  • automatically discover any services on the Docker host and let Træfik reconfigure itself automatically when containers get created (or shut down) so HTTP traffic can be routed accordingly.
  • use Træfik as a layer-7 load balancer with SSL termination for a set of micro-services used to run a web application.
  • Docker containers can only communicate with each other over TCP when they share at least one network.
  • ...15 more annotations...
  • Docker under the hood creates IPTable rules so containers can't reach other containers unless you'd want to
  • Træfik can listen to Docker events and reconfigure its own internal configuration when containers are created (or shut down).
  • Enable the Docker provider and listen for container events on the Docker unix socket we've mounted earlier.
  • Enable automatic request and configuration of SSL certificates using Let's Encrypt. These certificates will be stored in the acme.json file, which you can back-up yourself and store off-premises.
  • there isn't a single container that has any published ports to the host -- everything is routed through Docker networks.
  • Thanks to Docker labels, we can tell Træfik how to create its internal routing configuration.
  • container labels and service labels
  • With the traefik.enable label, we tell Træfik to include this container in its internal configuration.
  • tell Træfik to use the web network to route HTTP traffic to this container.
  • Service labels allow managing many routes for the same container.
  • When both container labels and service labels are defined, container labels are just used as default values for missing service labels but no frontend/backend are going to be defined only with these labels.
  • In the example, two service names are defined : basic and admin. They allow creating two frontends and two backends.
  • Always specify the correct port where the container expects HTTP traffic using traefik.port label.
  • all containers that are placed in the same network as Træfik will automatically be reachable from the outside world
  • With the traefik.frontend.auth.basic label, it's possible for Træfik to provide a HTTP basic-auth challenge for the endpoints you provide the label for.
張 旭

Baseimage-docker: A minimal Ubuntu base image modified for Docker-friendliness - 0 views

phusion.github.io/baseimage-docker

docker base system linux

shared by 張 旭 on 29 Oct 14 - No Cached
  • We encourage you to use multiple processes.
  • Baseimage-docker is a special Docker image that is configured for correct use within Docker containers.
  • When your Docker container starts, only the CMD command is run.
  • ...16 more annotations...
  • You're not running them, you're only running your app.
  • You have Ubuntu installed in Docker. The files are there. But that doesn't mean Ubuntu's running as it should.
  • The only processes that will be running inside the container is the CMD command, and all processes that it spawns.
  • A proper Unix system should run all kinds of important system services.
  • Ubuntu is not designed to be run inside Docker
  • When a system is started, the first process in the system is called the init process, with PID 1. The system halts when this processs halts.
  • Runit (written in C) is much lighter weight than supervisord (written in Python).
  • Docker runs fine with multiple processes in a container.
  • Baseimage-docker encourages you to run multiple processes through the use of runit.
  • If your init process is your app, then it'll probably only shut down itself, not all the other processes in the container.
  • a Docker container, which is a locked down environment with e.g. no direct access to many kernel resources.
  • Used for service supervision and management.
  • A custom tool for running a command as another user.
  • add additional daemons (e.g. your own app) to the image by creating runit entries.
  • write a small shell script which runs your daemon, and runit will keep it up and running for you, restarting it when it crashes, etc.
  • the shell script must run the daemon without letting it daemonize/fork it.
張 旭

The Twelve-Factor App - 0 views

12factor.net/concurrency

development application programming

shared by 張 旭 on 20 Feb 19 - No Cached
  • PHP processes run as child processes of Apache, started on demand as needed by request volume.
  • Java processes take the opposite approach, with the JVM providing one massive uberprocess that reserves a large block of system resources (CPU and memory) on startup, with concurrency managed internally via threads
  • Processes in the twelve-factor app take strong cues from the unix process model for running service daemons.
  • ...3 more annotations...
  • application must also be able to span multiple processes running on multiple physical machines.
  • The array of process types and number of processes of each type is known as the process formation.
  • Twelve-factor app processes should never daemonize or write PID files.
crazylion lee

Idle Time » Blog Archive » Colorized man pages: Understood and customized - 0 views

boredzo.org/...ages-understood-and-customized

shell unix linux pages man zsh

shared by crazylion lee on 24 Aug 16 - No Cached
  • crazylion lee on 24 Aug 16
     
    "Colorized man pages: Understood and customized"
張 旭

Rails Environment Variables · RailsApps - 1 views

railsapps.github.io/ils-environment-variables.html

rails environment system variable

shared by 張 旭 on 13 Mar 17 - No Cached
  • You can pass local configuration settings to an application using environment variables.
  • Operating systems (Linux, Mac OS X, Windows) provide mechanisms to set local environment variables, as does Heroku and other deployment platforms.
  • In general, you shouldn’t save email account credentials or private API keys to a shared git repository.
  • ...10 more annotations...
  • You could “hardcode” your Gmail username and password into the file but that would expose it to everyone who has access to your git repository.
  • It’s important to learn to use the Unix shell if you’re commited to improving your skills as a developer.
  • The gem reads a config/application.yml file and sets environment variables before anything else is configured in the Rails application.
  • make sure this file is listed in the .gitignore file so it isn’t checked into the git repository
  • Rails provides a config.before_configuration
  • YAML.load(File.open(env_file)).each do |key, value| ENV[key.to_s] = value end if File.exists?(env_file)
  • Heroku is a popular choice for low cost, easily configured Rails application hosting.
  • heroku config:add
  • the dotenv Ruby gem
  • Foreman is a tool for starting and configuring multiple processes in a complex application
1 - 20 of 26 Next ›
Showing 20▼ items per page