Skip to main content

Home/ Larvata/ Group items tagged processing

Rss Feed Group items tagged

Filter: All | Bookmarks | Topics Simple Middle
張 旭

Queues - Laravel - The PHP Framework For Web Artisans - 0 views

laravel.com/queues

php programming laravel queue worker framework

shared by 張 旭 on 02 Mar 19 - No Cached
  • Laravel queues provide a unified API across a variety of different queue backends, such as Beanstalk, Amazon SQS, Redis, or even a relational database.
  • The queue configuration file is stored in config/queue.php
  • a synchronous driver that will execute jobs immediately (for local use)
  • ...56 more annotations...
  • A null queue driver is also included which discards queued jobs.
  • In your config/queue.php configuration file, there is a connections configuration option.
  • any given queue connection may have multiple "queues" which may be thought of as different stacks or piles of queued jobs.
  • each connection configuration example in the queue configuration file contains a queue attribute.
  • if you dispatch a job without explicitly defining which queue it should be dispatched to, the job will be placed on the queue that is defined in the queue attribute of the connection configuration
  • pushing jobs to multiple queues can be especially useful for applications that wish to prioritize or segment how jobs are processed
  • specify which queues it should process by priority.
  • If your Redis queue connection uses a Redis Cluster, your queue names must contain a key hash tag.
  • ensure all of the Redis keys for a given queue are placed into the same hash slot
  • all of the queueable jobs for your application are stored in the app/Jobs directory.
  • Job classes are very simple, normally containing only a handle method which is called when the job is processed by the queue.
  • we were able to pass an Eloquent model directly into the queued job's constructor. Because of the SerializesModels trait that the job is using, Eloquent models will be gracefully serialized and unserialized when the job is processing.
  • When the job is actually handled, the queue system will automatically re-retrieve the full model instance from the database.
  • The handle method is called when the job is processed by the queue
  • The arguments passed to the dispatch method will be given to the job's constructor
  • delay the execution of a queued job, you may use the delay method when dispatching a job.
  • dispatch a job immediately (synchronously), you may use the dispatchNow method.
  • When using this method, the job will not be queued and will be run immediately within the current process
  • specify a list of queued jobs that should be run in sequence.
  • Deleting jobs using the $this->delete() method will not prevent chained jobs from being processed. The chain will only stop executing if a job in the chain fails.
  • this does not push jobs to different queue "connections" as defined by your queue configuration file, but only to specific queues within a single connection.
  • To specify the queue, use the onQueue method when dispatching the job
  • To specify the connection, use the onConnection method when dispatching the job
  • defining the maximum number of attempts on the job class itself.
  • to defining how many times a job may be attempted before it fails, you may define a time at which the job should timeout.
  • using the funnel method, you may limit jobs of a given type to only be processed by one worker at a time
  • using the throttle method, you may throttle a given type of job to only run 10 times every 60 seconds.
  • If an exception is thrown while the job is being processed, the job will automatically be released back onto the queue so it may be attempted again.
  • dispatch a Closure. This is great for quick, simple tasks that need to be executed outside of the current request cycle
  • When dispatching Closures to the queue, the Closure's code contents is cryptographically signed so it can not be modified in transit.
  • Laravel includes a queue worker that will process new jobs as they are pushed onto the queue.
  • once the queue:work command has started, it will continue to run until it is manually stopped or you close your terminal
  • queue workers are long-lived processes and store the booted application state in memory.
  • they will not notice changes in your code base after they have been started.
  • during your deployment process, be sure to restart your queue workers.
  • customize your queue worker even further by only processing particular queues for a given connection
  • The --once option may be used to instruct the worker to only process a single job from the queue
  • The --stop-when-empty option may be used to instruct the worker to process all jobs and then exit gracefully.
  • Daemon queue workers do not "reboot" the framework before processing each job.
  • you should free any heavy resources after each job completes.
  • Since queue workers are long-lived processes, they will not pick up changes to your code without being restarted.
  • restart the workers during your deployment process.
  • php artisan queue:restart
  • The queue uses the cache to store restart signals
  • the queue workers will die when the queue:restart command is executed, you should be running a process manager such as Supervisor to automatically restart the queue workers.
  • each queue connection defines a retry_after option. This option specifies how many seconds the queue connection should wait before retrying a job that is being processed.
  • The --timeout option specifies how long the Laravel queue master process will wait before killing off a child queue worker that is processing a job.
  • When jobs are available on the queue, the worker will keep processing jobs with no delay in between them.
  • While sleeping, the worker will not process any new jobs - the jobs will be processed after the worker wakes up again
  • the numprocs directive will instruct Supervisor to run 8 queue:work processes and monitor all of them, automatically restarting them if they fail.
  • Laravel includes a convenient way to specify the maximum number of times a job should be attempted.
  • define a failed method directly on your job class, allowing you to perform job specific clean-up when a failure occurs.
  • a great opportunity to notify your team via email or Slack.
  • php artisan queue:retry all
  • php artisan queue:flush
  • When injecting an Eloquent model into a job, it is automatically serialized before being placed on the queue and restored when the job is processed
張 旭

phusion/baseimage-docker - 1 views

github.com/...baseimage-docker

docker ubuntu base system

shared by 張 旭 on 24 Jun 15 - No Cached
    • 張 旭
      張 旭 on 24 Jun 15
       
      原始的 docker 在執行命令時，預設就是將傳入的 COMMAND 當成 PID 1 的程序，執行完畢就結束這個  docker，其他的 daemons 並不會執行，而 baseimage 解決了這個問題。
    • crazylion lee
      crazylion lee on 17 Dec 15
       
      好棒棒
  • docker exec
  • Through SSH
  • ...57 more annotations...
  • docker exec -t -i YOUR-CONTAINER-ID bash -l
  • Login to the container
  • Baseimage-docker only advocates running multiple OS processes inside a single container.
  • Password and challenge-response authentication are disabled by default. Only key authentication is allowed.
  • A tool for running a command as another user
  • The Docker developers advocate the philosophy of running a single logical service per container. A logical service can consist of multiple OS processes.
  • All syslog messages are forwarded to "docker logs".
  • Baseimage-docker advocates running multiple OS processes inside a single container, and a single logical service can consist of multiple OS processes.
  • Baseimage-docker provides tools to encourage running processes as different users
  • sometimes it makes sense to run multiple services in a single container, and sometimes it doesn't.
  • Splitting your logical service into multiple OS processes also makes sense from a security standpoint.
  • using environment variables to pass parameters to containers is very much the "Docker way"
  • Baseimage-docker provides a facility to run a single one-shot command, while solving all of the aforementioned problems
  • the shell script must run the daemon without letting it daemonize/fork it.
  • All executable scripts in /etc/my_init.d, if this directory exists. The scripts are run in lexicographic order.
  • variables will also be passed to all child processes
  • Environment variables on Unix are inherited on a per-process basis
  • there is no good central place for defining environment variables for all applications and services
  • centrally defining environment variables
  • One of the ideas behind Docker is that containers should be stateless, easily restartable, and behave like a black box.
  • a one-shot command in a new container
  • immediately exit after the command exits,
  • However the downside of this approach is that the init system is not started. That is, while invoking COMMAND, important daemons such as cron and syslog are not running. Also, orphaned child processes are not properly reaped, because COMMAND is PID 1.
  • add additional daemons (e.g. your own app) to the image by creating runit entries.
  • Nginx is one such example: it removes all environment variables unless you explicitly instruct it to retain them through the env configuration option.
  • Mechanisms for easily running multiple processes, without violating the Docker philosophy
  • Ubuntu is not designed to be run inside Docker
  • According to the Unix process model, the init process -- PID 1 -- inherits all orphaned child processes and must reap them
  • Syslog-ng seems to be much more stable
  • cron daemon
  • Rotates and compresses logs
  • /sbin/setuser
  • A tool for installing apt packages that automatically cleans up after itself.
  • a single logical service inside a single container
  • A daemon is a program which runs in the background of its system, such as a web server.
  • The shell script must be called run, must be executable, and is to be placed in the directory /etc/service/<NAME>. runsv will switch to the directory and invoke ./run after your container starts.
  • If any script exits with a non-zero exit code, the booting will fail.
  • If your process is started with a shell script, make sure you exec the actual process, otherwise the shell will receive the signal and not your process.
  • any environment variables set with docker run --env or with the ENV command in the Dockerfile, will be picked up by my_init
  • not possible for a child process to change the environment variables of other processes
  • they will not see the environment variables that were originally passed by Docker.
  • We ignore HOME, SHELL, USER and a bunch of other environment variables on purpose, because not ignoring them will break multi-user containers.
  • my_init imports environment variables from the directory /etc/container_environment
  • /etc/container_environment.sh - a dump of the environment variables in Bash format.
  • modify the environment variables in my_init (and therefore the environment variables in all child processes that are spawned after that point in time), by altering the files in /etc/container_environment
  • my_init only activates changes in /etc/container_environment when running startup scripts
  • environment variables don't contain sensitive data, then you can also relax the permissions
  • Syslog messages are forwarded to the console
  • syslog-ng is started separately before the runit supervisor process, and shutdown after runit exits.
  • RUN apt-get update && apt-get upgrade -y -o Dpkg::Options::="--force-confold"
  • /sbin/my_init --skip-startup-files --quiet --
  • By default, no keys are installed, so nobody can login
  • provide a pregenerated, insecure key (PuTTY format)
  • RUN /usr/sbin/enable_insecure_key
  • docker run YOUR_IMAGE /sbin/my_init --enable-insecure-key
  • RUN cat /tmp/your_key.pub >> /root/.ssh/authorized_keys && rm -f /tmp/your_key.pub
  • The default baseimage-docker installs syslog-ng, cron and sshd services during the build process
張 旭

How To Use Bash's Job Control to Manage Foreground and Background Processes | DigitalOcean - 0 views

www.digitalocean.com/...round-and-background-processes

linux system server command

shared by 張 旭 on 06 Feb 18 - No Cached
  • Most processes that you start on a Linux machine will run in the foreground. The command will begin execution, blocking use of the shell for the duration of the process.
  • By default, processes are started in the foreground. Until the program exits or changes state, you will not be able to interact with the shell.
  • stop the process by sending it a signal
  • ...17 more annotations...
  • Linux terminals are usually configured to send the "SIGINT" signal (typically signal number 2) to current foreground process when the CTRL-C key combination is pressed.
  • Another signal that we can send is the "SIGTSTP" signal (typically signal number 20).
  • A background process is associated with the specific terminal that started it, but does not block access to the shell
  • start a background process by appending an ampersand character ("&") to the end of your commands.
  • type commands at the same time.
  • The [1] represents the command's "job spec" or job number. We can reference this with other job and process control commands, like kill, fg, and bg by preceding the job number with a percentage sign. In this case, we'd reference this job as %1.
  • Once the process is stopped, we can use the bg command to start it again in the background
  • By default, the bg command operates on the most recently stopped process.
  • Whether a process is in the background or in the foreground, it is rather tightly tied with the terminal instance that started it
  • When a terminal closes, it typically sends a SIGHUP signal to all of the processes (foreground, background, or stopped) that are tied to the terminal.
  • a terminal multiplexer
  • start it using the nohup command
  • appending output to ‘nohup.out’
  • pgrep -a
  • The disown command, in its default configuration, removes a job from the jobs queue of a terminal.
  • You can pass the -h flag to the disown process instead in order to mark the process to ignore SIGHUP signals, but to otherwise continue on as a regular job
  • The huponexit shell option controls whether bash will send its child processes the SIGHUP signal when it exits.
張 旭

Probably Done Before: Visualizing Docker Containers and Images - 0 views

merrigrove.blogspot.tw/...ker-containers-and-images.html

docker technology system

shared by 張 旭 on 07 Apr 17 - No Cached
  •  In my opinion, understanding how a technology works under the hood is the best way to achieve learning speed and to build confidence that you are using the tool in the correct way.
  • union view
    • 張 旭
      張 旭 on 07 Apr 17
       
      把多層 image layer 串接起來，看上去就像是在讀一個 image 檔案而已。
  • The top-level layer may be read by a union-ing file system (AUFS on my docker implementation) to present a single cohesive view of all the changes as one read-only file system
  • ...36 more annotations...
  • it is nearly the same thing as an image, except that the top layer is read-write
  • A container is defined only as a read-write layer atop an image (of read-only layers itself).  It does not have to be running.
  • a running container
    • 張 旭
      張 旭 on 07 Apr 17
       
      之前一直搞錯了！不是 run 起來的才會叫 container，只要有 read-write layer 就是了！
  • the the isolated process-space and processes within
  • A running container is defined as a read-write "union view" and
  • kernel-level technologies like cgroups, namespaces
  • The processes within this process-space may change, delete or create files within the "union view" file that will be captured in the read-write layer
  • there is no longer a running container
    • 張 旭
      張 旭 on 07 Apr 17
       
      這行指令執行結束之後，running container 就停掉了，但是該 container 還在！
  • each layer contains a pointer to a parent layer using the Id
  • The 'docker create' command adds a read-write layer to the top stack based on the image id.  It does not run this container.
  • The command 'docker start' creates a process space around the union view of the container's layers.
  • can only be one process space per container.
  • the docker run command starts with an image, creates a container, and starts the container
  • 'git pull' (which is a combination of 'git fetch' and 'git merge')
  • 'docker ps' lists out the inventory of running containers on your system
  • 'docker ps -a' where the 'a' is short for 'all' lists out all the containers on your system, whether stopped or running.
  • Only those images that have containers attached to them or that have been pulled are considered top-level.
  • 'docker stop' issues a SIGTERM to a running container which politely stops all the processes in that process-space.
  • results is a normal, but non-running, container
  • 'docker kill' issues a non-polite SIGKILL command to all the processes in a running container.
  • 'docker stop' and 'docker kill' which send actual UNIX signals to a running process
  • 'docker pause' uses a special cgroups feature to freeze/pause a running process-space
  • 'docker rm' removes the read-write layer that defines a container from your host system
  • It effectively deletes files
  • 'docker rmi' removes the read-layer that defines a "union view" of an image.
  • 'docker commit' takes a container's top-level read-write layer and burns it into a read-only layer.
  • turns a container (whether running or stopped) into an immutable image
  • uses the FROM directive in the Dockerfile file as the starting image and iteratively 1) runs (create and start) 2) modifies and 3) commits.
  • At each step in the iteration a new layer is created.
  • 'docker exec' command runs on a running container and executes a process in that running container's process space
  • 'docker inspect' fetches the metadata that has been associated with the top-layer of the container or image
  • 'docker save' creates a single tar file that can be used to import on a different host system
  • only be run on an image
  • 'docker export' command creates a tar file of the contents of the "union view" and flattens it for consumption for non-Docker usages
  • This command removes the metadata and the layers.  This command can only be run on containers.
  • 'docker history' command takes an image-id and recursively prints out the read-only layers
張 旭

The Twelve-Factor App - 0 views

12factor.net/concurrency

development application programming

shared by 張 旭 on 20 Feb 19 - No Cached
  • PHP processes run as child processes of Apache, started on demand as needed by request volume.
  • Java processes take the opposite approach, with the JVM providing one massive uberprocess that reserves a large block of system resources (CPU and memory) on startup, with concurrency managed internally via threads
  • Processes in the twelve-factor app take strong cues from the unix process model for running service daemons.
  • ...3 more annotations...
  • application must also be able to span multiple processes running on multiple physical machines.
  • The array of process types and number of processes of each type is known as the process formation.
  • Twelve-factor app processes should never daemonize or write PID files.
張 旭

Gracefully Shutdown Docker Container - Kakashi's Blog - 1 views

kkc.github.io/...ully-shutdown-docker-container

docker process nodejs

shared by 張 旭 on 17 Nov 20 - No Cached
  • The initial idea is to make application invokes deconstructor of each component as soon as the application receives specific signals such as SIGTERM and SIGINT
  • When you run a docker container, by default it has a PID namespace, which means the docker process is isolated from other processes on your host.
  • The PID namespace has an important task to reap zombie processes.
  • ...11 more annotations...
  • This uses /bin/bash as PID1 and runs your program as the subprocess.
  • When a signal is sent to a shell, the signal actually won’t be forwarded to subprocesses.
  • By using the exec form, we can run our program as PID1
  • if you use exec form to run a shell script to spawn your application, remember to use exec syscall to overwrite /usr/bin/bash otherwise it will act as senario1
  • /bin/bash can handle repeating zombie process
  • with Tini, SIGTERM properly terminates your process even if you didn’t explicitly install a signal handler for it.
  • run tini as PID1 and it will forward the signal for subprocesses.
  • tini is a signal proxy and it also can deal with zombie process issue automatically.
  • run your program with tini by passing --init flag to docker run
  • use docker stop, docker will wait for 10s for stopping container before killing a process (by default). The main process inside the container will receive SIGTERM, then docker daemon will wait for 10s and send SIGKILL to terminate process.
  • kill running containers immediately. it’s more like kill -9 and kill --SIGKILL
張 旭

Share Process Namespace between Containers in a Pod | Kubernetes - 0 views

kubernetes.io/...share-process-namespace

kubernetes pod namespace

shared by 張 旭 on 19 Aug 22 - No Cached
  • When process namespace sharing is enabled, processes in a container are visible to all other containers in the same pod.
  • It's even possible to access the file system of another container using the /proc/$pid/root link.
  • Pods share many resources so it makes sense they would also share a process namespace.
  • ...2 more annotations...
  • Processes are visible to other containers in the pod. This includes all information visible in /proc, such as passwords that were passed as arguments or environment variables. These are protected only by regular Unix permissions.
  • Container filesystems are visible to other containers in the pod through the /proc/$pid/root link. This makes debugging easier, but it also means that filesystem secrets are protected only by filesystem permissions.
  • 張 旭 on 19 Aug 22
     
    "When process namespace sharing is enabled, processes in a container are visible to all other containers in the same pod. "
張 旭

Baseimage-docker: A minimal Ubuntu base image modified for Docker-friendliness - 0 views

phusion.github.io/baseimage-docker

docker base system linux

shared by 張 旭 on 29 Oct 14 - No Cached
  • We encourage you to use multiple processes.
  • Baseimage-docker is a special Docker image that is configured for correct use within Docker containers.
  • When your Docker container starts, only the CMD command is run.
  • ...16 more annotations...
  • You're not running them, you're only running your app.
  • You have Ubuntu installed in Docker. The files are there. But that doesn't mean Ubuntu's running as it should.
  • The only processes that will be running inside the container is the CMD command, and all processes that it spawns.
  • A proper Unix system should run all kinds of important system services.
  • Ubuntu is not designed to be run inside Docker
  • When a system is started, the first process in the system is called the init process, with PID 1. The system halts when this processs halts.
  • Runit (written in C) is much lighter weight than supervisord (written in Python).
  • Docker runs fine with multiple processes in a container.
  • Baseimage-docker encourages you to run multiple processes through the use of runit.
  • If your init process is your app, then it'll probably only shut down itself, not all the other processes in the container.
  • a Docker container, which is a locked down environment with e.g. no direct access to many kernel resources.
  • Used for service supervision and management.
  • A custom tool for running a command as another user.
  • add additional daemons (e.g. your own app) to the image by creating runit entries.
  • write a small shell script which runs your daemon, and runit will keep it up and running for you, restarting it when it crashes, etc.
  • the shell script must run the daemon without letting it daemonize/fork it.
張 旭

The Twelve-Factor App - 0 views

12factor.net/processes

development application programming

shared by 張 旭 on 20 Feb 19 - No Cached
  • stateless processes
  • a production deploy of a sophisticated app may use many process types, instantiated into zero or more running processes.
  • Twelve-factor processes are stateless and share-nothing.
  • ...6 more annotations...
  • Any data that needs to persist must be stored in a stateful backing service, typically a database.
  • The memory space or filesystem of the process can be used as a brief, single-transaction cache.
  • wipe out all local (e.g., memory and filesystem) state
  • compiling during the build stage
  • “sticky sessions” – that is, caching user session data in memory of the app’s process and expecting future requests from the same visitor to be routed to the same process.
  • Sticky sessions are a violation of twelve-factor and should never be used or relied upon
crazylion lee

Supervisor: A Process Control System - supervisor 3.1a1-dev documentation - 1 views

supervisord.org

monitoring sysadmin python linux unix supervisor

shared by crazylion lee on 13 Feb 14 - Cached
  • crazylion lee on 13 Feb 14
     
    Supervisor is a client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems. It shares some of the same goals of programs like launchd, daemontools, and runit. Unlike some of these programs, it is not meant to be run as a substitute for init as "process id 1". Instead it is meant to be used to control processes related to a project or a customer, and is meant to start like any other program at boot time.
張 旭

The Rubyist's Guide to Environment Variables - Honeybadger.io Blog - 0 views

blog.honeybadger.io/by-guide-environment-variables

ruby environment development shell

shared by 張 旭 on 12 Jun 15 - No Cached
  • If you want to be able to effectively manage web apps in development and in production, you have to understand environment variables.
  • Every program you run on your server has at least one process. That process gets its own set of environment variables.
  • Once it has them, nothing outside of that process can change them.
  • ...9 more annotations...
  • Every program you run on your server gets its own set of environment variables at the moment you launch it.
  • Environment variables die with their process
  • Every process has a parent. That’s because every program has to be started by some other program.
  • Child processes inherit env vars from their parent
  • By default a child will get copies of every environment variable that its parent has. But the parent has control over this.
  • Changes to the environment don’t sync between running processes
  • shells do provide their own “local” shell variable systems
  • Environment variables are NOT the same as shell variables
  • use the export command to convert the local variable into an environment variable.
張 旭

Running rootless Podman as a non-root user | Enable Sysadmin - 0 views

www.redhat.com/...rootless-podman-makes-sense

docker podman linux system

shared by 張 旭 on 04 Aug 20 - No Cached
  • By default, rootless Podman runs as root within the container.
  • the processes in the container have the default list of namespaced capabilities which allow the processes to act like root inside of the user namespace
  • the directory is owned by UID 26, but UID 26 is not mapped into the container and is not the same UID that Postgres runs with while in the container.
  • ...8 more annotations...
  • Podman launches a container inside of the user namespace, which is mapped with the range of UIDs defined for the user in /etc/subuid and /etc/subgid
  • The easy solution to this problem is to chown the html directory to match the UID that Postgresql runs with inside of the container.
  • use the podman unshare command, which drops you into the same user namespace that rootless Podman uses
  • This setup also means that the processes inside of the container are running as the user’s UID. If the container process escaped the container, the process would have full access to files in your home directory based on UID separation.
  • SELinux would still block the access, but I have heard that some people disable SELinux.
  • If you run the processes within the container as a different non-root UID, however, then those processes will run as that UID. If they escape the container, they would only have world access to content in your home directory.
  • run a podman unshare command, or set up the directories' group ownership as owned by your UID (root inside of the container).
  • running containers as non-root should always be your top priority for security reasons.
crazylion lee

BFH ImagePlay - Rapid Prototyping for Image Processing - 0 views

imageplay.io

processing image

shared by crazylion lee on 06 Aug 15 - No Cached
  • crazylion lee on 06 Aug 15
     
    "ImagePlay is a rapid prototyping tool for building and testing image processing algorithms. It comes with a variety of over 70 individual image processors which can be combined into complex process chains. ImagePlay is completely open source and can be built for Windows, Mac and Linux."
張 旭

The Twelve-Factor App - 0 views

12factor.net/admin-processes

development application programming

shared by 張 旭 on 20 Feb 19 - No Cached
  • The process formation is the array of processes that are used to do the app’s regular business
  • one-off administrative or maintenance tasks for the app
  • One-off admin processes should be run in an identical environment as the regular long-running processes of the app.
  • ...2 more annotations...
  • Admin code must ship with application code to avoid synchronization issues.
  • Twelve-factor strongly favors languages which provide a REPL shell out of the box, and which make it easy to run one-off scripts.
crazylion lee

anthonynsimon/bild: A collection of parallel image processing algorithms in pure Go - 0 views

github.com/bild

image-processing go processing image

shared by crazylion lee on 04 Sep 16 - No Cached
  • crazylion lee on 04 Sep 16
     
    "A collection of parallel image processing algorithms in pure Go"
張 旭

Pods - Kubernetes - 0 views

kubernetes.io/...pod

kubernetes pod system

shared by 張 旭 on 05 Jul 19 - No Cached
  • Pods are the smallest deployable units of computing
  • A Pod (as in a pod of whales or pea pod) is a group of one or more containersA lightweight and portable executable image that contains software and all of its dependencies. (such as Docker containers), with shared storage/network, and a specification for how to run the containers.
  • A Pod’s contents are always co-located and co-scheduled, and run in a shared context.
  • ...32 more annotations...
  • A Pod models an application-specific “logical host”
  • application containers which are relatively tightly coupled
  • being executed on the same physical or virtual machine would mean being executed on the same logical host.
  • The shared context of a Pod is a set of Linux namespaces, cgroups, and potentially other facets of isolation
  • Containers within a Pod share an IP address and port space, and can find each other via localhost
  • Containers in different Pods have distinct IP addresses and can not communicate by IPC without special configuration. These containers usually communicate with each other via Pod IP addresses.
  • Applications within a Pod also have access to shared volumesA directory containing data, accessible to the containers in a pod. , which are defined as part of a Pod and are made available to be mounted into each application’s filesystem.
  • a Pod is modelled as a group of Docker containers with shared namespaces and shared filesystem volumes
    • 張 旭
      張 旭 on 05 Jul 19
       
      類似 docker-compose 裡面宣告的同一坨？
  • Pods are considered to be relatively ephemeral (rather than durable) entities.
  • Pods are created, assigned a unique ID (UID), and scheduled to nodes where they remain until termination (according to restart policy) or deletion.
  • it can be replaced by an identical Pod
  • When something is said to have the same lifetime as a Pod, such as a volume, that means that it exists as long as that Pod (with that UID) exists.
  • uses a persistent volume for shared storage between the containers
  • Pods serve as unit of deployment, horizontal scaling, and replication
  • The applications in a Pod all use the same network namespace (same IP and port space), and can thus “find” each other and communicate using localhost
  • flat shared networking space
  • Containers within the Pod see the system hostname as being the same as the configured name for the Pod.
  • Volumes enable data to survive container restarts and to be shared among the applications within the Pod.
  • Individual Pods are not intended to run multiple instances of the same application
  • The individual containers may be versioned, rebuilt and redeployed independently.
  • Pods aren’t intended to be treated as durable entities.
  • Controllers like StatefulSet can also provide support to stateful Pods.
  • When a user requests deletion of a Pod, the system records the intended grace period before the Pod is allowed to be forcefully killed, and a TERM signal is sent to the main process in each container.
  • Once the grace period has expired, the KILL signal is sent to those processes, and the Pod is then deleted from the API server.
  • grace period
  • Pod is removed from endpoints list for service, and are no longer considered part of the set of running Pods for replication controllers.
  • When the grace period expires, any processes still running in the Pod are killed with SIGKILL.
  • By default, all deletes are graceful within 30 seconds.
  • You must specify an additional flag --force along with --grace-period=0 in order to perform force deletions.
  • Force deletion of a Pod is defined as deletion of a Pod from the cluster state and etcd immediately.
  • StatefulSet Pods
  • Processes within the container get almost the same privileges that are available to processes outside a container.
crazylion lee

GitHub - codeforequity-at/botium-speech-processing: Botium Speech Processing - 0 views

github.com/...botium-speech-processing

speech

shared by crazylion lee on 18 Feb 20 - No Cached
  • crazylion lee on 18 Feb 20
     
    "Botium Speech Processing"
張 旭

Kubernetes Deployments: The Ultimate Guide - Semaphore - 1 views

semaphoreci.com/...kubernetes-deployment

docker kubernetes devops system

shared by 張 旭 on 21 Oct 19 - No Cached
  • Continuous integration gives you confidence in your code. To extend that confidence to the release process, your deployment operations need to come with a safety belt.
  • these Kubernetes objects ensure that you can progressively deploy, roll back and scale your applications without downtime.
  • A pod is just a group of containers (it can be a group of one container) that run on the same machine, and share a few things together.
  • ...34 more annotations...
  • the containers within a pod can communicate with each other over localhost
  • From a network perspective, all the processes in these containers are local.
  • we can never create a standalone container: the closest we can do is create a pod, with a single container in it.
  • Kubernetes is a declarative system (by opposition to imperative systems).
  • All we can do, is describe what we want to have, and wait for Kubernetes to take action to reconcile what we have, with what we want to have.
  • In other words, we can say, “I would like a 40-feet long blue container with yellow doors“, and Kubernetes will find such a container for us. If it doesn’t exist, it will build it; if there is already one but it’s green with red doors, it will paint it for us; if there is already a container of the right size and color, Kubernetes will do nothing, since what we have already matches what we want.
  • The specification of a replica set looks very much like the specification of a pod, except that it carries a number, indicating how many replicas
  • What happens if we change that definition? Suddenly, there are zero pods matching the new specification.
  • the creation of new pods could happen in a more gradual manner.
  • the specification for a deployment looks very much like the one for a replica set: it features a pod specification, and a number of replicas.
  • Deployments, however, don’t create or delete pods directly.
  • When we update a deployment and adjust the number of replicas, it passes that update down to the replica set.
  • When we update the pod specification, the deployment creates a new replica set with the updated pod specification. That replica set has an initial size of zero. Then, the size of that replica set is progressively increased, while decreasing the size of the other replica set.
  • we are going to fade in (turn up the volume) on the new replica set, while we fade out (turn down the volume) on the old one.
  • During the whole process, requests are sent to pods of both the old and new replica sets, without any downtime for our users.
  • A readiness probe is a test that we add to a container specification.
  • Kubernetes supports three ways of implementing readiness probes:Running a command inside a container;Making an HTTP(S) request against a container; orOpening a TCP socket against a container.
  • When we roll out a new version, Kubernetes will wait for the new pod to mark itself as “ready” before moving on to the next one.
  • If there is no readiness probe, then the container is considered as ready, as long as it could be started.
  • MaxSurge indicates how many extra pods we are willing to run during a rolling update, while MaxUnavailable indicates how many pods we can lose during the rolling update.
  • Setting MaxUnavailable to 0 means, “do not shutdown any old pod before a new one is up and ready to serve traffic“.
  • Setting MaxSurge to 100% means, “immediately start all the new pods“, implying that we have enough spare capacity on our cluster, and that we want to go as fast as possible.
  • kubectl rollout undo deployment web
  • the replica set doesn’t look at the pods’ specifications, but only at their labels.
  • A replica set contains a selector, which is a logical expression that “selects” (just like a SELECT query in SQL) a number of pods.
  • it is absolutely possible to manually create pods with these labels, but running a different image (or with different settings), and fool our replica set.
  • Selectors are also used by services, which act as the load balancers for Kubernetes traffic, internal and external.
  • internal IP address (denoted by the name ClusterIP)
  • during a rollout, the deployment doesn’t reconfigure or inform the load balancer that pods are started and stopped. It happens automatically through the selector of the service associated to the load balancer.
  • a pod is added as a valid endpoint for a service only if all its containers pass their readiness check. In other words, a pod starts receiving traffic only once it’s actually ready for it.
  • In blue/green deployment, we want to instantly switch over all the traffic from the old version to the new, instead of doing it progressively
  • We can achieve blue/green deployment by creating multiple deployments (in the Kubernetes sense), and then switching from one to another by changing the selector of our service
  • kubectl label pods -l app=blue,version=v1.5 status=enabled
  • kubectl label pods -l app=blue,version=v1.4 status-
  • 張 旭 on 21 Oct 19
     
    "Continuous integration gives you confidence in your code. To extend that confidence to the release process, your deployment operations need to come with a safety belt."
crazylion lee

Introduction to the Personal Software Process(sm) 1, Watts S. Humphrey, eBook - Amazon.com - 0 views

www.amazon.com/...ref=mt_kindle

engineering software books personal

shared by crazylion lee on 30 Jan 17 - No Cached
  • crazylion lee on 30 Jan 17
     
    "Introduction to the Personal Software Process(sm) 1st Edition, Kindle Edition"
crazylion lee

How to record your terminal session on Linux - 0 views

www.howtoforge.com/...your-terminal-session-on-linux

session linux record terminal cwb

shared by crazylion lee on 15 Jun 16 - No Cached
  • crazylion lee on 15 Jun 16
     
    "Recording a terminal session may be important in helping someone learn a process, sharing information in an understandable way, and also presenting a series of commands in a proper manner. Whatever the purpose, there are many times when copy-pasting text from the terminal won't be very helpful while capturing a video of the process is quite far-fetched and may not be always possible. In this quick guide, we will take a look at the easiest way to record and share a terminal session in .gif format."
1 - 20 of 105 Next › Last »
Showing 20▼ items per page