Skip to main content

Home/ Larvata/ Group items tagged programming

Rss Feed Group items tagged

Filter: All | Bookmarks | Topics Simple Middle
張 旭

The Twelve-Factor App - 1 views

12factor.net/build-release-run

development application programming

shared by 張 旭 on 20 Feb 19 - No Cached
  • separate build and run
  • The build stage is a transform which converts a code repo into an executable bundle known as a build.
  • the build stage fetches vendors dependencies and compiles binaries and assets.
  • ...7 more annotations...
  • The release stage takes the build produced by the build stage and combines it with the deploy’s current config.
  • is ready for immediate execution in the execution environment.
  • The run stage (also known as “runtime”) runs the app in the execution environment
  • strict separation between the build, release, and run stages.
  • the Capistrano deployment tool stores releases in a subdirectory named releases, where the current release is a symlink to the current release directory.
  • Every release should always have a unique release ID
  • Releases are an append-only ledger and a release cannot be mutated once it is created.
張 旭

The Twelve-Factor App - 0 views

12factor.net/processes

development application programming

shared by 張 旭 on 20 Feb 19 - No Cached
  • stateless processes
  • a production deploy of a sophisticated app may use many process types, instantiated into zero or more running processes.
  • Twelve-factor processes are stateless and share-nothing.
  • ...6 more annotations...
  • Any data that needs to persist must be stored in a stateful backing service, typically a database.
  • The memory space or filesystem of the process can be used as a brief, single-transaction cache.
  • wipe out all local (e.g., memory and filesystem) state
  • compiling during the build stage
  • “sticky sessions” – that is, caching user session data in memory of the app’s process and expecting future requests from the same visitor to be routed to the same process.
  • Sticky sessions are a violation of twelve-factor and should never be used or relied upon
張 旭

The Twelve-Factor App - 0 views

12factor.net/concurrency

development application programming

shared by 張 旭 on 20 Feb 19 - No Cached
  • PHP processes run as child processes of Apache, started on demand as needed by request volume.
  • Java processes take the opposite approach, with the JVM providing one massive uberprocess that reserves a large block of system resources (CPU and memory) on startup, with concurrency managed internally via threads
  • Processes in the twelve-factor app take strong cues from the unix process model for running service daemons.
  • ...3 more annotations...
  • application must also be able to span multiple processes running on multiple physical machines.
  • The array of process types and number of processes of each type is known as the process formation.
  • Twelve-factor app processes should never daemonize or write PID files.
張 旭

The Twelve-Factor App - 0 views

12factor.net/dev-prod-parity

development application programming

shared by 張 旭 on 20 Feb 19 - No Cached
  • Keep development, staging, and production as similar as possible
  • Developers write code, ops engineers deploy it.
  • The twelve-factor app is designed for continuous deployment by keeping the gap between development and production small
  • ...4 more annotations...
  • Backing services, such as the app’s database, queueing system, or cache, is one area where dev/prod parity is important
  • The twelve-factor developer resists the urge to use different backing services between development and production, even when adapters theoretically abstract away any differences in backing services.
  • declarative provisioning tools such as Chef and Puppet combined with light-weight virtual environments such as Docker and Vagrant allow developers to run local environments which closely approximate production environments.
  • all deploys of the app (developer environments, staging, production) should be using the same type and version of each of the backing services.
  • 張 旭 on 20 Feb 19
     
    "as similar as possible "
張 旭

The Twelve-Factor App - 0 views

12factor.net/admin-processes

development application programming

shared by 張 旭 on 20 Feb 19 - No Cached
  • The process formation is the array of processes that are used to do the app’s regular business
  • one-off administrative or maintenance tasks for the app
  • One-off admin processes should be run in an identical environment as the regular long-running processes of the app.
  • ...2 more annotations...
  • Admin code must ship with application code to avoid synchronization issues.
  • Twelve-factor strongly favors languages which provide a REPL shell out of the box, and which make it easy to run one-off scripts.
張 旭

Scalable architecture without magic (and how to build it if you're not Google) - DEV Co... - 0 views

dev.to/...ld-it-if-youre-not-google-336a

development architecture system google

shared by 張 旭 on 24 Feb 19 - No Cached
  • Don’t mess up write-first and read-first databases.
  • keep them stateless.
  • you should know how to make a scalable setup on bare metal.
  • ...29 more annotations...
  • Different programming languages are for different tasks.
  • Go or C which are compiled to run on bare metal.
  • To run NodeJS on multiple cores, you have to use something like PM2, but since this you have to keep your code stateless.
  • Python have very rich and sugary syntax that’s great for working with data while keeping your code small and expressive.
  • SQL is almost always slower than NoSQL
  • databases are often read-first or write-first
  • write-first, just like Cassandra.
  • store all of your data to your databases and leave nothing at backend
  • Functional code is stateless by default
  • It’s better to go for stateless right from the very beginning.
  • deliver exactly the same responses for same requests.
  • Sessions? Store them at Redis and allow all of your servers to access it.
  • Only the first user will trigger a data query, and all others will be receiving exactly the same data straight from the RAM
  • never, never cache user input
  • Only the server output should be cached
  • Varnish is a great cache option that works with HTTP responses, so it may work with any backend.
  • a rate limiter – if there’s not enough time have passed since last request, the ongoing request will be denied.
  • different requests blasting every 10ms can bring your server down
  • Just set up entry relations and allow your database to calculate external keys for you
  • the query planner will always be faster than your backend.
  • Backend should have different responsibilities: hashing, building web pages from data and templates, managing sessions and so on.
  • For anything related to data management or data models, move it to your database as procedures or queries.
  • a distributed database.
  • your code has to be stateless
  • Move anything related to the data to the database.
  • For load-balancing a database, go for cluster.
  • DB is balancing requests, as well as your backend.
  • Users from different continents are separated with DNS.
  • Keep is scalable, keep is stateless.
  • 張 旭 on 24 Feb 19
     
    "Don't mess up write-first and read-first databases."
張 旭

你到底知不知道什麼是 Kubernetes? | Hwchiu Learning Note - 0 views

www.hwchiu.com/kubernetes-concept.html

kubernetes storage networking programming system

shared by 張 旭 on 03 Feb 19 - No Cached
  • Storage(儲存) 實際上一直都不是一個簡單處理的問題，從軟體面來看實際上牽扯到非常多的層級，譬如 Linux Kernel, FileSystem, Block/File-Level, Cache, Snapshot, Object Storage 等各式各樣的議題可以討論。
  • DRBD
  • 異地備援，容錯機制，快照，重複資料刪除等超多相關的議題基本上從來沒有一個完美的解法能夠滿足所有使用情境。
  • ...20 more annotations...
  • 管理者可能會直接在 NFS Server 上進行 MDADM 來設定相關的 Block Device 並且基於上面提供 Export 供 NFS 使用，甚至底層套用不同的檔案系統 (EXT4/BTF4) 來獲取不同的功能與效能。
  • Kubernetes 就只是 NFS Client 的角色
  • CSI(Container Storage Interface)。CSI 本身作為 Kubernetes 與 Storage Solution 的中介層。
  • 基本上 Pod 裡面每個 Container 會使用 Volume 這個物件來代表容器內的掛載點，而在外部實際上會透過 PVC 以及 PV 的方式來描述這個 Volume 背後的儲存方案伺服器的資訊。
  • 整體會透過 CSI 的元件們與最外面實際上的儲存設備連接，所有儲存相關的功能是否有實現，有支援全部都要仰賴最後面的實際提供者， kubernetes 只透過 CSI 的標準去執行。
  • 在網路部分也有與之對應的 CNI(Container Network Interface). kubernetes 透過 CNI 這個介面來與後方的 網路解決方案 溝通
  • CNI 最基本的要求就是在在對應的階段為對應的容器提供網路能力
  • 目前最常見也是 IPv4 + TCP/UDP 的傳輸方式，因此才會看到大部分的 CNI 都在講這些。
  • 希望所有容器彼此之間可以透過 IPv4 來互相存取彼此，不論是同節點或是跨節點的容器們都要可以滿足這個需求。
  • 容器間到底怎麼傳輸的，需不需要封裝，透過什麼網卡，要不要透過 NAT 處理? 這一切都是 CNI 介面背後的實現
  • 外部網路存取容器服務 (Service/Ingress)
  • kubernetes 在 Service/Ingress 中間自行實現了一個模組，大抵上稱為 kube-proxy, 其底層可以使用 iptables, IPVS, user-space software 等不同的實現方法，這部分是跟 CNI 完全無關。
  • CNI 跟 Service/Ingress 是會衝突的，也有可能彼此沒有配合，這中間沒有絕對的穩定整合。
  • CNI 一般會處理的部份，包含了容器內的 網卡數量,網卡名稱,網卡IP, 以及容器與外部節點的連接能力等
  • CRI (Container Runtime Interface) 或是 Device Plugin
  • 對於 kubernetes 來說，其實本身並不在意到底底下的容器化技術實際上是怎麼實現的，你要用 Docker, rkt, CRI-O 都無所謂，甚至背後是一個偽裝成 Container 的 Virtaul Machine virtlet 都可以。
  • 去思考到底為什麼自己本身的服務需要容器化，容器化可以帶來什麼優點
  • 太多太多的人都認為只要寫一個 Dockerfile 將原先的應用程式們全部包裝起來放在一起就是一個很好的容器 來使用了。
  • 最後就會發現根本把 Container 當作 Virtual Machine 來使用，然後再補一句 Contaienr 根本不好用啊
  • 容器化 不是把直接 Virtual Machine 的使用習慣換個環境使用就叫做 容器化，而是要從概念上去暸解與使用
張 旭

Introduction To The Queue System - Diving Laravel - 0 views

divinglaravel.com/troduction-to-the-queue-system

php programming laravel

shared by 張 旭 on 02 Mar 19 - No Cached
  • Laravel is shipped with a built-in queue system that helps you run tasks in the background
  • The QueueManager is registered into the container and it knows how to connect to the different built-in queue drivers
  • for example when we called the Queue::push() method, what happened is that the manager selected the desired queue driver, connected to it, and called the push method on that driver.
  • ...2 more annotations...
  • All calls to methods that don't exist in the QueueManager class will be sent to the loaded driver
  • when you do Queue::push() you're actually calling the push method on the queue driver you're using
  • 張 旭 on 02 Mar 19
     
    "Laravel is shipped with a built-in queue system that helps you run tasks in the background "
張 旭

Queue Workers: How they work - Diving Laravel - 0 views

divinglaravel.com/queue-workers-how-they-work

php programming laravel queue worker

shared by 張 旭 on 02 Mar 19 - No Cached
  • define workers as a simple PHP process that runs in the background with the purpose of extracting jobs from a storage space and run them with respect to several configuration options.
  • have to manually restart the worker to reflect any code change you made in your application.
  • avoiding booting up the whole app on every job
  • ...7 more annotations...
  • instruct Laravel to create an instance of your application and start executing jobs, this instance will stay alive indefinitely which means the action of starting your Laravel application happens only once when the command was run & the same instance will be used to execute your jobs
  • This will start an instance of the application, process a single job,
  • and then kill the script.
  • Using queue:listen ensures that a new instance of the app is created for every job, that means you don't have to manually restart the worker in case you made changes to your code, but also means more server resources will be consumed.
  • the queue:listen command runs the WorkCommand inside a loop
  • The connection this worker will be pulling jobs from
  • The queue the worker will use to find jobs
  • 張 旭 on 02 Mar 19
     
    "define workers as a simple PHP process that runs in the background with the purpose of extracting jobs from a storage space and run them with respect to several configuration options."
張 旭

Queues - Laravel - The PHP Framework For Web Artisans - 0 views

laravel.com/queues

php programming laravel queue worker framework

shared by 張 旭 on 02 Mar 19 - No Cached
  • Laravel queues provide a unified API across a variety of different queue backends, such as Beanstalk, Amazon SQS, Redis, or even a relational database.
  • The queue configuration file is stored in config/queue.php
  • a synchronous driver that will execute jobs immediately (for local use)
  • ...56 more annotations...
  • A null queue driver is also included which discards queued jobs.
  • In your config/queue.php configuration file, there is a connections configuration option.
  • any given queue connection may have multiple "queues" which may be thought of as different stacks or piles of queued jobs.
  • each connection configuration example in the queue configuration file contains a queue attribute.
  • if you dispatch a job without explicitly defining which queue it should be dispatched to, the job will be placed on the queue that is defined in the queue attribute of the connection configuration
  • pushing jobs to multiple queues can be especially useful for applications that wish to prioritize or segment how jobs are processed
  • specify which queues it should process by priority.
  • If your Redis queue connection uses a Redis Cluster, your queue names must contain a key hash tag.
  • ensure all of the Redis keys for a given queue are placed into the same hash slot
  • all of the queueable jobs for your application are stored in the app/Jobs directory.
  • Job classes are very simple, normally containing only a handle method which is called when the job is processed by the queue.
  • we were able to pass an Eloquent model directly into the queued job's constructor. Because of the SerializesModels trait that the job is using, Eloquent models will be gracefully serialized and unserialized when the job is processing.
  • When the job is actually handled, the queue system will automatically re-retrieve the full model instance from the database.
  • The handle method is called when the job is processed by the queue
  • The arguments passed to the dispatch method will be given to the job's constructor
  • delay the execution of a queued job, you may use the delay method when dispatching a job.
  • dispatch a job immediately (synchronously), you may use the dispatchNow method.
  • When using this method, the job will not be queued and will be run immediately within the current process
  • specify a list of queued jobs that should be run in sequence.
  • Deleting jobs using the $this->delete() method will not prevent chained jobs from being processed. The chain will only stop executing if a job in the chain fails.
  • this does not push jobs to different queue "connections" as defined by your queue configuration file, but only to specific queues within a single connection.
  • To specify the queue, use the onQueue method when dispatching the job
  • To specify the connection, use the onConnection method when dispatching the job
  • defining the maximum number of attempts on the job class itself.
  • to defining how many times a job may be attempted before it fails, you may define a time at which the job should timeout.
  • using the funnel method, you may limit jobs of a given type to only be processed by one worker at a time
  • using the throttle method, you may throttle a given type of job to only run 10 times every 60 seconds.
  • If an exception is thrown while the job is being processed, the job will automatically be released back onto the queue so it may be attempted again.
  • dispatch a Closure. This is great for quick, simple tasks that need to be executed outside of the current request cycle
  • When dispatching Closures to the queue, the Closure's code contents is cryptographically signed so it can not be modified in transit.
  • Laravel includes a queue worker that will process new jobs as they are pushed onto the queue.
  • once the queue:work command has started, it will continue to run until it is manually stopped or you close your terminal
  • queue workers are long-lived processes and store the booted application state in memory.
  • they will not notice changes in your code base after they have been started.
  • during your deployment process, be sure to restart your queue workers.
  • customize your queue worker even further by only processing particular queues for a given connection
  • The --once option may be used to instruct the worker to only process a single job from the queue
  • The --stop-when-empty option may be used to instruct the worker to process all jobs and then exit gracefully.
  • Daemon queue workers do not "reboot" the framework before processing each job.
  • you should free any heavy resources after each job completes.
  • Since queue workers are long-lived processes, they will not pick up changes to your code without being restarted.
  • restart the workers during your deployment process.
  • php artisan queue:restart
  • The queue uses the cache to store restart signals
  • the queue workers will die when the queue:restart command is executed, you should be running a process manager such as Supervisor to automatically restart the queue workers.
  • each queue connection defines a retry_after option. This option specifies how many seconds the queue connection should wait before retrying a job that is being processed.
  • The --timeout option specifies how long the Laravel queue master process will wait before killing off a child queue worker that is processing a job.
  • When jobs are available on the queue, the worker will keep processing jobs with no delay in between them.
  • While sleeping, the worker will not process any new jobs - the jobs will be processed after the worker wakes up again
  • the numprocs directive will instruct Supervisor to run 8 queue:work processes and monitor all of them, automatically restarting them if they fail.
  • Laravel includes a convenient way to specify the maximum number of times a job should be attempted.
  • define a failed method directly on your job class, allowing you to perform job specific clean-up when a failure occurs.
  • a great opportunity to notify your team via email or Slack.
  • php artisan queue:retry all
  • php artisan queue:flush
  • When injecting an Eloquent model into a job, it is automatically serialized before being placed on the queue and restored when the job is processed
張 旭

The Twelve-Factor App - 0 views

12factor.net/codebase

development programming system software

shared by 張 旭 on 01 Jan 19 - No Cached
  • A copy of the revision tracking database is known as a code repository, often shortened to code repo or just repo.
  • always a one-to-one correlation between the codebase and the app
  • If there are multiple codebases, it’s not an app – it’s a distributed system.
  • ...4 more annotations...
  • Each component in a distributed system is an app
  • only one codebase per app, but there will be many deploys of the app.
  • A deploy is a running instance of the app.
  • The codebase is the same across all deploys, although different versions may be active in each deploy.
  • 張 旭 on 01 Jan 19
     
    "A copy of the revision tracking database is known as a code repository, often shortened to code repo or just repo."
張 旭

The Twelve-Factor App - 0 views

12factor.net/dependencies

development programming system software

shared by 張 旭 on 01 Jan 19 - No Cached
  • Libraries installed through a packaging system can be installed system-wide (known as “site packages”) or scoped into the directory containing the app (known as “vendoring” or “bundling”).
  • A twelve-factor app never relies on implicit existence of system-wide packages.
  • declares all dependencies, completely and exactly, via a dependency declaration manifest.
  • ...8 more annotations...
  • The full and explicit dependency specification is applied uniformly to both production and development.
  • Bundler for Ruby offers the Gemfile manifest format for dependency declaration and bundle exec for dependency isolation.
  • Pip is used for declaration and Virtualenv for isolation.
  • No matter what the toolchain, dependency declaration and isolation must always be used together
  • requiring only the language runtime and dependency manager installed as prerequisites.
  • set up everything needed to run the app’s code with a deterministic build command.
  • If the app needs to shell out to a system tool, that tool should be vendored into the app.
  • do not rely on the implicit existence of any system tools
張 旭

The Twelve-Factor App - 0 views

12factor.net/config

development programming system software

shared by 張 旭 on 01 Jan 19 - No Cached
  • An app’s config is everything that is likely to vary between deploys (staging, production, developer environments, etc)
  • Resource handles
  • Credentials
  • ...8 more annotations...
  • Per-deploy values
  • trict separation of config from code.
  • Config varies substantially across deploys, code does not.
  • he codebase could be made open source at any moment, without compromising any credentials.
  • “config” does not include internal application config
  • stores config in environment variables (often shortened to env vars or env).
  • env vars are granular controls, each fully orthogonal to other env vars
  • They are never grouped together as “environments”
張 旭

Ruby on Rails 實戰聖經 | 網站效能 - 0 views

ihower.tw/performance.html

rails programming mysql performance

shared by 張 旭 on 28 Nov 18 - No Cached
  • 依照慣例是_count結尾，型別是integer，有預設值0。
  • lol_dba提供了Rake任務可以幫忙找忘記加的索引。
  • Bullet是一個外掛可以在開發時偵測N+1 queries問題。
  • ...19 more annotations...
  • 存取資料庫是一種相對很慢的I/O的操作：每一條SQL query都得耗上時間、執行回傳的結果也會被轉成ActiveRecord物件全部放進記憶體
  • 如果需要撈出全部的資料做處理，強烈建議最好不要用all方法，因為這樣會把全部的資料一次放進記憶體中，如果資料有成千上萬筆的話，效能就墜毀了。
  • .find_each( :batch_size => 100 )
  • .find_in_batches( :batch_size => 100 )
  • 在Transaction交易範圍內的SQL效能會加快，因為最後只需要COMMIT一次即可
  • Elasticsearch全文搜尋引擎和elasticsearch-rails gem
  • QueryReviewer這個套件透過SQL EXPLAIN分析SQL query的效率
  • 必要時可以採用逆正規化的設計。犧牲空間，增加修改的麻煩，但是讓讀取這事件變得更快更簡單。
  • 將成本轉嫁到寫入，而最佳化了讀取時間
  • 在效能還沒有造成問題前，就為了優化效能而修改程式和架構，只會讓程式更混亂不好維護
  • 當效能還不會造成問題時，程式的維護性比考慮效能重要
  • 會拖慢整體效能的程式，只佔全部程式的一小部分而已，所以我們只最佳化會造成問題的程式。
  • 善用分析工具找效能瓶頸，最佳化前需要測量，最佳化後也要測量比較。
  • rack-mini-profiler在頁面的左上角顯示花了多少時間，並且提供報表，推薦安裝
  • 如果是不需要權限控管的靜態檔案，可以直接放在public目錄下讓使用者下載。
  • Web伺服器得先安裝好x_sendfile功能
  • 如果要讓你的Assets例如CSS, JavaScript, Images也讓使用者透過CDN下載，只要修改config/environments/production.rb的config.action_controller.asset_host為CDN網址即可。
  • 有時候「執行速度較快」的程式碼不代表好維護、好除錯的程式碼
  • Ruby不是萬能，有時候直接呼叫外部程式是最快的作法
« First ‹ Previous 121 - 140 of 174 Next › Last »
Showing 20▼ items per page