Skip to main content

Home/ Larvata/ Group items tagged lock

Rss Feed Group items tagged

Filter: All | Bookmarks | Topics Simple Middle
張 旭

Dependency Lock File (.terraform.lock.hcl) - Configuration Language | Terraform | Hashi... - 0 views

developer.hashicorp.com/...dependency-lock

devops terraform

shared by 張 旭 on 21 Jan 23 - No Cached
  • Version constraints within the configuration itself determine which versions of dependencies are potentially compatible, but after selecting a specific version of each dependency Terraform remembers the decisions it made in a dependency lock file
  • At present, the dependency lock file tracks only provider dependencies.
  • Terraform does not remember version selections for remote modules, and so Terraform will always select the newest available module version that meets the specified version constraints.
  • ...14 more annotations...
  • The lock file is always named .terraform.lock.hcl, and this name is intended to signify that it is a lock file for various items that Terraform caches in the .terraform
  • Terraform automatically creates or updates the dependency lock file each time you run the terraform init command.
  • You should include this file in your version control repository
  • If a particular provider has no existing recorded selection, Terraform will select the newest available version that matches the given version constraint, and then update the lock file to include that selection.
  • the "trust on first use" model
  • you can pre-populate checksums for a variety of different platforms in your lock file using the terraform providers lock command, which will then allow future calls to terraform init to verify that the packages available in your chosen mirror match the official packages from the provider's origin registry.
  • The h1: and zh: prefixes on these values represent different hashing schemes, each of which represents calculating a checksum using a different algorithm.
  • zh:: a mnemonic for "zip hash"
  • h1:: a mnemonic for "hash scheme 1", which is the current preferred hashing scheme.
  • To determine whether there still exists a dependency on a given provider, Terraform uses two sources of truth: the configuration itself, and the state.
  • Version constraints within the configuration itself determine which versions of dependencies are potentially compatible, but after selecting a specific version of each dependency Terraform remembers the decisions it made in a dependency lock file so that it can (by default) make the same decisions again in future.
  • At present, the dependency lock file tracks only provider dependencies.
  • Terraform will always select the newest available module version that meets the specified version constraints.
  • The lock file is always named .terraform.lock.hcl
  • 張 旭 on 21 Jan 23
     
    "the overriding effect is compounded, with later blocks taking precedence over earlier blocks."
張 旭

MongoDB Performance Tuning: Everything You Need to Know - Stackify - 0 views

stackify.com/mongodb-performance-tuning

database mongodb

shared by 張 旭 on 15 Apr 21 - No Cached
  • db.serverStatus().globalLock
  • db.serverStatus().locks
  • globalLock.currentQueue.total: This number can indicate a possible concurrency issue if it’s consistently high. This can happen if a lot of requests are waiting for a lock to be released.
  • ...35 more annotations...
  • globalLock.totalTime: If this is higher than the total database uptime, the database has been in a lock state for too long.
  • Unlike relational databases such as MySQL or PostgreSQL, MongoDB uses JSON-like documents for storing data.
  • Databases operate in an environment that consists of numerous reads, writes, and updates.
  • When a lock occurs, no other operation can read or modify the data until the operation that initiated the lock is finished.
  • locks.deadlockCount: Number of times the lock acquisitions have encountered deadlocks
  • Is the database frequently locking from queries? This might indicate issues with the schema design, query structure, or system architecture.
  • For version 3.2 on, WiredTiger is the default.
  • MMAPv1 locks whole collections, not individual documents.
  • WiredTiger performs locking at the document level.
  • When the MMAPv1 storage engine is in use, MongoDB will use memory-mapped files to store data.
  • All available memory will be allocated for this usage if the data set is large enough.
  • db.serverStatus().mem
  • mem.resident: Roughly equivalent to the amount of RAM in megabytes that the database process uses
  • If mem.resident exceeds the value of system memory and there’s a large amount of unmapped data on disk, we’ve most likely exceeded system capacity.
  • If the value of mem.mapped is greater than the amount of system memory, some operations will experience page faults.
  • The WiredTiger storage engine is a significant improvement over MMAPv1 in performance and concurrency.
  • By default, MongoDB will reserve 50 percent of the available memory for the WiredTiger data cache.
  • wiredTiger.cache.bytes currently in the cache – This is the size of the data currently in the cache.
  • wiredTiger.cache.tracked dirty bytes in the cache – This is the size of the dirty data in the cache.
  • we can look at the wiredTiger.cache.bytes read into cache value for read-heavy applications. If this value is consistently high, increasing the cache size may improve overall read performance.
  • check whether the application is read-heavy. If it is, increase the size of the replica set and distribute the read operations to secondary members of the set.
  • write-heavy, use sharding within a sharded cluster to distribute the load.
  • Replication is the propagation of data from one node to another
  • Replication sets handle this replication.
  • Sometimes, data isn’t replicated as quickly as we’d like.
  • a particularly thorny problem if the lag between a primary and secondary node is high and the secondary becomes the primary
  • use the db.printSlaveReplicationInfo() or the rs.printSlaveReplicationInfo() command to see the status of a replica set from the perspective of the secondary member of the set.
  • shows how far behind the secondary members are from the primary. This number should be as low as possible.
  • monitor this metric closely.
  • watch for any spikes in replication delay.
  • Always investigate these issues to understand the reasons for the lag.
  • One replica set is primary. All others are secondary.
  • it’s not normal for nodes to change back and forth between primary and secondary.
  • use the profiler to gain a deeper understanding of the database’s behavior.
  • Enabling the profiler can affect system performance, due to the additional activity.
  • 張 旭 on 15 Apr 21
     
    "globalLock.currentQueue.total: This number can indicate a possible concurrency issue if it's consistently high. This can happen if a lot of requests are waiting for a lock to be released."
張 旭

MongoDB Performance - MongoDB Manual - 0 views

docs.mongodb.com/...analyzing-mongodb-performance

database mongodb performance

shared by 張 旭 on 20 Apr 21 - No Cached
  • MongoDB uses a locking system to ensure data set consistency. If certain operations are long-running or a queue forms, performance will degrade as requests and operations wait for the lock.
  • performance limitations as a result of inadequate or inappropriate indexing strategies, or as a consequence of poor schema design patterns.
  • performance issues may be temporary and related to abnormal traffic load.
  • ...9 more annotations...
  • Lock-related slowdowns can be intermittent.
  • If globalLock.currentQueue.total is consistently high, then there is a chance that a large number of requests are waiting for a lock.
  • If globalLock.totalTime is high relative to uptime, the database has existed in a lock state for a significant amount of time.
  • For write-heavy applications, deploy sharding and add one or more shards to a sharded cluster to distribute load among mongod instances.
  • Unless constrained by system-wide limits, the maximum number of incoming connections supported by MongoDB is configured with the maxIncomingConnections setting.
  • When logLevel is set to 0, MongoDB records slow operations to the diagnostic log at a rate determined by slowOpSampleRate.
  • At higher logLevel settings, all operations appear in the diagnostic log regardless of their latency with the following exception
  • Full Time Diagnostic Data Collection (FTDC) mechanism. FTDC data files are compressed, are not human-readable, and inherit the same file access permissions as the MongoDB data files.
  • mongod processes store FTDC data files in a diagnostic.data directory under the instances storage.dbPath.
  • 張 旭 on 20 Apr 21
     
    "MongoDB uses a locking system to ensure data set consistency. If certain operations are long-running or a queue forms, performance will degrade as requests and operations wait for the lock."
張 旭

How Percona XtraBackup Works - 0 views

www.percona.com/...how_xtrabackup_works.html

mysql database backup percona xtradb

shared by 張 旭 on 30 Nov 19 - No Cached
  • Percona XtraBackup is based on InnoDB‘s crash-recovery functionality.
  • it performs crash recovery on the files to make them a consistent, usable database again
  • InnoDB maintains a redo log, also called the transaction log. This contains a record of every change to InnoDB data.
  • ...14 more annotations...
  • When InnoDB starts, it inspects the data files and the transaction log, and performs two steps. It applies committed transaction log entries to the data files, and it performs an undo operation on any transactions that modified data but did not commit.
  • Percona XtraBackup works by remembering the log sequence number (LSN) when it starts, and then copying away the data files.
  • Percona XtraBackup runs a background process that watches the transaction log files, and copies changes from it.
  • Percona XtraBackup needs to do this continually
  • Percona XtraBackup needs the transaction log records for every change to the data files since it began execution.
  • Percona XtraBackup uses Backup locks where available as a lightweight alternative to FLUSH TABLES WITH READ LOCK.
  • Locking is only done for MyISAM and other non-InnoDB tables after Percona XtraBackup finishes backing up all InnoDB/XtraDB data and logs.
  • xtrabackup tries to avoid backup locks and FLUSH TABLES WITH READ LOCK when the instance contains only InnoDB tables. In this case, xtrabackup obtains binary log coordinates from performance_schema.log_status
  • When backup locks are supported by the server, xtrabackup first copies InnoDB data, runs the LOCK TABLES FOR BACKUP and then copies the MyISAM tables.
  • the STDERR of xtrabackup is not written in any file. You will have to redirect it to a file, e.g., xtrabackup OPTIONS 2> backupout.log
  • During the prepare phase, Percona XtraBackup performs crash recovery against the copied data files, using the copied transaction log file. After this is done, the database is ready to restore and use.
  • the tools enable you to do operations such as streaming and incremental backups with various combinations of copying the data files, copying the log files, and applying the logs to the data.
  • To restore a backup with xtrabackup you can use the --copy-back or --move-back options.
  • you may have to change the files’ ownership to mysql before starting the database server, as they will be owned by the user who created the backup.
  • 張 旭 on 30 Nov 19
     
    "Percona XtraBackup is based on InnoDB's crash-recovery functionality."
張 旭

Providers - Configuration Language | Terraform | HashiCorp Developer - 0 views

developer.hashicorp.com/...providers

terraform devops

shared by 張 旭 on 23 Jan 23 - No Cached
  • Terraform relies on plugins called providers to interact with cloud providers, SaaS providers, and other APIs.
  • Terraform configurations must declare which providers they require so that Terraform can install and use them.
  • Each provider adds a set of resource types and/or data sources that Terraform can manage.
  • ...6 more annotations...
  • Every resource type is implemented by a provider; without providers, Terraform can't manage any kind of infrastructure.
  • The Terraform Registry is the main directory of publicly available Terraform providers, and hosts providers for most major infrastructure platforms.
  • Dependency Lock File documents an additional HCL file that can be included with a configuration, which tells Terraform to always use a specific set of provider versions.
  • Terraform CLI finds and installs providers when initializing a working directory. It can automatically download providers from a Terraform registry, or load them from a local mirror or cache.
  • To save time and bandwidth, Terraform CLI supports an optional plugin cache. You can enable the cache using the plugin_cache_dir setting in the CLI configuration file.
  • you can use Terraform CLI to create a dependency lock file and commit it to version control along with your configuration.
張 旭

The package-lock.json file - 0 views

nodejs.dev/...the-package-lock-json-file

nodejs

shared by 張 旭 on 22 May 22 - No Cached
  • You don't commit to Git your node_modules folder, which is generally huge, and when you try to replicate the project on another machine by using the npm install command,
  • Even if a patch or minor release should not introduce breaking changes
  • The package-lock.json sets your currently installed version of each package in stone, and npm will use those exact versions when running npm ci
  • ...1 more annotation...
  • The package-lock.json file needs to be committed to your Git repository
  • 張 旭 on 22 May 22
     
    "You don't commit to Git your node_modules folder, which is generally huge, and when you try to replicate the project on another machine by using the npm install command,"
crazylion lee

pt-online-schema-change - 0 views

www.percona.com/...pt-online-schema-change.html

mysql lock alter

shared by crazylion lee on 03 Oct 15 - No Cached
  • crazylion lee on 03 Oct 15
     
    "pt-online-schema-change - ALTER tables without locking them"
張 旭

Backends: State Storage and Locking - Terraform by HashiCorp - 0 views

www.terraform.io/...state.html

terraform devops system storage status

shared by 張 旭 on 07 Apr 19 - No Cached
  • Backends determine where state is stored.
  • backends happen to provide locking: local via system APIs and Consul via locking APIs.
  • manually retrieve the state from the remote state using the terraform state pull command
  • ...3 more annotations...
  • manually write state with terraform state push. This is extremely dangerous and should be avoided if possible. This will overwrite the remote state.
  • The "lineage" is a unique ID assigned to a state when it is created.
  • Every state has a monotonically increasing "serial" number.
  • 張 旭 on 07 Apr 19
     
    "Backends determine where state is stored."
張 旭

Ask HN: What are the best practises for using SSH keys? | Hacker News - 0 views

news.ycombinator.com/item

security ssh key server

shared by 張 旭 on 14 Oct 16 - No Cached
  • Make sure you use full disk encryption and never stand up from your machine without locking it, and make sure you keep your local machine patched.
  • I'm more focused on just stealing your keys from you regardless of length
  • attacks that aren't after your keys specifically, e.g. your home directory gets stolen.
  • ...19 more annotations...
  • ED25519 is more vulnerable to quantum computation than is RSA
  • best practice to be using a hardware token
  • to use a yubikey via gpg: with this method you use your gpg subkey as an ssh key
  • sit down and spend an hour thinking about your backup and recovery strategy first
  • never share a private keys between physical devices
  • allows you to revoke a single credential if you lose (control over) that device
  • If a private key ever turns up on the wrong machine, you *know* the key and both source and destination machines have been compromised.
  • centralized management of authentication/authorization
  • I have setup a VPS, disabled passwords, and setup a key with a passphrase to gain access. At this point my greatest worry is losing this private key, as that means I can't access the server.What is a reasonable way to backup my private key?
  • a mountable disk image that's encrypted
  • a system that can update/rotate your keys across all of your servers on the fly in case one is compromised or assumed to be compromised.
  • different keys for different purposes per client device
  • fall back to password plus OTP
  • relying completely on the security of your disk, against either physical or cyber.
  • It is better to use a different passphrase for each key but it is also less convenient unless you're using a password manager (personally, I'm using KeePass)
  • - RSA is pretty standard, and generally speaking is fairly secure for key lengths >=2048. RSA-2048 is the default for ssh-keygen, and is compatible with just about everything.
  • public-key authentication has somewhat unexpected side effect of preventing MITM per this security consulting firm
  • Disable passwords and only allow keys even for root with PermitRootLogin without-password
  • You should definitely use a different passphrase for keys stored on separate computers,
  • 張 旭 on 14 Oct 16
     
    "Make sure you use full disk encryption and never stand up from your machine without locking it, and make sure you keep your local machine patched"
crazylion lee

pypa/pipfile - 0 views

github.com/pipfile

python pip

shared by crazylion lee on 03 Dec 16 - No Cached
  • crazylion lee on 03 Dec 16
     
    "A Pipfile, and its related Pipfile.lock, are a new (and much better!) replacement for pip's requirements.txt files."
crazylion lee

shrinkwrap | npm Documentation - 0 views

docs.npmjs.com/shrinkwrap

nodejs npm node

shared by crazylion lee on 11 Jul 16 - No Cached
  • crazylion lee on 11 Jul 16
     
    "Lock down dependency versions"
crazylion lee

TMSU - 0 views

tmsu.org

filesystem linux tag file tagging

shared by crazylion lee on 27 Jun 16 - No Cached
  • crazylion lee on 27 Jun 16
     
    "TMSU is a tool for tagging your files. It provides a simple command-line tool for applying tags and a virtual filesystem so that you can get a tag-based view of your files from within any other program. TMSU does not alter your files in any way: they remain unchanged on disk, or on the network, wherever you put them. TMSU maintains its own database and you simply gain an additional view, which you can mount, based upon the tags you set up. The only commitment required is your time and there's absolutely no lock-in."
crazylion lee

soundcloud/lhm - 1 views

github.com/lhm

mysql migration

shared by crazylion lee on 02 Oct 15 - No Cached
  • crazylion lee on 02 Oct 15
     
    "Online MySQL schema migrations" # 可以不用lock table
crazylion lee

YubiKey 4 and YubiKey 4 Nano | U2F, OTP, PIV | Yubico - 0 views

www.yubico.com/...yubikey4

key lock hardware

shared by crazylion lee on 26 Apr 16 - No Cached
  • crazylion lee on 26 Apr 16
     
    "The YubiKey 4 is the strong authentication bullseye the industry has been aiming at for years, enabling one single key to secure an unlimited number of applications. Yubico's 4th generation YubiKey is built on high-performance secure elements. It includes the same range of one-time password and public key authentication protocols as in the YubiKey NEO, excluding NFC, but with stronger public/private keys, faster crypto operations and the world's first touch-to-sign feature. With the YubiKey 4 platform, we have further improved our manufacturing and ordering process, enabling customers to order exactly what functions they want in 500+ unit volumes, with no secrets stored at Yubico or shared with a third-party organization. The best part? An organization can securely customize 1,000 YubiKeys in less than 10 minutes. For customers who require NFC, the YubiKey NEO is our full-featured key with both contact (USB) and contactless (NFC, MIFARE) communications."
張 旭

Secrets Management with Terraform - 0 views

www.linode.com/...rets-management-with-terraform

terraform config devops security

shared by 張 旭 on 12 Apr 19 - No Cached
  • Terraform is an Infrastructure as Code (IaC) tool that allows you to write declarative code to manage your infrastructure.
  • Keeping Secrets Out of .tf Files
  • .tf files contain the declarative code used to create, manage, and destroy infrastructure.
  • ...17 more annotations...
  • .tf files can accept values from input variables.
  • variable definitions can have default values assigned to them.
  • values are stored in separate files with the .tfvars extension.
  • looks through the working directory for a file named terraform.tfvars, or for files with the .auto.tfvars extension.
  • add the terraform.tfvars file to your .gitignore file and keep it out of version control.
  • include an example terraform.tfvars.example in your Git repository with all of the variable names recorded (but none of the values entered).
  • terraform apply -var-file=myvars.tfvars
  • Terraform allows you to keep input variable values in environment variables.
  • the prefix TF_VAR_
  • If Terraform does not find a default value for a defined variable; or a value from a .tfvars file, environment variable, or CLI flag; it will prompt you for a value before running an action
  • state file contains a JSON object that holds your managed infrastructure’s current state
  • state is a snapshot of the various attributes of your infrastructure at the time it was last modified
  • sensitive information used to generate your Terraform state can be stored as plain text in the terraform.tfstate file.
  • Avoid checking your terraform.tfstate file into your version control repository.
  • Some backends, like Consul, also allow for state locking. If one user is applying a state, another user will be unable to make any changes.
  • Terraform backends allow the user to securely store their state in a remote location, such as a key/value store like Consul, or an S3 compatible bucket storage like Minio.
  • at minimum the repository should be private.
張 旭

Baseimage-docker: A minimal Ubuntu base image modified for Docker-friendliness - 0 views

phusion.github.io/baseimage-docker

docker base system linux

shared by 張 旭 on 29 Oct 14 - No Cached
  • We encourage you to use multiple processes.
  • Baseimage-docker is a special Docker image that is configured for correct use within Docker containers.
  • When your Docker container starts, only the CMD command is run.
  • ...16 more annotations...
  • You're not running them, you're only running your app.
  • You have Ubuntu installed in Docker. The files are there. But that doesn't mean Ubuntu's running as it should.
  • The only processes that will be running inside the container is the CMD command, and all processes that it spawns.
  • A proper Unix system should run all kinds of important system services.
  • Ubuntu is not designed to be run inside Docker
  • When a system is started, the first process in the system is called the init process, with PID 1. The system halts when this processs halts.
  • Runit (written in C) is much lighter weight than supervisord (written in Python).
  • Docker runs fine with multiple processes in a container.
  • Baseimage-docker encourages you to run multiple processes through the use of runit.
  • If your init process is your app, then it'll probably only shut down itself, not all the other processes in the container.
  • a Docker container, which is a locked down environment with e.g. no direct access to many kernel resources.
  • Used for service supervision and management.
  • A custom tool for running a command as another user.
  • add additional daemons (e.g. your own app) to the image by creating runit entries.
  • write a small shell script which runs your daemon, and runit will keep it up and running for you, restarting it when it crashes, etc.
  • the shell script must run the daemon without letting it daemonize/fork it.
張 旭

bbatsov/rails-style-guide: A community-driven Ruby on Rails 4 style guide - 0 views

github.com/...rails-style-guide

rails programming code style

shared by 張 旭 on 07 May 17 - No Cached
  • custom initialization code in config/initializers. The code in initializers executes on application startup
  • Keep initialization code for each gem in a separate file with the same name as the gem
  • Mark additional assets for precompilation
  • ...90 more annotations...
  • config/environments/production.rb
  • Create an additional staging environment that closely resembles the production one
  • Keep any additional configuration in YAML files under the config/ directory
  • Rails::Application.config_for(:yaml_file)
  • Use nested routes to express better the relationship between ActiveRecord models
  • nest routes more than 1 level deep then use the shallow: true option
  • namespaced routes to group related actions
  • Don't use match to define any routes unless there is need to map multiple request types among [:get, :post, :patch, :put, :delete] to a single action using :via option.
  • Keep the controllers skinny
  • all the business logic should naturally reside in the model
  • Share no more than two instance variables between a controller and a view.
  • using a template
  • Prefer render plain: over render text
  • Prefer corresponding symbols to numeric HTTP status codes
  • without abbreviations
  • Keep your models for business logic and data-persistence only
  • Avoid altering ActiveRecord defaults (table names, primary key, etc)
  • Group macro-style methods (has_many, validates, etc) in the beginning of the class definition
  • Prefer has_many :through to has_and_belongs_to_many
  • self[:attribute]
  • self[:attribute] = value
  • validates
  • Keep custom validators under app/validators
  • Consider extracting custom validators to a shared gem
  • preferable to make a class method instead which serves the same purpose of the named scope
  • returns an ActiveRecord::Relation object
  • .update_attributes
  • Override the to_param method of the model
  • Use the friendly_id gem. It allows creation of human-readable URLs by using some descriptive attribute of the model instead of its id
  • find_each to iterate over a collection of AR objects
  • .find_each
  • .find_each
  • Looping through a collection of records from the database (using the all method, for example) is very inefficient since it will try to instantiate all the objects at once
  • always call before_destroy callbacks that perform validation with prepend: true
  • Define the dependent option to the has_many and has_one associations
  • always use the exception raising bang! method or handle the method return value.
  • When persisting AR objects
  • Avoid string interpolation in queries
  • param will be properly escaped
  • Consider using named placeholders instead of positional placeholders
  • use of find over where when you need to retrieve a single record by id
  • use of find_by over where and find_by_attribute
  • use of where.not over SQL
  • use heredocs with squish
  • Keep the schema.rb (or structure.sql) under version control.
  • Use rake db:schema:load instead of rake db:migrate to initialize an empty database
  • Enforce default values in the migrations themselves instead of in the application layer
  • change_column_default
  • imposing data integrity from the Rails app is impossible
  • use the change method instead of up and down methods.
  • constructive migrations
  • use models in migrations, make sure you define them so that you don't end up with broken migrations in the future
  • Don't use non-reversible migration commands in the change method.
  • In this case, block will be used by create_table in rollback
  • Never call the model layer directly from a view
  • Never make complex formatting in the views, export the formatting to a method in the view helper or the model.
  • When the labels of an ActiveRecord model need to be translated, use the activerecord scope
  • Separate the texts used in the views from translations of ActiveRecord attributes
  • Place the locale files for the models in a folder locales/models
  • the texts used in the views in folder locales/views
  • config/application.rb config.i18n.load_path += Dir[Rails.root.join('config', 'locales', '**', '*.{rb,yml}')]
  • I18n.t
  • I18n.l
  • Use "lazy" lookup for the texts used in views.
  • Use the dot-separated keys in the controllers and models
  • Reserve app/assets for custom stylesheets, javascripts, or images
  • Third party code such as jQuery or bootstrap should be placed in vendor/assets
  • Provide both HTML and plain-text view templates
  • config.action_mailer.raise_delivery_errors = true
  • Use a local SMTP server like Mailcatcher in the development environment
  • Provide default settings for the host name
  • The _url methods include the host name and the _path methods don't
  • _url
  • Format the from and to addresses properly
  • default from:
  • sending html emails all styles should be inline
  • Sending emails while generating page response should be avoided. It causes delays in loading of the page and request can timeout if multiple email are sent.
  • .start_with?
  • .end_with?
  • &.
  • Config your timezone accordingly in application.rb
  • config.active_record.default_timezone = :local
  • it can be only :utc or :local
  • Don't use Time.parse
  • Time.zone.parse
  • Don't use Time.now
  • Time.zone.now
  • Put gems used only for development or testing in the appropriate group in the Gemfile
  • Add all OS X specific gems to a darwin group in the Gemfile, and all Linux specific gems to a linux group
  • Do not remove the Gemfile.lock from version control.
張 旭

作業系統 - 維基百科，自由的百科全書 - 0 views

zh.wikipedia.org/...%8D%E4%BD%9C%E7%B3%BB%E7%BB%9F

system linux

shared by 張 旭 on 22 Jun 21 - No Cached
  • 作業系統位於底層硬體與使用者之間，是兩者溝通的橋樑。
  • 行程管理（Processing management）
  • 安全機制（Security）
  • ...20 more annotations...
  • 記憶體管理（Memory management）
  • 核心 - 作業系統之最核心部分，通常執行在最高特權級，負責提供基礎性、結構性的功能。
  • 驅動程式 - 最底層的、直接控制和監視各類硬體的部分，它們的職責是隱藏硬體的具體細節，並向其他部分提供一個抽象的、通用的介面。
  • 作業系統的分類沒有一個單一的標準，可以根據工作方式分為批次處理作業系統、分時作業系統、即時作業系統、網路作業系統和分散式作業系統等
  • 根據帕金森定律：「你給程式再多記憶體，程式也會想盡辦法耗光」
  • 大部分的現代電腦記憶體架構都是階層式的，最快且數量最少的暫存器為首，然後是快取、記憶體以及最慢的磁碟儲存裝置。
  • 虛擬記憶體管理的功能大幅增加每個行程可獲得的記憶空間
  • 當年運用馮·諾伊曼結構建造電腦時，每個中央處理器最多只能同時執行一個行程。
  • 現代的作業系統，即使只擁有一個CPU，也可以利用多行程（multitask）功能同時執行多個行程。行程管理指的是作業系統調整多個行程的功能。
  • 作業系統尚有擔負起行程間通訊（IPC）、行程異常終止處理以及死結（Dead Lock）偵測及處理等較為艱深的問題。
  • 檔案系統，通常指稱管理磁碟資料的系統，可將資料以目錄或檔案的型式儲存。每個檔案系統都有自己的特殊格式與功能，例如日誌管理或不需磁碟重整。
  • 現代的作業系統都具備操作主流網路通訊協定TCP/IP的能力。也就是說這樣的作業系統可以進入網路世界，並且與其他系統分享諸如檔案、印表機與掃描器等資源。
  • 作業系統提供外界直接或間接存取數種資源的管道
  • 作業系統有能力認證資源存取的請求
  • 通常是一個正在執行的程式發出的資源請求。在某些系統上，一個程式一旦可執行就可做任何事情（例如DOS時代的病毒），但通常作業系統會給程式一個識別代號，並且在此程式發出請求時，檢查其代號與所需資源的存取權限關係。
  • 一個高安全等級的系統也會提供記錄選項，允許記錄各種請求對資源存取的行為（例如「誰曾經讀了這個檔案？」）
  • 大部分的作業系統都包含圖形化使用者介面（GUI）。有幾類較舊的作業系統將圖形化使用者介面與核心緊密結合，例如最早的Windows與Mac OS實作產品。
  • 驅動程式（Device driver）是指某類設計來與硬體互動的電腦軟體。通常是一設計完善的裝置互動介面，利用與此硬體連接的電腦匯排流或通訊子系統，提供對此裝置下令與接收資訊的功能；以及最終目的，將訊息提供給作業系統或應用程式。
  • 驅動程式是針對特定硬體與特定作業系統設計的軟體，通常以作業系統核心模組、應用軟體包或普通電腦程式的形式在作業系統核心底下執行，以達到通透順暢地與硬體互動的效果
  • 適合的驅動程式一旦安裝，相對應的新裝置就可以無誤地執行。此新驅動程式可以讓此裝置完美地切合在作業系統中，讓使用者察覺不到這是作業系統原本沒有的功能。
  • 張 旭 on 22 Jun 21
     
    "作業系統位於底層硬體與使用者之間，是兩者溝通的橋樑。"
張 旭

Speeding up Docker image build process of a Rails application | BigBinary Blog - 1 views

blog.bigbinary.com/...ss-of-a-rails-application.html

docker kubernetes rails devops system

shared by 張 旭 on 23 Oct 19 - No Cached
  • we do not want to execute bundle install and rake assets:precompile tasks while starting a container in each pod which would prevent that pod from accepting any requests until these tasks are finished.
  • run bundle install and rake assets:precompile tasks while or before containerizing the Rails application.
  • Kubernetes pulls the image, starts a Docker container using that image inside the pod and runs puma server immediately.
  • ...7 more annotations...
  • Since source code changes often, the previously cached layer for the ADD instruction is invalidated due to the mismatching checksums.
  • The ARG instruction in the Dockerfile defines RAILS_ENV variable and is implicitly used as an environment variable by the rest of the instructions defined just after that ARG instruction.
  • RUN instructions are used to install gems and precompile static assets using sprockets
  • Instead, Docker automatically re-uses the previously built layer for the RUN bundle install instruction if the Gemfile.lock file remains unchanged.
  • everyday we need to build a lot of Docker images containing source code from varying Git branches as well as with varying environments.
  • it is hard for Docker to cache layers for bundle install and rake assets:precompile tasks and re-use those layers during every docker build command run with different application source code and a different environment.
  • By default, Bundler installs gems at the location which is set by Rubygems.
  • 張 旭 on 23 Oct 19
     
    "we do not want to execute bundle install and rake assets:precompile tasks while starting a container in each pod which would prevent that pod from accepting any requests until these tasks are finished."
張 旭

Optimizing Gitlab pipelines - Basics (1) | PrinsFrank.nl - 0 views

prinsfrank.nl/...g-gitlab-pipelines-pt-1-basics

devops gitlab cicd proxy performance

shared by 張 旭 on 23 Feb 22 - No Cached
  • When you use specific docker image, make sure you have the Dependency Proxy enabled so the image doesn’t have to be downloaded again for every job.
  • stages are used to group items that can run at the same time.
  • Instead of waiting for all jobs to finish, you can mark jobs as interruptible which signals a job to cancel when a new pipeline starts for the same branch
  • ...8 more annotations...
  • mark all jobs as interruptible as it doesn’t make sense to wait for builds and tests based on old information.
  • Deployment jobs are the main exception as they should probably finish.
  • only running it when specific files have changed
  • To prevent the ‘vendor’ and ‘node_modules’ folder from being regenerated in every job, we can configure a build job for composer and npm assets.
  • To share assets between multiple stages, Gitlab has caches and artifacts. For dependencies we should use caches.
  • The pull-push policy is the default, but specified here for clarity.
  • All consecutive runs for the build step with the same ‘composer.lock’ file don’t update the cache.
  • composer prevents this by caching packages in a global package cache,
1 - 20 of 20
Showing 20▼ items per page