Skip to main content

Home/ Larvata/ Group items tagged cache

Rss Feed Group items tagged

張 旭

Helm | - 0 views

  • A chart is a collection of files that describe a related set of Kubernetes resources.
  • A single chart might be used to deploy something simple, like a memcached pod, or something complex, like a full web app stack with HTTP servers, databases, caches, and so on.
  • Charts are created as files laid out in a particular directory tree, then they can be packaged into versioned archives to be deployed.
  • ...170 more annotations...
  • A chart is organized as a collection of files inside of a directory.
  • values.yaml # The default configuration values for this chart
  • charts/ # A directory containing any charts upon which this chart depends.
  • templates/ # A directory of templates that, when combined with values, # will generate valid Kubernetes manifest files.
  • version: A SemVer 2 version (required)
  • apiVersion: The chart API version, always "v1" (required)
  • Every chart must have a version number. A version must follow the SemVer 2 standard.
  • non-SemVer names are explicitly disallowed by the system.
  • When generating a package, the helm package command will use the version that it finds in the Chart.yaml as a token in the package name.
  • the appVersion field is not related to the version field. It is a way of specifying the version of the application.
  • appVersion: The version of the app that this contains (optional). This needn't be SemVer.
  • If the latest version of a chart in the repository is marked as deprecated, then the chart as a whole is considered to be deprecated.
  • deprecated: Whether this chart is deprecated (optional, boolean)
  • one chart may depend on any number of other charts.
  • dependencies can be dynamically linked through the requirements.yaml file or brought in to the charts/ directory and managed manually.
  • the preferred method of declaring dependencies is by using a requirements.yaml file inside of your chart.
  • A requirements.yaml file is a simple file for listing your dependencies.
  • The repository field is the full URL to the chart repository.
  • you must also use helm repo add to add that repo locally.
  • helm dependency update and it will use your dependency file to download all the specified charts into your charts/ directory for you.
  • When helm dependency update retrieves charts, it will store them as chart archives in the charts/ directory.
  • Managing charts with requirements.yaml is a good way to easily keep charts updated, and also share requirements information throughout a team.
  • All charts are loaded by default.
  • The condition field holds one or more YAML paths (delimited by commas). If this path exists in the top parent’s values and resolves to a boolean value, the chart will be enabled or disabled based on that boolean value.
  • The tags field is a YAML list of labels to associate with this chart.
  • all charts with tags can be enabled or disabled by specifying the tag and a boolean value.
  • The --set parameter can be used as usual to alter tag and condition values.
  • Conditions (when set in values) always override tags.
  • The first condition path that exists wins and subsequent ones for that chart are ignored.
  • The keys containing the values to be imported can be specified in the parent chart’s requirements.yaml file using a YAML list. Each item in the list is a key which is imported from the child chart’s exports field.
  • specifying the key data in our import list, Helm looks in the exports field of the child chart for data key and imports its contents.
  • the parent key data is not contained in the parent’s final values. If you need to specify the parent key, use the ‘child-parent’ format.
  • To access values that are not contained in the exports key of the child chart’s values, you will need to specify the source key of the values to be imported (child) and the destination path in the parent chart’s values (parent).
  • To drop a dependency into your charts/ directory, use the helm fetch command
  • A dependency can be either a chart archive (foo-1.2.3.tgz) or an unpacked chart directory.
  • name cannot start with _ or .. Such files are ignored by the chart loader.
  • a single release is created with all the objects for the chart and its dependencies.
  • Helm Chart templates are written in the Go template language, with the addition of 50 or so add-on template functions from the Sprig library and a few other specialized functions
  • When Helm renders the charts, it will pass every file in that directory through the template engine.
  • Chart developers may supply a file called values.yaml inside of a chart. This file can contain default values.
  • Chart users may supply a YAML file that contains values. This can be provided on the command line with helm install.
  • When a user supplies custom values, these values will override the values in the chart’s values.yaml file.
  • Template files follow the standard conventions for writing Go templates
  • {{default "minio" .Values.storage}}
  • Values that are supplied via a values.yaml file (or via the --set flag) are accessible from the .Values object in a template.
  • pre-defined, are available to every template, and cannot be overridden
  • the names are case sensitive
  • Release.Name: The name of the release (not the chart)
  • Release.IsUpgrade: This is set to true if the current operation is an upgrade or rollback.
  • Release.Revision: The revision number. It begins at 1, and increments with each helm upgrade
  • Chart: The contents of the Chart.yaml
  • Files: A map-like object containing all non-special files in the chart.
  • Files can be accessed using {{index .Files "file.name"}} or using the {{.Files.Get name}} or {{.Files.GetString name}} functions.
  • .helmignore
  • access the contents of the file as []byte using {{.Files.GetBytes}}
  • Any unknown Chart.yaml fields will be dropped
  • Chart.yaml cannot be used to pass arbitrarily structured data into the template.
  • A values file is formatted in YAML.
  • A chart may include a default values.yaml file
  • be merged into the default values file.
  • The default values file included inside of a chart must be named values.yaml
  • accessible inside of templates using the .Values object
  • Values files can declare values for the top-level chart, as well as for any of the charts that are included in that chart’s charts/ directory.
  • Charts at a higher level have access to all of the variables defined beneath.
  • lower level charts cannot access things in parent charts
  • Values are namespaced, but namespaces are pruned.
  • the scope of the values has been reduced and the namespace prefix removed
  • Helm supports special “global” value.
  • a way of sharing one top-level variable with all subcharts, which is useful for things like setting metadata properties like labels.
  • If a subchart declares a global variable, that global will be passed downward (to the subchart’s subcharts), but not upward to the parent chart.
  • global variables of parent charts take precedence over the global variables from subcharts.
  • helm lint
  • A chart repository is an HTTP server that houses one or more packaged charts
  • Any HTTP server that can serve YAML files and tar files and can answer GET requests can be used as a repository server.
  • Helm does not provide tools for uploading charts to remote repository servers.
  • the only way to add a chart to $HELM_HOME/starters is to manually copy it there.
  • Helm provides a hook mechanism to allow chart developers to intervene at certain points in a release’s life cycle.
  • Execute a Job to back up a database before installing a new chart, and then execute a second job after the upgrade in order to restore data.
  • Hooks are declared as an annotation in the metadata section of a manifest
  • Hooks work like regular templates, but they have special annotations
  • pre-install
  • post-install: Executes after all resources are loaded into Kubernetes
  • pre-delete
  • post-delete: Executes on a deletion request after all of the release’s resources have been deleted.
  • pre-upgrade
  • post-upgrade
  • pre-rollback
  • post-rollback: Executes on a rollback request after all resources have been modified.
  • crd-install
  • test-success: Executes when running helm test and expects the pod to return successfully (return code == 0).
  • test-failure: Executes when running helm test and expects the pod to fail (return code != 0).
  • Hooks allow you, the chart developer, an opportunity to perform operations at strategic points in a release lifecycle
  • Tiller then loads the hook with the lowest weight first (negative to positive)
  • Tiller returns the release name (and other data) to the client
  • If the resources is a Job kind, Tiller will wait until the job successfully runs to completion.
  • if the job fails, the release will fail. This is a blocking operation, so the Helm client will pause while the Job is run.
  • If they have hook weights (see below), they are executed in weighted order. Otherwise, ordering is not guaranteed.
  • good practice to add a hook weight, and set it to 0 if weight is not important.
  • The resources that a hook creates are not tracked or managed as part of the release.
  • leave the hook resource alone.
  • To destroy such resources, you need to either write code to perform this operation in a pre-delete or post-delete hook or add "helm.sh/hook-delete-policy" annotation to the hook template file.
  • Hooks are just Kubernetes manifest files with special annotations in the metadata section
  • One resource can implement multiple hooks
  • no limit to the number of different resources that may implement a given hook.
  • When subcharts declare hooks, those are also evaluated. There is no way for a top-level chart to disable the hooks declared by subcharts.
  • Hook weights can be positive or negative numbers but must be represented as strings.
  • sort those hooks in ascending order.
  • Hook deletion policies
  • "before-hook-creation" specifies Tiller should delete the previous hook before the new hook is launched.
  • By default Tiller will wait for 60 seconds for a deleted hook to no longer exist in the API server before timing out.
  • Custom Resource Definitions (CRDs) are a special kind in Kubernetes.
  • The crd-install hook is executed very early during an installation, before the rest of the manifests are verified.
  • A common reason why the hook resource might already exist is that it was not deleted following use on a previous install/upgrade.
  • Helm uses Go templates for templating your resource files.
  • two special template functions: include and required
  • include function allows you to bring in another template, and then pass the results to other template functions.
  • The required function allows you to declare a particular values entry as required for template rendering.
  • If the value is empty, the template rendering will fail with a user submitted error message.
  • When you are working with string data, you are always safer quoting the strings than leaving them as bare words
  • Quote Strings, Don’t Quote Integers
  • when working with integers do not quote the values
  • env variables values which are expected to be string
  • to include a template, and then perform an operation on that template’s output, Helm has a special include function
  • The above includes a template called toYaml, passes it $value, and then passes the output of that template to the nindent function.
  • Go provides a way for setting template options to control behavior when a map is indexed with a key that’s not present in the map
  • The required function gives developers the ability to declare a value entry as required for template rendering.
  • The tpl function allows developers to evaluate strings as templates inside a template.
  • Rendering a external configuration file
  • (.Files.Get "conf/app.conf")
  • Image pull secrets are essentially a combination of registry, username, and password.
  • Automatically Roll Deployments When ConfigMaps or Secrets change
  • configmaps or secrets are injected as configuration files in containers
  • a restart may be required should those be updated with a subsequent helm upgrade
  • The sha256sum function can be used to ensure a deployment’s annotation section is updated if another file changes
  • checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
  • helm upgrade --recreate-pods
  • "helm.sh/resource-policy": keep
  • resources that should not be deleted when Helm runs a helm delete
  • this resource becomes orphaned. Helm will no longer manage it in any way.
  • create some reusable parts in your chart
  • In the templates/ directory, any file that begins with an underscore(_) is not expected to output a Kubernetes manifest file.
  • by convention, helper templates and partials are placed in a _helpers.tpl file.
  • The current best practice for composing a complex application from discrete parts is to create a top-level umbrella chart that exposes the global configurations, and then use the charts/ subdirectory to embed each of the components.
  • SAP’s Converged charts: These charts install SAP Converged Cloud a full OpenStack IaaS on Kubernetes. All of the charts are collected together in one GitHub repository, except for a few submodules.
  • Deis’s Workflow: This chart exposes the entire Deis PaaS system with one chart. But it’s different from the SAP chart in that this umbrella chart is built from each component, and each component is tracked in a different Git repository.
  • YAML is a superset of JSON
  • any valid JSON structure ought to be valid in YAML.
  • As a best practice, templates should follow a YAML-like syntax unless the JSON syntax substantially reduces the risk of a formatting issue.
  • There are functions in Helm that allow you to generate random data, cryptographic keys, and so on.
  • a chart repository is a location where packaged charts can be stored and shared.
  • A chart repository is an HTTP server that houses an index.yaml file and optionally some packaged charts.
  • Because a chart repository can be any HTTP server that can serve YAML and tar files and can answer GET requests, you have a plethora of options when it comes down to hosting your own chart repository.
  • It is not required that a chart package be located on the same server as the index.yaml file.
  • A valid chart repository must have an index file. The index file contains information about each chart in the chart repository.
  • The Helm project provides an open-source Helm repository server called ChartMuseum that you can host yourself.
  • $ helm repo index fantastic-charts --url https://fantastic-charts.storage.googleapis.com
  • A repository will not be added if it does not contain a valid index.yaml
  • add the repository to their helm client via the helm repo add [NAME] [URL] command with any name they would like to use to reference the repository.
  • Helm has provenance tools which help chart users verify the integrity and origin of a package.
  • Integrity is established by comparing a chart to a provenance record
  • The provenance file contains a chart’s YAML file plus several pieces of verification information
  • Chart repositories serve as a centralized collection of Helm charts.
  • Chart repositories must make it possible to serve provenance files over HTTP via a specific request, and must make them available at the same URI path as the chart.
  • We don’t want to be “the certificate authority” for all chart signers. Instead, we strongly favor a decentralized model, which is part of the reason we chose OpenPGP as our foundational technology.
  • The Keybase platform provides a public centralized repository for trust information.
  • A chart contains a number of Kubernetes resources and components that work together.
  • A test in a helm chart lives under the templates/ directory and is a pod definition that specifies a container with a given command to run.
  • The pod definition must contain one of the helm test hook annotations: helm.sh/hook: test-success or helm.sh/hook: test-failure
  • helm test
  • nest your test suite under a tests/ directory like <chart-name>/templates/tests/
張 旭

Forwarding (DNS and BIND, 4th Edition) - 0 views

  • Forwarders are also useful if you need to shunt name resolution to a particular name server
  • If you designate one or more servers at your site as forwarders, your name servers will send all their off-site queries to the forwarders first.
  • A primary master or slave name server's mode of operation changes slightly when it is configured to use a forwarder.
  • ...2 more annotations...
  • If a resolver requests records that are already in the name server's authoritative data or cached data, the name server answers with that information
  • if the records aren't in its database, the name server sends the query to a forwarder and waits a short period for an answer before resuming normal operation and contacting the remote name servers itself. What the name server is doing differently here is sending a recursive query to the forwarder, expecting it to find the answer.
張 旭

MySQL :: MySQL 5.7 Reference Manual :: 20.2 Introducing InnoDB Cluster - 0 views

  • A group of MySQL servers can be configured to create a cluster using MySQL Shell
  • The cluster of servers has a single master, called the primary, which acts as the read-write master.
  • Multiple secondary servers are replicas of the master
  • ...6 more annotations...
  • A client application is connected to the primary via MySQL Router
  • MySQL Shell also requires Python 2.7 and above to run cluster provisioning scripts
  • AdminAPI, which enables you to create and administer an InnoDB cluster, using either JavaScript or Python scripting
  • Caches the metadata of the InnoDB cluster and performs high availability routing to the MySQL Server instances which make up the cluster
  • Group Replication mechanism to allow data to be replicated from the primary to the secondaries in the cluster
  • AdminAPI is available as of MySQL Shell 1.0.8.
張 旭

[Elasticsearch] 分散式特性 & 分散式搜尋的機制 | 小信豬的原始部落 - 0 views

  • 水平擴展儲存空間
  • Data HA:若有 node 掛掉,資料不會遺失
  • 若是要查詢 cluster 中的 node 狀態,可以使用 GET /_cat/nodes API
  • ...39 more annotations...
  • 決定每個 shard 要被分配到哪個 data node 上
  • 為 cluster 設置多個 master node
  • 一旦發現被選中的 master node 出現問題,就會選出新的 master node
  • 每個 node 啟動時就預設是一個 master eligible node,可以透過設定 node.master: false 取消此預設設定
  • 處理 request 的 node 稱為 Coordinating Node,其功能是將 request 轉發到合適的 node 上
  • 所有的 node 都預設是 Coordinating Node
  • coordinating node 可以直接接收 search request 並處理,不需要透過 master node 轉過來
  • 可以保存資料的 node,每個 node 啟動後都會預設是 data node,可以透過設定 node.data: false 停用 data node 功能
  • 由 master node 決定如何把分片分發到不同的 data node 上
  • 每個 node 上都保存了 cluster state
  • 只有 master 才可以修改 cluster state 並負責同步給其他 node
  • 每個 node 都會詳細紀錄本身的狀態資訊
  • shard 是 Elasticsearch 分散式儲存的基礎,包含 primary shard & replica shard
  • 每一個 shard 就是一個 Lucene instance
  • primary shard 功能是將一份被索引後的資料,分散到多個 data node 上存放,實現儲存方面的水平擴展
  • primary shard 的數量在建立 index 時就會指定,後續是無法修改的,若要修改就必須要進行 reindex
  • 當 primary shard 遺失時,replica shard 就可以被 promote 成 primary shard 來保持資料完整性
  • replica shard 數量可以動態調整,讓每個 data node 上都有完整的資料
  • ES 7.0 開始,primary shard 預設為 1,replica shard 預設為 0
  • replica shard 若設定過多,會降低 cluster 整體的寫入效能
  • replica shard 必須和 primary shard 被分配在不同的 data node 上
  • 所有的 primary shard 可以在同一個 data node 上
  • 透過 GET _cluster/health/<target> 可以取得目前 cluster 的健康狀態
  • Yellow:表示 primary shard 可以正常分配,但 replica shard 分配有問題
  • 透過 GET /_cat/shards/<target> 可以取得目前的 shard 狀態
  • replica shard 無法被分配,因此 cluster 健康狀態為黃色
  • 若是擔心 reboot 機器造成 failover 動作開始執行,可以設定將 replication 延遲一段時間後再執行(透過調整 settings 中的 index.unassigned.node_left.delayed_timeout 參數),避免無謂的 data copy 動作 (此功能稱為 delay allocation)
  • 集群變紅,代表有 primary shard 丟失,這個時候會影響讀寫。
  • 如果 node 重新回來,會從 translog 中恢復沒有寫入的資料
  • 設定 index settings 之後,primary shard 數量無法隨意變更
  • 不建議直接發送請求到master節點,雖然也會工作,但是大量請求發送到 master,會有潛在的性能問題
  • shard 是 ES 中最小的工作單元
  • shard 是一個 Lucene 的 index
  • 將 Index Buffer 中的內容寫入 Segment,而這寫入的過程就稱為 Refresh
  • 當 document 被 refresh 進入到 segment 之後,就可以被搜尋到了
  • 在進行 refresh 時先將 segment 寫入 cache 以開放查詢
  • 將 document 進行索引時,同時也會寫入 transaction log,且預設都會寫入磁碟中
  • 每個 shard 都會有對應的 transaction log
  • 由於 transaction log 都會寫入磁碟中,因此當 node 從故障中恢復時,就會優先讀取 transaction log 來恢復資料
張 旭

鳥哥的 Linux 私房菜 -- 第零章、計算機概論 - 0 views

  • 但因為 CPU 的運算速度比其他的設備都要來的快,又為了要滿足 FSB 的頻率,因此廠商就在 CPU 內部再進行加速, 於是就有所謂的外頻與倍頻了。
  • 中央處理器 (Central Processing Unit, CPU),CPU 為一個具有特定功能的晶片, 裡頭含有微指令集,如果你想要讓主機進行什麼特異的功能,就得要參考這顆 CPU 是否有相關內建的微指令集才可以。
  • CPU 內又可分為兩個主要的單元,分別是: 算數邏輯單元與控制單元。
  • ...63 more annotations...
  • CPU 讀取的資料都是從主記憶體來的! 主記憶體內的資料則是從輸入單元所傳輸進來!而 CPU 處理完畢的資料也必須要先寫回主記憶體中,最後資料才從主記憶體傳輸到輸出單元。
  • 重點在於 CPU 與主記憶體。 特別要看的是實線部分的傳輸方向,基本上資料都是流經過主記憶體再轉出去的!
  • CPU 實際要處理的資料則完全來自於主記憶體 (不管是程式還是一般文件資料)!這是個很重要的概念喔! 這也是為什麼當你的記憶體不足時,系統的效能就很糟糕!
  • 常見到的兩種主要 CPU 架構, 分別是:精簡指令集 (RISC) 與複雜指令集 (CISC) 系統。
  • 微指令集較為精簡,每個指令的執行時間都很短,完成的動作也很單純,指令的執行效能較佳; 但是若要做複雜的事情,就要由多個指令來完成。
  • CISC在微指令集的每個小指令可以執行一些較低階的硬體操作,指令數目多而且複雜, 每條指令的長度並不相同。因為指令執行較為複雜所以每條指令花費的時間較長, 但每條個別指令可以處理的工作較為豐富。
  • 多媒體微指令集:MMX, SSE, SSE2, SSE3, SSE4, AMD-3DNow! 虛擬化微指令集:Intel-VT, AMD-SVM 省電功能:Intel-SpeedStep, AMD-PowerNow! 64/32位元相容技術:AMD-AMD64, Intel-EM64T
  • 若光以效能來說,目前的個人電腦效能已經夠快了,甚至已經比工作站等級以上的電腦運算速度還要快! 但是工作站電腦強調的是穩定不當機,並且運算過程要完全正確,因此工作站以上等級的電腦在設計時的考量與個人電腦並不相同啦
  • 1 Byte = 8 bits
  • 檔案容量使用的是二進位的方式,所以 1 GBytes 的檔案大小實際上為:1024x1024x1024 Bytes 這麼大! 速度單位則常使用十進位,例如 1GHz 就是 1000x1000x1000 Hz 的意思。
  • CPU的運算速度常使用 MHz 或者是 GHz 之類的單位,這個 Hz 其實就是秒分之一
  • 在網路傳輸方面,由於網路使用的是 bit 為單位,因此網路常使用的單位為 Mbps 是 Mbits per second,亦即是每秒多少 Mbit
  • (1)北橋:負責連結速度較快的CPU、主記憶體與顯示卡界面等元件
  • (2)南橋:負責連接速度較慢的裝置介面, 包括硬碟、USB、網路卡等等
  • CPU內部含有微指令集,不同的微指令集會導致CPU工作效率的優劣
  • 時脈就是CPU每秒鐘可以進行的工作次數。 所以時脈越高表示這顆CPU單位時間內可以作更多的事情。
  • 早期的 CPU 架構主要透過北橋來連結系統最重要的 CPU、主記憶體與顯示卡裝置。因為所有的設備都得掉透過北橋來連結,因此每個設備的工作頻率應該要相同。
  • 前端匯流排 (FSB)
  • 外頻指的是CPU與外部元件進行資料傳輸時的速度
  • 倍頻則是 CPU 內部用來加速工作效能的一個倍數
  • 新的 CPU 設計中, 已經將記憶體控制器整合到 CPU 內部,而連結 CPU 與記憶體、顯示卡的控制器的設計,在Intel部份使用 QPI (Quick Path Interconnect) 與 DMI 技術,而 AMD 部份則使用 Hyper Transport 了,這些技術都可以讓 CPU 直接與主記憶體、顯示卡等設備分別進行溝通,而不需要透過外部的連結晶片了。
  • 如何知道主記憶體能提供的資料量呢?此時還是得要藉由 CPU 內的記憶體控制晶片與主記憶體間的傳輸速度『前端匯流排速度(Front Side Bus, FSB)
  • 主記憶體也是有其工作的時脈,這個時脈限制還是來自於 CPU 內的記憶體控制器所決定的。
  • CPU每次能夠處理的資料量稱為字組大小(word size), 字組大小依據CPU的設計而有32位元與64位元。我們現在所稱的電腦是32或64位元主要是依據這個 CPU解析的字組大小而來的
  • 早期的32位元CPU中,因為CPU每次能夠解析的資料量有限, 因此由主記憶體傳來的資料量就有所限制了。這也導致32位元的CPU最多只能支援最大到4GBytes的記憶體。
  • 在每一個 CPU 內部將重要的暫存器 (register) 分成兩群, 而讓程序分別使用這兩群暫存器。
  • 可以有兩個程序『同時競爭 CPU 的運算單元』,而非透過作業系統的多工切換!
  • 大多發現 HT 雖然可以提昇效能,不過,有些情況下卻可能導致效能降低喔!因為,實際上明明就僅有一個運算單元
  • 個人電腦的主記憶體主要元件為動態隨機存取記憶體(Dynamic Random Access Memory, DRAM), 隨機存取記憶體只有在通電時才能記錄與使用,斷電後資料就消失了。因此我們也稱這種RAM為揮發性記憶體。
  • 要啟用雙通道的功能你必須要安插兩支(或四支)主記憶體,這兩支記憶體最好連型號都一模一樣比較好, 這是因為啟動雙通道記憶體功能時,資料是同步寫入/讀出這一對主記憶體中,如此才能夠提升整體的頻寬啊!
  • 第二層快取(L2 cache)整合到CPU內部,因此這個L2記憶體的速度必須要CPU時脈相同。 使用DRAM是無法達到這個時脈速度的,此時就需要靜態隨機存取記憶體(Static Random Access Memory, SRAM)的幫忙了。
  • BIOS(Basic Input Output System)是一套程式,這套程式是寫死到主機板上面的一個記憶體晶片中, 這個記憶體晶片在沒有通電時也能夠將資料記錄下來,那就是唯讀記憶體(Read Only Memory, ROM)。
  • BIOS對於個人電腦來說是非常重要的, 因為他是系統在開機的時候首先會去讀取的一個小程式
  • 由於磁碟盤是圓的,並且透過機器手臂去讀寫資料,磁碟盤要轉動才能夠讓機器手臂讀寫。因此,通常資料寫入當然就是以圓圈轉圈的方式讀寫囉! 所以,當初設計就是在類似磁碟盤同心圓上面切出一個一個的小區塊,這些小區塊整合成一個圓形,讓機器手臂上的讀寫頭去存取。 這個小區塊就是磁碟的最小物理儲存單位,稱之為磁區 (sector),那同一個同心圓的磁區組合成的圓就是所謂的磁軌(track)。 由於磁碟裡面可能會有多個磁碟盤,因此在所有磁碟盤上面的同一個磁軌可以組合成所謂的磁柱 (cylinder)。
  • 原本硬碟的磁區都是設計成 512byte 的容量,但因為近期以來硬碟的容量越來越大,為了減少資料量的拆解,所以新的高容量硬碟已經有 4Kbyte 的磁區設計
  • 拿快閃記憶體去製作成高容量的設備,這些設備的連接界面也是透過 SATA 或 SAS,而且外型還做的跟傳統磁碟一樣
  • 固態硬碟最大的好處是,它沒有馬達不需要轉動,而是透過記憶體直接讀寫的特性,因此除了沒資料延遲且快速之外,還很省電
  • 硬碟主要是利用主軸馬達轉動磁碟盤來存取,因此轉速的快慢會影響到效能
  • 使用作業系統的正常關機方式,才能夠有比較好的硬碟保養啊!因為他會讓硬碟的機械手臂歸回原位啊!
  • I/O位址有點類似每個裝置的門牌號碼,每個裝置都有他自己的位址,一般來說,不能有兩個裝置使用同一個I/O位址, 否則系統就會不曉得該如何運作這兩個裝置了。
  • IRQ就可以想成是各個門牌連接到郵件中心(CPU)的專門路徑囉! 各裝置可以透過IRQ中斷通道來告知CPU該裝置的工作情況,以方便CPU進行工作分配的任務。
  • BIOS為寫入到主機板上某一塊 flash 或 EEPROM 的程式,他可以在開機的時候執行,以載入CMOS當中的參數, 並嘗試呼叫儲存裝置中的開機程式,進一步進入作業系統當中。
  • 電腦都只有記錄0/1而已,甚至記錄的資料都是使用byte/bit等單位來記錄的
  • 常用的英文編碼表為ASCII系統,這個編碼系統中, 每個符號(英文、數字或符號等)都會佔用1bytes的記錄, 因此總共會有28=256種變化
  • 中文字當中的編碼系統早期最常用的就是big5這個編碼表了。 每個中文字會佔用2bytes,理論上最多可以有216=65536,亦即最多可達6萬多個中文字。
  • 國際組織ISO/IEC跳出來制訂了所謂的Unicode編碼系統, 我們常常稱呼的UTF8或萬國碼的編碼
  • CPU其實是具有微指令集的。因此,我們需要CPU幫忙工作時,就得要參考微指令集的內容, 然後撰寫讓CPU讀的懂的指令碼給CPU執行,這樣就能夠讓CPU運作了。
  • 編譯器』來將這些人類能夠寫的程式語言轉譯成為機器能看懂得機器碼
  • 當你需要將運作的資料寫入記憶體中,你就得要自行分配一個記憶體區塊出來讓自己的資料能夠填上去, 所以你還得要瞭解到記憶體的位址是如何定位的,啊!眼淚還是不知不覺的流了下來... 怎麼寫程式這麼麻煩啊!
  • 作業系統(Operating System, OS)其實也是一組程式, 這組程式的重點在於管理電腦的所有活動以及驅動系統中的所有硬體。
  • 作業系統的功能就是讓CPU可以開始判斷邏輯與運算數值、 讓主記憶體可以開始載入/讀出資料與程式碼、讓硬碟可以開始被存取、讓網路卡可以開始傳輸資料、 讓所有周邊可以開始運轉等等。
  • 只有核心有提供的功能,你的電腦系統才能幫你完成!舉例來說,你的核心並不支援TCP/IP的網路協定, 那麼無論你購買了什麼樣的網卡,這個核心都無法提供網路能力的!
  • 核心程式所放置到記憶體當中的區塊是受保護的! 並且開機後就一直常駐在記憶體當中。
  • 作業系統通常會提供一整組的開發介面給工程師來開發軟體! 工程師只要遵守該開發介面那就很容易開發軟體了!
  • 系統呼叫介面(System call interface)
  • 程序管理(Process control)
  • 記憶體管理(Memory management)
  • 檔案系統管理(Filesystem management)
  • 通常核心會提供虛擬記憶體的功能,當記憶體不足時可以提供記憶體置換(swap)的功能
  • 裝置的驅動(Device drivers)
  • 『可載入模組』功能,可以將驅動程式編輯成模組,就不需要重新的編譯核心
  • 驅動程式可以說是作業系統裡面相當重要的一環
  • 作業系統通常會提供一個開發介面給硬體開發商, 讓他們可以根據這個介面設計可以驅動他們硬體的『驅動程式』,如此一來,只要使用者安裝驅動程式後, 自然就可以在他們的作業系統上面驅動這塊顯示卡了。
  •  
    "但因為 CPU 的運算速度比其他的設備都要來的快,又為了要滿足 FSB 的頻率,因此廠商就在 CPU 內部再進行加速, 於是就有所謂的外頻與倍頻了。"
張 旭

How to Benchmark Performance of MySQL & MariaDB Using SysBench | Severalnines - 1 views

  • SysBench is a C binary which uses LUA scripts to execute benchmarks
  • support for parallelization in the LUA scripts, multiple queries can be executed in parallel
  • by default, benchmarks which cover most of the cases - OLTP workloads, read-only or read-write, primary key lookups and primary key updates.
  • ...21 more annotations...
  • SysBench is not a tool which you can use to tune configurations of your MySQL servers (unless you prepared LUA scripts with custom workload or your workload happen to be very similar to the benchmark workloads that SysBench comes with)
  • it is great for is to compare performance of different hardware.
  • Every new server acquired should go through a warm-up period during which you will stress it to pinpoint potential hardware defects
  • by executing OLTP workload which overloads the server, or you can also use dedicated benchmarks for CPU, disk and memory.
  • bulk_insert.lua. This test can be used to benchmark the ability of MySQL to perform multi-row inserts.
  • All oltp_* scripts share a common table structure. First two of them (oltp_delete.lua and oltp_insert.lua) execute single DELETE and INSERT statements.
  • oltp_point_select, oltp_update_index and oltp_update_non_index. These will execute a subset of queries - primary key-based selects, index-based updates and non-index-based updates.
  • you can run different workload patterns using the same benchmark.
  • Warmup helps to identify “regular” throughput by executing benchmark for a predefined time, allowing to warm up the cache, buffer pools etc.
  • By default SysBench will attempt to execute queries as fast as possible. To simulate slower traffic this option may be used. You can define here how many transactions should be executed per second.
  • SysBench gives you ability to generate different types of data distribution.
  • decide if SysBench should use prepared statements (as long as they are available in the given datastore - for MySQL it means PS will be enabled by default) or not.
  • sysbench ./sysbench/src/lua/oltp_read_write.lua  help
  • By default, SysBench will attempt to execute queries in explicit transaction. This way the dataset will stay consistent and not affected: SysBench will, for example, execute INSERT and DELETE on the same row, making sure the data set will not grow (impacting your ability to reproduce results).
  • specify error codes from MySQL which SysBench should ignore (and not kill the connection).
  • the two most popular benchmarks - OLTP read only and OLTP read/write.
  • 1 million rows will result in ~240 MB of data. Ten tables, 1000 000 rows each equals to 2.4GB
  • by default, SysBench looks for ‘sbtest’ schema which has to exist before you prepare the data set. You may have to create it manually.
  • pass ‘--histogram’ argument to SysBench
  • ~48GB of data (20 tables, 10 000 000 rows each).
  • if you don’t understand why the performance was like it was, you may draw incorrect conclusions out of the benchmarks.
張 旭

Introducing CNAME Flattening: RFC-Compliant CNAMEs at a Domain's Root - 0 views

  • you can now safely use a CNAME record, as opposed to an A record that points to a fixed IP address, as your root record in CloudFlare DNS without triggering a number of edge case error conditions because you’re violating the DNS spec.
  • CNAME Flattening allowed us to use a root domain while still maintaining DNS fault-tolerance across multiple IP addresses.
  • Traditionally, the root record of a domain needed to point to an IP address (known as an A -- for "address" -- Record).
  • ...13 more annotations...
  • WordPlumblr allows its users to use custom domains that point to the WordPlumblr infrastructure
  • A CNAME is an alias. It allows one domain to point to another domain which, eventually if you follow the CNAME chain, will resolve to an A record and IP address.
  • For example, WordPlumblr might have assigned the CNAME 6equj5.wordplumblr.com for Foo.com. Foo.com and the other customers may have all initially resolved, at the end of the CNAME chain, to the same IP address.
  • you usually don't want to address memory directly but, instead, you set up a pointer to a block of memory where you're going to store something. If the operating system needs to move the memory around then it just updates the pointer to point to wherever the chunk of memory has been moved to.
  • CNAMEs work great for subdomains like www.foo.com or blog.foo.com. Unfortunately, they don't work for a naked domain like foo.com itself.
  • the DNS spec enshrined that the root record -- the naked domain without any subdomain -- could not be a CNAME.
  • Technically, the root could be a CNAME but the RFCs state that once a record has a CNAME it can't have any other entries associated with it
  • a way to support a CNAME at the root, but still follow the RFC and return an IP address for any query for the root record.
  • extended our authoritative DNS infrastructure to, in certain cases, act as a kind of DNS resolver.
  • if there's a CNAME at the root, rather than returning that record directly we recurse through the CNAME chain ourselves until we find an A Record.
  • allows the flexibility of having CNAMEs at the root without breaking the DNS specification.
  • We cache the CNAME responses -- respecting the DNS TTLs, just like a recursor should -- which means often we have the answer without having to traverse the chain.
  • CNAME flattening solved email resolution errors for us which was very key.
張 旭

Dependency Lock File (.terraform.lock.hcl) - Configuration Language | Terraform | Hashi... - 0 views

  • Version constraints within the configuration itself determine which versions of dependencies are potentially compatible, but after selecting a specific version of each dependency Terraform remembers the decisions it made in a dependency lock file
  • At present, the dependency lock file tracks only provider dependencies.
  • Terraform does not remember version selections for remote modules, and so Terraform will always select the newest available module version that meets the specified version constraints.
  • ...14 more annotations...
  • The lock file is always named .terraform.lock.hcl, and this name is intended to signify that it is a lock file for various items that Terraform caches in the .terraform
  • Terraform automatically creates or updates the dependency lock file each time you run the terraform init command.
  • You should include this file in your version control repository
  • If a particular provider has no existing recorded selection, Terraform will select the newest available version that matches the given version constraint, and then update the lock file to include that selection.
  • the "trust on first use" model
  • you can pre-populate checksums for a variety of different platforms in your lock file using the terraform providers lock command, which will then allow future calls to terraform init to verify that the packages available in your chosen mirror match the official packages from the provider's origin registry.
  • The h1: and zh: prefixes on these values represent different hashing schemes, each of which represents calculating a checksum using a different algorithm.
  • zh:: a mnemonic for "zip hash"
  • h1:: a mnemonic for "hash scheme 1", which is the current preferred hashing scheme.
  • To determine whether there still exists a dependency on a given provider, Terraform uses two sources of truth: the configuration itself, and the state.
  • Version constraints within the configuration itself determine which versions of dependencies are potentially compatible, but after selecting a specific version of each dependency Terraform remembers the decisions it made in a dependency lock file so that it can (by default) make the same decisions again in future.
  • At present, the dependency lock file tracks only provider dependencies.
  • Terraform will always select the newest available module version that meets the specified version constraints.
  • The lock file is always named .terraform.lock.hcl
  •  
    "the overriding effect is compounded, with later blocks taking precedence over earlier blocks."
« First ‹ Previous 41 - 49 of 49
Showing 20 items per page