Skip to main content

Home/ Larvata/ Group items tagged HA

Rss Feed Group items tagged

Filter: All | Bookmarks | Topics Simple Middle
張 旭

MongoDB Performance - MongoDB Manual - 0 views

docs.mongodb.com/...analyzing-mongodb-performance

database mongodb performance

shared by 張 旭 on 20 Apr 21 - No Cached
  • MongoDB uses a locking system to ensure data set consistency. If certain operations are long-running or a queue forms, performance will degrade as requests and operations wait for the lock.
  • performance limitations as a result of inadequate or inappropriate indexing strategies, or as a consequence of poor schema design patterns.
  • performance issues may be temporary and related to abnormal traffic load.
  • ...9 more annotations...
  • Lock-related slowdowns can be intermittent.
  • If globalLock.currentQueue.total is consistently high, then there is a chance that a large number of requests are waiting for a lock.
  • If globalLock.totalTime is high relative to uptime, the database has existed in a lock state for a significant amount of time.
  • For write-heavy applications, deploy sharding and add one or more shards to a sharded cluster to distribute load among mongod instances.
  • Unless constrained by system-wide limits, the maximum number of incoming connections supported by MongoDB is configured with the maxIncomingConnections setting.
  • When logLevel is set to 0, MongoDB records slow operations to the diagnostic log at a rate determined by slowOpSampleRate.
  • At higher logLevel settings, all operations appear in the diagnostic log regardless of their latency with the following exception
  • Full Time Diagnostic Data Collection (FTDC) mechanism. FTDC data files are compressed, are not human-readable, and inherit the same file access permissions as the MongoDB data files.
  • mongod processes store FTDC data files in a diagnostic.data directory under the instances storage.dbPath.
  • 張 旭 on 20 Apr 21
     
    "MongoDB uses a locking system to ensure data set consistency. If certain operations are long-running or a queue forms, performance will degrade as requests and operations wait for the lock."
張 旭

Production Notes - MongoDB Manual - 0 views

docs.mongodb.com/...production-notes

mongodb database performance

shared by 張 旭 on 29 Apr 21 - No Cached
  • mongod will not start if dbPath contains data files created by a storage engine other than the one specified by --storageEngine.
  • mongod must possess read and write permissions for the specified dbPath.
  • WiredTiger supports concurrent access by readers and writers to the documents in a collection
  • ...9 more annotations...
  • Journaling guarantees that MongoDB can quickly recover write operations that were written to the journal but not written to data files in cases where mongod terminated due to a crash or other serious failure.
  • To use read concern level of "majority", replica sets must use WiredTiger storage engine.
  • Write concern describes the level of acknowledgement requested from MongoDB for write operations.
  • With stronger write concerns, clients must wait after sending a write operation until MongoDB confirms the write operation at the requested write concern level.
  • By default, authorization is not enabled, and mongod assumes a trusted environment
  • The HTTP interface is disabled by default. Do not enable the HTTP interface in production environments.
  • Avoid overloading the connection resources of a mongod or mongos instance by adjusting the connection pool size to suit your use case.
  • ensure that each mongod or mongos instance has access to two real cores or one multi-core physical CPU.
  • The WiredTiger storage engine is multithreaded and can take advantage of additional CPU cores
張 旭

ALB vs ELB | Differences Between an ELB and an ALB on AWS | Sumo Logic - 0 views

www.sumologic.com/...aws-elb-alb

aws HA proxy

shared by 張 旭 on 06 Apr 21 - No Cached
  • If you use AWS, you have two load-balancing options: ELB and ALB.
  • An ELB is a software-based load balancer which can be set up and configured in front of a collection of AWS Elastic Compute (EC2) instances.
  • The load balancer serves as a single entry point for consumers of the EC2 instances and distributes incoming traffic across all machines available to receive requests.
  • ...14 more annotations...
  • the ELB also performs a vital role in improving the fault tolerance of the services which it fronts.
  • he Open Systems Interconnection Model, or OSI Model, is a conceptual model which is used to facilitate communications between different computing systems.
  • Layer 1 is the physical layer, and represents the physical medium across which the request is sent.
  • Layer 2 describes the data link layer
  • Layer 3 (the network layer)
  • Layer 7, which serves the application layer.
  • The Classic ELB operates at Layer 4. Layer 4 represents the transport layer, and is controlled by the protocol being used to transmit the request.
  • A network device, of which the Classic ELB is an example, reads the protocol and port of the incoming request, and then routes it to one or more backend servers.
  • the ALB operates at Layer 7. Layer 7 represents the application layer, and as such allows for the redirection of traffic based on the content of the request.
  • Whereas a request to a specific URL backed by a Classic ELB would only enable routing to a particular pool of homogeneous servers, the ALB can route based on the content of the URL, and direct to a specific subgroup of backing servers existing in a heterogeneous collection registered with the load balancer.
  • The Classic ELB is a simple load balancer, is easy to configure
  • As organizations move towards microservice architecture or adopt a container-based infrastructure, the ability to merely map a single address to a specific service becomes more complicated and harder to maintain.
  • the ALB manages routing based on user-defined rules.
  • oute traffic to different services based on either the host or the content of the path contained within that URL.
張 旭

Kubernetes Components | Kubernetes - 0 views

kubernetes.io/...components

kubernetes system

shared by 張 旭 on 17 May 21 - No Cached
  • A Kubernetes cluster consists of a set of worker machines, called nodes, that run containerized applications
  • Every cluster has at least one worker node.
  • The control plane manages the worker nodes and the Pods in the cluster.
  • ...29 more annotations...
  • The control plane's components make global decisions about the cluster
  • Control plane components can be run on any machine in the cluster.
  • for simplicity, set up scripts typically start all control plane components on the same machine, and do not run user containers on this machine
  • The API server is the front end for the Kubernetes control plane.
  • kube-apiserver is designed to scale horizontally—that is, it scales by deploying more instances. You can run several instances of kube-apiserver and balance traffic between those instances.
  • Kubernetes cluster uses etcd as its backing store, make sure you have a back up plan for those data.
  • watches for newly created Pods with no assigned node, and selects a node for them to run on.
  • Factors taken into account for scheduling decisions include: individual and collective resource requirements, hardware/software/policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference, and deadlines.
  • each controller is a separate process, but to reduce complexity, they are all compiled into a single binary and run in a single process.
  • Node controller
  • Job controller
  • Endpoints controller
  • Service Account & Token controllers
  • The cloud controller manager lets you link your cluster into your cloud provider's API, and separates out the components that interact with that cloud platform from components that only interact with your cluster.
  • If you are running Kubernetes on your own premises, or in a learning environment inside your own PC, the cluster does not have a cloud controller manager.
  • An agent that runs on each node in the cluster. It makes sure that containers are running in a Pod.
  • The kubelet takes a set of PodSpecs that are provided through various mechanisms and ensures that the containers described in those PodSpecs are running and healthy.
  • The kubelet doesn't manage containers which were not created by Kubernetes.
  • kube-proxy is a network proxy that runs on each node in your cluster, implementing part of the Kubernetes Service concept.
  • kube-proxy maintains network rules on nodes. These network rules allow network communication to your Pods from network sessions inside or outside of your cluster.
  • kube-proxy uses the operating system packet filtering layer if there is one and it's available.
  • Kubernetes supports several container runtimes: Docker, containerd, CRI-O, and any implementation of the Kubernetes CRI (Container Runtime Interface).
  • Addons use Kubernetes resources (DaemonSet, Deployment, etc) to implement cluster features
  • namespaced resources for addons belong within the kube-system namespace.
  • all Kubernetes clusters should have cluster DNS,
  • Cluster DNS is a DNS server, in addition to the other DNS server(s) in your environment, which serves DNS records for Kubernetes services.
  • Containers started by Kubernetes automatically include this DNS server in their DNS searches.
  • Container Resource Monitoring records generic time-series metrics about containers in a central database, and provides a UI for browsing that data.
  • A cluster-level logging mechanism is responsible for saving container logs to a central log store with search/browsing interface.
張 旭

Think Before you NodePort in Kubernetes - Oteemo - 0 views

oteemo.com/think-nodeport-kubernetes

kubernetes networking

shared by 張 旭 on 23 Jun 21 - No Cached
  • Two options are provided for Services intended for external use: a NodePort, or a LoadBalancer
  • no built-in cloud load balancers for Kubernetes in bare-metal environments
  • NodePort may not be your best choice.
  • ...15 more annotations...
  • NodePort, by design, bypasses almost all network security in Kubernetes.
  • NetworkPolicy resources can currently only control NodePorts by allowing or disallowing all traffic on them.
  • put a network filter in front of all the nodes
  • if a Nodeport-ranged Service is advertised to the public, it may serve as an invitation to black-hats to scan and probe
  • When Kubernetes creates a NodePort service, it allocates a port from a range specified in the flags that define your Kubernetes cluster. (By default, these are ports ranging from 30000-32767.)
  • By design, Kubernetes NodePort cannot expose standard low-numbered ports like 80 and 443, or even 8080 and 8443.
  • A port in the NodePort range can be specified manually, but this would mean the creation of a list of non-standard ports, cross-referenced with the applications they map to
  • if you want the exposed application to be highly available, everything contacting the application has to know all of your node addresses, or at least more than one.
  • non-standard ports.
  • Ingress resources use an Ingress controller (the nginx one is common but not by any means the only choice) and an external load balancer or public IP to enable path-based routing of external requests to internal Services.
  • With a single point of entry to expose and secure
  • get simpler TLS management!
  • consider putting a real load balancer in front of your NodePort Services before opening them up to the world
  • Google very recently released an alpha-stage bare-metal load balancer that, once installed in your cluster, will load-balance using BGP
  • NodePort Services are easy to create but hard to secure, hard to manage, and not especially friendly to others
張 旭

The differences between Docker, containerd, CRI-O and runc - Tutorial Works - 0 views

www.tutorialworks.com/ocker-containerd-runc-crio-oci

docker container devops

shared by 張 旭 on 28 Jun 21 - No Cached
  • Docker isn’t the only container contender on the block.
  • Container Runtime Interface (CRI), which defines an API between Kubernetes and the container runtime
  • Open Container Initiative (OCI) which publishes specifications for images and containers.
  • ...20 more annotations...
  • for a lot of people, the name “Docker” itself is synonymous with the word “container”.
  • Docker created a very ergonomic (nice-to-use) tool for working with containers – also called docker.
  • docker is designed to be installed on a workstation or server and comes with a bunch of tools to make it easy to build and run containers as a developer, or DevOps person.
  • containerd: This is a daemon process that manages and runs containers.
  • runc: This is the low-level container runtime (the thing that actually creates and runs containers).
  • libcontainer, a native Go-based implementation for creating containers.
  • Kubernetes includes a component called dockershim, which allows it to support Docker.
  • Kubernetes prefers to run containers through any container runtime which supports its Container Runtime Interface (CRI).
  • Kubernetes will remove support for Docker directly, and prefer to use only container runtimes that implement its Container Runtime Interface.
  • Both containerd and CRI-O can run Docker-formatted (actually OCI-formatted) images, they just do it without having to use the docker command or the Docker daemon.
  • Docker images, are actually images packaged in the Open Container Initiative (OCI) format.
  • CRI is the API that Kubernetes uses to control the different runtimes that create and manage containers.
  • CRI makes it easier for Kubernetes to use different container runtimes
  • containerd is a high-level container runtime that came from Docker, and implements the CRI spec
  • containerd was separated out of the Docker project, to make Docker more modular.
  • CRI-O is another high-level container runtime which implements the Container Runtime Interface (CRI).
  • The idea behind the OCI is that you can choose between different runtimes which conform to the spec.
  • runc is an OCI-compatible container runtime.
  • A reference implementation is a piece of software that has implemented all the requirements of a specification or standard.
  • runc provides all of the low-level functionality for containers, interacting with existing low-level Linux features, like namespaces and control groups.
張 旭

Kubernetes Deployments: The Ultimate Guide - Semaphore - 1 views

semaphoreci.com/...kubernetes-deployment

docker kubernetes devops system

shared by 張 旭 on 21 Oct 19 - No Cached
  • Continuous integration gives you confidence in your code. To extend that confidence to the release process, your deployment operations need to come with a safety belt.
  • these Kubernetes objects ensure that you can progressively deploy, roll back and scale your applications without downtime.
  • A pod is just a group of containers (it can be a group of one container) that run on the same machine, and share a few things together.
  • ...34 more annotations...
  • the containers within a pod can communicate with each other over localhost
  • From a network perspective, all the processes in these containers are local.
  • we can never create a standalone container: the closest we can do is create a pod, with a single container in it.
  • Kubernetes is a declarative system (by opposition to imperative systems).
  • All we can do, is describe what we want to have, and wait for Kubernetes to take action to reconcile what we have, with what we want to have.
  • In other words, we can say, “I would like a 40-feet long blue container with yellow doors“, and Kubernetes will find such a container for us. If it doesn’t exist, it will build it; if there is already one but it’s green with red doors, it will paint it for us; if there is already a container of the right size and color, Kubernetes will do nothing, since what we have already matches what we want.
  • The specification of a replica set looks very much like the specification of a pod, except that it carries a number, indicating how many replicas
  • What happens if we change that definition? Suddenly, there are zero pods matching the new specification.
  • the creation of new pods could happen in a more gradual manner.
  • the specification for a deployment looks very much like the one for a replica set: it features a pod specification, and a number of replicas.
  • Deployments, however, don’t create or delete pods directly.
  • When we update a deployment and adjust the number of replicas, it passes that update down to the replica set.
  • When we update the pod specification, the deployment creates a new replica set with the updated pod specification. That replica set has an initial size of zero. Then, the size of that replica set is progressively increased, while decreasing the size of the other replica set.
  • we are going to fade in (turn up the volume) on the new replica set, while we fade out (turn down the volume) on the old one.
  • During the whole process, requests are sent to pods of both the old and new replica sets, without any downtime for our users.
  • A readiness probe is a test that we add to a container specification.
  • Kubernetes supports three ways of implementing readiness probes:Running a command inside a container;Making an HTTP(S) request against a container; orOpening a TCP socket against a container.
  • When we roll out a new version, Kubernetes will wait for the new pod to mark itself as “ready” before moving on to the next one.
  • If there is no readiness probe, then the container is considered as ready, as long as it could be started.
  • MaxSurge indicates how many extra pods we are willing to run during a rolling update, while MaxUnavailable indicates how many pods we can lose during the rolling update.
  • Setting MaxUnavailable to 0 means, “do not shutdown any old pod before a new one is up and ready to serve traffic“.
  • Setting MaxSurge to 100% means, “immediately start all the new pods“, implying that we have enough spare capacity on our cluster, and that we want to go as fast as possible.
  • kubectl rollout undo deployment web
  • the replica set doesn’t look at the pods’ specifications, but only at their labels.
  • A replica set contains a selector, which is a logical expression that “selects” (just like a SELECT query in SQL) a number of pods.
  • it is absolutely possible to manually create pods with these labels, but running a different image (or with different settings), and fool our replica set.
  • Selectors are also used by services, which act as the load balancers for Kubernetes traffic, internal and external.
  • internal IP address (denoted by the name ClusterIP)
  • during a rollout, the deployment doesn’t reconfigure or inform the load balancer that pods are started and stopped. It happens automatically through the selector of the service associated to the load balancer.
  • a pod is added as a valid endpoint for a service only if all its containers pass their readiness check. In other words, a pod starts receiving traffic only once it’s actually ready for it.
  • In blue/green deployment, we want to instantly switch over all the traffic from the old version to the new, instead of doing it progressively
  • We can achieve blue/green deployment by creating multiple deployments (in the Kubernetes sense), and then switching from one to another by changing the selector of our service
  • kubectl label pods -l app=blue,version=v1.5 status=enabled
  • kubectl label pods -l app=blue,version=v1.4 status-
  • 張 旭 on 21 Oct 19
     
    "Continuous integration gives you confidence in your code. To extend that confidence to the release process, your deployment operations need to come with a safety belt."
張 旭

Helm | - 0 views

helm.sh/...chart_template_guide

helm kubernetes chart devops development

shared by 張 旭 on 06 Aug 19 - No Cached
  • Templates generate manifest files, which are YAML-formatted resource descriptions that Kubernetes can understand.
  • service.yaml: A basic manifest for creating a service endpoint for your deployment
  • In Kubernetes, a ConfigMap is simply a container for storing configuration data.
  • ...88 more annotations...
  • deployment.yaml: A basic manifest for creating a Kubernetes deployment
  • using the suffix .yaml for YAML files and .tpl for helpers.
  • It is just fine to put a plain YAML file like this in the templates/ directory.
  • helm get manifest
  • The helm get manifest command takes a release name (full-coral) and prints out all of the Kubernetes resources that were uploaded to the server. Each file begins with --- to indicate the start of a YAML document
  • Names should be unique to a release
  • The name: field is limited to 63 characters because of limitations to the DNS system.
  • release names are limited to 53 characters
  • {{ .Release.Name }}
  • A template directive is enclosed in {{ and }} blocks.
  • The values that are passed into a template can be thought of as namespaced objects, where a dot (.) separates each namespaced element.
  • The leading dot before Release indicates that we start with the top-most namespace for this scope
  • The Release object is one of the built-in objects for Helm
  • When you want to test the template rendering, but not actually install anything, you can use helm install ./mychart --debug --dry-run
  • Using --dry-run will make it easier to test your code, but it won’t ensure that Kubernetes itself will accept the templates you generate.
  • Objects are passed into a template from the template engine.
  • create new objects within your templates
  • Objects can be simple, and have just one value. Or they can contain other objects or functions.
  • Release is one of the top-level objects that you can access in your templates.
  • Release.Namespace: The namespace to be released into (if the manifest doesn’t override)
  • Values: Values passed into the template from the values.yaml file and from user-supplied files. By default, Values is empty.
  • Chart: The contents of the Chart.yaml file.
  • Files: This provides access to all non-special files in a chart.
  • Files.Get is a function for getting a file by name
  • Files.GetBytes is a function for getting the contents of a file as an array of bytes instead of as a string. This is useful for things like images.
  • Template: Contains information about the current template that is being executed
  • BasePath: The namespaced path to the templates directory of the current chart
  • The built-in values always begin with a capital letter.
  • Go’s naming convention
  • use only initial lower case letters in order to distinguish local names from those built-in.
  • If this is a subchart, the values.yaml file of a parent chart
  • Individual parameters passed with --set
  • values.yaml is the default, which can be overridden by a parent chart’s values.yaml, which can in turn be overridden by a user-supplied values file, which can in turn be overridden by --set parameters.
  • While structuring data this way is possible, the recommendation is that you keep your values trees shallow, favoring flatness.
  • If you need to delete a key from the default values, you may override the value of the key to be null, in which case Helm will remove the key from the overridden values merge.
  • Kubernetes would then fail because you can not declare more than one livenessProbe handler.
  • When injecting strings from the .Values object into the template, we ought to quote these strings.
  • quote
  • Template functions follow the syntax functionName arg1 arg2...
  • While we talk about the “Helm template language” as if it is Helm-specific, it is actually a combination of the Go template language, some extra functions, and a variety of wrappers to expose certain objects to the templates.
  • Drawing on a concept from UNIX, pipelines are a tool for chaining together a series of template commands to compactly express a series of transformations.
  • pipelines are an efficient way of getting several things done in sequence
  • The repeat function will echo the given string the given number of times
  • default DEFAULT_VALUE GIVEN_VALUE. This function allows you to specify a default value inside of the template, in case the value is omitted.
  • all static default values should live in the values.yaml, and should not be repeated using the default command
  • Operators are implemented as functions that return a boolean value.
  • To use eq, ne, lt, gt, and, or, not etcetera place the operator at the front of the statement followed by its parameters just as you would a function.
  • if and
  • if or
  • with to specify a scope
  • range, which provides a “for each”-style loop
  • block declares a special kind of fillable template area
  • A pipeline is evaluated as false if the value is: a boolean false a numeric zero an empty string a nil (empty or null) an empty collection (map, slice, tuple, dict, array)
  • incorrect YAML because of the whitespacing
  • When the template engine runs, it removes the contents inside of {{ and }}, but it leaves the remaining whitespace exactly as is.
  • {{- (with the dash and space added) indicates that whitespace should be chomped left, while -}} means whitespace to the right should be consumed.
  • Newlines are whitespace!
  • an * at the end of the line indicates a newline character that would be removed
  • Be careful with the chomping modifiers.
  • the indent function
  • Scopes can be changed. with can allow you to set the current scope (.) to a particular object.
  • Inside of the restricted scope, you will not be able to access the other objects from the parent scope.
  • range
  • The range function will “range over” (iterate through) the pizzaToppings list.
  • Just like with sets the scope of ., so does a range operator.
  • The toppings: |- line is declaring a multi-line string.
  • not a YAML list. It’s a big string.
  • the data in ConfigMaps data is composed of key/value pairs, where both the key and the value are simple strings.
  • The |- marker in YAML takes a multi-line string.
  • range can be used to iterate over collections that have a key and a value (like a map or dict).
  • In Helm templates, a variable is a named reference to another object. It follows the form $name
  • Variables are assigned with a special assignment operator: :=
  • {{- $relname := .Release.Name -}}
  • capture both the index and the value
  • the integer index (starting from zero) to $index and the value to $topping
  • For data structures that have both a key and a value, we can use range to get both
  • Variables are normally not “global”. They are scoped to the block in which they are declared.
  • one variable that is always global - $ - this variable will always point to the root context.
  • $.
  • $.
  • Helm template language is its ability to declare multiple templates and use them together.
  • A named template (sometimes called a partial or a subtemplate) is simply a template defined inside of a file, and given a name.
  • when naming templates: template names are global.
  • If you declare two templates with the same name, whichever one is loaded last will be the one used.
  • you should be careful to name your templates with chart-specific names.
  • templates in subcharts are compiled together with top-level templates
  • naming convention is to prefix each defined template with the name of the chart: {{ define "mychart.labels" }}
  • Helm has over 60 available functions.
張 旭

Load balancing with ProxySQL - 0 views

www.percona.com/...proxysql.html

proxysql mysql database HA

shared by 張 旭 on 07 Nov 19 - No Cached
  • accepts incoming traffic from MySQL clients and forwards it to backend MySQL servers.
張 旭

A visual guide on troubleshooting Kubernetes deployments - 0 views

learnk8s.io/troubleshooting-deployments

kubernetes pod container service HA proxy

shared by 張 旭 on 06 Dec 19 - No Cached
  • Service and Deployment aren't connected at all.
  • the Service points to the Pods directly and skips the Deployment altogether.
張 旭

HowTo/LDAP - FreeIPA - 0 views

www.freeipa.org/LDAP

ldap authorization authentication auth security

shared by 張 旭 on 03 Dec 19 - No Cached
  • The basedn in an IPA installation consists of a set of domain components (dc) for the initial domain that IPA was configured with.
  • You will only ever have one basedn, the one defined during installation.
  • find your basedn, and other interesting things, in /etc/ipa/default.conf
  • ...8 more annotations...
  • IPA uses a flat structure, storing like objects in what we call containers.
  • Users: cn=users,cn=accounts,$SUFFIX Groups: cn=groups,cn=accounts,$SUFFIX
  • Do not use the Directory Manager account to authenticate remote services to the IPA LDAP server. Use a system account
  • The reason to use an account like this rather than creating a normal user account in IPA and using that is that the system account exists only for binding to LDAP. It is not a real POSIX user, can't log into any systems and doesn't own any files.
  • This use also has no special rights and is unable to write any data in the IPA LDAP server, only read.
  • When possible, configure your LDAP client to communicate over SSL/TLS.
  • The IPA CA certificate can be found in /etc/ipa/ca.crt
  • /etc/openldap/ldap.conf
張 旭

How to Benchmark Performance of MySQL & MariaDB Using SysBench | Severalnines - 1 views

severalnines.com/...e-mysql-mariadb-using-sysbench

database mysql performance

shared by 張 旭 on 22 Oct 20 - No Cached
  • SysBench is a C binary which uses LUA scripts to execute benchmarks
  • support for parallelization in the LUA scripts, multiple queries can be executed in parallel
  • by default, benchmarks which cover most of the cases - OLTP workloads, read-only or read-write, primary key lookups and primary key updates.
  • ...21 more annotations...
  • SysBench is not a tool which you can use to tune configurations of your MySQL servers (unless you prepared LUA scripts with custom workload or your workload happen to be very similar to the benchmark workloads that SysBench comes with)
  • it is great for is to compare performance of different hardware.
  • Every new server acquired should go through a warm-up period during which you will stress it to pinpoint potential hardware defects
  • by executing OLTP workload which overloads the server, or you can also use dedicated benchmarks for CPU, disk and memory.
  • bulk_insert.lua. This test can be used to benchmark the ability of MySQL to perform multi-row inserts.
  • All oltp_* scripts share a common table structure. First two of them (oltp_delete.lua and oltp_insert.lua) execute single DELETE and INSERT statements.
  • oltp_point_select, oltp_update_index and oltp_update_non_index. These will execute a subset of queries - primary key-based selects, index-based updates and non-index-based updates.
  • you can run different workload patterns using the same benchmark.
  • Warmup helps to identify “regular” throughput by executing benchmark for a predefined time, allowing to warm up the cache, buffer pools etc.
  • By default SysBench will attempt to execute queries as fast as possible. To simulate slower traffic this option may be used. You can define here how many transactions should be executed per second.
  • SysBench gives you ability to generate different types of data distribution.
  • decide if SysBench should use prepared statements (as long as they are available in the given datastore - for MySQL it means PS will be enabled by default) or not.
  • sysbench ./sysbench/src/lua/oltp_read_write.lua  help
  • By default, SysBench will attempt to execute queries in explicit transaction. This way the dataset will stay consistent and not affected: SysBench will, for example, execute INSERT and DELETE on the same row, making sure the data set will not grow (impacting your ability to reproduce results).
  • specify error codes from MySQL which SysBench should ignore (and not kill the connection).
  • the two most popular benchmarks - OLTP read only and OLTP read/write.
  • 1 million rows will result in ~240 MB of data. Ten tables, 1000 000 rows each equals to 2.4GB
  • by default, SysBench looks for ‘sbtest’ schema which has to exist before you prepare the data set. You may have to create it manually.
  • pass ‘--histogram’ argument to SysBench
  • ~48GB of data (20 tables, 10 000 000 rows each).
  • if you don’t understand why the performance was like it was, you may draw incorrect conclusions out of the benchmarks.
張 旭

podman/rootless.md at master · containers/podman - 0 views

github.com/...rootless.md

docker kubernetes

shared by 張 旭 on 03 Aug 20 - No Cached
  • Podman can not create containers that bind to ports < 1024
  • If /etc/subuid and /etc/subgid are not setup for a user, then podman commands can easily fail
  • Fedora 31 defaults to cgroup V2, which has full support of rootless cgroup management.
  • ...6 more annotations...
  • Some system unit configuration options do not work in the rootless container
  • it's better to create an override.conf drop-in that sets PrivateNetwork=no
  • Difficult to use additional stores for sharing content
  • Can not use overlayfs driver, but does support fuse-overlayfs
  • No CNI Support
  • Making device nodes within a container fails, even when running --privileged.
張 旭

mvn clean install - a short guide to Maven - 0 views

www.marcobehler.com/...install-a-short-guide-to-maven

programming java

shared by 張 旭 on 12 Aug 20 - No Cached
  • An equivalent in other languages would be Javascript’s npm, Ruby’s gems or PHP’s composer.
  • Maven expects a certain directory structure for your Java source code to live in and when you later do a mvn clean install , the whole compilation and packaging work will be done for you.
  • any directory that contains a pom.xml file is also a valid Maven project.
  • ...17 more annotations...
  • A pom.xml file contains everything needed to describe your Java project.
  • Java source code is to be meant to live in the "/src/main/java" folder
  • Maven will put compiled Java classes into the "target/classes" folder
  • Maven will also build a .jar or .war file, depending on your project, that lives in the "target" folder.
  • Maven has the concept of a build lifecycle, which is made up of different phases.
  • clean is not part of Maven’s default lifecycle, you end up with commands like mvn clean install or mvn clean package. Install or package will trigger all preceding phases, but you need to specify clean in addition.
  • Maven will always download your project dependencies into your local maven repository first and then reference them for your build.
  • local repositories (in your user’s home directory: ~/.m2/)
  • clean: deletes the /target folder.
  • mvn clean package
  • mvn clean install
  • package: Converts your .java source code into a .jar/.war file and puts it into the /target folder.
  • install: First, it does a package(!). Then it takes that .jar/.war file and puts it into your local Maven repository, which lives in ~/.m2/repository.
  • calling 'mvn install' would be enough if Maven was smart enough to do reliable, incremental builds.
  • figuring out what Java source files/modules changed and only compile those.
  • developers got it ingrained to always call 'mvn clean install' (even though this increases build time a lot in bigger projects).
  • make sure that Maven always tries to download the latest snapshot dependency versions
張 旭

Helm | Values Files - 0 views

helm.sh/...values_files

helm

shared by 張 旭 on 02 Oct 21 - No Cached
  • a subchart, the values.yaml file of a parent chart
  • Individual parameters passed with --set
  • The list above is in order of specificity: values.yaml is the default, which can be overridden by a parent chart's values.yaml, which can in turn be overridden by a user-supplied values file, which can in turn be overridden by --set parameters.
  • ...4 more annotations...
  • --set has a higher precedence than the default values.yaml file
  • Values files can contain more structured content
  • If you need to delete a key from the default values, you may override the value of the key to be null, in which case Helm will remove the key from the overridden values merge.
  • Kubernetes would then fail because you can not declare more than one livenessProbe handler.
張 旭

Helm | Template Functions and Pipelines - 0 views

helm.sh/...functions_and_pipelines

helm

shared by 張 旭 on 02 Oct 21 - No Cached
  • When injecting strings from the .Values object into the template, we ought to quote these strings.
  • Helm has over 60 available functions. Some of them are defined by the Go template language itself. Most of the others are part of the Sprig template library
  • the "Helm template language" as if it is Helm-specific, it is actually a combination of the Go template language, some extra functions, and a variety of wrappers to expose certain objects to the templates.
  • ...10 more annotations...
  • Drawing on a concept from UNIX, pipelines are a tool for chaining together a series of template commands to compactly express a series of transformations.
  • the default function: default DEFAULT_VALUE GIVEN_VALUE
  • all static default values should live in the values.yaml, and should not be repeated using the default command (otherwise they would be redundant).
  • the default command is perfect for computed values, which can not be declared inside values.yaml.
  • When lookup returns an object, it will return a dictionary.
  • The synopsis of the lookup function is lookup apiVersion, kind, namespace, name -> resource or resource list
  • When no object is found, an empty value is returned. This can be used to check for the existence of an object.
  • The lookup function uses Helm's existing Kubernetes connection configuration to query Kubernetes.
  • Helm is not supposed to contact the Kubernetes API Server during a helm template or a helm install|update|delete|rollback --dry-run, so the lookup function will return an empty list (i.e. dict) in such a case.
  • the operators (eq, ne, lt, gt, and, or and so on) are all implemented as functions. In pipelines, operations can be grouped with parentheses ((, and )).
  • 張 旭 on 02 Oct 21
     
    "When injecting strings from the .Values object into the template, we ought to quote these strings. "
張 旭

Helm | Named Templates - 0 views

helm.sh/...named_templates

helm

shared by 張 旭 on 03 Oct 21 - No Cached
  • a special-purpose include function that works similarly to the template action.
  • when naming templates: template names are global.
  • templates in subcharts are compiled together with top-level templates, you should be careful to name your templates with chart-specific names.
  • ...14 more annotations...
  • One popular naming convention is to prefix each defined template with the name of the chart: {{ define "mychart.labels" }}
  • using the specific chart name as a prefix we can avoid any conflicts
  • But files whose name begins with an underscore (_) are assumed to not have a manifest inside.
  • The define action allows us to create a named template inside of a template file.
  • include it with the template action
  • a define does not produce output unless it is called with a template
  • define functions should have a simple documentation block ({{/* ... */}}) describing what they do.
  • template names are global.
  • A popular naming convention is to prefix each defined template with the name of the chart
  • When a named template (created with define) is rendered, it will receive the scope passed in by the template call.
  • No scope was passed in, so within the template we cannot access anything in .
  • Note that we pass . at the end of the template call. We could just as easily pass .Values or .Values.favorite or whatever scope we want
  • the template that is substituted in has the text aligned to the left. Because template is an action, and not a function, there is no way to pass the output of a template call to other functions; the data is simply inserted inline.
  • use indent to indent
  • 張 旭 on 03 Oct 21
     
    "a special-purpose include function that works similarly to the template action."
張 旭

Run your CI/CD jobs in Docker containers | GitLab - 0 views

docs.gitlab.com/...using_docker_images.html

docker cicd gitlab

shared by 張 旭 on 20 Jan 22 - No Cached
  • If you run Docker on your local machine, you can run tests in the container, rather than testing on a dedicated CI/CD server.
  • Run other services, like MySQL, in containers. Do this by specifying services in your .gitlab-ci.yml file.
  • By default, the executor pulls images from Docker Hub
  • ...10 more annotations...
  • Maps must contain at least the name option, which is the same image name as used for the string setting.
  • When a CI job runs in a Docker container, the before_script, script, and after_script commands run in the /builds/<project-path>/ directory. Your image may have a different default WORKDIR defined. To move to your WORKDIR, save the WORKDIR as an environment variable so you can reference it in the container during the job’s runtime.
  • The runner starts a Docker container using the defined entrypoint. The default from Dockerfile that may be overridden in the .gitlab-ci.yml file.
  • attaches itself to a running container.
  • sends the script to the container’s shell stdin and receives the output.
  • To override the entrypoint of a Docker image, define an empty entrypoint in the .gitlab-ci.yml file, so the runner does not start a useless shell layer. However, that does not work for all Docker versions. For Docker 17.06 and later, the entrypoint can be set to an empty value. For Docker 17.03 and earlier, the entrypoint can be set to /bin/sh -c, /bin/bash -c, or an equivalent shell available in the image.
  • The runner expects that the image has no entrypoint or that the entrypoint is prepared to start a shell command.
  • entrypoint: [""]
  • entrypoint: ["/bin/sh", "-c"]
  • A DOCKER_AUTH_CONFIG CI/CD variable
  • 張 旭 on 20 Jan 22
     
    "If you run Docker on your local machine, you can run tests in the container, rather than testing on a dedicated CI/CD server. "
張 旭

phusion/passenger-docker: Docker base images for Ruby, Python, Node.js and Meteor web apps - 0 views

github.com/...passenger-docker

docker ruby rails

shared by 張 旭 on 14 Jan 22 - No Cached
  • Ubuntu 20.04 LTS as base system
  • 2.7.5 is configured as the default.
  • Python 3.8
  • ...23 more annotations...
  • A build system, git, and development headers for many popular libraries, so that the most popular Ruby, Python and Node.js native extensions can be compiled without problems.
  • Nginx 1.18. Disabled by default
  • production-grade features, such as process monitoring, administration and status inspection.
  • Redis 5.0. Not installed by default.
  • The image has an app user with UID 9999 and home directory /home/app. Your application is supposed to run as this user.
  • running applications without root privileges is good security practice.
  • Your application should be placed inside /home/app.
  • COPY --chown=app:app
  • Passenger works like a mod_ruby, mod_nodejs, etc. It changes Nginx into an application server and runs your app from Nginx.
  • placing a .conf file in the directory /etc/nginx/sites-enabled
  • The best way to configure Nginx is by adding .conf files to /etc/nginx/main.d and /etc/nginx/conf.d
  • files in conf.d are included in the Nginx configuration's http context.
  • any environment variables you set with docker run -e, Docker linking and /etc/container_environment, won't reach Nginx.
  • To preserve these variables, place an Nginx config file ending with *.conf in the directory /etc/nginx/main.d, in which you tell Nginx to preserve these variables.
  • By default, Phusion Passenger sets all of the following environment variables to the value production
  • Setting these environment variables yourself (e.g. using docker run -e RAILS_ENV=...) will not have any effect, because Phusion Passenger overrides all of these environment variables.
  • PASSENGER_APP_ENV environment variable
  • passenger-docker autogenerates an Nginx configuration file (/etc/nginx/conf.d/00_app_env.conf) during container boot.
  • The configuration file is in /etc/redis/redis.conf. Modify it as you see fit, but make sure daemonize no is set.
  • You can add additional daemons to the image by creating runit entries.
  • The shell script must be called run, must be executable
  • the shell script must run the daemon without letting it daemonize/fork it.
  • We use RVM to install and to manage Ruby interpreters.
« First ‹ Previous 121 - 140 of 147 Next ›
Showing 20▼ items per page