Skip to main content

Home/ Larvata/ Contents contributed and discussions participated by 張 旭

Contents contributed and discussions participated by 張 旭

Simple Middle Filter: All | Bookmarks | Topics
張 旭

How to Benchmark Performance of MySQL & MariaDB Using SysBench | Severalnines - 1 views

severalnines.com/...e-mysql-mariadb-using-sysbench

database mysql performance

shared by 張 旭 on 22 Oct 20 - No Cached
  • SysBench is a C binary which uses LUA scripts to execute benchmarks
  • support for parallelization in the LUA scripts, multiple queries can be executed in parallel
  • by default, benchmarks which cover most of the cases - OLTP workloads, read-only or read-write, primary key lookups and primary key updates.
  • ...21 more annotations...
  • SysBench is not a tool which you can use to tune configurations of your MySQL servers (unless you prepared LUA scripts with custom workload or your workload happen to be very similar to the benchmark workloads that SysBench comes with)
  • it is great for is to compare performance of different hardware.
  • Every new server acquired should go through a warm-up period during which you will stress it to pinpoint potential hardware defects
  • by executing OLTP workload which overloads the server, or you can also use dedicated benchmarks for CPU, disk and memory.
  • bulk_insert.lua. This test can be used to benchmark the ability of MySQL to perform multi-row inserts.
  • All oltp_* scripts share a common table structure. First two of them (oltp_delete.lua and oltp_insert.lua) execute single DELETE and INSERT statements.
  • oltp_point_select, oltp_update_index and oltp_update_non_index. These will execute a subset of queries - primary key-based selects, index-based updates and non-index-based updates.
  • you can run different workload patterns using the same benchmark.
  • Warmup helps to identify “regular” throughput by executing benchmark for a predefined time, allowing to warm up the cache, buffer pools etc.
  • By default SysBench will attempt to execute queries as fast as possible. To simulate slower traffic this option may be used. You can define here how many transactions should be executed per second.
  • SysBench gives you ability to generate different types of data distribution.
  • decide if SysBench should use prepared statements (as long as they are available in the given datastore - for MySQL it means PS will be enabled by default) or not.
  • sysbench ./sysbench/src/lua/oltp_read_write.lua  help
  • By default, SysBench will attempt to execute queries in explicit transaction. This way the dataset will stay consistent and not affected: SysBench will, for example, execute INSERT and DELETE on the same row, making sure the data set will not grow (impacting your ability to reproduce results).
  • specify error codes from MySQL which SysBench should ignore (and not kill the connection).
  • the two most popular benchmarks - OLTP read only and OLTP read/write.
  • 1 million rows will result in ~240 MB of data. Ten tables, 1000 000 rows each equals to 2.4GB
  • by default, SysBench looks for ‘sbtest’ schema which has to exist before you prepare the data set. You may have to create it manually.
  • pass ‘--histogram’ argument to SysBench
  • ~48GB of data (20 tables, 10 000 000 rows each).
  • if you don’t understand why the performance was like it was, you may draw incorrect conclusions out of the benchmarks.
張 旭

Deploy Replica Set With Keyfile Authentication - MongoDB Manual - 0 views

docs.mongodb.com/...et-with-keyfile-access-control

database mongodb

shared by 張 旭 on 13 Oct 20 - No Cached
  • Keyfiles are bare-minimum forms of security and are best suited for testing or development environments.
  • With keyfile authentication, each mongod instances in the replica set uses the contents of the keyfile as the shared password for authenticating other members in the deployment.
  • On UNIX systems, the keyfile must not have group or world permissions.
  • ...3 more annotations...
  • Copy the keyfile to each server hosting the replica set members.
  • the user running the mongod instances is the owner of the file and can access the keyfile.
  • For each member in the replica set, start the mongod with either the security.keyFile configuration file setting or the --keyFile command-line option.
張 旭

Internal/Membership Authentication - MongoDB Manual - 0 views

docs.mongodb.com/...curity-internal-authentication

database mongodb

shared by 張 旭 on 13 Oct 20 - No Cached
  • equire that members of replica sets and sharded clusters authenticate to each other.
  • Enabling internal authentication also enables client authorization.
張 旭

Convert a Standalone to a Replica Set - MongoDB Manual - 0 views

docs.mongodb.com/...vert-standalone-to-replica-set

database mongodb

shared by 張 旭 on 13 Oct 20 - No Cached
  • Restart the instance. Use the --replSet option to specify the name of the new replica set.
  • 張 旭 on 13 Oct 20
     
    "Restart the instance. Use the --replSet option to specify the name of the new replica set. "
張 旭

Deploy a Replica Set - MongoDB Manual - 0 views

docs.mongodb.com/...deploy-replica-set

database mongodb

shared by 張 旭 on 13 Oct 20 - No Cached
  • Three member replica sets provide enough redundancy to survive most network partitions and other system failures.
張 旭

Replication - MongoDB Manual - 0 views

docs.mongodb.com/...replication

database mongodb

shared by 張 旭 on 13 Oct 20 - No Cached
  • A replica set in MongoDB is a group of mongod processes that maintain the same data set.
  • Replica sets provide redundancy and high availability, and are the basis for all production deployments.
  • With multiple copies of data on different database servers, replication provides a level of fault tolerance against the loss of a single database server.
  • ...18 more annotations...
  • replication can provide increased read capacity as clients can send read operations to different servers.
  • A replica set is a group of mongod instances that maintain the same data set.
  • A replica set contains several data bearing nodes and optionally one arbiter node.
  • one and only one member is deemed the primary node, while the other nodes are deemed secondary nodes.
  • A replica set can have only one primary capable of confirming writes with { w: "majority" } write concern; although in some circumstances, another mongod instance may transiently believe itself to also be primary.
  • The secondaries replicate the primary’s oplog and apply the operations to their data sets such that the secondaries’ data sets reflect the primary’s data set
  • add a mongod instance to a replica set as an arbiter. An arbiter participates in elections but does not hold data
  • An arbiter will always be an arbiter whereas a primary may step down and become a secondary and a secondary may become the primary during an election.
  • Secondaries replicate the primary’s oplog and apply the operations to their data sets asynchronously.
  • These slow oplog messages are logged for the secondaries in the diagnostic log under the REPL component with the text applied op: <oplog entry> took <num>ms.
  • Replication lag refers to the amount of time that it takes to copy (i.e. replicate) a write operation on the primary to a secondary.
  • When a primary does not communicate with the other members of the set for more than the configured electionTimeoutMillis period (10 seconds by default), an eligible secondary calls for an election to nominate itself as the new primary.
  • The replica set cannot process write operations until the election completes successfully.
  • The median time before a cluster elects a new primary should not typically exceed 12 seconds, assuming default replica configuration settings.
  • Factors such as network latency may extend the time required for replica set elections to complete, which in turn affects the amount of time your cluster may operate without a primary.
  • Your application connection logic should include tolerance for automatic failovers and the subsequent elections.
  • MongoDB drivers can detect the loss of the primary and automatically retry certain write operations a single time, providing additional built-in handling of automatic failovers and elections
  • By default, clients read from the primary [1]; however, clients can specify a read preference to send read operations to secondaries.
張 旭

MySQL cluster vs Galera - How to make the right choice - 0 views

bobcares.com/...mysql-cluster-vs-galera

database mysql cluster

shared by 張 旭 on 27 Sep 20 - No Cached
  • there is no “one size fits all” solution when coming to database clustering.
  • MySQL cluster contains the data nodes that store the cluster data and management node that store the cluster’s configuration.
  • MySQL clients first communicate with the management node and then connect directly to these data nodes.
  • ...13 more annotations...
  • For synchronization of data in the data nodes, MySQL cluster uses a special data engine called NDB (Network Database).
  • it uses automatic shrading aka splitting of a large database into small units.
  • MySQL cluster avoids single point failure and ensures 99.99% availability.
  • MySQL cluster can provide a response time as low as less than 3 ms.
  • Galera Cluster consists of a database server and uses the Galera Replication Plugin to manage replication.
  • a multi-master database cluster that supports synchronous replication.
  • it provides multiple, up-to-date copies of the data.
  • there is a need for instant fail-over.
  • Galera cluster allows the read and write of data in any node.
  • Galera cluster include guaranteed write consistency, automatic node provisioning, etc.
  • Upon restoring the connection, the separated nodes will sync back and rejoin the cluster automatically.
  • there is no need to have management node like MySQL cluster.
  • it gives best results with the InnoDB storage engine.
張 旭

mvn clean install - a short guide to Maven - 0 views

www.marcobehler.com/...install-a-short-guide-to-maven

programming java

shared by 張 旭 on 12 Aug 20 - No Cached
  • An equivalent in other languages would be Javascript’s npm, Ruby’s gems or PHP’s composer.
  • Maven expects a certain directory structure for your Java source code to live in and when you later do a mvn clean install , the whole compilation and packaging work will be done for you.
  • any directory that contains a pom.xml file is also a valid Maven project.
  • ...17 more annotations...
  • A pom.xml file contains everything needed to describe your Java project.
  • Java source code is to be meant to live in the "/src/main/java" folder
  • Maven will put compiled Java classes into the "target/classes" folder
  • Maven will also build a .jar or .war file, depending on your project, that lives in the "target" folder.
  • Maven has the concept of a build lifecycle, which is made up of different phases.
  • clean is not part of Maven’s default lifecycle, you end up with commands like mvn clean install or mvn clean package. Install or package will trigger all preceding phases, but you need to specify clean in addition.
  • Maven will always download your project dependencies into your local maven repository first and then reference them for your build.
  • local repositories (in your user’s home directory: ~/.m2/)
  • clean: deletes the /target folder.
  • mvn clean package
  • mvn clean install
  • package: Converts your .java source code into a .jar/.war file and puts it into the /target folder.
  • install: First, it does a package(!). Then it takes that .jar/.war file and puts it into your local Maven repository, which lives in ~/.m2/repository.
  • calling 'mvn install' would be enough if Maven was smart enough to do reliable, incremental builds.
  • figuring out what Java source files/modules changed and only compile those.
  • developers got it ingrained to always call 'mvn clean install' (even though this increases build time a lot in bigger projects).
  • make sure that Maven always tries to download the latest snapshot dependency versions
張 旭

Running rootless Podman as a non-root user | Enable Sysadmin - 0 views

www.redhat.com/...rootless-podman-makes-sense

docker podman linux system

shared by 張 旭 on 04 Aug 20 - No Cached
  • By default, rootless Podman runs as root within the container.
  • the processes in the container have the default list of namespaced capabilities which allow the processes to act like root inside of the user namespace
  • the directory is owned by UID 26, but UID 26 is not mapped into the container and is not the same UID that Postgres runs with while in the container.
  • ...8 more annotations...
  • Podman launches a container inside of the user namespace, which is mapped with the range of UIDs defined for the user in /etc/subuid and /etc/subgid
  • The easy solution to this problem is to chown the html directory to match the UID that Postgresql runs with inside of the container.
  • use the podman unshare command, which drops you into the same user namespace that rootless Podman uses
  • This setup also means that the processes inside of the container are running as the user’s UID. If the container process escaped the container, the process would have full access to files in your home directory based on UID separation.
  • SELinux would still block the access, but I have heard that some people disable SELinux.
  • If you run the processes within the container as a different non-root UID, however, then those processes will run as that UID. If they escape the container, they would only have world access to content in your home directory.
  • run a podman unshare command, or set up the directories' group ownership as owned by your UID (root inside of the container).
  • running containers as non-root should always be your top priority for security reasons.
張 旭

Run the Docker daemon as a non-root user (Rootless mode) | Docker Documentation - 0 views

docs.docker.com/...rootless

docker kubernetes system linux

shared by 張 旭 on 04 Aug 20 - No Cached
  • running the Docker daemon and containers as a non-root user
  • Rootless mode does not require root privileges even during the installation of the Docker daemon
  • Rootless mode executes the Docker daemon and containers inside a user namespace.
  • ...9 more annotations...
  • in rootless mode, both the daemon and the container are running without root privileges.
  • Rootless mode does not use binaries with SETUID bits or file capabilities, except newuidmap and newgidmap, which are needed to allow multiple UIDs/GIDs to be used in the user namespace.
  • expose privileged ports (< 1024)
  • add net.ipv4.ip_unprivileged_port_start=0 to /etc/sysctl.conf (or /etc/sysctl.d) and run sudo sysctl --system
  • dockerd-rootless.sh uses slirp4netns (if installed) or VPNKit as the network stack by default.
  • These network stacks run in userspace and might have performance overhead
  • This error occurs when the number of available entries in /etc/subuid or /etc/subgid is not sufficient.
  • This error occurs mostly when the host is running in cgroup v2. See the section Fedora 31 or later for information on switching the host to use cgroup v1.
  • --net=host doesn’t listen ports on the host network namespace This is an expected behavior, as the daemon is namespaced inside RootlessKit’s network namespace. Use docker run -p instead.
張 旭

podman/rootless.md at master · containers/podman - 0 views

github.com/...rootless.md

docker kubernetes

shared by 張 旭 on 03 Aug 20 - No Cached
  • Podman can not create containers that bind to ports < 1024
  • If /etc/subuid and /etc/subgid are not setup for a user, then podman commands can easily fail
  • Fedora 31 defaults to cgroup V2, which has full support of rootless cgroup management.
  • ...6 more annotations...
  • Some system unit configuration options do not work in the rootless container
  • it's better to create an override.conf drop-in that sets PrivateNetwork=no
  • Difficult to use additional stores for sharing content
  • Can not use overlayfs driver, but does support fuse-overlayfs
  • No CNI Support
  • Making device nodes within a container fails, even when running --privileged.
張 旭

Full Cycle Developers at Netflix - Operate What You Build - 1 views

netflixtechblog.com/lopers-at-netflix-a08c31f83249

devops system

shared by 張 旭 on 27 May 20 - No Cached
  • Researching issues felt like bouncing a rubber ball between teams, hard to catch the root cause and harder yet to stop from bouncing between one another.
  • In the past, Edge Engineering had ops-focused teams and SRE specialists who owned the deploy+operate+support parts of the software life cycle
  • hearing about those problems second-hand
  • ...17 more annotations...
  • devs could push code themselves when needed, and also were responsible for off-hours production issues and support requests
  • What were we trying to accomplish and why weren’t we being successful?
  • These specialized roles create efficiencies within each segment while potentially creating inefficiencies across the entire life cycle.
  • Grouping differing specialists together into one team can reduce silos, but having different people do each role adds communication overhead, introduces bottlenecks, and inhibits the effectiveness of feedback loops.
  • devops principles
  • develops a system also be responsible for operating and supporting that system
  • Each development team owns deployment issues, performance bugs, capacity planning, alerting gaps, partner support, and so on.
  • Those centralized teams act as force multipliers by turning their specialized knowledge into reusable building blocks.
  • Communication and alignment are the keys to success.
  • Full cycle developers are expected to be knowledgeable and effective in all areas of the software life cycle.
  • ramping up on areas they haven’t focused on before
  • We run dev bootcamps and other forms of ongoing training to impart this knowledge and build up these skills
  • “how can I automate what is needed to operate this system?”
  • “what self-service tool will enable my partners to answer their questions without needing me to be involved?”
  • A full cycle developer thinks and acts like an SWE, SDET, and SRE. At times they create software that solves business problems, at other times they write test cases for that, and still other times they automate operational aspects of that system.
  • the need for continuous delivery pipelines, monitoring/observability, and so on.
  • Tooling and automation help to scale expertise, but no tool will solve every problem in the developer productivity and operations space
張 旭

- 0 views

dev.to/...lized-useful-git-commands-37p1

git programming code

shared by 張 旭 on 07 May 20 - No Cached
  • A fast-forward merge can happen when the current branch has no extra commits compared to the branch we’re merging.
  • With a no-fast-forward merge, Git creates a new merging commit on the active branch.
  • We can manually remove the changes we don't want to keep, save the changes, add the changed file again, and commit the changes
  • ...14 more annotations...
  • A git rebase copies the commits from the current branch, and puts these copied commits on top of the specified branch.
  • The branch that we're rebasing always has the latest changes that we want to keep!
  • A git rebase changes the history of the project as new hashes are created for the copied commits!
  • Rebasing is great whenever you're working on a feature branch, and the master branch has been updated.
  • An interactive rebase can also be useful on the branch you're currently working on, and want to modify some commits.
  • A git reset gets rid of all the current staged files and gives us control over where HEAD should point to.
  • A soft reset moves HEAD to the specified commit (or the index of the commit compared to HEAD)
  • Git should simply reset its state back to where it was on the specified commit: this even includes the changes in your working directory and staged files!
  • By reverting a certain commit, we create a new commit that contains the reverted changes!
  • Performing a git revert is very useful in order to undo a certain commit, without modifying the history of the branch.
  • By cherry-picking a commit, we create a new commit on our active branch that contains the changes that were introduced by the cherry-picked commit.
  • a fetch simply downloads new data.
  • A git pull is actually two commands in one: a git fetch, and a git merge
  • git reflog is a very useful command in order to show a log of all the actions that have been taken
張 旭

AskF5 | Manual Chapter: Working with Partitions - 0 views

techdocs.f5.com/...8.html

bigip network

shared by 張 旭 on 05 May 20 - No Cached
  • During BIG-IP® system installation, the system automatically creates a partition named Common
  • An administrative partition is a logical container that you create, containing a defined set of BIG-IP® system objects.
  • No user can delete partition Common itself.
  • ...9 more annotations...
  • With respect to permissions, all users on the system except those with a user role of No Access have read access to objects in partition Common, and by default, partition Common is their current partition.
  • The current partition is the specific partition to which the system is currently set for a logged-in user.
  • A partition access assignment gives a user some level of access to the specified partition.
  • assigning partition access to a user does not necessarily give the user full access to all objects in the partition
  • user account objects also reside in partitions
  • when you first install the BIG-IP system, every existing user account (root and admin) resides in partition Common
  • the partition in which a user account object resides does not affect the partition or partitions to which that user is granted access to manage other BIG-IP objects
  • the object it references resides in partition Common
  • a referenced object must reside either in the same partition as the object that is referencing it
張 旭

Warnings, Notes, & Tips - 0 views

clouddocs.f5.com/...tips-warnings.html

terraform networking network firewall bigip

shared by 張 旭 on 04 May 20 - No Cached
  • AS3 manages topology records globally in /Common, it is required that records only be managed through AS3, as it will treat the records declaratively.
  • If a record is added outside of AS3, it will be removed if it is not included in the next AS3 declaration for topology records (AS3 completely overwrites non-AS3 topologies when a declaration is submitted).
  • using AS3 to delete a tenant (for example, sending DELETE to the /declare/<TENANT> endpoint) that contains GSLB topologies will completely remove ALL GSLB topologies from the BIG-IP.
  • ...12 more annotations...
  • When posting a large declaration (hundreds of application services in a single declaration), you may experience a 500 error stating that the save sys config operation failed.
  • Even if you have asynchronous mode set to false, after 45 seconds AS3 sets asynchronous mode to true (API swap), and returns an async response.
  • When creating a new tenant using AS3, it must not use the same name as a partition you separately create on the target BIG-IP system.
  • If you use the same name and then post the declaration, AS3 overwrites (or removes) the existing partition completely, including all configuration objects in that partition.
  • use AS3 to create a tenant (which creates a BIG-IP partition), manually adding configuration objects to the partition created by AS3 can have unexpected results
  • When you delete the Tenant using AS3, the system deletes both virtual servers.
  • if a Firewall_Address_List contains zero addresses, a dummy IPv6 address of ::1:5ee:bad:c0de is added in order to maintain a valid Firewall_Address_List. If an address is added to the list, the dummy address is removed.
  • use /mgmt/shared/appsvcs/declare?async=true if you have a particularly large declaration which will take a long time to process.
  • reviewing the Sizing BIG-IP Virtual Editions section (page 7) of Deploying BIG-IP VEs in a Hyper-Converged Infrastructure
  • To test whether your system has AS3 installed or not, use GET with the /mgmt/shared/appsvcs/info URI.
  • You may find it more convenient to put multi-line texts such as iRules into AS3 declarations by first encoding them in Base64.
  • no matter your BIG-IP user account name, audit logs show all messages from admin and not the specific user name.
張 旭

Forwarding (DNS and BIND, 4th Edition) - 0 views

docstore.mik.ua/...ch10_05.htm

dns networking network

shared by 張 旭 on 16 Apr 20 - Cached
  • Forwarders are also useful if you need to shunt name resolution to a particular name server
  • If you designate one or more servers at your site as forwarders, your name servers will send all their off-site queries to the forwarders first.
  • A primary master or slave name server's mode of operation changes slightly when it is configured to use a forwarder.
  • ...2 more annotations...
  • If a resolver requests records that are already in the name server's authoritative data or cached data, the name server answers with that information
  • if the records aren't in its database, the name server sends the query to a forwarder and waits a short period for an answer before resuming normal operation and contacting the remote name servers itself. What the name server is doing differently here is sending a recursive query to the forwarder, expecting it to find the answer.
張 旭

如何在 Ubuntu 18.04 下正确配置网络 - 运维之美 - 0 views

www.hi-linux.com/...49513.html

networking ubuntu server network

shared by 張 旭 on 09 Apr 20 - No Cached
  • 張 旭 on 09 Apr 20
     
    "systemd-resolve --status"
« First ‹ Previous 141 - 160 of 596 Next › Last »
Showing 20▼ items per page