Group items tagged

Nginx is one of the most popular web servers in the world. It can successfully handle high loads with many concurrent client connections, and can easily function as a web server, a mail server, or a reverse proxy server.

The main server block directives that Nginx is concerned with during this process are the listen directive, and the server_name directive.

The listen directive typically defines which IP address and port that the server block will respond to.

Nginx translates all “incomplete” listen directives by substituting missing values with their default values so that each block can be evaluated by its IP address and port.

In any case, the port must be matched exactly.

If there are multiple server blocks with the same level of specificity matching, Nginx then begins to evaluate the server_name directive of each server block.

Nginx checks the request’s “Host” header. This value holds the domain or IP address that the client was actually trying to reach.

Nginx will first try to find a server block with a server_name that matches the value in the “Host” header of the request exactly.

If no exact match is found, Nginx will then try to find a server block with a server_name that matches using a leading wildcard (indicated by a * at the beginning of the name in the config).

If no match is found using a leading wildcard, Nginx then looks for a server block with a server_name that matches using a trailing wildcard (indicated by a server name ending with a * in the config)

If no match is found using a trailing wildcard, Nginx then evaluates server blocks that define the server_name using regular expressions (indicated by a ~ before the name).

If no regular expression match is found, Nginx then selects the default server block for that IP address and port.

If no modifiers are present, the location is interpreted as a prefix match.

=: If an equal sign is used, this block will be considered a match if the request URI exactly matches the location given.

~: If a tilde modifier is present, this location will be interpreted as a case-sensitive regular expression match.

~*: If a tilde and asterisk modifier is used, the location block will be interpreted as a case-insensitive regular expression match.

^~: If a carat and tilde modifier is present, and if this block is selected as the best non-regular expression match, regular expression matching will not take place.

Keep in mind that if this block is selected and the request is fulfilled using an index page, an internal redirect will take place to another location that will be the actual handler of the request

Keeping in mind the types of location declarations we described above, Nginx evaluates the possible location contexts by comparing the request URI to each of the locations.

Nginx begins by checking all prefix-based location matches (all location types not involving a regular expression).

First, Nginx looks for an exact match.

If no exact (with the = modifier) location block matches are found, Nginx then moves on to evaluating non-exact prefixes.

After the longest matching prefix location is determined and stored, Nginx moves on to evaluating the regular expression locations (both case sensitive and insensitive).

by default, Nginx will serve regular expression matches in preference to prefix matches.

regular expression matches within the longest prefix match will “jump the line” when Nginx evaluates regex locations.

The exceptions to the “only one location block” rule may have implications on how the request is actually served and may not align with the expectations you had when designing your location blocks.

The index directive always leads to an internal redirect if it is used to handle the request.

In the case above, if you really need the execution to stay in the first block, you will have to come up with a different method of satisfying the request to the directory.

one way of preventing an index from switching contexts, but it’s probably not useful for most configurations

the try_files directive. This directive tells Nginx to check for the existence of a named set of files or directories.

the rewrite directive. When using the last parameter with the rewrite directive, or when using no parameter at all, Nginx will search for a new matching location based on the results of the rewrite.

The error_page directive can lead to an internal redirect similar to that created by try_files.

when certain status codes are encountered.

application’s logic is a great example of a component

Aspects cross-cut our application - when we use some kind of persistence (e.g. a database) or network communication (such as ZMQ sockets) our components need to know about it.

It’s responsible for pushing snippets scenario

SRP-conformant object

the join points in Ruby

advice

In most cases after and before advice are sufficient.

to provide a join point

You’ll often see empty methods in code written in AOP paradigm

provide aspect code to link with our use case

use case is a pure domain object, without even knowing it’s connected with some kind of persistence and logging layer.

Aspect-oriented programming is fixing the problem with polluting pure logic objects with technical context of our applications.

we treat our glues as a configuration part, not the logic part of our apps.

name-based virtual hosting

Edge routerA router that enforces the firewall policy for your cluster.

A Kubernetes ServiceA way to expose an application running on a set of Pods as a network service. that identifies a set of Pods using labelTags objects with identifying attributes that are meaningful and relevant to users. selectors.

Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster.

An Ingress can be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name based virtual hosting.

Exposing services other than HTTP and HTTPS to the internet typically uses a service of type Service.Type=NodePort or Service.Type=LoadBalancer.

Ingress frequently uses annotations to configure some options depending on the Ingress controller,

An optional host.

A list of paths

A backend is a combination of Service and port names

has an associated backend

Both the host and path must match the content of an incoming request before the load balancer directs traffic to the referenced Service.

HTTP (and HTTPS) requests to the Ingress that matches the host and path of the rule are sent to the listed backend.

An Ingress with no rules sends all traffic to a single default backend.

Ingress controllers and load balancers may take a minute or two to allocate an IP address.

A fanout configuration routes traffic from a single IP address to more than one Service, based on the HTTP URI being requested.

nginx.ingress.kubernetes.io/rewrite-target: /

describe ingress

get ingress

Name-based virtual hosts support routing HTTP traffic to multiple host names at the same IP address.

an Ingress resource without any hosts defined in the rules, then any web traffic to the IP address of your Ingress controller can be matched without a name based virtual host being required.

secure an Ingress by specifying a SecretStores sensitive information, such as passwords, OAuth tokens, and ssh keys. that contains a TLS private key and certificate.

An Ingress controller is bootstrapped with some load balancing policy settings that it applies to all Ingress, such as the load balancing algorithm, backend weight scheme, and others.

review the controller specific documentation to see how they handle health checks

edit ingress

After you save your changes, kubectl updates the resource in the API server, which tells the Ingress controller to reconfigure the load balancer.

kubectl replace -f on a modified Ingress YAML file.

Node: A worker machine in Kubernetes, part of a cluster.

in most common Kubernetes deployments, nodes in the cluster are not part of the public internet.

Edge router: A router that enforces the firewall policy for your cluster.

Service: A Kubernetes Service that identifies a set of Pods using label selectors.

An Ingress may be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name-based virtual hosting.

The Ingress spec has all the information needed to configure a load balancer or proxy server.

An Ingress with no rules sends all traffic to a single default backend and .spec.defaultBackend is the backend that should handle requests in that case.

If defaultBackend is not set, the handling of requests that do not match any of the rules will be up to the ingress controller

A common usage for a Resource backend is to ingress data to an object storage backend with static assets.

Exact: Matches the URL path exactly and with case sensitivity.

Prefix: Matches based on a URL path prefix split by /. Matching is case sensitive and done on a path element by element basis.

multiple paths within an Ingress will match a request. In those cases precedence will be given first to the longest matching path.

Hosts can be precise matches (for example “foo.bar.com”) or a wildcard (for example “*.foo.com”).

Each Ingress should specify a class, a reference to an IngressClass resource that contains additional configuration including the name of the controller that should implement the class.

secure an Ingress by specifying a Secret that contains a TLS private key and certificate.

The Ingress resource only supports a single TLS port, 443, and assumes TLS termination at the ingress point (traffic to the Service and its Pods is in plaintext).

hosts in the tls section need to explicitly match the host in the rules section.

deployment.yaml: A basic manifest for creating a Kubernetes deployment

using the suffix .yaml for YAML files and .tpl for helpers.

helm get manifest

The helm get manifest command takes a release name (full-coral) and prints out all of the Kubernetes resources that were uploaded to the server. Each file begins with --- to indicate the start of a YAML document

Names should be unique to a release

The name: field is limited to 63 characters because of limitations to the DNS system.

release names are limited to 53 characters

{{ .Release.Name }}

The values that are passed into a template can be thought of as namespaced objects, where a dot (.) separates each namespaced element.

The Release object is one of the built-in objects for Helm

Using --dry-run will make it easier to test your code, but it won’t ensure that Kubernetes itself will accept the templates you generate.

Objects are passed into a template from the template engine.

create new objects within your templates

Objects can be simple, and have just one value. Or they can contain other objects or functions.

Release is one of the top-level objects that you can access in your templates.

Release.Namespace: The namespace to be released into (if the manifest doesn’t override)

Chart: The contents of the Chart.yaml file.

Files: This provides access to all non-special files in a chart.

Files.Get is a function for getting a file by name

Files.GetBytes is a function for getting the contents of a file as an array of bytes instead of as a string. This is useful for things like images.

Template: Contains information about the current template that is being executed

BasePath: The namespaced path to the templates directory of the current chart

Go’s naming convention

use only initial lower case letters in order to distinguish local names from those built-in.

If this is a subchart, the values.yaml file of a parent chart

Individual parameters passed with --set

While structuring data this way is possible, the recommendation is that you keep your values trees shallow, favoring flatness.

If you need to delete a key from the default values, you may override the value of the key to be null, in which case Helm will remove the key from the overridden values merge.

Kubernetes would then fail because you can not declare more than one livenessProbe handler.

When injecting strings from the .Values object into the template, we ought to quote these strings.

quote

Template functions follow the syntax functionName arg1 arg2...

While we talk about the “Helm template language” as if it is Helm-specific, it is actually a combination of the Go template language, some extra functions, and a variety of wrappers to expose certain objects to the templates.

Drawing on a concept from UNIX, pipelines are a tool for chaining together a series of template commands to compactly express a series of transformations.

pipelines are an efficient way of getting several things done in sequence

The repeat function will echo the given string the given number of times

default DEFAULT_VALUE GIVEN_VALUE. This function allows you to specify a default value inside of the template, in case the value is omitted.

all static default values should live in the values.yaml, and should not be repeated using the default command

Operators are implemented as functions that return a boolean value.

To use eq, ne, lt, gt, and, or, not etcetera place the operator at the front of the statement followed by its parameters just as you would a function.

if and

if or

with to specify a scope

range, which provides a “for each”-style loop

block declares a special kind of fillable template area

A pipeline is evaluated as false if the value is: a boolean false a numeric zero an empty string a nil (empty or null) an empty collection (map, slice, tuple, dict, array)

incorrect YAML because of the whitespacing

When the template engine runs, it removes the contents inside of {{ and }}, but it leaves the remaining whitespace exactly as is.

{{- (with the dash and space added) indicates that whitespace should be chomped left, while -}} means whitespace to the right should be consumed.

Newlines are whitespace!

an * at the end of the line indicates a newline character that would be removed

the indent function

Scopes can be changed. with can allow you to set the current scope (.) to a particular object.

range

The range function will “range over” (iterate through) the pizzaToppings list.

Just like with sets the scope of ., so does a range operator.

The toppings: |- line is declaring a multi-line string.

not a YAML list. It’s a big string.

the data in ConfigMaps data is composed of key/value pairs, where both the key and the value are simple strings.

The |- marker in YAML takes a multi-line string.

range can be used to iterate over collections that have a key and a value (like a map or dict).

In Helm templates, a variable is a named reference to another object. It follows the form $name

Variables are assigned with a special assignment operator: :=

{{- $relname := .Release.Name -}}

capture both the index and the value

the integer index (starting from zero) to $index and the value to $topping

For data structures that have both a key and a value, we can use range to get both

one variable that is always global - $ - this variable will always point to the root context.

$.

$.

Helm template language is its ability to declare multiple templates and use them together.

A named template (sometimes called a partial or a subtemplate) is simply a template defined inside of a file, and given a name.

If you declare two templates with the same name, whichever one is loaded last will be the one used.

naming convention is to prefix each defined template with the name of the chart: {{ define "mychart.labels" }}

Tags are accessible to many AWS services, including billing.

personally identifiable information (PII)

Use automated tools to help manage resource tags.

Tag policies let you specify tagging rules that define valid key names and the values that are valid for each key.

Name – Identify individual resources

Environment – Distinguish between development, test, and production resources

Project – Identify projects that the resource supports

Owner – Identify who is responsible for the resource

Each resource can have a maximum of 50 user created tags.

For each resource, each tag key must be unique, and each tag key can have only one value.

decide on a strategy for capitalizing tags, and consistently implement that strategy across all resource types.

AWS Cost Explorer and detailed billing reports let you break down AWS costs by tag.

An effective tagging strategy uses standardized tags and applies them consistently and programmatically across AWS resources.

pulling from git is slow.

Support for containers has existed in the Linux kernel since version 2.6.24 when cgroup support was added

All of the logic that used to live in your cookbooks/playbooks/manifests/etc now lives in a Dockerfile that resides directly in the repository for the application it is designed to build

All of the dependencies of the application are bundled with the container which means no need to build on the fly on every server during deployment.

Containers bring standardization which allows for systems like centralized logging, monitoring, and metrics to easily snap into place no matter what is running in the container.

Dockerfiles do not give you the same level of control over configuration as your application transitions between environments, like dev, staging, and production.

configuration management systems now have hooks for docker integration.

Config management will only be used to install Docker, an orchestration system, configure PAM/SSH auth, and tune OS sysctl values.

view templates are written in a language called ERB (Embedded Ruby) which is converted by the request cycle in Rails before being sent to the user.

Each action's purpose is to collect information to provide it to a view.

A view's purpose is to display this information in a human readable format.

routing file which holds entries in a special DSL (domain-specific language) that tells Rails how to connect incoming requests to controllers and actions.

You can create, read, update and destroy items for a resource and these operations are referred to as CRUD operations

A controller is simply a class that is defined to inherit from ApplicationController.

If not found, then it will attempt to load a template called application/new. It looks for one here because the PostsController inherits from ApplicationController

:formats specifies the format of template to be served in response. The default format is :html, and so Rails is looking for an HTML template.

:handlers, is telling us what template handlers could be used to render our template.

When you call form_for, you pass it an identifying object for this form. In this case, it's the symbol :post. This tells the form_for helper what this form is for.

that the action attribute for the form is pointing at /posts/new

When a form is submitted, the fields of the form are sent to Rails as parameters.

parameters can then be referenced inside the controller actions, typically to perform a particular task

params method is the object which represents the parameters (or fields) coming in from the form.

Active Record is smart enough to automatically map column names to model attributes,

Rails uses rake commands to run migrations, and it's possible to undo a migration after it's been applied to your database

every Rails model can be initialized with its respective attributes, which are automatically mapped to the respective database columns.

migration creates a method named change which will be called when you run this migration.

The action defined in this method is also reversible, which means Rails knows how to reverse the change made by this migration, in case you want to reverse it later

Migration filenames include a timestamp to ensure that they're processed in the order that they were created.

@post.save returns a boolean indicating whether the model was saved or not.

prevents an attacker from setting the model's attributes by manipulating the hash passed to the model.

If you want to link to an action in the same controller, you don't need to specify the :controller option, as Rails will use the current controller by default.

inherits from ActiveRecord::Base

Rails includes methods to help you validate the data that you send to models

Rails can validate a variety of conditions in a model, including the presence or uniqueness of columns, their format, and the existence of associated objects.

redirect_to will tell the browser to issue another request.

rendering is done within the same request as the form submission

Each request for a comment has to keep track of the post to which the comment is attached, thus the initial call to the find method of the Post model to get the post in question.

pluralize is a rails helper that takes a number and a string as its arguments. If the number is greater than one, the string will be automatically pluralized.

The render method is used so that the @post object is passed back to the new template when it is rendered.

it accepts a hash containing the attributes that you want to update.

field_with_errors. You can define a css rule to make them standout

belongs_to :post, which sets up an Active Record association

creates comments as a nested resource within posts

call destroy on Active Record objects when you want to delete them from the database.

Rails allows you to use the dependent option of an association to achieve this.

store all external data as UTF-8

you're better off ensuring that all external data is UTF-8

use UTF-8 as the internal storage of your database

Rails defaults to converting data from your database into UTF-8 at the boundary.

:patch

By default forms built with the form_for helper are sent via POST

The :method and :'data-confirm' options are used as HTML5 attributes so that when the link is clicked, Rails will first show a confirm dialog to the user, and then submit the link with method delete. This is done via the JavaScript file jquery_ujs which is automatically included into your application's layout (app/views/layouts/application.html.erb) when you generated the application.

Without this file, the confirmation dialog box wouldn't appear.

just defines the partial template we want to render

As the render method iterates over the @post.comments collection, it assigns each comment to

use the authentication system

require and permit

the method is often made private to make sure it can't be called outside its intended context.

all resources created for a particular provider configuration must be destroyed before that provider configuration is removed, unless the related resources are re-configured to use a different provider configuration first.

recommended in the common case where only a single configuration is needed for each provider across the entire configuration.

Journaling guarantees that MongoDB can quickly recover write operations that were written to the journal but not written to data files in cases where mongod terminated due to a crash or other serious failure.

To use read concern level of "majority", replica sets must use WiredTiger storage engine.

Write concern describes the level of acknowledgement requested from MongoDB for write operations.

With stronger write concerns, clients must wait after sending a write operation until MongoDB confirms the write operation at the requested write concern level.

The HTTP interface is disabled by default. Do not enable the HTTP interface in production environments.

Avoid overloading the connection resources of a mongod or mongos instance by adjusting the connection pool size to suit your use case.

ensure that each mongod or mongos instance has access to two real cores or one multi-core physical CPU.

The WiredTiger storage engine is multithreaded and can take advantage of additional CPU cores

WordPlumblr allows its users to use custom domains that point to the WordPlumblr infrastructure

A CNAME is an alias. It allows one domain to point to another domain which, eventually if you follow the CNAME chain, will resolve to an A record and IP address.

For example, WordPlumblr might have assigned the CNAME 6equj5.wordplumblr.com for Foo.com. Foo.com and the other customers may have all initially resolved, at the end of the CNAME chain, to the same IP address.

you usually don't want to address memory directly but, instead, you set up a pointer to a block of memory where you're going to store something. If the operating system needs to move the memory around then it just updates the pointer to point to wherever the chunk of memory has been moved to.

the DNS spec enshrined that the root record -- the naked domain without any subdomain -- could not be a CNAME.

a way to support a CNAME at the root, but still follow the RFC and return an IP address for any query for the root record.

extended our authoritative DNS infrastructure to, in certain cases, act as a kind of DNS resolver.

allows the flexibility of having CNAMEs at the root without breaking the DNS specification.

We cache the CNAME responses -- respecting the DNS TTLs, just like a recursor should -- which means often we have the answer without having to traverse the chain.

CNAME flattening solved email resolution errors for us which was very key.

use all rules keywords, like if, changes, and exists, in the same rule. The rule evaluates to true only when all included keywords evaluate to true.

use parentheses with && and || to build more complicated variable expressions.

Use workflow to specify which types of pipelines can run.

every push to an open merge request’s source branch causes duplicated pipelines.

avoid duplicate pipelines by changing the job rules to avoid either push (branch) pipelines or merge request pipelines.

For behavior similar to the only/except keywords, you can check the value of the $CI_PIPELINE_SOURCE variable

commonly used variables for if clauses

rules:changes expressions to determine when to add jobs to a pipeline

Use !reference tags to reuse rules in different jobs.

Use except to define when a job does not run.

only or except used without refs is the same as only:refs / except/refs

If you change multiple files, but only one file ends in .md, the build job is still skipped.

If you use multiple keywords with only or except, the keywords are evaluated as a single conjoined expression.

only includes the job if all of the keys have at least one condition that matches.

except excludes the job if any of the keys have at least one condition that matches.

To specify a job as manual, add when: manual to the job in the .gitlab-ci.yml file.

Use protected environments to define a list of users authorized to run a manual job.

Use when: delayed to execute scripts after a waiting period, or if you want to avoid jobs immediately entering the pending state.

To split a large job into multiple smaller jobs that run in parallel, use the parallel keyword

run a trigger job multiple times in parallel in a single pipeline, but with different variable values for each instance of the job.

The @ symbol denotes the beginning of a ref’s repository path. To match a ref name that contains the @ character in a regular expression, you must use the hex character code match \x40.

Compare a variable to a string

Check if a variable is undefined

Check if a variable is empty

Check if a variable exists

Check if a variable is empty

Matches are found when using =~.

Matches are not found when using !~.

Join variable expressions together with && or ||

models directory is meant to hold tests for your models

controllers directory is meant to hold tests for your controllers

integration directory is meant to hold tests that involve any number of controllers interacting

Fixtures are a way of organizing test data; they reside in the fixtures folder

The test_helper.rb file holds the default configuration for your tests

Fixtures allow you to populate your testing database with predefined data before your tests run

Fixtures are database independent written in YAML.

one file per model.

Each fixture is given a name followed by an indented list of colon-separated key/value pairs.

Keys which resemble YAML keywords such as 'yes' and 'no' are quoted so that the YAML Parser correctly interprets them.

define a reference node between two different fixtures.

ERB allows you to embed Ruby code within templates

The YAML fixture format is pre-processed with ERB when Rails loads fixtures.

Rails by default automatically loads all fixtures from the test/fixtures folder for your models and controllers test.

Fixtures are instances of Active Record.

access the object directly

test_helper.rb specifies the default configuration to run our tests. This is included with all the tests, so any methods added to this file are available to all your tests.

test with method names prefixed with test_.

An assertion is a line of code that evaluates an object (or expression) for expected results.

bin/rake db:test:prepare

Every test contains one or more assertions. Only when all the assertions are successful will the test pass.

rake test command

run a particular test method from the test case by running the test and providing the test method name.

The . (dot) above indicates a passing test. When a test fails you see an F; when a test throws an error you see an E in its place.

we first wrote a test which fails for a desired functionality, then we wrote some code which adds the functionality and finally we ensured that our test passes. This approach to software development is referred to as Test-Driven Development (TDD).