Group items tagged

allowing your schema and changes to be database independent.

each migration as being a new 'version' of the database

each migration modifies it to add or remove tables, columns, or entries

roll this migration back, it will remove the table

timestamps macro adds two columns, created_at and updated_at

On databases that support transactions with statements that change the schema, migrations are wrapped in a transaction

reversible

use up and down instead of change

Migrations are stored as files in the db/migrate directory, one for each migration class.

a UTC timestamp identifying

Rails uses this timestamp to determine which migration should be run and in what order

"AddXXXToYYY" or "RemoveXXXFromYYY"

use a Ruby DSL

column type as references

part_number:string:index

a migration to remove a column

"CreateXXX"

change_column_null

AddUserRefToProducts

:references

produce join tables if JoinTable is part of the name

CreateJoinTable

The model and scaffold generators will create migrations appropriate for adding a new model.

enclosed by curly braces and follow the field type

create_table

By default, create_table will create a primary key called id

add an index on the new column

when using MySQL, the default is ENGINE=InnoDB

create_join_table creates an HABTM (has and belongs to many) join table

create_join_table also accepts a block

change_table, used for changing existing tables

remove

rename

add_column

change_column

remove_column

change_column_default

place an SQL fragment in the :options option.

limit

precision

scale

polymorphic

default

index

add_foreign_key

use the old style of migration using up and down methods instead of the change method.

.connection.execute

remove_column is reversible if you supply the column type as the third argument

Complex migrations may require processing that Active Record doesn't know how to reverse

reversible

Using reversible will ensure that the instructions are executed in the right order too.

add_column add_foreign_key add_index add_reference add_timestamps change_column_default (must supply a :from and :to option) change_column_null create_join_table create_table disable_extension drop_join_table drop_table (must supply a block) enable_extension remove_column (must supply a type) remove_foreign_key (must supply a second table) remove_index remove_reference remove_timestamps rename_column rename_index rename_table

:column_options option

have the option :null set to false by default

By default, the name of the join table comes from the union of the first two arguments provided to create_join_table

If the column names can not be derived from the table names, you can use the :column and :primary_key options.

figure out the column name

foreign key for a specific column

foreign key by name

should use reversible or write the up and down methods instead of using the change method

If your migration is irreversible, you should raise ActiveRecord::IrreversibleMigration from your down method.

DontUseConstraintForZipcodeValidationMigration

rails db:migrate

specify a target version

all migrations up to and including 20080906120000

run the down method on all the migrations down to, but not including, 20080906120000

rails db:rollback

db:migrate:redo task is a shortcut for doing a rollback and then migrating back up again

STEP parameter

db:setup task will create the database, load the schema and initialize it with the seed data

db:reset task will drop the database and set it up again. This is functionally equivalent to rails db:drop db:setup.

run a specific migration up or down, the db:migrate:up and db:migrate:down

the RAILS_ENV environment variable

db:migrate VERBOSE=false will suppress all output.

must rollback the migration (for example with bin/rails db:rollback), edit your migration and then run rails db:migrate to run the corrected version.

editing existing migrations is not a good idea.

revert method can be helpful when writing a new migration to undo previous migrations in whole or in part

require_relative

revert

Schema Files for

Schema files are also useful if you want a quick look at what attributes an Active Record object has

annotate_models gem automatically adds and updates comments at the top of each model summarizing the schema if you desire that functionality.

database-independent

multiple databases

you can execute custom SQL statements, the schema dumper cannot reconstitute those statements from the database

db:structure:dump

set in config/application.rb by the config.active_record.schema_format setting, which may be either :sql or :ruby.

check them into source control.

db/schema.rb contains the current version number of the database

Validations such as validates :foreign_key, uniqueness: true are one way in which models can enforce data integrity

The :dependent option on associations allows models to automatically destroy child objects when the parent is destroyed.

Migrations can also be used to add or modify data

Initial

To add initial data after a database is created, Rails has a built-in 'seeds' feature that makes the process quick and easy.

db/seeds.rb

rails db:seed

An env_file, is a convenient way to pass many environment variables to a single command in one batch.

If you have a file named .env in your project, it’s only used to put values into the docker-compose.yml file which is in the same folder. Those are used with Docker Compose and Docker Stack.

Just type docker-compose config. This way you’ll see how the docker-compose.yml file content looks after the substitution step has been performed without running anything else.

ARG are also known as build-time variables. They are only available from the moment they are ‘announced’ in the Dockerfile with an ARG instruction up to the moment when the image is built.

ENV variables are also available during the build, as soon as you introduce them with an ENV instruction. However, unlike ARG, they are also accessible by containers started from the final image.

ENV values can be overridden when starting a container,

If you don’t provide a value to expected ARG variables which don’t have a default, you’ll get an error message.

args block

You can use ARG to set the default values of ENV vars.

dynamic on-build env values

2. Pass environment variable values from your host

1. Provide values one by one

3. Take values from a file (env_file)

for each RUN statement, a new container is launched from an intermediate image.

An image is saved by the end of the command, but environment variables do not persist that way.

The precedence is, from stronger to less-strong: stuff the containerized application sets, values from single environment entries, values from the env_file(s) and finally Dockerfile defaults.

DevOps is a culture, a movement, a philosophy.

a firm handshake between development and operations

DevOps isn’t magic, and transformations don’t happen overnight.

Culture is the #1 success factor in DevOps.

Building a culture of shared responsibility, transparency and faster feedback is the foundation of every high performing DevOps team.

 'not our problem' mentality

DevOps is that change in mindset of looking at the development process holistically and breaking down the barrier between Dev and Ops.

Speed is everything.

Open communication helps Dev and Ops teams swarm on issues, fix incidents, and unblock the release pipeline faster.

Unplanned work is a reality that every team faces–a reality that most often impacts team productivity.

“cross-functional collaboration.”

All the tooling and automation in the world are useless if they aren’t accompanied by a genuine desire on the part of development and IT/Ops professionals to work together.

Forming project- or product-oriented teams to replace function-based teams is a step in the right direction.

sharing a common goal and having a plan to reach it together

join sprint planning sessions, daily stand-ups, and sprint demos.

DevOps culture across every department

open channels of communication, and talk regularly

automation eliminates repetitive manual work, yields repeatable processes, and creates reliable systems.

continuous delivery: the practice of running each code change through a gauntlet of automated tests, often facilitated by cloud-based infrastructure, then packaging up successful builds and promoting them up toward production using automated deploys.

automated deploys alert IT/Ops to server “drift” between environments, which reduces or eliminates surprises when it’s time to release.

when DevOps uses automated deploys to send thoroughly tested code to identically provisioned environments, “Works on my machine!” becomes irrelevant.

A DevOps mindset sees opportunities for continuous improvement everywhere.

regular retrospectives

A/B testing

failure is inevitable. So you might as well set up your team to absorb it, recover, and learn from it (some call this “being anti-fragile”).

Postmortems focus on where processes fell down and how to strengthen them – not on which team member f'ed up the code.

Our engineers are responsible for QA, writing, and running their own tests to get the software out to customers.

How long did it take to go from development to deployment? 

How long does it take to recover after a system failure?

service level agreements (SLAs)

DevOps is big on the idea that the same people who build an application should be involved in shipping and running it.

developers and operators pair with each other in each phase of the application’s lifecycle.

All subsequent make calls must become a NOP unless sources or dependencies change or have been removed.

Do not use directories as a dependency for generated targets, ever.

Parallel make: add an explicit timestamp dependency (.done) that make can synchronize threaded calls on to avoid a race condition.

Wrapper check/unit tests with a ENABLE_TESTS conditional

Modifiers and matchers

Matcher rules determine if a particular request should be forwarded to a backend

if any rule matches

if all rules match

Use a *Prefix* matcher if your backend listens on a particular base path but also serves requests on sub-paths. For instance, PathPrefix: /products would match /products but also /products/shoes and /products/shirts. Since the path is forwarded as-is, your backend is expected to listen on /products

Use Path if your backend listens on the exact path only. For instance, Path: /products would match /products but not /products/shoes.

A backend is responsible to load-balance the traffic coming from one or more frontends to a set of http servers

wrr: Weighted Round Robin

drr: Dynamic Round Robin: increases weights on servers that perform better than others.

A circuit breaker can also be applied to a backend, preventing high loads on failing servers.

To proactively prevent backends from being overwhelmed with high load, a maximum connection limit can also be applied to each backend.

Sticky sessions are supported with both load balancers.

When sticky sessions are enabled, a cookie is set on the initial request.

The check is defined by a path appended to the backend URL and an interval (given in a format understood by time.ParseDuration) specifying how often the health check should be executed (the default being 30 seconds). Each backend must respond to the health check within 5 seconds.

The static configuration is the global configuration which is setting up connections to configuration backends and entrypoints.

We only need to enable watch option to make Træfik watch configuration backend changes and generate its configuration automatically.

a comma-separated key/value pair where both key and value must be literals.

namespacing of your backends happens on the basis of hosts in addition to paths

Modifiers will be applied in a pre-determined order regardless of their order in the rule configuration section.

customize priority

Custom headers can be configured through the frontends, to add headers to either requests or responses that match the frontend's rules.

Security related headers (HSTS headers, SSL redirection, Browser XSS filter, etc) can be added and configured per frontend in a similar manner to the custom headers above.

Servers are simply defined using a url. You can also apply a custom weight to each server (this will be used by load-balancing).

Maximum connections can be configured by specifying an integer value for maxconn.amount and maxconn.extractorfunc which is a strategy used to determine how to categorize requests in order to evaluate the maximum connections.

To use an internet gateway, your subnet's route table must contain a route that directs internet-bound traffic to the internet gateway.

Your instance is only aware of the private (internal) IP address space defined within the VPC and subnet

internet gateway logically provides the one-to-one NAT on behalf of your instance

To enable communication over the internet for IPv6, your VPC and subnet must have an associated IPv6 CIDR block, and your instance must be assigned an IPv6 address from the range of the subnet.

When you create a subnet, we automatically associate it with the main route table for the VPC.

the main route table doesn't contain a route to an internet gateway

Each instance that you launch into a VPC is automatically associated with its default security group.

a default security group allow no inbound traffic from the internet and allow all outbound traffic to the internet.

Communication between pods is more complicated, however, and requires a separate networking component that can transparently route traffic from a pod on one node to a pod on another.

pod network plugins. For this cluster, you will use Flannel, a stable and performant option.

Passing the argument --pod-network-cidr=10.244.0.0/16 specifies the private subnet that the pod IPs will be assigned from.

kubectl apply -f descriptor.[yml|json] is the syntax for telling kubectl to create the objects described in the descriptor.[yml|json] file.

deploy Nginx using Deployments and Services

NodePort, a scheme that will make the pod accessible through an arbitrary port opened on each node of the cluster

load balancing requests to multiple pods

Pods are ubiquitous in Kubernetes, so understanding them will facilitate your work

how controllers such as deployments work since they are used frequently in stateless applications for scaling and the automated healing of unhealthy applications.

Understanding the types of services and the options they have is essential for running both stateless and stateful applications.

Functions perform operations on and within strings

the {{timestamp}} function can be used in any string to generate the current timestamp.

pwd - The working directory while executing Packer.

template_dir - The directory to the template for the build.

uuid - Returns a random UUID.

user - Specifies a user variable.

Template variables are special variables automatically set by Packer at build time.

production stack (docker-compose.yml) does not need any changes and would look like this (and is portable, no docker-sync adjustments).

docker-compose-dev.yml ( it needs to be called that way, look like this ) will override

add the external volume and the mount here

enforce the content from our host on the initial wiring

set your environment variables by creating a .env file at the root of your project

{% ... %} for Statements

{{ ... }} for Expressions to print to the template output

use a dot (.) to access attributes of a variable

the outer double-curly braces are not part of the variable, but the print statement.

if an object has an item and attribute with the same name. Additionally, the attr() filter only looks up attributes.

Variables can be modified by filters. Filters are separated from the variable by a pipe symbol (|) and may have optional arguments in parentheses.

Multiple filters can be chained

Tests can be used to test a variable against a common expression.

add is plus the name of the test after the variable.

to find out if a variable is defined, you can do name is defined, which will then return true or false depending on whether name is defined in the current template context.

strip whitespace in templates by hand. If you add a minus sign (-) to the start or end of a block (e.g. a For tag), a comment, or a variable expression, the whitespaces before or after that block will be removed

not add whitespace between the tag and the minus sign

mark a block raw

Template inheritance allows you to build a base “skeleton” template that contains all the common elements of your site and defines blocks that child templates can override.

The {% extends %} tag is the key here. It tells the template engine that this template “extends” another template.

access templates in subdirectories with a slash

can’t define multiple {% block %} tags with the same name in the same template

put the name of the block after the end tag for better readability

if the block is replaced by a child template, a variable would appear that was not defined in the block or passed to the context.

setting the block to “scoped” by adding the scoped modifier to a block declaration

Jinja2 functions (macros, super, self.BLOCKNAME) always return template data that is marked as safe.

With the default syntax, control structures appear inside {% ... %} blocks.

the dictsort filter

loop.cycle

use loops recursively

whether the value changed at all,

use it to test if a variable is defined, not empty and not false

Macros are comparable with functions in regular programming languages.

If a macro name starts with an underscore, it’s not exported and can’t be imported.

pass a macro to another macro

caller()

a single trailing newline is stripped if present

other whitespace (spaces, tabs, newlines etc.) is returned unchanged

caller(user)

call(user)

This is a simple dialog rendered by using a macro and a call block.

Filter sections allow you to apply regular Jinja2 filters on a block of template data.

Assignments at top level (outside of blocks, macros or loops) are exported from the template like top level macros and can be imported by other templates.

using namespace objects which allow propagating of changes across scopes

use block assignments to capture the contents of a block into a variable name.

The extends tag can be used to extend one template from another.

Blocks are used for inheritance and act as both placeholders and replacements at the same time.

The include statement is useful to include a template and return the rendered contents of that file into the current namespace

Included templates have access to the variables of the active context by default.

putting often used code into macros

imports are cached and imported templates don’t have access to the current template variables, just the globals by default.

Macros and variables starting with one or more underscores are private and cannot be imported.

By default, included templates are passed the current context and imported templates are not.

Integers and floating point numbers are created by just writing the number down

Everything between two brackets is a list.

Tuples are like lists that cannot be modified (“immutable”).

A dict in Python is a structure that combines keys and values.

// Divide two numbers and return the truncated integer result

(expr) group an expression.

The is and in operators support negation using an infix notation

in Perform a sequence / mapping containment test.

| Applies a filter.

~ Converts all operands into strings and concatenates them.

use inline if expressions.

always an attribute is returned and items are not looked up.

default(value, default_value=u'', boolean=False)¶ If the value is undefined it will return the passed default value, otherwise the value of the variable

dictsort(value, case_sensitive=False, by='key', reverse=False)¶ Sort a dict and yield (key, value) pairs.

format(value, *args, **kwargs)¶ Apply python string formatting on an object

indent(s, width=4, first=False, blank=False, indentfirst=None)¶ Return a copy of the string with each line indented by 4 spaces. The first line and blank lines are not indented by default.

join(value, d=u'', attribute=None)¶ Return a string which is the concatenation of the strings in the sequence.

map()¶ Applies a filter on a sequence of objects or looks up an attribute.

pprint(value, verbose=False)¶ Pretty print a variable. Useful for debugging.

reject()¶ Filters a sequence of objects by applying a test to each object, and rejecting the objects with the test succeeding.

replace(s, old, new, count=None)¶ Return a copy of the value with all occurrences of a substring replaced with a new one.

round(value, precision=0, method='common')¶ Round the number to a given precision

even if rounded to 0 precision, a float is returned.

select()¶ Filters a sequence of objects by applying a test to each object, and only selecting the objects with the test succeeding.

sort(value, reverse=False, case_sensitive=False, attribute=None)¶ Sort an iterable. Per default it sorts ascending, if you pass it true as first argument it will reverse the sorting.

striptags(value)¶ Strip SGML/XML tags and replace adjacent whitespace by one space.

tojson(value, indent=None)¶ Dumps a structure to JSON so that it’s safe to use in <script> tags.

trim(value)¶ Strip leading and trailing whitespace.

unique(value, case_sensitive=False, attribute=None)¶ Returns a list of unique items from the the given iterable

urlize(value, trim_url_limit=None, nofollow=False, target=None, rel=None)¶ Converts URLs in plain text into clickable links.

defined(value)¶ Return true if the variable is defined

in(value, seq)¶ Check if value is in seq.

mapping(value)¶ Return true if the object is a mapping (dict etc.).

number(value)¶ Return true if the variable is a number.

sameas(value, other)¶ Check if an object points to the same memory address than another object

undefined(value)¶ Like defined() but the other way round.

A joiner is passed a string and will return that string every time it’s called, except the first time (in which case it returns an empty string).