Group items tagged

Return an ActiveRecord::Relation

Filtering in Ruby is slower

trigger the query and therefore, we lose our Relation

leaving trivial ordering out of scopes all together.

where

where

ActiveRecord provides an easy API for doing many things with our database, but it also makes it pretty easy to do things inefficiently. The layer of abstraction hides what’s really happening.

first pure SQL, then ActiveRecord

Databases can only do fast lookups for columns with indexes, otherwise it’s doing a sequential scan

Add an index on every id column as well as any column that is used in a where clause.

use a Query class to encapsulate the potentially gnarly query.

subqueries

Single Responsibility Principle

Avoid ad-hoc queries outside of Scopes and Query Objects

encapsulate data access into scopes and Query objects

An ad-hoc query embedded in a controller (or view, task, etc) is harder to test in isolation and cannot be reused

to scopes and Query objects

Every databases provides more datatypes than your ORM might have you believe

Both Postgres and MySQL have full-text search capabilities

Associations are implemented using macro-style calls, so that you can declaratively add features to your models

A belongs_to association sets up a one-to-one connection with another model, such that each instance of the declaring model "belongs to" one instance of the other model.

belongs_to

A has_one association also sets up a one-to-one connection with another model, but with somewhat different semantics (and consequences).

belongs_to

A has_many association indicates a one-to-many connection with another model.

This association indicates that each instance of the model has zero or more instances of another model.

belongs_to

A has_many :through association is often used to set up a many-to-many connection with another model

This association indicates that the declaring model can be matched with zero or more instances of another model by proceeding through a third model.

through:

through:

The collection of join models can be managed via the API

new join models are created for newly associated objects, and if some are gone their rows are deleted.

The has_many :through association is also useful for setting up "shortcuts" through nested has_many associations

A has_one :through association sets up a one-to-one connection with another model. This association indicates that the declaring model can be matched with one instance of another model by proceeding through a third model.

A has_and_belongs_to_many association creates a direct many-to-many connection with another model, with no intervening model.

id: false

The has_one relationship says that one of something is yours

using t.references :supplier instead.

declare a many-to-many relationship is to use has_many :through. This makes the association indirectly, through a join model

set up a has_many :through relationship if you need to work with the relationship model as an independent entity

set up a has_and_belongs_to_many relationship (though you'll need to remember to create the joining table in the database).

use has_many :through if you need validations, callbacks, or extra attributes on the join model

With polymorphic associations, a model can belong to more than one other model, on a single association.

a polymorphic belongs_to declaration as setting up an interface that any other model can use.

In designing a data model, you will sometimes find a model that should have a relation to itself

add a references column to the model itself

All of the association methods are built around caching, which keeps the result of the most recent query available for further operations.

it is a bad idea to give an association a name that is already used for an instance method of ActiveRecord::Base. The association method would override the base method and break things.

belongs_to associations you need to create foreign keys

has_and_belongs_to_many associations you need to create the appropriate join table

So a join between customer and order models will give the default join table name of "customers_orders" because "c" outranks "o" in lexical ordering.

For example, one would expect the tables "paper_boxes" and "papers" to generate a join table name of "papers_paper_boxes" because of the length of the name "paper_boxes", but it in fact generates a join table name of "paper_boxes_papers" (because the underscore '' is lexicographically _less than 's' in common encodings).

id: false

pass id: false to create_table because that table does not represent a model

By default, associations look for objects only within the current module's scope.

will work fine, because both the Supplier and the Account class are defined within the same scope.

To associate a model with a model in a different namespace, you must specify the complete class name in your association declaration:

class_name

class_name

Active Record provides the :inverse_of option

Every association will attempt to automatically find the inverse association and set the :inverse_of option heuristically (based on the association name)

In database terms, this association says that this class contains the foreign key.

In all of these methods, association is replaced with the symbol passed as the first argument to belongs_to.

(force_reload = false)

The association method returns the associated object, if any. If no associated object is found, it returns nil.

the cached version will be returned.

The association= method assigns an associated object to this object.

Behind the scenes, this means extracting the primary key from the associate object and setting this object's foreign key to the same value.

The build_association method returns a new object of the associated type

The create_association method returns a new object of the associated type

raises ActiveRecord::RecordInvalid if the record is invalid.

dependent

counter_cache

:autosave :class_name :counter_cache :dependent :foreign_key :inverse_of :polymorphic :touch :validate

finding the number of belonging objects more efficient.

Although the :counter_cache option is specified on the model that includes the belongs_to declaration, the actual column must be added to the associated model.

:destroy, when the object is destroyed, destroy will be called on its associated objects.

The :inverse_of option specifies the name of the has_many or has_one association that is the inverse of this association

set the :touch option to :true, then the updated_at or updated_on timestamp on the associated object will be set to the current time whenever this object is saved or destroyed

specify a particular timestamp attribute to update

If you set the :validate option to true, then associated objects will be validated whenever you save this object

where includes readonly select

make your code somewhat more efficient

no need to use includes for immediate associations

will be read-only when retrieved via the association

The select method lets you override the SQL SELECT clause that is used to retrieve data about the associated object

using the association.nil?

In database terms, this association says that the other class contains the foreign key.

the cached version will be returned.

:as :autosave :class_name :dependent :foreign_key :inverse_of :primary_key :source :source_type :through :validate

Setting the :as option indicates that this is a polymorphic association

:nullify causes the foreign key to be set to NULL. Callbacks are not executed.

It's necessary not to set or leave :nullify option for those associations that have NOT NULL database constraints.

The :source_type option specifies the source association type for a has_one :through association that proceeds through a polymorphic association.

The :source option specifies the source association name for a has_one :through association.

The :through option specifies a join model through which to perform the query

more efficient by including representatives in the association from suppliers to accounts

When you assign an object to a has_one association, that object is automatically saved (in order to update its foreign key).

If either of these saves fails due to validation errors, then the assignment statement returns false and the assignment itself is cancelled.

If the parent object (the one declaring the has_one association) is unsaved (that is, new_record? returns true) then the child objects are not saved.

If you want to assign an object to a has_one association without saving the object, use the association.build method

collection(force_reload = false) collection<<(object, ...) collection.delete(object, ...) collection.destroy(object, ...) collection=(objects) collection_singular_ids collection_singular_ids=(ids) collection.clear collection.empty? collection.size collection.find(...) collection.where(...) collection.exists?(...) collection.build(attributes = {}, ...) collection.create(attributes = {}) collection.create!(attributes = {})

In all of these methods, collection is replaced with the symbol passed as the first argument to has_many, and collection_singular is replaced with the singularized version of that symbol.

The collection<< method adds one or more objects to the collection by setting their foreign keys to the primary key of the calling model

The collection.delete method removes one or more objects from the collection by setting their foreign keys to NULL.

objects will be destroyed if they're associated with dependent: :destroy, and deleted if they're associated with dependent: :delete_all

The collection.destroy method removes one or more objects from the collection by running destroy on each object.

The collection_singular_ids method returns an array of the ids of the objects in the collection.

The collection_singular_ids= method makes the collection contain only the objects identified by the supplied primary key values, by adding and deleting as appropriate

The default strategy for has_many :through associations is delete_all, and for has_many associations is to set the foreign keys to NULL.

The collection.clear method removes all objects from the collection according to the strategy specified by the dependent option

uses the same syntax and options as ActiveRecord::Base.find

The collection.where method finds objects within the collection based on the conditions supplied but the objects are loaded lazily meaning that the database is queried only when the object(s) are accessed.

The collection.build method returns one or more new objects of the associated type. These objects will be instantiated from the passed attributes, and the link through their foreign key will be created, but the associated objects will not yet be saved.

The collection.create method returns a new object of the associated type. This object will be instantiated from the passed attributes, the link through its foreign key will be created, and, once it passes all of the validations specified on the associated model, the associated object will be saved.

:as :autosave :class_name :dependent :foreign_key :inverse_of :primary_key :source :source_type :through :validate

:nullify causes the foreign keys to be set to NULL. Callbacks are not executed.

where includes readonly select

:conditions :through :polymorphic :foreign_key

By convention, Rails assumes that the column used to hold the primary key of the association is id. You can override this and explicitly specify the primary key with the :primary_key option.

The :source option specifies the source association name for a has_many :through association.

You only need to use this option if the name of the source association cannot be automatically inferred from the association name.

The :source_type option specifies the source association type for a has_many :through association that proceeds through a polymorphic association.

The :through option specifies a join model through which to perform the query.

has_many :through associations provide a way to implement many-to-many relationships,

By default, this is true: associated objects will be validated when this object is saved.

where extending group includes limit offset order readonly select uniq

If you use a hash-style where option, then record creation via this association will be automatically scoped using the hash

The extending method specifies a named module to extend the association proxy.

Association extensions

The group method supplies an attribute name to group the result set by, using a GROUP BY clause in the finder SQL.

more efficient by including line items in the association from customers to orders

The limit method lets you restrict the total number of objects that will be fetched through an association.

The offset method lets you specify the starting offset for fetching objects via an association

The order method dictates the order in which associated objects will be received (in the syntax used by an SQL ORDER BY clause).

Use the distinct method to keep the collection free of duplicates.

mostly useful together with the :through option

-> { distinct }

.all.inspect

If you want to make sure that, upon insertion, all of the records in the persisted association are distinct (so that you can be sure that when you inspect the association that you will never find duplicate records), you should add a unique index on the table itself

Do not attempt to use include? to enforce distinctness in an association.

When you assign an object to a has_many association, that object is automatically saved (in order to update its foreign key).

If any of these saves fails due to validation errors, then the assignment statement returns false and the assignment itself is cancelled.

If the parent object (the one declaring the has_many association) is unsaved (that is, new_record? returns true) then the child objects are not saved when they are added

assign an object to a has_many association without saving the object, use the collection.build method

collection(force_reload = false) collection<<(object, ...) collection.delete(object, ...) collection.destroy(object, ...) collection=(objects) collection_singular_ids collection_singular_ids=(ids) collection.clear collection.empty? collection.size collection.find(...) collection.where(...) collection.exists?(...) collection.build(attributes = {}) collection.create(attributes = {}) collection.create!(attributes = {})

If the join table for a has_and_belongs_to_many association has additional columns beyond the two foreign keys, these columns will be added as attributes to records retrieved via that association.

If you require this sort of complex behavior on the table that joins two models in a many-to-many relationship, you should use a has_many :through association instead of has_and_belongs_to_many.

The collection.delete method removes one or more objects from the collection by deleting records in the join table

The collection.destroy method removes one or more objects from the collection by running destroy on each record in the join table, including running callbacks.

The collection.clear method removes every object from the collection by deleting the rows from the joining table.

The collection.find method finds objects within the collection. It uses the same syntax and options as ActiveRecord::Base.find.

The collection.where method finds objects within the collection based on the conditions supplied but the objects are loaded lazily meaning that the database is queried only when the object(s) are accessed.

The collection.exists? method checks whether an object meeting the supplied conditions exists in the collection.

The collection.build method returns a new object of the associated type.

The collection.create method returns a new object of the associated type.

it passes all of the validations specified on the associated model

:association_foreign_key :autosave :class_name :foreign_key :join_table :validate

The :foreign_key and :association_foreign_key options are useful when setting up a many-to-many self-join.

Rails assumes that the column in the join table used to hold the foreign key pointing to the other model is the name of that model with the suffix _id added.

If you set the :autosave option to true, Rails will save any loaded members and destroy members that are marked for destruction whenever you save the parent object.

By convention, Rails assumes that the column in the join table used to hold the foreign key pointing to this model is the name of this model with the suffix _id added.

By default, this is true: associated objects will be validated when this object is saved.

where extending group includes limit offset order readonly select uniq

set conditions via a hash

In this case, using @parts.assemblies.create or @parts.assemblies.build will create orders where the factory column has the value "Seattle"

using a GROUP BY clause in the finder SQL.

Use the uniq method to remove duplicates from the collection.

assign an object to a has_and_belongs_to_many association, that object is automatically saved (in order to update the join table).

If any of these saves fails due to validation errors, then the assignment statement returns false and the assignment itself is cancelled.

If the parent object (the one declaring the has_and_belongs_to_many association) is unsaved (that is, new_record? returns true) then the child objects are not saved when they are added.

If you want to assign an object to a has_and_belongs_to_many association without saving the object, use the collection.build method.

Normal callbacks hook into the life cycle of Active Record objects, allowing you to work with those objects at various points

define association callbacks by adding options to the association declaration

stack callbacks on a single event by passing them as an array

If a before_add callback throws an exception, the object does not get added to the collection.

if a before_remove callback throws an exception, the object does not get removed from the collection

extend these objects through anonymous modules, adding new finders, creators, or other methods.

order_number

use a named extension module

proxy_association.owner returns the object that the association is a part of.

SysBench is not a tool which you can use to tune configurations of your MySQL servers (unless you prepared LUA scripts with custom workload or your workload happen to be very similar to the benchmark workloads that SysBench comes with)

it is great for is to compare performance of different hardware.

Every new server acquired should go through a warm-up period during which you will stress it to pinpoint potential hardware defects

by executing OLTP workload which overloads the server, or you can also use dedicated benchmarks for CPU, disk and memory.

bulk_insert.lua. This test can be used to benchmark the ability of MySQL to perform multi-row inserts.

All oltp_* scripts share a common table structure. First two of them (oltp_delete.lua and oltp_insert.lua) execute single DELETE and INSERT statements.

oltp_point_select, oltp_update_index and oltp_update_non_index. These will execute a subset of queries - primary key-based selects, index-based updates and non-index-based updates.

you can run different workload patterns using the same benchmark.

Warmup helps to identify “regular” throughput by executing benchmark for a predefined time, allowing to warm up the cache, buffer pools etc.

By default SysBench will attempt to execute queries as fast as possible. To simulate slower traffic this option may be used. You can define here how many transactions should be executed per second.

SysBench gives you ability to generate different types of data distribution.

decide if SysBench should use prepared statements (as long as they are available in the given datastore - for MySQL it means PS will be enabled by default) or not.

sysbench ./sysbench/src/lua/oltp_read_write.lua  help

By default, SysBench will attempt to execute queries in explicit transaction. This way the dataset will stay consistent and not affected: SysBench will, for example, execute INSERT and DELETE on the same row, making sure the data set will not grow (impacting your ability to reproduce results).

specify error codes from MySQL which SysBench should ignore (and not kill the connection).

the two most popular benchmarks - OLTP read only and OLTP read/write.

1 million rows will result in ~240 MB of data. Ten tables, 1000 000 rows each equals to 2.4GB

by default, SysBench looks for ‘sbtest’ schema which has to exist before you prepare the data set. You may have to create it manually.

pass ‘--histogram’ argument to SysBench

~48GB of data (20 tables, 10 000 000 rows each).

if you don’t understand why the performance was like it was, you may draw incorrect conclusions out of the benchmarks.

Different programming languages are for different tasks.

Go or C which are compiled to run on bare metal.

To run NodeJS on multiple cores, you have to use something like PM2, but since this you have to keep your code stateless.

Python have very rich and sugary syntax that’s great for working with data while keeping your code small and expressive.

SQL is almost always slower than NoSQL

databases are often read-first or write-first

write-first, just like Cassandra.

store all of your data to your databases and leave nothing at backend

Functional code is stateless by default

It’s better to go for stateless right from the very beginning.

deliver exactly the same responses for same requests.

Sessions? Store them at Redis and allow all of your servers to access it.

Only the first user will trigger a data query, and all others will be receiving exactly the same data straight from the RAM

Only the server output should be cached

Varnish is a great cache option that works with HTTP responses, so it may work with any backend.

a rate limiter – if there’s not enough time have passed since last request, the ongoing request will be denied.

Just set up entry relations and allow your database to calculate external keys for you

Backend should have different responsibilities: hashing, building web pages from data and templates, managing sessions and so on.

a distributed database.

your code has to be stateless

Move anything related to the data to the database.

For load-balancing a database, go for cluster.

DB is balancing requests, as well as your backend.

Users from different continents are separated with DNS.

Keep is scalable, keep is stateless.

存取資料庫是一種相對很慢的I/O的操作：每一條SQL query都得耗上時間、執行回傳的結果也會被轉成ActiveRecord物件全部放進記憶體

如果需要撈出全部的資料做處理，強烈建議最好不要用all方法，因為這樣會把全部的資料一次放進記憶體中，如果資料有成千上萬筆的話，效能就墜毀了。

.find_each( :batch_size => 100 )

.find_in_batches( :batch_size => 100 )

在Transaction交易範圍內的SQL效能會加快，因為最後只需要COMMIT一次即可

Elasticsearch全文搜尋引擎和elasticsearch-rails gem

QueryReviewer這個套件透過SQL EXPLAIN分析SQL query的效率

必要時可以採用逆正規化的設計。犧牲空間，增加修改的麻煩，但是讓讀取這事件變得更快更簡單。

將成本轉嫁到寫入，而最佳化了讀取時間

在效能還沒有造成問題前，就為了優化效能而修改程式和架構，只會讓程式更混亂不好維護

當效能還不會造成問題時，程式的維護性比考慮效能重要

會拖慢整體效能的程式，只佔全部程式的一小部分而已，所以我們只最佳化會造成問題的程式。

善用分析工具找效能瓶頸，最佳化前需要測量，最佳化後也要測量比較。

rack-mini-profiler在頁面的左上角顯示花了多少時間，並且提供報表，推薦安裝

如果是不需要權限控管的靜態檔案，可以直接放在public目錄下讓使用者下載。

Web伺服器得先安裝好x_sendfile功能

如果要讓你的Assets例如CSS, JavaScript, Images也讓使用者透過CDN下載，只要修改config/environments/production.rb的config.action_controller.asset_host為CDN網址即可。

有時候「執行速度較快」的程式碼不代表好維護、好除錯的程式碼

Ruby不是萬能，有時候直接呼叫外部程式是最快的作法

Data Manipulation Language (DML), commands are used to modify data in a database. DML statements control access to the database data.

DDL deals with descriptions of the database schema and is useful for creating new tables, indexes, sequences, stogroups, etc. and to define the attributes of these objects, such as data type, field length and alternate table names (aliases).

Data Query Language (DQL) is used to get data within the schema objects of a database and also to query it and impose order upon it.

DQL is also a subset of SQL. One of the most common commands in DQL is SELECT.

The most common command types in DDL are CREATE, ALTER and DROP.

Unique string used to identify the operation.

The id MUST be unique among all operations described in the API.

A list of MIME types the operation can consume.

A list of MIME types the operation can produce

A unique parameter is defined by a combination of a name and location.

There can be one "body" parameter at most.

Required. The list of possible responses as they are returned from executing this operation.

The transfer protocol for the operation. Values MUST be from the list: "http", "https", "ws", "wss".

Declares this operation to be deprecated. Usage of the declared operation should be refrained. Default value is

A unique parameter is defined by a combination of a name and location.

Path

Query

Header

Body

Form

Required. The location of the parameter. Possible values are "query", "header", "path", "formData" or "body".

the parameter value is actually part of the operation's URL

Parameters that are appended to the URL

The payload that's appended to the HTTP request.

The name of the body parameter has no effect on the parameter itself and is used for documentation purposes only

This is the only parameter type that can be used to send files, thus supporting the file type.

If the parameter is in "path", this property is required and its value MUST be true.

default value is false.

The schema defining the type used for the body parameter.

The value MUST be one of "string", "number", "integer", "boolean", "array" or "file"

Default value is false

Required if type is "array". Describes the type of items in the array.

Determines the format of the array if type array is used

enum

pattern

each action also maps to particular CRUD operations in a database

One way to avoid deep nesting (as recommended above) is to generate the collection actions scoped under the parent, so as to get a sense of the hierarchy, but to not nest the member actions.

to only build routes with the minimal amount of information to uniquely identify the resource

The shallow method of the DSL creates a scope inside of which every nesting is shallow

These concerns can be used in resources to avoid code duplication and share behavior across routes

add a member route, just add a member block into the resource block

You can leave out the :on option, this will create the same member route except that the resource id value will be available in params[:photo_id] instead of params[:id].

Singular Resources

use a singular resource to map /profile (rather than /profile/:id) to the show action

Passing a String to get will expect a controller#action format

workaround

organize groups of controllers under a namespace

route /articles (without the prefix /admin) to Admin::ArticlesController

route /admin/articles to ArticlesController (without the Admin:: module prefix)

Nested routes allow you to capture this relationship in your routing.

Resources should never be nested more than 1 level deep.

via the :shallow option

a balance between descriptive routes and deep nesting

:shallow_path prefixes member paths with the specified parameter

Routing Concerns allows you to declare common routes that can be reused inside other resources and routes

Rails can also create paths and URLs from an array of parameters.

use url_for with a set of objects

In helpers like link_to, you can specify just the object in place of the full url_for call

insert the action name as the first element of the array

pass :on to a route, eliminating the block:

Collection Routes

simple routing makes it very easy to map legacy URLs to new Rails actions

add an alternate new action using the :on shortcut

When you set up a regular route, you supply a series of symbols that Rails maps to parts of an incoming HTTP request.

:controller maps to the name of a controller in your application

:action maps to the name of an action within that controller

This route will also route the incoming request of /photos to PhotosController#index, since :action and :id are

use a constraint on :controller that matches the namespace you require

dynamic segments don't accept dots

The params will also include any parameters from the query string

:defaults option.

set params[:format] to "jpg"

specify a name for any route using the :as option

create logout_path and logout_url as named helpers in your application.

Inside the show action of UsersController, params[:username] will contain the username for the user.

should use the get, post, put, patch and delete methods to constrain a route to a particular verb.

use the match method with the :via option to match multiple verbs at once

use the :constraints option to enforce a format for a dynamic segment

constraints

don't need to use anchors

the same name as the hash key and then compare the return value with the hash value.

constraint values should match the corresponding Request object method return type