Skip to main content

Home/ Larvata/ Group items tagged data

Rss Feed Group items tagged

Filter: All | Bookmarks | Topics Simple Middle
crazylion lee

Lecture Videos | Design and Analysis of Algorithms | Electrical Engineering and Compute... - 0 views

ocw.mit.edu/...lecture-videos

data Analysis

shared by crazylion lee on 11 Jul 16 - No Cached
  • crazylion lee on 11 Jul 16
     
    "Design and Analysis of Algorithms "
張 旭

Serverless Architectures - 0 views

martinfowler.com/...serverless.html

development architecture system server backend

shared by 張 旭 on 12 Jul 16 - No Cached
  • Serverless was first used to describe applications that significantly or fully depend on 3rd party applications / services (‘in the cloud’) to manage server-side logic and state.
  • ‘rich client’ applications (think single page web apps, or mobile apps) that use the vast ecosystem of cloud accessible databases (like Parse, Firebase), authentication services (Auth0, AWS Cognito), etc.
  • ‘(Mobile) Backend as a Service’
  • ...33 more annotations...
  • Serverless can also mean applications where some amount of server-side logic is still written by the application developer but unlike traditional architectures is run in stateless compute containers that are event-triggered, ephemeral (may only last for one invocation), and fully managed by a 3rd party.
  • ‘Functions as a service
  • AWS Lambda is one of the most popular implementations of FaaS at present,
  • A good example is Auth0 - they started initially with BaaS ‘Authentication as a Service’, but with Auth0 Webtask they are entering the FaaS space.
  • a typical ecommerce app
  • a backend data-processing service
  • with zero administration.
  • FaaS offerings do not require coding to a specific framework or library.
  • Horizontal scaling is completely automatic, elastic, and managed by the provider
  • Functions in FaaS are triggered by event types defined by the provider.
  • a FaaS-supported message broker
  • from a deployment-unit point of view FaaS functions are stateless.
  • allowed the client direct access to a subset of our database
  • deleted the authentication logic in the original application and have replaced it with a third party BaaS service
  • The client is in fact well on its way to becoming a Single Page Application.
  • implement a FaaS function that responds to http requests via an API Gateway
  • port the search code from the Pet Store server to the Pet Store Search function
  • replaced a long lived consumer application with a FaaS function that runs within the event driven context
  • server applications - is a key difference when comparing with other modern architectural trends like containers and PaaS
  • the only code that needs to change when moving to FaaS is the ‘main method / startup’ code, in that it is deleted, and likely the specific code that is the top-level message handler (the ‘message listener interface’ implementation), but this might only be a change in method signature
  • With FaaS you need to write the function ahead of time to assume parallelism
  • Most providers also allow functions to be triggered as a response to inbound http requests, typically in some kind of API gateway
  • you should assume that for any given invocation of a function none of the in-process or host state that you create will be available to any subsequent invocation.
  • FaaS functions are either naturally stateless
  • store state across requests or for further input to handle a request.
  • certain classes of long lived task are not suited to FaaS functions without re-architecture
  • if you were writing a low-latency trading application you probably wouldn’t want to use FaaS systems at this time
  • An API Gateway is an HTTP server where routes / endpoints are defined in configuration and each route is associated with a FaaS function.
  • API Gateway will allow mapping from http request parameters to inputs arguments for the FaaS function
  • API Gateways may also perform authentication, input validation, response code mapping, etc.
  • the Serverless Framework makes working with API Gateway + Lambda significantly easier than using the first principles provided by AWS.
  • Apex - a project to ‘Build, deploy, and manage AWS Lambda functions with ease.'
  • 'Serverless' to mean the union of a couple of other ideas - 'Backend as a Service' and 'Functions as a Service'.
張 旭

Transactions - RSpec Rails - RSpec - Relish - 0 views

relishapp.com/...transactions

rspec test rails programming

shared by 張 旭 on 03 Apr 17 - No Cached
  • before(:all) hooks are invoked before the transaction is opened. You can use this to speed things up by creating data once before any example in a group is run
張 旭

MonolithFirst - 0 views

martinfowler.com/...MonolithFirst.html

system architecture deploy

shared by 張 旭 on 05 Jun 15 - No Cached
  • Microservices are a useful architecture, but even their advocates say that using them incurs a significant MicroservicePremium, which means they are only useful with more complex systems.
  • you should build a new application as a monolith initially, even if you think it's likely that it will benefit from a microservices architecture later on.
  • Any refactoring of functionality between services is much harder than it is in a monolith.
  • ...4 more annotations...
  • By building a monolith first, you can figure out what the right boundaries are, before a microservices design brushes a layer of treacle over them.
  • The logical way is to design a monolith carefully, paying attention to modularity within the software, both at the API boundaries and how the data is stored.
  • start with a monolith and gradually peel off microservices at the edges
  • Don't be afraid of building a monolith that you will discard, particularly if a monolith can get you to market quickly
張 旭

David Verhasselt - Different Ways to Set Attributes in ActiveRecord - 0 views

www.davidverhasselt.com/set-attributes-in-activerecord

rails model ORM

shared by 張 旭 on 18 Mar 15 - No Cached
  • update
  • attribute will be marked as dirty and the change will not be sent to the database yet
  • will change the attribute in the model and pass it straight to the database, without running any validations
  • ...9 more annotations...
  • end up with invalid data
  • Any other changed attributes are also saved
  • set all the attributes you pass it. The changes are not saved to the database. Any attributes you don’t pass will be left unchanged
  • also saves other changed attributes to the database
  • executes a direct SQL UPDATE query and bypasses any validations or callbacks.
  • finds the object with the specified ID and updates it’s attributes with the passed in hash
  • pass in an array of ID’s and parameters
  • runs an SQL UPDATE query that updates the attributes of all objects without running any validations or callbacks
  • call this method on a scoped relation
張 旭

The Rails Command Line - Ruby on Rails Guides - 0 views

guides.rubyonrails.org/command_line.html

rails console command

shared by 張 旭 on 28 Nov 14 - Cached
  • rake --tasks
  • Think of destroy as the opposite of generate.
  • runner runs Ruby code in the context of Rails non-interactively
  • ...28 more annotations...
  • rails dbconsole figures out which database you're using and drops you into whichever command line interface you would use with it
  • The console command lets you interact with your Rails application from the command line. On the underside, rails console uses IRB
  • rake about gives information about version numbers for Ruby, RubyGems, Rails, the Rails subcomponents, your application's folder, the current Rails environment name, your app's database adapter, and schema version
  • You can precompile the assets in app/assets using rake assets:precompile and remove those compiled assets using rake assets:clean.
  • rake db:version is useful when troubleshooting
  • The doc: namespace has the tools to generate documentation for your app, API documentation, guides.
  • rake notes will search through your code for comments beginning with FIXME, OPTIMIZE or TODO.
  • You can also use custom annotations in your code and list them using rake notes:custom by specifying the annotation using an environment variable ANNOTATION.
  • rake routes will list all of your defined routes, which is useful for tracking down routing problems in your app, or giving you a good overview of the URLs in an app you're trying to get familiar with.
  • rake secret will give you a pseudo-random key to use for your session secret.
  • Custom rake tasks have a .rake extension and are placed in Rails.root/lib/tasks.
  • rails new . --git --database=postgresql
  • All commands can run with -h or --help to list more information
  • The rails server command launches a small web server named WEBrick which comes bundled with Ruby
  • rails server -e production -p 4000
  • You can run a server as a daemon by passing a -d option
  • The rails generate command uses templates to create a whole lot of things.
  • Using generators will save you a large amount of time by writing boilerplate code, code that is necessary for the app to work.
  • All Rails console utilities have help text.
  • generate controller ControllerName action1 action2.
  • With a normal, plain-old Rails application, your URLs will generally follow the pattern of http://(host)/(controller)/(action), and a URL like http://(host)/(controller) will hit the index action of that controller.
  • A scaffold in Rails is a full set of model, database migration for that model, controller to manipulate it, views to view and manipulate the data, and a test suite for each of the above.
  • Unit tests are code that tests and makes assertions about code.
  • Unit tests are your friend.
  • rails console --sandbox
  • rails db
  • Each task has a description, and should help you find the thing you need.
  • rake tmp:clear clears all the three: cache, sessions and sockets.
張 旭

Asset Pipeline - Ruby on Rails 指南 - 0 views

guides.ruby-china.org/asset_pipeline.html

shared by 張 旭 on 30 Jan 15 - No Cached
  • 清单文件或帮助方法
    • 張 旭
      張 旭 on 30 Jan 15
       
      清單文件是指：application.css 跟 application.js
  • Sprockets 会按照搜索路径中各路径出现的顺序进行搜索。默认情况下，这意味着 app/assets 文件夹中的静态资源优先级较高，会遮盖 lib 和 vendor 文件夹中的相应文件
  • 如果静态资源不会在清单文件中引入，就要添加到预编译的文件列表中，否则在生产环境中就无法访问文件。
  • ...36 more annotations...
  • 程序中使用了 jQuery 代码库和许多模块，都保存在 lib/assets/javascripts/library_name 文件夹中，那么 lib/assets/javascripts/library_name/index.js 文件的作用就是这个代码库的清单。清单文件中可以按顺序列出所需的文件，或者干脆使用 require_tree 指令。
  • 如果使用 Turbolinks（Rails 4 默认启用），加上 data-turbolinks-track 选项后，Turbolinks 会检查静态资源是否有更新，如果更新了就会将其载入页面
  • config.assets.paths 包含标准路径和其他 Rails 引擎添加的路径。
  • 链接不存在的资源（也包括链接到空字符串的情况）会在调用页面抛出异常。
  • 关闭标签不能使用 -%> 形式
  • Sprockets 通过清单文件决定要引入和伺服哪些静态资源
  • 在 JavaScript 文件中，Sprockets 的指令以 //= 开头。在上面的文件中，用到了 require 和 the require_tree 指令。
  • app/assets/javascripts/application.js
  • require_tree 指令告知 Sprockets 递归引入指定文件夹中的所有 JavaScript 文件。文件夹的路径必须相对于清单文件。也可使用 require_directory 指令加载指定文件夹中的所有 JavaScript 文件，但不会递归。
  • Sprockets 会按照从上至下的顺序处理指令，但 require_tree 引入的文件顺序是不可预期的，不要设想能得到一个期望的顺序。
  • app/assets/stylesheets/application.css
  • 不管创建新程序时有没有指定 --skip-sprockets 选项，Rails 4 都会生成 app/assets/javascripts/application.js 和 app/assets/stylesheets/application.css
  • 如果多次调用 require_self，只有最后一次调用有效
  • 如果想使用多个 Sass 文件，应该使用 Sass 中的 @import 规则，不要使用 Sprockets 指令。
  • 清单文件可以有多个。
  • 如果使用默认的 gem，生成控制器或脚手架时，会生成 CoffeeScript 和 SCSS 文件，而不是普通的 JavaScript 和 CSS 文件。
  • 在开发环境中，或者禁用 Asset Pipeline 时，这些文件会使用 coffee-script 和 sass 提供的预处理器处理，然后再发给浏览器
  • 启用 Asset Pipeline 时，这些文件会先使用预处理器处理，然后保存到 public/assets 文件夹中，再由 Rails 程序或网页服务器伺服
  • 添加额外的扩展名可以增加预处理次数，预处理程序会按照扩展名从右至左的顺序处理文件内容。所以，扩展名的顺序要和处理的顺序一致
  • 预处理器的执行顺序很重要
  • 在开发环境中也可启用压缩功能，检查是否能正常运行。需要调试时再禁用压缩即可。
  • 默认情况下，Rails 认为静态资源已经事先编译好了，直接由网页服务器伺服。
  • 般情况下，请勿修改 config.assets.digest 的默认值
  • 可在部署时编译静态资源
  • 在多次部署之间共用这个文件夹是十分重要的，这样只要缓存的页面可用，其中引用的编译后的静态资源就能正常使用。
  • 默认编译的文件包括 application.js、application.css 以及 gem 中 app/assets 文件夹中的所有非 JS/CSS 文件（会自动加载所有图片）
  • 如果想编译其他清单，或者单独的样式表和 JavaScript，可以添加到 config/application.rb 文件中的 precompile 选项
  • 设置编译所有静态资源
  • manifest-md5hash.json 的文件，列出所有静态资源和对应的指纹
  • 把 Expires 报头设置为很久以后
  • 在本地预编译后，可以把编译好的文件纳入版本控制系统，再按照常规的方式部署
  • 实时编译消耗的内存更多，比默认的编译方式性能更低，因此不推荐使用
  • 如果用 CDN 分发静态资源，要确保文件不会被缓存，因为缓存会导致问题。如果设置了 config.action_controller.perform_caching = true，Rack::Cache 会使用 Rails.cache 存储静态文件，很快缓存空间就会用完。
  • Sprockets 默认使用的公开路径是 /assets
  • X-Sendfile 报头的作用是让服务器忽略程序的响应，直接从硬盘上伺服指定的文件
  • 为 Rails 提供标准 JavaScript 代码库的 jquery-rails gem 是个很好的例子。这个 gem 中有个引擎类，继承自 Rails::Engine。添加这层继承关系后，Rails 就知道这个 gem 中可能包含静态资源文件，会把这个引擎中的 app/assets、lib/assets 和 vendor/assets 三个文件夹加入 Sprockets 的搜索路径中。
張 旭

Developing with Docker - 1 views

engineering.ifttt.com/...developing-with-docker

shared by 張 旭 on 04 Dec 15 - No Cached
  • Before moving our production infrastructure over however, we decided that we wanted to start developing with them locally first. We could shake out any issues with our applications before risking the production environment.
  • using Chef and Vagrant to provision local VMs
  • Engineers at IFTTT currently all use Apple computers
  • ...7 more annotations...
  • /bin/true
    • 張 旭
      張 旭 on 04 Dec 15
       
      如果使用 docker create 就不用跑這個， 不過目前 docker-compose 沒有支援 volume-only 的 container
  • it will install gems onto the data volume from the bundler-cache container.
  • dev rm bundler-cache
    • 張 旭
      張 旭 on 04 Dec 15
       
      要完全刪除干淨，後面的指令可能是： docker rm -v bundler-cache
  • if you accidentally delete bundler-cache, you then have to install all your gems over again.
  • Containerization and Docker are powerful tools in your infrastructure toolbox.
  • highly recommend starting off in your developer environment first
  • the onboarding time for new developers go from a couple days or more to a matter of hours.
張 旭

Active Record Callbacks - Ruby on Rails Guides - 0 views

guides.rubyonrails.org/active_record_callbacks.html

ruby rails model callback

shared by 張 旭 on 12 Apr 16 - No Cached
  • Active Record provides hooks into this object life cycle so that you can control your application and its data.
  • Callbacks allow you to trigger logic before or after an alteration of an object's state.
  • Callbacks are methods that get called at certain moments of an object's life cycle.
  • ...42 more annotations...
  • created
  • saved
  • updated
  • deleted
  • validated
  • loaded
  • use a macro-style class method to register them as callbacks
  • self.name = login.capitalize if name.blank?
  • registered to only fire on certain life cycle events
  • considered good practice to declare callback methods as protected or private
  • all the available Active Record callbacks,
  • after_initialize callback will be called whenever an Active Record object is instantiated, either by directly using new or when a record is loaded from the database
  • after_find callback will be called whenever Active Record loads a record from the database.
  • after_find is called before after_initialize if both are defined
  • after_touch callback will be called whenever an Active Record object is touched.
  • belongs_to :company, touch: true
  • methods trigger callbacks
  • after_find callback is triggered by the following finder methods
  • after_initialize callback is triggered every time a new object of the class is initialized
  • should be used with caution, however, because important business rules and application logic may be kept in callbacks.
  • As you start registering new callbacks for your models, they will be queued for execution
  • The whole callback chain is wrapped in a transaction
  • Callbacks work through model relationships, and can even be defined by them.
  • As with validations, we can also make the calling of a callback method conditional on the satisfaction of a given predicate
  • When using the :if option, the callback won't be executed if the predicate method returns false; when using the :unless option, the callback won't be executed if the predicate method returns true.
  • with a Symbol
  • with a String
  • with a Proc
  • using eval and hence needs to contain valid Ruby code.
  • mix both :if and :unless in the same callback declaration
  • needed to instantiate a new PictureFileCallbacks object, since we declared our callback as an instance method.
  • Active Record makes it possible to create classes that encapsulate the callback methods, so it becomes very easy to reuse them.
  • won't be necessary to instantiate
  • after_commit
  • after_rollback
  • very similar to the after_save callback except that they don't execute until after database changes have either been committed or rolled back
  • delete_picture_file_from_disk
  • after_commit
  • If anything raises an exception after the after_destroy callback is called and the transaction rolls back, the file will have been deleted and the model will be left in an inconsistent state
    • 張 旭
      張 旭 on 12 Apr 16
       
      刪除檔案這種動作，要在資料庫的變動正確執行完成之後。
  • don't supply the :on option the callback will fire for every action.
  • The after_commit and after_rollback callbacks are guaranteed to be called for all models created, updated, or destroyed within a transaction block.
張 旭

Virtual Private Cloud (VPC)  |  Virtual Private Cloud  |  Google Cloud - 0 views

cloud.google.com/vpc

google gcp cloud system

shared by 張 旭 on 09 Nov 18 - No Cached
  • A single Google Cloud VPC can span multiple regions without communicating across the public Internet.
  • Google Cloud VPCs let you increase the IP space of any subnets without any workload shutdown or downtime.
  • Get private access to Google services, such as storage, big data, analytics, or machine learning, without having to give your service a public IP address.
  • ...3 more annotations...
  • Enable dynamic Border Gateway Protocol (BGP) route updates between your VPC network and your non-Google network with our virtual router.
  • Configure a VPC Network to be shared across several projects in your organization.
  • Hosting globally distributed multi-tier applications, by creating a VPC with subnets.
張 旭

我所认为的RESTful API最佳实践 · ScienJus's Blog - 0 views

www.scienjus.com/my-restful-api-best-practices

api REST

shared by 張 旭 on 29 Oct 18 - No Cached
  • 最好将过滤、分页和排序的相关信息全权交给客户端，包括过滤条件、页数或是游标、每页的数量、排序方式、升降序等，这样可以使 API 更加灵活。
  • 过度纠结如何遵守规范只是徒增烦恼
  • API 的版本号和客户端 APP 的版本号是毫无关系的
  • ...13 more annotations...
  • 除了 POST 其他三种请求都具备幂等性（多次请求的效果相同）
  • PUT 也可以用于创建操作，只要在创建前就可以确定资源的 id。
  • 永远去使用可以指向资源的的最短 URL 路径
  • RESTful 中不建议出现动词
  • 这里我选择了 PUT 而不是 POST，因为我觉得点赞这种行为应该是幂等的，多次操作的结果应该相同。
  • Token 的状态保持一般有两种方式实现：一种是在用户每次操作都会延长或重置 TOKEN 的生存时间（类似于缓存的机制），另一种是 Token 的生存时间固定不变，但是同时返回一个刷新用的 Token，当 Token 过期时可以将其刷新而不是重新登录。
  • 一般客户端会把请求参数拼接后并加密作为 Sign 传给服务端，这样即使被抓包了，对方只修改参数而无法生成对应的 Sign 也会被服务端识破。
  • 尽量使用 HTTP 状态码
  • data是真正需要返回的数据，并且只会在请求成功时才存在，msg只用在开发环境，并且只为了开发人员识别。客户端逻辑只允许识别code，并且不允许直接将msg的内容展示给用户。
  • JSON 比 XML 可视化更好，也更加节约流量，所以尽量不要使用 XML。
  • 创建和修改操作成功后，需要返回该资源的全部信息。
  • 即使客户端一个页面需要展示多个资源，也不要在一个接口中全部返回，而是让客户端分别请求多个接口。
  • 推荐使用游标分页
張 旭

Home · sysown/proxysql Wiki - 0 views

github.com/wiki

mysql database cluster mariadb proxy

shared by 張 旭 on 19 Nov 18 - No Cached
  • bear in mind that the best way to configure ProxySQL is through its admin interface.
  • llow you to control the list of the backend servers, how traffic is routed to them, and other important settings (such as caching, access control, etc)
  • Once you've made modifications to the in-memory data structure, you must load the new configuration to the runtime, or persist the new settings to disk
  • ...4 more annotations...
  • mysql_variables: contains global variables that control the functionality for handling the incoming MySQL traffic.
  • mysql_users: contains rows for the mysql_users table from the admin interface. Basically, these define the users which can connect to the proxy, and the users with which the proxy can connect to the backend servers.
  • mysql_servers: contains rows for the mysql_servers table from the admin interface. Basically, these define the backend servers towards which the incoming MySQL traffic is routed.
  • mysql_query_rules: contains rows for the mysql_query_rules table from the admin interface. Basically, these define the rules used to classify and route the incoming MySQL traffic, according to various criteria (patterns matched, user used to run the query, etc.).
張 旭

What's the Docker Swarm "-advertise-addr"? - Blog | BoxBoat - 0 views

boxboat.com/...ts-docker-swarm-advertise-addr

docker swarm networking

shared by 張 旭 on 11 Sep 18 - No Cached
  • To put it simply, the --advertise-addr is the address other nodes in the Docker swarm use to connect into your node.
  • a port number which defaults to 2377
  • The --listen-addr is the address that the swarm service listens on for incoming connections.
  • ...2 more annotations...
  • The default for --listen-addr is to listen on all interfaces on TCP port 2377 (0.0.0.0:2377)
  • Depending on your network architecture, you may want your swarm management interface only accessible on a management network that could be separate from a data and/or public network that are each attached to a physical server.
張 旭

10 Common Git Problems and How to Fix Them - DEV Community - 0 views

dev.to/...blems-and-how-to-fix-them-234o

git

shared by 張 旭 on 03 Sep 18 - No Cached
  • Please keep in mind that --amend actually will create a new commit which replaces the previous one, so don’t use it for modifying commits which already have been pushed to a central repository.
  • git rebase --interactive
  • Just pick the commit(s) you want to update, change pick to reword (or r for short), and you will be taken to a new view where you can edit the message.
  • ...8 more annotations...
  • you can completely remove commits by deleting them from the list, as well as edit, reorder, and squash them.
  • Squashing allows you to merge several commits into one
  • In case you don’t want to create additional revert commits but only apply the necessary changes to your working tree, you can use the --no-commit/-n option.
  • reuse recorded resolution
  • Unfortunately it turns out that one of the branches isn’t quite there yet, so you decide to un-merge it again. Several days (or weeks) later when the branch is finally ready you merge it again, but thanks to the recorded resolutions, you won’t have to resolve the same merge conflicts again.
  • You can also define global hooks to use in all your projects by creating a template directory that git will use when initializing a new repository
  • removing sensitive data
  • Keep in mind that this will rewrite your project’s entire history, which can be very disruptive in a distributed workflow.
張 旭

drapergem/draper: Decorators/View-Models for Rails Applications - 0 views

github.com/draper

rails model programming decorator

shared by 張 旭 on 04 Jul 17 - No Cached
  • The decorator wraps the model, and deals only with presentational concerns.
  • In the controller, you decorate the article before handing it off to the view
  • whenever you start needing logic in the view or start thinking about a helper method, you can implement a method on the decorator instead.
  • ...5 more annotations...
  • convert an existing Rails helper to a decorator method
  • That method is presentation-centric, and thus does not belong in a model.
  • Where does that come from? It's a method of the source Article, whose methods have been made available on the decorator by the delegate_all call above.
  • a great way to replace procedural helpers like the one above with "real" object-oriented programming
  • format complex data for user display
張 旭

phusion/baseimage-docker - 1 views

github.com/...baseimage-docker

docker ubuntu base system

shared by 張 旭 on 24 Jun 15 - No Cached
    • 張 旭
      張 旭 on 24 Jun 15
       
      原始的 docker 在執行命令時，預設就是將傳入的 COMMAND 當成 PID 1 的程序，執行完畢就結束這個  docker，其他的 daemons 並不會執行，而 baseimage 解決了這個問題。
    • crazylion lee
      crazylion lee on 17 Dec 15
       
      好棒棒
  • docker exec
  • Through SSH
  • ...57 more annotations...
  • docker exec -t -i YOUR-CONTAINER-ID bash -l
  • Login to the container
  • Baseimage-docker only advocates running multiple OS processes inside a single container.
  • Password and challenge-response authentication are disabled by default. Only key authentication is allowed.
  • A tool for running a command as another user
  • The Docker developers advocate the philosophy of running a single logical service per container. A logical service can consist of multiple OS processes.
  • All syslog messages are forwarded to "docker logs".
  • Baseimage-docker advocates running multiple OS processes inside a single container, and a single logical service can consist of multiple OS processes.
  • Baseimage-docker provides tools to encourage running processes as different users
  • sometimes it makes sense to run multiple services in a single container, and sometimes it doesn't.
  • Splitting your logical service into multiple OS processes also makes sense from a security standpoint.
  • using environment variables to pass parameters to containers is very much the "Docker way"
  • Baseimage-docker provides a facility to run a single one-shot command, while solving all of the aforementioned problems
  • the shell script must run the daemon without letting it daemonize/fork it.
  • All executable scripts in /etc/my_init.d, if this directory exists. The scripts are run in lexicographic order.
  • variables will also be passed to all child processes
  • Environment variables on Unix are inherited on a per-process basis
  • there is no good central place for defining environment variables for all applications and services
  • centrally defining environment variables
  • One of the ideas behind Docker is that containers should be stateless, easily restartable, and behave like a black box.
  • a one-shot command in a new container
  • immediately exit after the command exits,
  • However the downside of this approach is that the init system is not started. That is, while invoking COMMAND, important daemons such as cron and syslog are not running. Also, orphaned child processes are not properly reaped, because COMMAND is PID 1.
  • add additional daemons (e.g. your own app) to the image by creating runit entries.
  • Nginx is one such example: it removes all environment variables unless you explicitly instruct it to retain them through the env configuration option.
  • Mechanisms for easily running multiple processes, without violating the Docker philosophy
  • Ubuntu is not designed to be run inside Docker
  • According to the Unix process model, the init process -- PID 1 -- inherits all orphaned child processes and must reap them
  • Syslog-ng seems to be much more stable
  • cron daemon
  • Rotates and compresses logs
  • /sbin/setuser
  • A tool for installing apt packages that automatically cleans up after itself.
  • a single logical service inside a single container
  • A daemon is a program which runs in the background of its system, such as a web server.
  • The shell script must be called run, must be executable, and is to be placed in the directory /etc/service/<NAME>. runsv will switch to the directory and invoke ./run after your container starts.
  • If any script exits with a non-zero exit code, the booting will fail.
  • If your process is started with a shell script, make sure you exec the actual process, otherwise the shell will receive the signal and not your process.
  • any environment variables set with docker run --env or with the ENV command in the Dockerfile, will be picked up by my_init
  • not possible for a child process to change the environment variables of other processes
  • they will not see the environment variables that were originally passed by Docker.
  • We ignore HOME, SHELL, USER and a bunch of other environment variables on purpose, because not ignoring them will break multi-user containers.
  • my_init imports environment variables from the directory /etc/container_environment
  • /etc/container_environment.sh - a dump of the environment variables in Bash format.
  • modify the environment variables in my_init (and therefore the environment variables in all child processes that are spawned after that point in time), by altering the files in /etc/container_environment
  • my_init only activates changes in /etc/container_environment when running startup scripts
  • environment variables don't contain sensitive data, then you can also relax the permissions
  • Syslog messages are forwarded to the console
  • syslog-ng is started separately before the runit supervisor process, and shutdown after runit exits.
  • RUN apt-get update && apt-get upgrade -y -o Dpkg::Options::="--force-confold"
  • /sbin/my_init --skip-startup-files --quiet --
  • By default, no keys are installed, so nobody can login
  • provide a pregenerated, insecure key (PuTTY format)
  • RUN /usr/sbin/enable_insecure_key
  • docker run YOUR_IMAGE /sbin/my_init --enable-insecure-key
  • RUN cat /tmp/your_key.pub >> /root/.ssh/authorized_keys && rm -f /tmp/your_key.pub
  • The default baseimage-docker installs syslog-ng, cron and sshd services during the build process
張 旭

The Twelve-Factor App - 0 views

12factor.net/logs

development application programming

shared by 張 旭 on 20 Feb 19 - No Cached
  • Logs are the stream of aggregated, time-ordered events collected from the output streams of all running processes and backing services.
  • Logs have no fixed beginning or end, but flow continuously as long as the app is operating.
  • each running process writes its event stream, unbuffered, to stdout.
  • ...2 more annotations...
  • long-term archival. These archival destinations are not visible to or configurable by the app, and instead are completely managed by the execution environment.
  • Most significantly, the stream can be sent to a log indexing and analysis system such as Splunk, or a general-purpose data warehousing system such as Hadoop/Hive.
張 旭

Storing Sessions in a Database, by Chris Shiflett - 0 views

shiflett.org/...storing-sessions-in-a-database

php session database

shared by 張 旭 on 14 Jan 19 - Cached
  • to store sessions in a database rather than the filesystem
  • server affinity (methods that direct requests from the same client to the same server)
  • store sessions in a central database that is common to all servers
  • ...6 more annotations...
  • security concerns
  • PHP provides a function that lets you override the default session mechanism by specifying the names of your own functions for taking care of the distinct tasks
  • The handler PHP uses to handle data serialization is defined by the session.serialize_handler configuration directive. It is set to php by default.
  • REPLACE
  • REPLACE, which behaves exactly like INSERT, except that it handles cases where a record already exists with the same session identifier by first deleting that record.
  • the _write() function keeps the timestamp of the last access in the access column for each record, this can be used to determine which records to delete.
張 旭

ProxySQL Experimental Feature: Native ProxySQL Clustering - Percona Database Performanc... - 0 views

www.percona.com/...ntal-feature-native-clustering

mysql proxysql cluster HA database

shared by 張 旭 on 07 Aug 19 - No Cached
  • several ProxySQL instances to communicate with and share configuration updates with each other.
  • 4 tables where you can make changes and propagate the configuration
  • When you make a change like INSERT/DELETE/UPDATE on any of these tables, after running the command LOAD … TO RUNTIME , ProxySQL creates a new checksum of the table’s data and increments the version number in the table runtime_checksums_values
  • ...2 more annotations...
  • all nodes are monitoring and communicating with all the other ProxySQL nodes. When another node detects a change in the checksum and version (both at the same time), each node will get a copy of the table that was modified, make the same changes locally, and apply the new config to RUNTIME to refresh the new config, make it visible to the applications connected and automatically save it to DISK for persistence.
  • a “synchronous cluster” so any changes to these 4 tables on any ProxySQL server will be replicated to all other ProxySQL nodes.
« First ‹ Previous 121 - 140 of 166 Next › Last »
Showing 20▼ items per page