Skip to main content

Home/ Larvata/ Group items tagged tasks

Rss Feed Group items tagged

張 旭

Deploying Rails Apps, Part 6: Writing Capistrano Tasks - Vladi Gleba - 0 views

  • we can write our own tasks to help us automate various things.
  • organizing all of the tasks here under a namespace
  • upload a file from our local computer.
  • ...27 more annotations...
  • learn about is SSHKit and the various methods it provides
  • SSHKit was actually developed and released with Capistrano 3, and it’s basically a lower-level tool that provides methods for connecting and interacting with remote servers
  • on(): specifies the server to run on
  • within(): specifies the directory path to run in
  • with(): specifies the environment variables to run with
  • run on the application server
  • within the path specified
  • with certain environment variables set
  • execute(): the workhorse that runs the commands on your server
  • upload(): uploads a file from your local computer to your remote server
  • capture(): executes a command and returns its output as a string
    • 張 旭
       
      capture 是跑在遠端伺服器上
  • upload() has the bang symbol (!) because that’s how it’s defined in SSHKit, and it’s just a convention letting us know that the method will block until it finishes.
  • But in order to ensure rake runs with the proper environment variables set, we have to use rake as a symbol and pass db:seed as a string
  • This format will also be necessary whenever you’re running any other Rails-specific commands that rely on certain environment variables being set
  • I recommend you take a look at SSHKit’s example page to learn more
  • make sure we pushed all our local changes to the remote master branch
  • run this task before Capistrano runs its own deploy task
  • actually creates three separate tasks
  • I created a namespace called deploy to contain these tasks since that’s what they’re related to.
  • we’re using the callbacks inside a namespace to make sure Capistrano knows which tasks the callbacks are referencing.
  • custom recipe (a Capistrano term meaning a series of tasks)
  • /shared: holds files and directories that persist throughout deploys
  • When you run cap production deploy, you’re actually calling a Capistrano task called deploy, which then sequentially invokes other tasks
  • your favorite browser (I hope it’s not Internet Explorer)
  • Deployment is hard and takes a while to sink in.
  • the most important thing is to not get discouraged
  • I didn’t want other people going through the same thing
張 旭

Swarm mode key concepts | Docker Documentation - 0 views

  • The cluster management and orchestration features embedded in the Docker Engine are built using SwarmKit.
  • Docker engines participating in a cluster are running in swarm mode
  • A swarm is a cluster of Docker engines, or nodes, where you deploy services
  • ...19 more annotations...
  • When you run Docker without using swarm mode, you execute container commands.
  • When you run the Docker in swarm mode, you orchestrate services.
  • You can run swarm services and standalone containers on the same Docker instances.
  • A node is an instance of the Docker engine participating in the swarm
  • You can run one or more nodes on a single physical computer or cloud server
  • To deploy your application to a swarm, you submit a service definition to a manager node.
  • Manager nodes also perform the orchestration and cluster management functions required to maintain the desired state of the swarm.
  • Manager nodes elect a single leader to conduct orchestration tasks.
  • Worker nodes receive and execute tasks dispatched from manager nodes.
  • service is the definition of the tasks to execute on the worker nodes
  • When you create a service, you specify which container image to use and which commands to execute inside running containers.
  • replicated services model, the swarm manager distributes a specific number of replica tasks among the nodes based upon the scale you set in the desired state.
  • global services, the swarm runs one task for the service on every available node in the cluster.
  • A task carries a Docker container and the commands to run inside the container
  • Manager nodes assign tasks to worker nodes according to the number of replicas set in the service scale.
  • Once a task is assigned to a node, it cannot move to another node
  • If you do not specify a port, the swarm manager assigns the service a port in the 30000-32767 range.
  • External components, such as cloud load balancers, can access the service on the PublishedPort of any node in the cluster whether or not the node is currently running the task for the service.
  • Swarm mode has an internal DNS component that automatically assigns each service in the swarm a DNS entry.
張 旭

Docker for AWS persistent data volumes | Docker Documentation - 0 views

  • Cloudstor is a modern volume plugin built by Docker
  • Docker swarm mode tasks and regular Docker containers can use a volume created with Cloudstor to mount a persistent data volume.
  • Global shared Cloudstor volumes mounted by all tasks in a swarm service.
  • ...14 more annotations...
  • Workloads running in a Docker service that require access to low latency/high IOPs persistent storage, such as a database engine, can use a relocatable Cloudstor volume backed by EBS.
  • Each relocatable Cloudstor volume is backed by a single EBS volume.
  • If a swarm task using a relocatable Cloudstor volume gets rescheduled to another node within the same availability zone as the original node where the task was running, Cloudstor detaches the backing EBS volume from the original node and attaches it to the new target node automatically.
  • in a different availability zone,
  • Cloudstor transfers the contents of the backing EBS volume to the destination availability zone using a snapshot, and cleans up the EBS volume in the original availability zone.
  • Typically the snapshot-based transfer process across availability zones takes between 2 and 5 minutes unless the work load is write-heavy.
  • A swarm task is not started until the volume it mounts becomes available
  • Sharing/mounting the same Cloudstor volume backed by EBS among multiple tasks is not a supported scenario and leads to data loss.
  • a Cloudstor volume to share data between tasks, choose the appropriate EFS backed shared volume option.
  • When multiple swarm service tasks need to share data in a persistent storage volume, you can use a shared Cloudstor volume backed by EFS.
  • a volume and its contents can be mounted by multiple swarm service tasks without the risk of data loss
  • over NFS
  • the persistent data backed by EFS volumes is always available.
  • shared Cloudstor volumes only work in those AWS regions where EFS is supported.
張 旭

How services work | Docker Documentation - 0 views

  • a service is the image for a microservice within the context of some larger application.
  • When you create a service, you specify which container image to use and which commands to execute inside running containers.
  • an overlay network for the service to connect to other services in the swarm
  • ...13 more annotations...
  • In the swarm mode model, each task invokes exactly one container
  • A task is analogous to a “slot” where the scheduler places a container.
  • A task is the atomic unit of scheduling within a swarm.
  • A task is a one-directional mechanism. It progresses monotonically through a series of states: assigned, prepared, running, etc.
  • Docker swarm mode is a general purpose scheduler and orchestrator.
  • Hypothetically, you could implement other types of tasks such as virtual machine tasks or non-containerized process tasks.
  • If all nodes are paused or drained, and you create a service, it is pending until a node becomes available.
  • reserve a specific amount of memory for a service.
  • impose placement constraints on the service
  • As the administrator of a swarm, you declare the desired state of your swarm, and the manager works with the nodes in the swarm to create that state.
  • two types of service deployments, replicated and global.
  • A global service is a service that runs one task on every node.
  • Good candidates for global services are monitoring agents, an anti-virus scanners or other types of containers that you want to run on every node in the swarm.
張 旭

Speeding up Docker image build process of a Rails application | BigBinary Blog - 1 views

  • we do not want to execute bundle install and rake assets:precompile tasks while starting a container in each pod which would prevent that pod from accepting any requests until these tasks are finished.
  • run bundle install and rake assets:precompile tasks while or before containerizing the Rails application.
  • Kubernetes pulls the image, starts a Docker container using that image inside the pod and runs puma server immediately.
  • ...7 more annotations...
  • Since source code changes often, the previously cached layer for the ADD instruction is invalidated due to the mismatching checksums.
  • The ARG instruction in the Dockerfile defines RAILS_ENV variable and is implicitly used as an environment variable by the rest of the instructions defined just after that ARG instruction.
  • RUN instructions are used to install gems and precompile static assets using sprockets
  • Instead, Docker automatically re-uses the previously built layer for the RUN bundle install instruction if the Gemfile.lock file remains unchanged.
  • everyday we need to build a lot of Docker images containing source code from varying Git branches as well as with varying environments.
  • it is hard for Docker to cache layers for bundle install and rake assets:precompile tasks and re-use those layers during every docker build command run with different application source code and a different environment.
  • By default, Bundler installs gems at the location which is set by Rubygems.
  •  
    "we do not want to execute bundle install and rake assets:precompile tasks while starting a container in each pod which would prevent that pod from accepting any requests until these tasks are finished."
crazylion lee

Homepage | Celery: Distributed Task Queue - 0 views

  •  
    "Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready)."
張 旭

Why we should stop using Grunt & Gulp - 0 views

  • All of these task runners (or build systems if you want to call them that) try to abstract some kind of task paradigm away into their own siloed incantations.
  • Gulp isn't the only culprit either; Jake, Broccoli, Brunch and Mimosa all require plugins to install as well, which means when using any of these task runners, you're essentially just installing 1 more dependency (the task runner) than if you had no task runner, and just used each of the projects own binaries
張 旭

Active Record Migrations - Ruby on Rails Guides - 0 views

    • 張 旭
       
       跟 belongs_to 與 has_many 設定對應的 Migrattion
    • 張 旭
       
      has_and_belongs_to_many 的對應?
  • add_column and remove_column
  • ...114 more annotations...
  • allowing your schema and changes to be database independent.
  • each migration as being a new 'version' of the database
  • each migration modifies it to add or remove tables, columns, or entries
  • Active Record will also update your db/schema.rb file to match the up-to-date structure of your database.
  • A primary key column called id will also be added implicitly, as it's the default primary key for all Active Record models
  • roll this migration back, it will remove the table
  • timestamps macro adds two columns, created_at and updated_at
  • On databases that support transactions with statements that change the schema, migrations are wrapped in a transaction
  • reversible
  • use up and down instead of change
  • Migrations are stored as files in the db/migrate directory, one for each migration class.
  • a UTC timestamp identifying
  • Rails uses this timestamp to determine which migration should be run and in what order
  • "AddXXXToYYY" or "RemoveXXXFromYYY"
  • use a Ruby DSL
  • column type as references
  • part_number:string:index
  • a migration to remove a column
  • "CreateXXX"
  • change_column_null
  • AddUserRefToProducts
  • :references
  • produce join tables if JoinTable is part of the name
  • CreateJoinTable
  • The model and scaffold generators will create migrations appropriate for adding a new model.
  • enclosed by curly braces and follow the field type
  • create_table
  • By default, create_table will create a primary key called id
  • add an index on the new column
  • when using MySQL, the default is ENGINE=InnoDB
  • create_join_table creates an HABTM (has and belongs to many) join table
  • To customize the name of the table, provide a :table_name option:
  • create_join_table also accepts a block
  • change_table, used for changing existing tables
  • remove
  • rename
  • add_column
  • change_column
  • remove_column
  • change_column_default
  • place an SQL fragment in the :options option.
  • limit
  • precision
  • scale
  • polymorphic
  • default
  • index
  • add_foreign_key
  • Active Record only supports single column foreign keys.
  • use the old style of migration using up and down methods instead of the change method.
  • .connection.execute
  • change_table is also reversible, as long as the block does not call change, change_default or remove.
  • remove_column is reversible if you supply the column type as the third argument
  • Complex migrations may require processing that Active Record doesn't know how to reverse
  • reversible
  • Using reversible will ensure that the instructions are executed in the right order too.
  • add_column add_foreign_key add_index add_reference add_timestamps change_column_default (must supply a :from and :to option) change_column_null create_join_table create_table disable_extension drop_join_table drop_table (must supply a block) enable_extension remove_column (must supply a type) remove_foreign_key (must supply a second table) remove_index remove_reference remove_timestamps rename_column rename_index rename_table
  • :column_options option
  • have the option :null set to false by default
  • By default, the name of the join table comes from the union of the first two arguments provided to create_join_table
  • in alphabetical order
  • change_column command is irreversible.
    • 張 旭
       
      關聯物在前,被關聯物在後。 A 關聯到 B
  • If the column names can not be derived from the table names, you can use the :column and :primary_key options.
  • figure out the column name
  • foreign key for a specific column
  • foreign key by name
    • 張 旭
       
      不懂 column 跟 name 的用法差異,基本上一樣。
  • Active Record knows how to reverse the migration automatically
    • 張 旭
       
      使用內建的 method,Rails 比較容易自動 rollback
    • 張 旭
       
      除了幾個特殊的 change_ 跟 remove_
  • should use reversible or write the up and down methods instead of using the change method
  • If your migration is irreversible, you should raise ActiveRecord::IrreversibleMigration from your down method.
  • DontUseConstraintForZipcodeValidationMigration
  • rails db:migrate
  • the db:migrate task also invokes the db:schema:dump task, which will update your db/schema.rb file to match the structure of your database.
  • specify a target version
  • all migrations up to and including 20080906120000
  • run the down method on all the migrations down to, but not including, 20080906120000
  • rails db:rollback
  • db:migrate:redo task is a shortcut for doing a rollback and then migrating back up again
    • 張 旭
       
      舊版的還是 rake!
  • STEP parameter
  • db:setup task will create the database, load the schema and initialize it with the seed data
  • db:reset task will drop the database and set it up again. This is functionally equivalent to rails db:drop db:setup.
  • run a specific migration up or down, the db:migrate:up and db:migrate:down
  • the RAILS_ENV environment variable
  • db:migrate VERBOSE=false will suppress all output.
  • If you have already run the migration, then you cannot just edit the migration and run the migration again: Rails thinks it has already run the migration and so will do nothing when you run rails db:migrate.
  • must rollback the migration (for example with bin/rails db:rollback), edit your migration and then run rails db:migrate to run the corrected version.
  • editing existing migrations is not a good idea.
  • should write a new migration that performs the changes you require
  • revert method can be helpful when writing a new migration to undo previous migrations in whole or in part
  • require_relative
  • revert
  • They are not designed to be edited, they just represent the current state of the database.
  • Schema Files for
  • Schema files are also useful if you want a quick look at what attributes an Active Record object has
  • annotate_models gem automatically adds and updates comments at the top of each model summarizing the schema if you desire that functionality.
  • database-independent
  • multiple databases
  • db/schema.rb cannot express database specific items such as triggers, stored procedures or check constraints
  • you can execute custom SQL statements, the schema dumper cannot reconstitute those statements from the database
  • db:structure:dump
    • 張 旭
       
      資料庫種類不相依的 schema 付出的代價就是有些特殊的資料庫特性無法描述出來,例如 trigger;如果有在 migration 寫 SQL 的,簡單說 schema dumper 這邊就要設定成 :sql 而不是預設的 :ruby
  • set in config/application.rb by the config.active_record.schema_format setting, which may be either :sql or :ruby.
  • check them into source control.
  • db/schema.rb contains the current version number of the database
  • Validations such as validates :foreign_key, uniqueness: true are one way in which models can enforce data integrity
  • The :dependent option on associations allows models to automatically destroy child objects when the parent is destroyed.
  • Migrations can also be used to add or modify data
  • Initial
  • To add initial data after a database is created, Rails has a built-in 'seeds' feature that makes the process quick and easy.
  • db/seeds.rb
  • rails db:seed
張 旭

Swarm task states | Docker Documentation - 0 views

  • Each service can start multiple tasks.
  • Tasks are execution units that run once to completion.
  • The task progresses forward through a number of states, and its state doesn’t go backward.
crazylion lee

Task Management for Teams - MeisterTask - 0 views

  •  
    "The most intuitive project and task management tool on the web"
crazylion lee

Machine Learning for Everyday Tasks - 0 views

  •  
    "Machine learning is often thought to be too complicated for everyday development tasks. We often associate it with things like big data, data mining, data science, and artificial intelligence. Sometimes it feels something like this:"
張 旭

The Rails Command Line - Ruby on Rails Guides - 0 views

  • rake --tasks
  • Think of destroy as the opposite of generate.
  • runner runs Ruby code in the context of Rails non-interactively
  • ...28 more annotations...
  • rails dbconsole figures out which database you're using and drops you into whichever command line interface you would use with it
  • The console command lets you interact with your Rails application from the command line. On the underside, rails console uses IRB
  • rake about gives information about version numbers for Ruby, RubyGems, Rails, the Rails subcomponents, your application's folder, the current Rails environment name, your app's database adapter, and schema version
  • You can precompile the assets in app/assets using rake assets:precompile and remove those compiled assets using rake assets:clean.
  • rake db:version is useful when troubleshooting
  • The doc: namespace has the tools to generate documentation for your app, API documentation, guides.
  • rake notes will search through your code for comments beginning with FIXME, OPTIMIZE or TODO.
  • You can also use custom annotations in your code and list them using rake notes:custom by specifying the annotation using an environment variable ANNOTATION.
  • rake routes will list all of your defined routes, which is useful for tracking down routing problems in your app, or giving you a good overview of the URLs in an app you're trying to get familiar with.
  • rake secret will give you a pseudo-random key to use for your session secret.
  • Custom rake tasks have a .rake extension and are placed in Rails.root/lib/tasks.
  • rails new . --git --database=postgresql
  • All commands can run with -h or --help to list more information
  • The rails server command launches a small web server named WEBrick which comes bundled with Ruby
  • rails server -e production -p 4000
  • You can run a server as a daemon by passing a -d option
  • The rails generate command uses templates to create a whole lot of things.
  • Using generators will save you a large amount of time by writing boilerplate code, code that is necessary for the app to work.
  • All Rails console utilities have help text.
  • generate controller ControllerName action1 action2.
  • With a normal, plain-old Rails application, your URLs will generally follow the pattern of http://(host)/(controller)/(action), and a URL like http://(host)/(controller) will hit the index action of that controller.
  • A scaffold in Rails is a full set of model, database migration for that model, controller to manipulate it, views to view and manipulate the data, and a test suite for each of the above.
  • Unit tests are code that tests and makes assertions about code.
  • Unit tests are your friend.
  • rails console --sandbox
  • rails db
  • Each task has a description, and should help you find the thing you need.
  • rake tmp:clear clears all the three: cache, sessions and sockets.
張 旭

Networking with overlay networks | Docker Documentation - 0 views

  • The manager host will function as both a manager and a worker, which means it can both run service tasks and manage the swarm.
  • connected together using an overlay network called ingress
  • each of them now has an overlay network called ingress and a bridge network called docker_gwbridge
  • ...7 more annotations...
  • The docker_gwbridge connects the ingress network to the Docker host’s network interface so that traffic can flow to and from swarm managers and workers
  • recommended that you use separate overlay networks for each application or group of applications which will work together
  • You don’t need to create the overlay network on the other nodes, beacause it will be automatically created when one of those nodes starts running a service task which requires it.
  • The default publish mode of ingress, which is used when you do not specify a mode for the --publish flag, means that if you browse to port 80 on manager, worker-1, or worker-2, you will be connected to port 80 on one of the 5 service tasks, even if no tasks are currently running on the node you browse to.
  • Even though overlay networks are automatically created on swarm worker nodes as needed, they are not automatically removed.
  • The -dit flags mean to start the container detached (in the background), interactive (with the ability to type into it), and with a TTY (so you can see the input and output).
  • alpine containers running ash, which is Alpine’s default shell rather than bash
張 旭

Manage nodes in a swarm | Docker Documentation - 0 views

  • Drain means the scheduler doesn’t assign new tasks to the node. The scheduler shuts down any existing tasks and schedules them on an available node.
  • Reachable means the node is a manager node participating in the Raft consensus quorum. If the leader node becomes unavailable, the node is eligible for election as the new leader.
  • If a manager node becomes unavailable, you should either join a new manager node to the swarm or promote a worker node to be a manager.
  • ...8 more annotations...
  • docker node inspect self --pretty
  • docker node update --availability drain node
  • use node labels in service constraints
  • The labels you set for nodes using docker node update apply only to the node entity within the swarm
  • node labels can be used to limit critical tasks to nodes that meet certain requirements
  • promote a worker node to the manager role
  • demote a manager node to the worker role
  • If the last manager node leaves the swarm, the swarm becomes unavailable requiring you to take disaster recovery measures.
張 旭

Introduction To The Queue System - Diving Laravel - 0 views

  • Laravel is shipped with a built-in queue system that helps you run tasks in the background
  • The QueueManager is registered into the container and it knows how to connect to the different built-in queue drivers
  • for example when we called the Queue::push() method, what happened is that the manager selected the desired queue driver, connected to it, and called the push method on that driver.
  • ...2 more annotations...
  • All calls to methods that don't exist in the QueueManager class will be sent to the loaded driver
  • when you do Queue::push() you're actually calling the push method on the queue driver you're using
  •  
    "Laravel is shipped with a built-in queue system that helps you run tasks in the background "
張 旭

Using Ansible and Ansible Tower with shared roles - 2 views

  • clearly defined roles for dedicated tasks
  • a predefined structure of folders and files to hold your automation code.
  • Roles can be part of your project repository.
  • ...8 more annotations...
  • a better way is to keep a role in its own repository.
  • to be available to a playbook, the role still needs to be included.
  • The best way to make shared roles available to your playbooks is to use a function built into Ansible itself: by using the command ansible-galaxy
  • ansible galaxy can read a file specifying which external roles need to be imported for a successful Ansible run: requirements.yml
  • requirements.yml ensures that the used role can be pinned to a certain release tag value, commit hash, or branch name.
  • Each time Ansible Tower checks out a project it looks for a roles/requirements.yml. If such a file is present, a new version of each listed role is copied to the local checkout of the project and thus available to the relevant playbooks.
  • stick to the directory name roles, sitting in the root of your project directory.
  • have one requirements.yml only, and keep it at roles/requirements.yml
  •  
    "clearly defined roles for dedicated tasks"
crazylion lee

rgcr/m-cli:  Swiss Army Knife for macOS - 0 views

shared by crazylion lee on 23 Nov 16 - No Cached
  •  
    " m-cli differs from other mac command line tools in that: Its main purpose is to manage administrative tasks and do it easier It doesn't install 3rd party tools because it doesn't have dependencies The installation is very easy and doesn't require intervention It only uses macOS commands"
張 旭

Which Ruby background job framework is right for you? - 0 views

  • A background or asynchronous job (or task) is one that is processed outside of the usual request/response workflow that is part of any modern web framework
  • Asynchronous tasks on the other hand, are those that may be started from a normal web request, but require a longer time to complete than the normal request.
crazylion lee

It's Magit! A Git Porcelain inside Emacs - 0 views

shared by crazylion lee on 15 Dec 16 - No Cached
  •  
    "Magit is an interface to the version control system Git, implemented as an Emacs package. Magit aspires to be a complete Git porcelain. While we cannot (yet) claim that Magit wraps and improves upon each and every Git command, it is complete enough to allow even experienced Git users to perform almost all of their daily version control tasks directly from within Emacs. While many fine Git clients exist, only Magit and Git itself deserve to be called porcelains. (more)"
1 - 20 of 59 Next › Last »
Showing 20 items per page