Skip to main content

Home/ Larvata/ Contents contributed and discussions participated by 張 旭

Contents contributed and discussions participated by 張 旭

Simple Middle Filter: All | Bookmarks | Topics
張 旭

Choosing an Executor Type - CircleCI - 0 views

circleci.com/...executor-types

circle CI

shared by 張 旭 on 14 Nov 18 - No Cached
  • Containers are an instance of the Docker Image you specify and the first image listed in your configuration is the primary container image in which all steps run.
  • In this example, all steps run in the container created by the first image listed under the build job
  • If you experience increases in your run times due to installing additional tools during execution, it is best practice to use the Building Custom Docker Images Documentation to create a custom image with tools that are pre-loaded in the container to meet the job requirements.
  • ...9 more annotations...
  • The machine option runs your jobs in a dedicated, ephemeral VM
  • Using the machine executor gives your application full access to OS resources and provides you with full control over the job environment.
  • Using machine may require additional fees in a future pricing update.
  • Using the macos executor allows you to run your job in a macOS environment on a VM.
  • In a multi-image configuration job, all steps are executed in the container created by the first image listed.
  • All containers run in a common network and every exposed port will be available on localhost from a primary container.
  • If you want to work with private images/registries, please refer to Using Private Images.
  • Docker also has built-in image caching and enables you to build, run, and publish Docker images via Remote Docker.
  • if you require low-level access to the network or need to mount external volumes consider using machine
張 旭

Using Orbs - CircleCI - 0 views

circleci.com/...using-orbs

circle CI

shared by 張 旭 on 14 Nov 18 - No Cached
  • Orbs enable you to share, standardize, and simplify config across your projects.
  • Jobs are comprised of two parts: a set of steps, and the environment they should be executed within.
  • Executors define the environment in which the steps of a job will be run.
  • ...12 more annotations...
  • Commands are reusable sets of steps that you can invoke with specific parameters within an existing job.
  • you can pass my-executor as the value of a name key under executor. This method is primarily employed when passing parameters to executor invocations.
  • Development orbs are mutable and expire after 90 days.
  • Production Orbs are immutable and durable.
  • CircleCI allows development orbs that have versions that start with dev:
  • Production orbs are immutable
  • Each installation of CircleCI, including circleci.com, has only one registry where orbs can be kept.
  • Organization Admins publish production orbs.
  • Organization members publish development orbs
  • You must invoke jobs in the workflow stanza of config.yml file, making sure to pass any necessary parameters as subkeys to the job.
  • When you declare an executor in a configuration outside of jobs, you can use these declarations for all jobs in the scope of that declaration, enabling you to reuse a single executor definition across multiple jobs.
  • Orbs are transparent - If you can execute an orb, you and anyone else can view the source of that orb.
張 旭

Orbs, Jobs, Steps, and Workflows - CircleCI - 0 views

circleci.com/...jobs-steps

circle CI

shared by 張 旭 on 14 Nov 18 - No Cached
  • Orbs are packages of config that you either import by name or configure inline to simplify your config, share, and reuse config within and across projects.
  • Jobs are a collection of Steps.
  • All of the steps in the job are executed in a single unit which consumes a CircleCI container from your plan while it’s running.
  • ...11 more annotations...
  • Workspaces persist data between jobs in a single Workflow.
  • Caching persists data between the same job in different Workflow builds.
  • Artifacts persist data after a Workflow has finished.
  • run using the machine executor which enables reuse of recently used machine executor runs,
  • docker executor which can compose Docker containers to run your tests and any services they require
  • macos executor
  • Steps are a collection of executable commands which are run during a job
  • In addition to the run: key, keys for save_cache:, restore_cache:, deploy:, store_artifacts:, store_test_results: and add_ssh_keys are nested under Steps.
  • checkout: key is required to checkout your code
  • run: enables addition of arbitrary, multi-line shell command scripting
  • orchestrating job runs with parallel, sequential, and manual approval workflows.
張 旭

Overview - CircleCI - 0 views

circleci.com/...about-circleci

circle CI

shared by 張 旭 on 14 Nov 18 - No Cached
  • every code change triggers automated tests in a clean container or VM
  • CircleCI may be configured to deploy code to various environments
  • Other cloud service deployments are easily scripted using SSH or by installing the API client of the service with your job configuration.
  • ...1 more annotation...
  • Continuous integration is a practice that encourages developers to integrate their code into a master branch of a shared repository early and often.
  • 張 旭 on 14 Nov 18
     
    "every code change triggers automated tests in a clean container or VM"
張 旭

What is DevOps? | Atlassian - 0 views

www.atlassian.com/devops

devops system

shared by 張 旭 on 13 Nov 18 - No Cached
  • DevOps is a set of practices that automates the processes between software development and IT teams, in order that they can build, test, and release software faster and more reliably.
  • increased trust, faster software releases, ability to solve critical issues quickly, and better manage unplanned work.
  • bringing together the best of software development and IT operations.
  • ...39 more annotations...
  • DevOps is a culture, a movement, a philosophy.
  • a firm handshake between development and operations
  • DevOps isn’t magic, and transformations don’t happen overnight.
  • Infrastructure as code
  • Culture is the #1 success factor in DevOps.
  • Building a culture of shared responsibility, transparency and faster feedback is the foundation of every high performing DevOps team.
  •  'not our problem' mentality
  • DevOps is that change in mindset of looking at the development process holistically and breaking down the barrier between Dev and Ops.
  • Speed is everything.
  • Lack of automated test and review cycles block the release to production and poor incident response time kills velocity and team confidence
  • Open communication helps Dev and Ops teams swarm on issues, fix incidents, and unblock the release pipeline faster.
  • Unplanned work is a reality that every team faces–a reality that most often impacts team productivity.
  • “cross-functional collaboration.”
  • All the tooling and automation in the world are useless if they aren’t accompanied by a genuine desire on the part of development and IT/Ops professionals to work together.
  • DevOps doesn’t solve tooling problems. It solves human problems.
  • Forming project- or product-oriented teams to replace function-based teams is a step in the right direction.
  • sharing a common goal and having a plan to reach it together
  • join sprint planning sessions, daily stand-ups, and sprint demos.
  • DevOps culture across every department
  • open channels of communication, and talk regularly
  • DevOps isn’t one team’s job. It’s everyone’s job.
  • automation eliminates repetitive manual work, yields repeatable processes, and creates reliable systems.
  • Build, test, deploy, and provisioning automation
  • continuous delivery: the practice of running each code change through a gauntlet of automated tests, often facilitated by cloud-based infrastructure, then packaging up successful builds and promoting them up toward production using automated deploys.
  • automated deploys alert IT/Ops to server “drift” between environments, which reduces or eliminates surprises when it’s time to release.
  • “configuration as code.”
  • when DevOps uses automated deploys to send thoroughly tested code to identically provisioned environments, “Works on my machine!” becomes irrelevant.
  • A DevOps mindset sees opportunities for continuous improvement everywhere.
  • regular retrospectives
  • A/B testing
  • failure is inevitable. So you might as well set up your team to absorb it, recover, and learn from it (some call this “being anti-fragile”).
  • Postmortems focus on where processes fell down and how to strengthen them – not on which team member f'ed up the code.
  • Our engineers are responsible for QA, writing, and running their own tests to get the software out to customers.
  • How long did it take to go from development to deployment? 
  • How long does it take to recover after a system failure?
  • service level agreements (SLAs)
  • Devops isn't any single person's job. It's everyone's job.
  • DevOps is big on the idea that the same people who build an application should be involved in shipping and running it.
  • developers and operators pair with each other in each phase of the application’s lifecycle.
張 旭

Overview of Virtual Private Cloud  |  VPC  |  Google Cloud - 0 views

cloud.google.com/...overview

google gcp cloud system networking

shared by 張 旭 on 09 Nov 18 - No Cached
  • a VPC network the same way you'd think of a physical network, except that it is virtualized within GCP.
  • VPC networks are logically isolated from each other in GCP.
  • The network connects the resources to each other and to the Internet.
  • 張 旭 on 09 Nov 18
     
    "a VPC network the same way you'd think of a physical network, except that it is virtualized within GCP."
張 旭

Virtual Private Cloud (VPC)  |  Virtual Private Cloud  |  Google Cloud - 0 views

cloud.google.com/vpc

google gcp cloud system

shared by 張 旭 on 09 Nov 18 - No Cached
  • A single Google Cloud VPC can span multiple regions without communicating across the public Internet.
  • Google Cloud VPCs let you increase the IP space of any subnets without any workload shutdown or downtime.
  • Get private access to Google services, such as storage, big data, analytics, or machine learning, without having to give your service a public IP address.
  • ...3 more annotations...
  • Enable dynamic Border Gateway Protocol (BGP) route updates between your VPC network and your non-Google network with our virtual router.
  • Configure a VPC Network to be shared across several projects in your organization.
  • Hosting globally distributed multi-tier applications, by creating a VPC with subnets.
張 旭

2.0 Project Tutorial - CircleCI - 0 views

circleci.com/...project-walkthrough

circle CI deploy

shared by 張 旭 on 09 Nov 18 - No Cached
  • The .circleci/config.yml file may be comprised of several Jobs.
  • a job is comprised of several Steps
  • which are commands that execute in the container that is defined in the first image: key in the file. This first image is also referred to as the primary container.
  • ...5 more annotations...
  • Every .circleci/config.yml file must have a job named build
  • Executor of the underlying technology
  • Image is a Docker image
  • Steps starting with a required checkout Step and followed by run: keys that execute commands sequentially on the primary container.
  • Docker images are typically configured using environment variables,
張 旭

Scrum懶人包 - 10分鐘讀懂Scrum與敏捷軟體開發入門（含中文英文名詞對照） - 敏捷進化趣 Agile FunEvo - 1 views

funevo.com/...hao-xin-shou-zhi-nan-introduce

project scrum

shared by 張 旭 on 08 Nov 18 - No Cached
  • Daily Scrum（每日站立會議）：每天10-15分鐘不能超時
  • Sprint（衝刺）：顧名思義，當團隊決定要哪些 Item 後，就著手去衝。
  • 大原則是 Sprint 內的 Sprint Backlog 不改變（有原則就有例外）。
  • ...6 more annotations...
  • Sprint 開始時，討論一下這個 Sprint 團隊可以交付哪些Item。
  • Sprint 結束時針對產品的會議，PO邀請利害關係人對產出給意見，是要可用的軟體才算產出。
  • Sprint Review 後，Scrum Team 成員（Dev Team 或包含 PO），針對這個 Sprint 團隊的工作模式討論改善，并定出下個 Sprint 改善事項。
  • Product Owner（PO，產品負責人）
  • ScrumMaster
  • Development Team（Dev Team，開發團隊）
張 旭

今天拜飛機了沒？ - 談敏捷開發中的貨物崇拜 - 敏捷進化趣 Agile FunEvo - 0 views

funevo.com/...-cult-agile-min-jie-bai-fei-ji

project

shared by 張 旭 on 08 Nov 18 - No Cached
  • 張 旭 on 08 Nov 18
     
    "沒有溝通（產品的）願景和策略 產品藍圖和發佈日期在一年前就由CTO規劃好了 組織內沒有人跟顧客對談 CTO和利害關係人堅持所有的改動都要經由他們批准 因為保密資安等理由，禁止使用實體的看板或告示 利害關係人直接跟CTO對談，跳過Product Owner 由利害關係人來決定交付產品增量，而不是Product Owner 專案/產品只有在完成時才交付，而不是增量式的交付 避免利害關係人直接跟開發團隊對談 Product Backlog是由一個產品委員會決定的 就算對功能的價值有所懷疑，但還是硬上了（為了年終獎金…） 業務為了成交，答應客戶增加目前不存在的功能，而Product Owner不知情 就算是不重要的問題，也有固定的進度表和期限 負責產品管理的角色沒有取得商業智能（BI）資訊的權限，沒有充足的資訊和數據幫助決定 利害關係人使用需求文件來和產品與工程部門溝通 Product Owner大部分的時間都在撰寫和管理使用者故事（User Stories） 在Sprint開始後不久Sprint Backlog就變了 專門成立一個開發團隊來修Bugs和處理小的需求 利害關係人沒有參加過Scrum活動（例如Sprint Planning和Sprint Review） 主要是用『速率（Velocity）符合當初的承諾』來當指標評估Scrum是否成功 開發人員沒有參與創造使用者故事 同時處理的專案數量和工作會改變開發團隊的人數跟組成方式 在Daily Stand up中團隊成員對ScrumMaster報告 定期舉行自省會議（Retrospectives），但沒有改變隨之發生 開發團隊並不是跨功能（Cross Functional），而要靠其他團隊或部門才能完成工作"
張 旭

我所认为的RESTful API最佳实践 · ScienJus's Blog - 0 views

www.scienjus.com/my-restful-api-best-practices

api REST

shared by 張 旭 on 29 Oct 18 - No Cached
  • 最好将过滤、分页和排序的相关信息全权交给客户端，包括过滤条件、页数或是游标、每页的数量、排序方式、升降序等，这样可以使 API 更加灵活。
  • 过度纠结如何遵守规范只是徒增烦恼
  • API 的版本号和客户端 APP 的版本号是毫无关系的
  • ...13 more annotations...
  • 除了 POST 其他三种请求都具备幂等性（多次请求的效果相同）
  • PUT 也可以用于创建操作，只要在创建前就可以确定资源的 id。
  • 永远去使用可以指向资源的的最短 URL 路径
  • RESTful 中不建议出现动词
  • 这里我选择了 PUT 而不是 POST，因为我觉得点赞这种行为应该是幂等的，多次操作的结果应该相同。
  • Token 的状态保持一般有两种方式实现：一种是在用户每次操作都会延长或重置 TOKEN 的生存时间（类似于缓存的机制），另一种是 Token 的生存时间固定不变，但是同时返回一个刷新用的 Token，当 Token 过期时可以将其刷新而不是重新登录。
  • 一般客户端会把请求参数拼接后并加密作为 Sign 传给服务端，这样即使被抓包了，对方只修改参数而无法生成对应的 Sign 也会被服务端识破。
  • 尽量使用 HTTP 状态码
  • data是真正需要返回的数据，并且只会在请求成功时才存在，msg只用在开发环境，并且只为了开发人员识别。客户端逻辑只允许识别code，并且不允许直接将msg的内容展示给用户。
  • JSON 比 XML 可视化更好，也更加节约流量，所以尽量不要使用 XML。
  • 创建和修改操作成功后，需要返回该资源的全部信息。
  • 即使客户端一个页面需要展示多个资源，也不要在一个接口中全部返回，而是让客户端分别请求多个接口。
  • 推荐使用游标分页
張 旭

数据库水平分片心得 · ScienJus's Blog - 0 views

www.scienjus.com/database-sharding-review

database mysql system

shared by 張 旭 on 29 Oct 18 - No Cached
  • 水平分片（也叫水平分库）指的是将整体存储在单个数据库中的数据，通过某种策略分摊到多个表结构与其相同的数据库中，这样每个数据库中的数据量就会相对减少很多，并且可以部署在不同物理服务器上，理论上能够实现数据库的无限横向拓展。
  • 当遇到第一次数据库性能问题时，最先想到的方案应该是读写分离，将所有写操作都放在主数据库上，所有读操作都放在从数据库上
  • 配置一主多从
  • ...16 more annotations...
  • 主从关系（Master-Slave），此时所有操作还是由主数据库完成，主数据库再同步到从数据库上，而从数据库只需要在主数据库挂掉之后代替其工作。
  • 一般来说读写分离加上缓存已经可以应付绝大多数情况了，并且几乎不需要对业务层面进行修改。
  • 对数据库进行垂直分库，将业务彼此无关的表放在单独的数据库中，分库后不同库中的表无法进行联合查询等操作，但是可以平摊压力，并且独立做读写分离。
  • 对数据库进行水平分表，建立多个结构相同的表分摊数据，使得每个表的数据量减少从而提升速度。
  • 分表却只能在单台机器的单个数据库上，如果是服务器本身的性能达到瓶颈，则分表不会有明显作用。
  • 分表后各个子表还是可以通过 union 等命令联合查询，分库后则不行
  • 基于 id 的区间分片，例如：将 id 为 1-2w 的数据存放在 A 数据库，2w-4w 的数据存放在 B 数据库。
  • 基于 id 的 hash 分片，例如：将 id%2=0 的数据存放在 A 数据库，id%2=1 的数据放在 B 数据库。
  • 基于时间的区间分片，大部分软件都会有一个特征：越新的数据被操作的几率越大，老数据几乎不会被操作。所以通过数据的插入时间进行分库（也称为冷热分离）
  • 基于检索表分片，通过额外建立一张检索表保存 id 与所在数据库节点的对应关系，优点是逻辑简单，自由且不会有迁移问题，缺点是每次查询都需要额外查询检索表，所以一般会选择将检索表缓存到内存中。
  • 基于地理位置分片
  • 分库策略
  • 在数据库表设计时，为了保证 id 唯一，大部分人都会将主键设为自增的 int 类型。但是由于 auto\_increment 是和表所绑定的，所以在分库后每个表的自增 id 也是独立的。这样就肯定会发生主键冲突
  • 但是很多人都希望主键即使不是连续自增，也是一个有序的整数，这样在排序等情况下会有用。这时候就需要自己实现一个 id 生成算法了，一般都会使用 unix 时间戳保持有序，混入 Mac 地址等保证唯一。
  • 在分库情况下，需要将大部分联合查询都替换为至少两次查询，先从关联的表中查询出符合条件的 id，在根据 id 去对应的数据库里查询主体信息。
  • 数据库水平分片作为数据库性能瓶颈的最终解决方案，确实可以有效的解决这个问题。但是它将业务逻辑变得非常复杂（主要是关联表冗余和字段冗余，以及这些数据的更新），并且有分布式事务这个难题。所以不到必要时刻，尽量不要轻易尝试数据库分片。
張 旭

Java microservices architecture by example - 0 views

www.scnsoft.com/...java-microservices-example

java micro service architecture programming

shared by 張 旭 on 26 Oct 18 - No Cached
baliar liked it
  • A microservices architecture is a particular case of a service-oriented architecture (SOA)
  • What sets microservices apart is the extent to which these modules are interconnected.
  • Every server comprises just one certain business process and never consists of several smaller servers.
  • ...12 more annotations...
  • Microservices also bring a set of additional benefits, such as easier scaling, the possibility to use multiple programming languages and technologies, and others.
  • Java is a frequent choice for building a microservices architecture as it is a mature language tested over decades and has a multitude of microservices-favorable frameworks, such as legendary Spring, Jersey, Play, and others.
  • A monolithic architecture keeps it all simple. An app has just one server and one database.
  • All the connections between units are inside-code calls.
  • split our application into microservices and got a set of units completely independent for deployment and maintenance.
  • Each of microservices responsible for a certain business function communicates either via sync HTTP/REST or async AMQP protocols.
  • ensure seamless communication between newly created distributed components.
  • The gateway became an entry point for all clients’ requests.
  • We also set the Zuul 2 framework for our gateway service so that the application could leverage the benefits of non-blocking HTTP calls.
  • we've implemented the Eureka server as our server discovery that keeps a list of utilized user profile and order servers to help them discover each other.
  • We also have a message broker (RabbitMQ) as an intermediary between the notification server and the rest of the servers to allow async messaging in-between.
  • microservices can definitely help when it comes to creating complex applications that deal with huge loads and need continuous improvement and scaling.
張 旭

Password management in Django | Django documentation | Django - 0 views

docs.djangoproject.com/...passwords

python django password

shared by 張 旭 on 18 Oct 18 - No Cached
  • Each password validator must provide a help text to explain the requirements to the user, validate a given password and return an error message if it does not meet the requirements, and optionally receive passwords that have been set.
  • By default, validators are used in the forms to reset or change passwords and in the createsuperuser and changepassword management commands
  • Validators aren’t applied at the model level,
張 旭

Why it took a long time to build that tiny link preview on Wikipedia - Wikimedia Blog - 0 views

blog.wikimedia.org/...tiny-link-preview-on-wikipedia

programming performance website image

shared by 張 旭 on 16 Oct 18 - No Cached
  • taken a few years for us to get this out to everyone
  • 張 旭 on 16 Oct 18
     
    "taken a few years for us to get this out to everyone"
張 旭

Managing files | Django documentation | Django - 0 views

docs.djangoproject.com/...files

python django web framework

shared by 張 旭 on 14 Oct 18 - No Cached
  • By default, Django stores files locally, using the MEDIA_ROOT and MEDIA_URL settings.
  • use a FileField or ImageField, Django provides a set of APIs you can use to deal with that file.
  • Behind the scenes, Django delegates decisions about how and where to store files to a file storage system.
  • ...1 more annotation...
  • Django uses a django.core.files.File instance any time it needs to represent a file.
張 旭

Docker for AWS persistent data volumes | Docker Documentation - 0 views

docs.docker.com/...persistent-data-volumes

aws docker system

shared by 張 旭 on 08 Oct 18 - No Cached
  • Cloudstor is a modern volume plugin built by Docker
  • Docker swarm mode tasks and regular Docker containers can use a volume created with Cloudstor to mount a persistent data volume.
  • Global shared Cloudstor volumes mounted by all tasks in a swarm service.
  • ...14 more annotations...
  • Workloads running in a Docker service that require access to low latency/high IOPs persistent storage, such as a database engine, can use a relocatable Cloudstor volume backed by EBS.
  • Each relocatable Cloudstor volume is backed by a single EBS volume.
  • If a swarm task using a relocatable Cloudstor volume gets rescheduled to another node within the same availability zone as the original node where the task was running, Cloudstor detaches the backing EBS volume from the original node and attaches it to the new target node automatically.
  • in a different availability zone,
  • Cloudstor transfers the contents of the backing EBS volume to the destination availability zone using a snapshot, and cleans up the EBS volume in the original availability zone.
  • Typically the snapshot-based transfer process across availability zones takes between 2 and 5 minutes unless the work load is write-heavy.
  • A swarm task is not started until the volume it mounts becomes available
  • Sharing/mounting the same Cloudstor volume backed by EBS among multiple tasks is not a supported scenario and leads to data loss.
  • a Cloudstor volume to share data between tasks, choose the appropriate EFS backed shared volume option.
  • When multiple swarm service tasks need to share data in a persistent storage volume, you can use a shared Cloudstor volume backed by EFS.
  • a volume and its contents can be mounted by multiple swarm service tasks without the risk of data loss
  • over NFS
  • the persistent data backed by EFS volumes is always available.
  • shared Cloudstor volumes only work in those AWS regions where EFS is supported.
張 旭

Use multi-stage builds | Docker Documentation - 0 views

docs.docker.com/...multistage-build

docker system

shared by 張 旭 on 29 Sep 18 - No Cached
  • Maintaining two Dockerfiles is not ideal.
  • This is failure-prone and hard to maintain. It’s easy to insert another command and forget to continue the line using the \ character
  • create a container from it to copy the artifact out
  • ...4 more annotations...
  • You only need the single Dockerfile. You don’t need a separate build script,
  • You don’t need to create any intermediate images and you don’t need to extract any artifacts to your local system at all.
  • Debugging a specific build stage
  • You can use the COPY --from instruction to copy from a separate image, either using the local image name, a tag available locally or on a Docker registry, or a tag ID.
« First ‹ Previous 321 - 340 of 596 Next › Last »
Showing 20▼ items per page