Group items tagged

Within your Terraform configuration, you may include the name of the current workspace using the ${terraform.workspace} interpolation sequence.

changing behavior based on the workspace.

Named workspaces allow conveniently switching between multiple instances of a single configuration within its single backend.

A common use for multiple workspaces is to create a parallel, distinct copy of a set of infrastructure in order to test a set of changes before modifying the main production infrastructure.

Non-default workspaces are often related to feature branches in version control.

In particular, organizations commonly want to create a strong separation between multiple deployments of the same infrastructure serving different development stages (e.g. staging vs. production) or different internal teams.

use one or more re-usable modules to represent the common elements, and then represent each instance as a separate configuration that instantiates those common elements in the context of a different backend.

If a Terraform state for one configuration is stored in a remote backend that is accessible to other configurations then terraform_remote_state can be used to directly consume its root module outputs from those other configurations.

For server addresses, use a provider-specific resource to create a DNS record with a predictable name and then either use that name directly or use the dns provider to retrieve the published addresses in other configurations.

using a remote backend instead is recommended when there are multiple collaborators.

When a Workspace is declared in a job, one or more files or directories can be added. Each addition creates a new layer in the Workspace filesystem. Downstreams jobs can then use this Workspace for its own needs or add more layers on top.

Unlike caching, Workspaces are not shared between runs as they no longer exists once a Workflow is complete.

A prime example is package dependency managers such as Yarn, Bundler, or Pip.

Caches are global within a project, a cache saved on one branch will be used by others so they should only be used for data that is OK to share across Branches

Artifacts are used for longer-term storage of the outputs of your build process.

If your project needs to be packaged in some form or fashion, say an Android app where the .apk file is uploaded to Google Play, that’s a great example of an artifact.

rerun just the failed job

Builds without workflows require a build job.

Refer the YAML Anchors/Aliases documentation for information about how to alias and reuse syntax to keep your .circleci/config.yml file small.

workflow orchestration with two parallel jobs

jobs run according to configured requirements, each job waiting to start until the required job finishes successfully

requires: key

fans-out to run a set of acceptance test jobs in parallel, and finally fans-in to run a common deploy job.

Workflows can be configured to wait for manual approval of a job before continuing to the next job

approval is a special job type that is only available to jobs under the workflow key

The name of the job to hold is arbitrary - it could be wait or pause, for example, as long as the job has a type: approval key in it.

schedule a workflow to run at a certain time for specific branches.

The triggers key is only added under your workflows key

using cron syntax to represent Coordinated Universal Time (UTC) for specified branches.

the commit workflow has no triggers key and will run on every git push

The nightly workflow has a triggers key and will run on the specified schedule

use a context to share environment variables

use the same shared environment variables when initiated by a user who is part of the organization.

CircleCI does not run workflows for tags unless you explicitly specify tag filters.

CircleCI branch and tag filters support the Java variant of regex pattern matching.

Each workflow has an associated workspace which can be used to transfer files to downstream jobs as the workflow progresses.

Downstream jobs can attach the workspace to their container filesystem.

Attaching the workspace downloads and unpacks each layer based on the ordering of the upstream jobs in the workflow graph.

Workflows that include jobs running on multiple branches may require data to be shared using workspaces

Files and directories named in the paths: property of persist_to_workspace will be uploaded to the workflow’s temporary workspace relative to the directory specified with the root key.

Configure a job to get saved data by configuring the attach_workspace key.

persist_to_workspace

attach_workspace

To rerun only a workflow’s failed jobs, click the Workflows icon in the app and select a workflow to see the status of each job, then click the Rerun button and select Rerun from failed.

if you do not see your workflows triggering, a configuration error is preventing the workflow from starting.

check your Workflows page of the CircleCI app (not the Job page)

The environment variables are defined as name/value pairs and are injected at runtime.

The CircleCI Docker Layer Caching feature allows builds to reuse Docker image layers

Image layers are stored in separate volumes in the cloud and are not shared between projects.

Environment variables store customer data that is used by a project.

Defines the underlying technology to run a job.

machine to run your job inside a full virtual machine.

docker to run your job inside a Docker container with a specified image

A job is a collection of steps.

The first image listed in config.yml

A CircleCI project shares the name of the code repository for which it automates workflows, tests, and deployment.

must be added with the Add Project button

Following a project enables a user to subscribe to email notifications for the project build status and adds the project to their CircleCI dashboard.

A step is a collection of executable commands

Users must be added to a GitHub or Bitbucket org to view or follow associated CircleCI projects.

A Workflow is a set of rules for defining a collection of jobs and their run order.

Workflows are implemented as a directed acyclic graph (DAG) of jobs for greatest flexibility.

A workspace is a workflows-aware storage mechanism.

docker build --cache-from "$CI_REGISTRY_IMAGE:latest" -t "$CI_REGISTRY_IMAGE:new-tag"

But if you maintain a CHANGELOG in this format, and/or your Git tags are also your Docker tags, you can get the previous version and use cache the this image version.

script: - export PREVIOUS_VERSION=$(perl -lne 'print "v${1}" if /^##\s\[(\d\.\d\.\d)\]\s-\s\d{4}(?:-\d{2}){2}\s*$/' CHANGELOG.md | sed -n '2 p') - docker build --cache-from "$CI_REGISTRY_IMAGE:$PREVIOUS_VERSION" -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_TAG" -f ./prod.Dockerfile .

« Docker layer caching » is enough to optimize the build time.

We're building a Docker image, dependencies are installed inside a container.We can't cache a dependencies directory if it doesn't exists in the job workspace.

Dependencies will always be installed from a container but will be extracted by the GitLab Runner in the job workspace. Our goal is to send the cached version in the build context.

- docker cp app:/var/www/html/vendor/ ./vendor

after_script

- docker cp app:/var/www/html/node_modules/ ./node_modules

To avoid old dependencies to be mixed with the new ones, at the risk of keeping unused dependencies in cache, which would make cache and images heavier.

If you need to cache directories in testing jobs, it's easier: use volumes !

version your cache keys !

sharing Docker image between jobs

In every job, we automatically get artifacts from previous stages.

docker save $DOCKER_CI_IMAGE | gzip > app.tar.gz

I personally use the « push / pull » technique,

Workspaces persist data between jobs in a single Workflow.

Caching persists data between the same job in different Workflow builds.

run using the machine executor which enables reuse of recently used machine executor runs,

docker executor which can compose Docker containers to run your tests and any services they require

macos executor

Steps are a collection of executable commands which are run during a job

checkout: key is required to checkout your code

run: enables addition of arbitrary, multi-line shell command scripting

orchestrating job runs with parallel, sequential, and manual approval workflows.