How to setup multiple environments for bitbucket pipeline without using deployments.

Bitbucket Pipelines is an integrated CI/CD service built into Bitbucket. It allows you to automatically build, test, and even deploy your code based on a configuration file (bitbucket-pipelines.yml) in your repository. Essentially, it create containers in the cloud for you. Inside these containers, you can run commands (like you might on a local machine) but with all the advantages of a fresh system, customized and configured for your needs.

With the help of deployments in Bitbucket, we are able to deploy the code to multiple environments hosted on different cloud (AWS in my case) accounts.

Here I would like to highlight an issue that we faced earlier while using multiple deployment environments more than once and will share a work-around the issue.

I have two AWS account, one for staging and another for production. For staging environment, two images (project-1:STAGING and project-2:STAGING) are being built and then pushed to AWS ECR. And for production environment, one image (project-1:PROD) is being built and then pushed to AWS ECR.

The bitbucket-pipelines.yml is as follows;

push-image-1-STAGING: &push-image-1-STAGING
  name: push-image-1-STAGING
  script:
    - export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
    - export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
    - docker push ************.dkr.ecr.us-east-1.amazonaws.com/project-1:STAGING

push-image-2-STAGING: &push-image-2-STAGING
  name: push-image-2-STAGING
  script:
    - export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
    - export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
    - docker push ************.dkr.ecr.us-east-1.amazonaws.com/project-2:STAGING         


push-image-PROD: &push-image-PROD
  name: push-image-PROD
  script:
    - export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
    - export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
    - docker push ************.dkr.ecr.us-east-1.amazonaws.com/project-1:PROD

pipelines:
  branches:
    development:
      - step: *push-image-1-STAGING
        deployment: STAGING
      - step: *push-image-2-STAGING
        deployment: STAGING

master:
  - step: *push-image-PROD
    deployment: PROD

Deployment environments are set as below (Your_project > Repository settings > Deployments);

AWS credentials are stored in each deployment environment as below;

Here you can see, we have used deployment:STAGING twice and the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are stored in deployments (STAGING and PROD). While running the pipeline, the following error appeared;

"Configuration error: The deployment environment 'STAGING' in your bitbucket-pipelines.yml file occurs multiple times in the pipeline."

A quick way to fix this issue is buy removing the AWS access credentials from the deployment and place them in repository variables with a unique tag name and replace the environment variables in the bitbucket-pipelines.yml with the same name from the repository variables.

Repository variables are stored as below (Your_project > Repository settings > Repository Variables) ;

Updated bitbucket-pipelines.yml ;

push-image-1-STAGING: &push-image-1-STAGING
  name: push-image-1-STAGING
  script:
    - export AWS_ACCESS_KEY_ID_STAGING=$AWS_ACCESS_KEY_ID_STAGING
    - export AWS_SECRET_ACCESS_KEY_STAGING=$AWS_SECRET_ACCESS_KEY_STAGING
    - docker push ************.dkr.ecr.us-east-1.amazonaws.com/project-1:STAGING

push-image-2-STAGING: &push-image-2-STAGING
  name: push-image-2-STAGING
  script:
    - cd demo-project
    - export AWS_ACCESS_KEY_ID_STAGING=$AWS_ACCESS_KEY_ID_STAGING
    - export AWS_SECRET_ACCESS_KEY_STAGING=$AWS_SECRET_ACCESS_KEY_STAGING
    - docker push ************.dkr.ecr.us-east-1.amazonaws.com/project-2:STAGING        


push-image-PROD: &push-image-PROD
  name: push-image-PROD
  script:
    - export AWS_ACCESS_KEY_ID_PROD=$AWS_ACCESS_KEY_ID_PROD
    - export AWS_SECRET_ACCESS_KEY_PROD=$AWS_SECRET_ACCESS_KEY_PROD
    - docker push ************.dkr.ecr.us-east-1.amazonaws.com/project-1:PROD

pipelines:
  branches:
    development:
      - step: *push-image-1-STAGING
      - step: *push-image-2-STAGING

    master:
      - step: *push-image-PROD 

As you can see, not much has been changed in the bitbucket-pipelines.yml.

For STAGING, AWS credentials has been changed to; AWS_ACCESS_KEY_ID_STAGING and AWS_SECRET_ACCESS_KEY_STAGING

For PROD, AWS credentials has been changed to;            AWS_ACCESS_KEY_ID_PROD and AWS_SECRET_ACCESS_KEY_PROD

Just by adding these unique tags and adding all keys in the repository variables will solve the problem and can replace the use of deployments for storing multiple environment credentials.