mrvitej.in

Archive · AWS

GCP to AWS Migration: Connectivity, Data, Workloads and Cutover

Executing a Google Cloud to AWS migration: the VPN, database replication with DMS, object copies, VM replication, moving GKE workloads to EKS, and a rehearsed cutover runbook.

7 min read

  • AWS DMS
  • AWS DataSync
  • AWS Application Migration Service
  • Amazon EKS
  • Cloud VPN
  • Transit Gateway

With the plan, landing zone and waves from the previous post in place, this is the execution: the moving parts, in the order they happen.

The migration architecture

flowchart LR
  subgraph GCP[Google Cloud]
    GVM[Compute Engine VMs]
    GSQL[Cloud SQL PostgreSQL]
    GCS[Cloud Storage]
    GKE[GKE workloads]
    GAR[Artifact Registry]
  end
  subgraph AWS[AWS ap-south-1]
    EC2[EC2]
    RDS[RDS PostgreSQL]
    S3[S3]
    EKS[EKS]
    ECR[ECR]
  end
  GCP -- HA VPN and BGP --- AWS
  GVM -- MGN replication --> EC2
  GSQL -- DMS full load and CDC --> RDS
  GCS -- DataSync --> S3
  GAR -- image copy --> ECR
  GKE -- manifests rewritten --> EKS

Step 1: Connect the clouds

On AWS, attach a site-to-site VPN to the Transit Gateway. On Google Cloud, create an HA VPN gateway and a Cloud Router, and use the AWS tunnel details for the peer.

# AWS: customer gateway points at the GCP HA VPN interface IP
aws ec2 create-customer-gateway --type ipsec.1 --bgp-asn 65010 --public-ip 34.0.0.10
aws ec2 create-vpn-connection --type ipsec.1 \
  --customer-gateway-id cgw-0abc --transit-gateway-id tgw-0def \
  --options TunnelOptions='[{TunnelInsideCidr=169.254.10.0/30},{TunnelInsideCidr=169.254.11.0/30}]'
# GCP: HA VPN gateway and router
gcloud compute vpn-gateways create gcp-to-aws --network=prod-vpc --region=asia-south1
gcloud compute routers create gcp-to-aws-router --network=prod-vpc --region=asia-south1 --asn=65010
gcloud compute external-vpn-gateways create aws-tgw \
  --interfaces 0=13.0.0.1,1=13.0.0.2
# then one vpn-tunnel and one BGP peer per AWS tunnel

Verify before going further:

  • Both tunnels up, BGP routes learned on both sides.
  • Security groups and GCP firewall rules allow the migration ports only.
  • Measure throughput with iperf3 between a VM on each side. Use the result to schedule data copies realistically.

Step 2: Databases with AWS DMS

DMS copies the existing data (full load) and then streams changes (CDC) until cutover, so the downtime is only the final switch.

Prepare Cloud SQL for PostgreSQL for logical replication:

gcloud sql instances patch shop-db --database-flags=cloudsql.logical_decoding=on
# a user with REPLICATION privilege for DMS
gcloud sql users create dms_user --instance=shop-db --password='...'
psql -h <cloud-sql-ip> -U postgres -c "ALTER USER dms_user WITH REPLICATION;"

Create the schema first, because DMS moves data well but not every object (functions, sequences ownership, some constraints):

pg_dump -h <cloud-sql-ip> -U postgres --schema-only --no-owner shop > schema.sql
psql -h shop-db.xxxx.ap-south-1.rds.amazonaws.com -U dbadmin -d shop -f schema.sql

Endpoints and task:

aws dms create-endpoint --endpoint-identifier gcp-src --endpoint-type source \
  --engine-name postgres --server-name 10.10.5.3 --port 5432 \
  --database-name shop --username dms_user --password '...'

aws dms create-endpoint --endpoint-identifier rds-tgt --endpoint-type target \
  --engine-name postgres --server-name shop-db.xxxx.ap-south-1.rds.amazonaws.com \
  --port 5432 --database-name shop --username dbadmin --password '...'

aws dms create-replication-task --replication-task-identifier shop-full-cdc \
  --source-endpoint-arn <src-arn> --target-endpoint-arn <tgt-arn> \
  --replication-instance-arn <ri-arn> --migration-type full-load-and-cdc \
  --table-mappings file://mappings.json \
  --replication-task-settings '{"FullLoadSettings":{"TargetTablePrepMode":"DO_NOTHING"}}'

mappings.json:

{
  "rules": [
    {
      "rule-type": "selection",
      "rule-id": "1",
      "rule-name": "all-public",
      "object-locator": { "schema-name": "public", "table-name": "%" },
      "rule-action": "include"
    }
  ]
}

DO_NOTHING keeps the schema you created. Turn on data validation in the task settings, and compare row counts and a few checksums per table yourself:

SELECT count(*), md5(string_agg(id::text, ',' ORDER BY id)) FROM orders WHERE created_at < now() - interval '1 hour';

After cutover, reset sequences on the target, since DMS does not carry their current values:

SELECT setval('orders_id_seq', (SELECT max(id) FROM orders));

Step 3: Object storage with DataSync

DataSync can read Google Cloud Storage through its S3-compatible XML API using an HMAC key, and write to S3.

# GCP: HMAC key for a service account with storage.objectViewer on the bucket
gcloud storage hmac create migration-sa@prod.iam.gserviceaccount.com

# AWS: source location (object storage type) and destination
aws datasync create-location-object-storage --server-hostname storage.googleapis.com \
  --bucket-name acme-uploads --access-key <HMAC_ID> --secret-key <HMAC_SECRET> \
  --agent-arns <agent-arn>
aws datasync create-location-s3 --s3-bucket-arn arn:aws:s3:::acme-uploads-aws \
  --s3-config BucketAccessRoleArn=arn:aws:iam::111122223333:role/datasync-s3
aws datasync create-task --source-location-arn <src> --destination-location-arn <dst> \
  --name gcs-to-s3 --options VerifyMode=ONLY_FILES_TRANSFERRED,OverwriteMode=ALWAYS

Run it once for the bulk copy, then again just before cutover for the delta. Check that content type and cache headers came across, because web clients depend on them.

Step 4: Virtual machines with Application Migration Service

MGN treats a Compute Engine VM as a source server. Install the agent on each VM:

wget -O ./aws-replication-installer-init \
  https://aws-application-migration-service-ap-south-1.s3.ap-south-1.amazonaws.com/latest/linux/aws-replication-installer-init
sudo chmod +x aws-replication-installer-init
sudo ./aws-replication-installer-init --region ap-south-1 --no-prompt

MGN copies the disks continuously into a staging subnet. For each server:

  1. Set the launch template: instance type, subnet, security groups, IAM role.
  2. Launch a test instance, check the application starts and can reach its dependencies.
  3. Mark it ready for cutover; launch the cutover instance during the window.

Step 5: Containers, from GKE to EKS

Copy images without pulling them through a laptop:

aws ecr get-login-password --region ap-south-1 | \
  crane auth login 111122223333.dkr.ecr.ap-south-1.amazonaws.com -u AWS --password-stdin
crane copy asia-south1-docker.pkg.dev/prod/apps/shop-api:2.14.0 \
  111122223333.dkr.ecr.ap-south-1.amazonaws.com/shop-api:2.14.0

Most Kubernetes manifests move unchanged. Four parts do not:

Ingress. Replace the GKE ingress class with the AWS Load Balancer Controller:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop-api
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:ap-south-1:111122223333:certificate/abc
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
spec:
  ingressClassName: alb
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend: { service: { name: shop-api, port: { number: 80 } } }

Storage. Replace the GCE persistent disk class with EBS gp3:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gp3
provisioner: ebs.csi.aws.com
parameters: { type: gp3, encrypted: "true" }
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true

Stateful data on volumes does not move with the manifest; migrate it at the application level or with a backup tool such as Velero.

Workload identity. On GKE a Kubernetes service account is annotated with a Google service account. On EKS, use Pod Identity (an association made outside the manifest) or IRSA (an annotation):

apiVersion: v1
kind: ServiceAccount
metadata:
  name: shop-api
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::111122223333:role/shop-api   # IRSA

Application code that calls Google APIs (Cloud Storage client, Pub/Sub, Secret Manager) must switch to the AWS SDK. Hide these behind a small interface so the switch is one module, not a search across the codebase.

Step 6: The cutover runbook

flowchart TB
  A[T minus 7 days<br/>lower DNS TTL to 60s] --> B[T minus 1 day<br/>final test launches, go or no-go]
  B --> C[T zero<br/>maintenance page, freeze writes]
  C --> D[Wait for DMS and DataSync<br/>to drain, validate counts]
  D --> E[Launch MGN cutover instances<br/>scale EKS workloads]
  E --> F[Switch DNS to AWS]
  F --> G[Smoke tests and<br/>watch dashboards]
  G --> H{Healthy after<br/>agreed window?}
  H -- yes --> I[Stop replication<br/>GCP read-only]
  H -- no --> J[Rollback<br/>DNS to GCP]

A typical window, written down with owners and times:

Time Step Owner
22:00 Maintenance page on; stop schedulers and consumers on GCP App
22:10 Confirm DMS CDC latency near zero; stop the task DBA
22:20 Validate row counts and checksums; reset sequences DBA
22:30 Final DataSync delta run DevOps
22:45 Launch cutover instances; scale EKS deployments DevOps
23:00 Switch Route 53 records; remove maintenance page DevOps
23:00 to 01:00 Watch error rate, latency, queues, logs All
01:00 Go or rollback decision Lead

Step 7: After cutover

  • Keep GCP read-only for the agreed rollback period, then export final backups and delete resources, including disks, snapshots, static IPs and buckets.
  • Remove the VPN and the migration IAM roles and HMAC keys.
  • Right-size on AWS from real metrics after two to four weeks, then consider Savings Plans for the steady baseline.
  • Update runbooks, dashboards and on-call alerts to point at AWS resources.

Lessons that save time

  • Rehearse the whole cutover once with a copy of production data.
  • Watch egress charges on the Google side; large data sets dominate the one-off cost.
  • Reset database sequences and check time zones; both are common causes of day-one bugs.
  • Keep the change scope small during the move; optimise after it.