Skip to content
Cloud, DevOps & Infrastructure

Google Cloud (GCP) interview questions and answers

79 questions, 21 with code. About 7 h 33 min to read them all.

By TestLab · Updated

Plan for an interview
79 questions

Track what you know and what to revise.

Sign in to track progress

You can open a few full answers without signing in. Sign in to read all

1

Which service categories make up Google Cloud Platform?

Google Cloud Platform (GCP) is Google’s public cloud platform for hosting applications, storing and analyzing data, running containers and virtual machines, connecting networks, and operating production systems. Its services are organized into major categories such as compute, storage, databases, networking, messaging, analytics, AI, security, and DevOps.

Read full answer

Applies toGoogle Cloud Platform current services and Google Cloud CLI current stable release

Resource Hierarchy and Identity

GCP uses a hierarchical resource model:

  • Organization: The top-level container for a Google Workspace or Cloud Identity organization. It can contain folders and directly managed projects.

  • Folders: Containers that organize projects by department, environment, or policy boundary.

  • Projects: Primary ownership containers for resources. They hold IAM policies, enabled service APIs, quotas, service usage, and an association with a billing account.

  • Resources: Individual service resources such as Compute Engine instances, Cloud Storage buckets, Pub/Sub topics, or BigQuery datasets.

Projects can be attached directly to an organization, placed in folders, or initially have no organization parent. A billing account is separate from this hierarchy and can be associated with projects in different organizations.

IAM grants are inherited and additive. A grant at an ancestor level is inherited by descendants and cannot be removed lower down; only an IAM deny policy can block it, and a narrower grant does not override a broader one. Organization policies separately constrain which configurations descendant resources may use.

Resource location affects availability and latency:

  • Zonal resources operate in one zone.

  • Regional resources span multiple zones within a region.

  • Cloud Storage buckets use regional, dual-region, or multi-region locations; a bucket is not itself a global resource.

  • Global external Application Load Balancers provide a global frontend backed by regional infrastructure.

Main Service Categories

Category Representative GCP services AWS equivalent Azure equivalent
Compute Compute Engine, GKE, Cloud Run, App Engine, Cloud Functions EC2, EKS, App Runner, Lambda Virtual Machines, AKS, Container Apps, Functions
Storage Cloud Storage, Filestore, Persistent Disk, Backup and DR S3, EFS, EBS, Backup Blob Storage, Azure Files, Managed Disks, Backup
Relational databases Cloud SQL, AlloyDB RDS, Aurora Database for PostgreSQL/MySQL
Distributed and NoSQL data Spanner, Firestore, Bigtable DynamoDB and other NoSQL services Cosmos DB
Data warehousing BigQuery Redshift Microsoft Fabric, Synapse
Networking VPC, Cloud Load Balancing, Cloud CDN, Cloud DNS, Cloud NAT, Cloud VPN, Cloud Interconnect VPC, ELB, CloudFront, Route 53 VNet, Azure Load Balancer, CDN, DNS
Messaging and events Pub/Sub, Cloud Tasks, Eventarc SNS, SQS, EventBridge Service Bus, Event Grid
DevOps and artifacts Cloud Build, Cloud Deploy, Artifact Registry CodeBuild, CodeDeploy, ECR, CodePipeline Azure DevOps, Container Registry, Azure Pipelines
Observability Cloud Monitoring, Cloud Logging, Cloud Trace, Error Reporting, Profiler CloudWatch, X-Ray Azure Monitor, Application Insights
Security and identity IAM, Cloud KMS, Secret Manager, Security Command Center, Cloud IDS, VPC Service Controls IAM, KMS, Secrets Manager, GuardDuty Entra ID, Key Vault, Defender for Cloud
AI and data science Vertex AI, Gemini models, Vertex AI Workbench, Data Catalog, Looker SageMaker, Bedrock, SageMaker Notebooks Azure AI Foundry, Azure Machine Learning

Choosing Compute and Data Services

  • Compute Engine: Choose for full control over the guest OS, machine image, networking, disks, and operating-system configuration. It fits legacy applications, specialized runtimes, and workloads that cannot use managed platforms.

  • GKE: Choose when the team needs Kubernetes APIs, scheduling control, custom operators, or Kubernetes portability. Google manages the GKE control plane, while the worker nodes run on Compute Engine.

  • Cloud Run: Choose for stateless containers that should scale automatically without operating a cluster. It supports services and jobs and can scale a service to zero when no requests are pending and the minimum instance count is zero.

  • App Engine: Choose for applications built around its supported runtime and deployment model, particularly managed web services.

  • Cloud Functions: Choose for small event-driven handlers triggered by Pub/Sub, Cloud Storage, and other supported events. Cloud Run is generally more flexible for larger containers or complex execution requirements.

For databases, Cloud SQL fits conventional relational applications, AlloyDB targets PostgreSQL-oriented workloads, Spanner provides globally distributed strongly consistent transactions, and Firestore or Bigtable suit NoSQL access patterns. BigQuery is a serverless analytics warehouse, not a row-oriented transactional database.

Production Operations and Failure Modes

Google operates the physical infrastructure and managed-service control planes, but customers remain responsible for application configuration, IAM, data handling, secrets, schemas, and workload logic.

Key production practices include:

  • Run important workloads across multiple zones and, where justified, regions. Use health checks, regional load balancing, and tested failover rather than assuming a zone will remain available.

  • Design Pub/Sub consumers for at-least-once delivery. Exactly-once delivery is available only for pull subscriptions within a region, so consumers should still be idempotent and use bounded retries and dead-letter handling where appropriate.

  • Configure database backups, point-in-time recovery, deletion protection where supported, and recurring restore tests. A backup that has never been restored is not a proven recovery strategy.

  • Use Cloud Monitoring dashboards and SLO-based alerts for availability, latency, saturation, and error-budget consumption. Correlate metrics with Cloud Logging and distributed traces.

  • Inventory resources with commands such as gcloud projects list and gcloud compute regions list. Keep configuration and dependencies in source control and apply changes through reviewed infrastructure-as-code pipelines.

  • Plan upgrades deliberately. Managed services reduce patching work, but customer-managed runtime images, client libraries, schemas, and deployment configurations still require updates.

Security and Cost Management

Use separate projects for production, staging, and development. Apply least-privilege IAM roles to groups and service accounts, avoid using the default Compute Engine service account as an application identity, and prefer Workload Identity Federation over downloadable service-account keys. Store sensitive values in Secret Manager.

Use private addresses, restrictive firewall rules, Private Google Access, and VPC Service Controls where the exfiltration risk justifies them. Cloud Armor can protect global external load-balancing endpoints with DDoS protection, web application firewall rules, and rate limiting.

Cost controls include budgets, billing alerts, labels, committed-use discounts, and reviews of idle resources. Budgets are alerts, not hard spending caps. Also monitor network egress, BigQuery bytes processed, storage classes, logging volume, and unused snapshots because these frequently cause unexpected cloud costs.

2

How is the GCP resource hierarchy organized?

The GCP resource hierarchy organizes an organization’s projects and service resources into a parent-child governance structure. It centralizes IAM and organization policy while retaining projects as the usual boundary for billing, quotas, API enablement, and service configuration.

Read full answer

Applies toGoogle Cloud Platform (all current versions)

Hierarchy Structure

The primary hierarchy is:

  1. Organization

  2. The root container for a Google Cloud organization.

  3. Represented as organizations/123456789.

  4. Usually associated with a Google Workspace or Cloud Identity enterprise.

  5. Folders

  6. Group projects and other folders.

  7. Can be nested to represent business units, environments, or governance domains.

  8. Represented as folders/123456789.

  9. Cannot directly contain service resources.

  10. Projects

  11. Required containers for most Google Cloud resources.

  12. Represented as projects/example-project.

  13. Have an immutable project number and a globally unique project ID.

  14. Are the main billing-account association, IAM, API-enablement, quota, and service-configuration boundary.

  15. Resources

  16. Service-specific objects owned by a project, such as Compute Engine instances, GKE clusters, Cloud Storage buckets, Cloud SQL instances, Pub/Sub topics, and BigQuery datasets.

  17. Can have global, regional, or zonal scope.

  18. Do not all implement the same child-resource hierarchy.

A project belongs to exactly one folder or directly to the organization. It cannot belong to multiple folders simultaneously. Organizations and folders are hierarchy nodes; IAM and organization policies attach to supported nodes or resources.

IAM and Organization Policies

Higher-level policies are inherited through the hierarchy.

Policy type Purpose Example
IAM allow policy Grants permissions on supported resources Allow a group to read project logs
IAM deny policy Denies permissions regardless of applicable allow policies Prevent service accounts from deleting production resources
Organization policy Restricts permitted configurations Restrict Cloud Storage locations or require uniform bucket-level access
Service Usage constraint Limits which services can be enabled in descendant projects Prevent unapproved APIs from being enabled

Important distinctions:

  • Principals such as users, groups, service accounts, and workload identities are not hierarchy nodes. They receive access through IAM bindings.

  • Permissions granted at organization, folder, or project level are inherited by supported descendant resources. A lower-level policy cannot subtract a permission granted by an ancestor.

  • IAM allow policies can be attached to organizations, folders, projects, and resource types that support resource-level IAM. IAM deny policies can be attached at organization, folder, or project level.

  • IAM permissions and conditional bindings are evaluated when a request is made. Organization-policy constraints are generally enforced during resource creation or update; list constraints can also restrict operations involving existing resources.

  • Organization policies constrain configuration but do not grant access.

Cloud Resource Manager manages the organization, folder, and project hierarchy. The IAM service manages IAM allow and deny policies, while individual service APIs create and operate resources.

Operational and Security Boundaries

Projects provide useful production boundaries:

  • Billing: Projects are associated with billing accounts, allowing charges to be consolidated while retaining project-level cost attribution.

  • Least privilege: Grant permissions at the narrowest practical folder, project, or resource level, while accounting for inherited access.

  • Environment isolation: Separate development, staging, and production projects, usually beneath separate folders.

  • Auditability: Cloud Audit Logs, Security Command Center, and Cloud Asset Inventory can operate across the hierarchy.

  • Data protection: VPC Service Controls and organization policies can protect data across projects.

  • Lifecycle: Deleting a project affects its resources, but restoration is available only during a limited post-deletion period. Inventory dependencies, export required logs and configuration, and coordinate project retirement before deletion.

The hierarchy does not automatically pool every service quota. Quotas remain service- and scope-specific, so production capacity must be checked for the relevant project, region, and service.

Comparison with AWS and Azure

GCP AWS Azure
Organization Organization Tenant-level management scope
Folder Organizational unit Management group
Project Account Subscription
Resources Service resources with global, regional, or zonal scope Resources in resource groups, with global or regional scope

All three platforms support nested administrative grouping, but their boundaries differ. AWS organizational units group accounts, while GCP folders group projects and folders. Azure management groups group subscriptions, while Azure resource groups live inside subscriptions and group resources rather than subscriptions.

Production Best Practices

  • Use the organization for enterprise-wide governance, folders for business units or environments, and projects for independently managed systems.

  • Apply organization-wide restrictions such as approved regions, uniform bucket-level access, and restricted service APIs.

  • Grant folder-level permissions only when every descendant project needs the same access; otherwise prefer project or resource scope.

  • Keep production, non-production, and security-tooling projects in separate folders to simplify policy auditing.

  • Inventory hierarchy changes with Cloud Asset Inventory and monitor administrative and policy changes through Cloud Audit Logs.

  • Review inherited IAM and organization-policy behavior during incidents because changes at a parent can affect many descendants.

  • Treat hierarchy as a governance mechanism, not a backup strategy; separately configure backups, replication, retention, and recovery testing for stateful services.

  • Check service-specific quotas before placing additional workloads in a project, and plan dependency cleanup before deleting projects or folders.

3

What are regions and zones in Google Cloud?

A region is a geographic area containing multiple isolated zones, while a zone is an independent deployment and fault domain within a region. Choose a zone for zonal resources and a region for services that replicate or operate across multiple zones.

Read full answer

Applies toGoogle Cloud Platform (all current versions)

Regions and Zones

  • Region: A geographic location identified by a code such as us-central1. It provides a defined area for placing resources while keeping traffic within a predictable geography.

  • Zone: A deployment area within a region, such as us-central1-a. A region typically contains three or more zones.

  • Fault isolation: Zones are engineered to be isolated in power, cooling, and network infrastructure. A failure affecting one zone should not directly affect another zone in the same region.

  • Independent placement: Zone identifiers are region-specific. The -a, -b, and -c suffixes do not imply the same physical location across different regions.

Google Cloud’s logical resource hierarchy—organization, folders, and projects—is separate from geography. A project does not become fault-isolated merely because its name or placement policy refers to a region.

Resource Placement Options

Placement Scope Failure behavior Examples
Zonal One zone May be unavailable during a zonal outage Compute Engine instance, zonal disk, zonal GKE cluster
Regional Multiple zones in one region Designed to tolerate individual zone failures Cloud SQL regional HA, regional persistent disk, regional GKE cluster
Multi-region Multiple regions in a broad geography Can support region-level resilience BigQuery multi-region dataset, Cloud Storage multi-region bucket
Global Global scope Depends on the individual global service Global external Application Load Balancer frontend

Regional resources do not all provide identical replication, failover, or SLA guarantees. Check the documentation for the specific service rather than assuming that “regional” means fully active-active.

Production Failure Behavior

For a highly available stateless application:

  1. Deploy instances across at least two zones in a region.

  2. Place them behind a suitable load balancer.

  3. Keep regional state in a regional or multi-region service.

  4. Monitor errors and latency separately for each zone.

Deploying multiple instances in one zone does not protect against a zonal outage. Conversely, placing a regional service’s database in only one zone can make the entire application unavailable despite multiple stateless frontends.

For example, a regional Cloud SQL instance with high availability maintains a standby in another zone and manages failover. A GKE regional cluster can distribute nodes across zones, while a zonal cluster depends on one zone. A regional outage requires a multi-region architecture, with additional concerns such as data replication, traffic routing, consistency, and cost.

Regions and Zones Compared with AWS and Azure

Google Cloud AWS Azure Meaning
Region Region Region Geographic area
Zone Availability Zone Availability Zone Isolated deployment area within a region
us-central1 us-east-1 eastus Region identifier
us-central1-a us-east-1a eastus-1 Zone identifier

These mappings are conceptual, not one-to-one. AWS and Azure availability-zone numbering has service- and region-specific interpretations, so infrastructure should never assume equivalent physical failure domains.

Operational Best Practices

  • Select by requirements: Prioritize user latency, service availability, data residency, compliance, quota, capacity, and regional pricing.

  • Prefer explicit placement: Include zone or region in infrastructure-as-code, deployment scripts, and incident runbooks.

  • Avoid shared dependencies: Check whether DNS, certificates, databases, secrets, or internal APIs remain tied to one zone.

  • Plan upgrades: Stagger maintenance and use service-supported rolling upgrades so one zone is not disrupted simultaneously.

  • Monitor by location: Alert on zonal error rates, saturation, network failures, and regional SLO burn rather than only aggregate metrics.

  • Test failover and restoration: Exercise regional database failover, backup restoration, and multi-region traffic routing before an incident.

  • Control costs: Multi-region replication, cross-region traffic, backups, and regional load balancing increase usage and operational complexity.

4

What is Google Cloud IAM and how are permissions granted?

Google Cloud IAM is Google Cloud’s authorization system: it grants identities access to resources by binding roles that contain permissions. Authentication establishes who the caller is; IAM determines what that identity may do, while organization policies and resource-specific policies provide additional controls.

Read full answer

Applies toGoogle Cloud IAM (all versions)

How Authorization Works

When a principal requests an action on a resource, IAM evaluates the applicable policies:

  1. The request identifies the principal, resource, and requested action.

  2. IAM checks for an allow binding that gives the principal a role containing the required permission.

  3. Any IAM Condition attached to that binding must also be satisfied.

  4. IAM evaluates applicable deny rules. A matching deny rule blocks access even when another policy allows it.

  5. If at least one applicable allow binding grants the permission and no deny rule blocks it, access is granted.

  6. If no policy grants the permission, access is denied by default.

Permissions are granted by adding a binding between a principal and a role at the appropriate scope. Allow policies are additive, so permissions inherited from multiple scopes can combine. Some resources, such as Cloud Storage buckets and Cloud Run services, also expose resource-level IAM controls.

Principals, Roles, and Permissions

  • Principal: The identity making the request. This can be a Google Workspace user, Google group, service account, workload identity principal, domain, allUsers, or allAuthenticatedUsers.

  • Permission: A specific operation, such as compute.instances.start or storage.objects.get.

  • Role: A collection of permissions assigned to a principal through a policy binding.

  • Binding: Connects a principal to a role and can optionally include an IAM Condition.

Role type Description Example
Basic Grants broad access across most services in the scope where granted roles/viewer, roles/editor
Predefined Grants permissions for a specific service or job roles/compute.viewer, roles/pubsub.subscriber
Custom Contains permissions selected by an administrator A role allowing instance creation but not deletion

Avoid roles/owner and roles/editor for routine access because they include broad permissions. Basic roles are generally granted at organization, folder, or project level and should be replaced by narrowly scoped predefined or custom roles where practical.

Resource Hierarchy and Inheritance

A common Google Cloud resource hierarchy is:

Organization → Folder → Project → Supported resource

Projects can also be attached directly to an organization. IAM allow policies are inherited downward: a role granted at the organization level applies to descendant folders, projects, and supported resources; a project-level grant applies to that project’s supported resources.

Deny policies have different inheritance behavior from allow policies. A deny policy is not automatically inherited by child folders, projects, or resources. Attach deny policies at each hierarchy level where denial is required, and verify the intended scope before applying them.

Organization Policy Service rules are separate from IAM. IAM determines who can act, while organization policies determine which configurations and locations are permitted.

Comparison With AWS and Azure

Platform Permission definition Access assignment
Google Cloud Roles contain permissions; policies contain bindings A principal and role are bound at an IAM-supported scope
AWS Identity and resource policies contain permissions Identity policies attach to principals or roles; resource policies attach to resources
Azure Role definitions contain actions Role assignments connect a principal, role, and scope

All three provide deny-by-default authorization, scoped role assignments, hierarchical controls, and audit logging, but their policy models and inheritance rules differ.

Production Best Practices and Failure Modes

  • Apply least privilege: Prefer narrowly scoped predefined roles, then create custom roles when no suitable predefined role exists.

  • Use groups: Bind groups instead of individual users so access can be changed centrally without editing many policies.

  • Use IAM Conditions: Restrict permissions by time, resource attributes, or request properties, including a condition that requires access to expire.

  • Avoid service-account keys: Use Workload Identity Federation or attached service accounts where supported. Long-lived keys can leak and are difficult to rotate safely.

  • Separate duties: Give operators, developers, auditors, and incident responders different roles. Use controlled, time-bound elevation for privileged tasks.

  • Monitor changes: Admin Activity Audit Logs record IAM policy changes. Enable relevant Data Access Audit Logs where resource access must be investigated.

  • Test changes before removal: Project-level bindings can immediately break CI/CD systems, automation accounts, and production workloads.

  • Plan for emergency access: Maintain a tightly controlled break-glass process with alerting, short-lived credentials where possible, and regular testing.

  • Review permissions continuously: Use IAM Recommender to identify unused or excessive permissions, then validate recommendations against workload and disaster-recovery requirements.

5

How do basic, predefined, and custom IAM roles differ in GCP?

IAM roles are bundles of permissions, but GCP’s basic, predefined, and custom roles differ in scope, precision, and maintenance. Basic roles provide broad access, predefined roles provide Google-maintained job-specific access, and custom roles provide customer-defined permissions.

Read full answer

Applies toGoogle Cloud IAM (current stable)

Core Differences

Type Examples Granularity Maintained by Typical use
Basic roles roles/viewer, roles/editor, roles/owner Very broad Google Broad access where accepted; controlled break-glass administration
Predefined roles roles/run.viewer, roles/compute.instanceAdmin.v1, roles/cloudsql.client Product- or job-specific Google Production workloads and teams
Custom roles projects/example-project/roles/serviceAccountOperator Organization- or project-specific Your organization Permissions not covered by a suitable predefined role
  • Basic roles:

  • Viewer generally reads resources across many Google Cloud products.

  • Editor generally creates and changes resources but has less administrative authority than Owner.

  • Owner includes broad administrative and IAM-management permissions.

  • They can be granted at the organization, folder, or project level, but not directly to an individual resource.

  • They expose unrelated data and workloads and are generally unsuitable for routine job access.

  • Predefined roles:

  • Google creates and maintains them as products evolve.

  • They represent common responsibilities, such as deploying a Cloud Run service or administering Compute Engine instances.

  • Google can update their permission lists, so administrators should monitor changes affecting sensitive permissions.

  • They are normally the preferred choice for production access.

  • Custom roles:

  • They contain only the permissions explicitly selected by the customer.

  • A custom role can contain at most 280 permissions and can be created at the organization or project level; folder-level custom roles cannot be created.

  • An organization-level custom role can be granted throughout that organization and on supported descendant resources. A project-level custom role can be granted on its project and supported project resources.

  • Only permissions supported for use in custom roles can be included. Custom roles are independent definitions, not subsets of basic roles.

  • Google does not add future permissions automatically, so administrators must review and update the role when services introduce required capabilities.

Resource Hierarchy and Effective Access

IAM evaluates role bindings across the Google Cloud resource hierarchy:

  1. A binding on an organization can apply to every descendant folder, project, and supported resource.

  2. A binding on a folder applies to its descendant folders, projects, and supported resources.

  3. A binding on a project applies to the project and its IAM-supporting resources.

  4. A resource-level binding applies only to that resource.

Allow policies are additive: a member receives the union of permissions granted by applicable bindings. A child project cannot subtract an inherited allow by replacing it with a narrower allow policy. An IAM Deny policy at an applicable organization, folder, or project scope can override allowed actions when that permission and resource support Deny. Organization policies and service-specific controls can impose additional restrictions.

A role therefore does not guarantee that an operation will succeed. VPC Service Controls, organization policies, resource policies, API enablement, and service-side authorization may still block access.

Comparison with AWS and Azure

Capability Google Cloud AWS Azure
Broad predefined access Basic roles Broad managed policies; no exact one-to-one role class Broad built-in roles
Job-specific predefined access Predefined roles AWS-managed and customer-managed policies Built-in roles
Customer-defined permissions Custom roles Customer-managed policies Custom roles
Who maintains the custom definition Your organization Your organization Your organization

The concepts are similar, but they are not identical. An AWS customer-managed policy or Azure custom role is not automatically equivalent to a GCP custom role because scope handling, permission behavior, and administrative safeguards differ.

Production Best Practices

  • Prefer predefined roles: select the narrowest predefined role that supports the workload, then use IAM Conditions or a custom role when necessary.

  • Avoid basic roles: especially avoid roles/owner; reserve it for narrowly controlled break-glass administration.

  • Grant to groups or service accounts, rather than individual users, and keep workload identities separate from human identities.

  • Review effective permissions, not just direct bindings, because inherited grants can expand access unexpectedly.

  • Monitor Cloud Audit Logs for IAM policy changes, impersonation, service-account key creation, and access-token generation.

  • Use IAM Recommender and Security Command Center to identify unused permissions, excessive access, and role assignments that should be reduced.

  • Test custom-role changes in a non-production project. Updates affect every principal bound to the role after IAM propagation completes.

  • Maintain an approval and review process for IAM changes, and keep emergency access outside normal CI/CD automation.

6

What are GCP service accounts, and how should they be used?

A Google Cloud service account is a non-human IAM identity used by workloads and automation. Treat it as a narrowly scoped workload identity: prefer short-lived attached credentials or federation, grant only required permissions, and avoid long-lived keys whenever possible.

Read full answer

Applies toGoogle Cloud IAM (current) and Google Cloud CLI (current stable release)

Identity and Authentication

A service account belongs to a project and appears as an IAM principal, for example service-account-name@project-id.iam.gserviceaccount.com. IAM policies bind principals to predefined or custom roles; IAM Conditions can narrow a grant by attributes, time, or resource. It is not a human login, although an authorized principal can impersonate it.

Choose an authentication method based on where the workload runs:

  1. Attached identity: Compute Engine, GKE, Cloud Run, App Engine, and Cloud Functions use the platform’s identity integration to obtain short-lived OAuth 2.0 access tokens without a key file.

  2. Workload Identity Federation: GitHub Actions and other external workloads exchange a configured external credential for a short-lived Google Cloud token.

  3. Impersonation: A principal with roles/iam.serviceAccountTokenCreator can obtain an access token for a service account.

  4. Service account key: A JSON key contains a private key capable of signing authentication assertions. It is a long-lived secret that can be copied or misused, so use it only when attached identities and federation are unavailable.

For every API request, Google Cloud evaluates the token identity, requested operation, resource, IAM policy, applicable IAM Conditions, organization policies, and other controls such as VPC Service Controls. A valid token can still receive 403 Forbidden when authorization is denied.

IAM Roles and Least Privilege

Grant a dedicated identity to each workload or environment rather than sharing platform default accounts.

Need Suitable role
Read one Secret Manager secret roles/secretmanager.secretAccessor
Read BigQuery data roles/bigquery.dataViewer; add roles/bigquery.jobUser if it creates query jobs
Invoke one Cloud Run service roles/run.invoker
Attach or act as a service account roles/iam.serviceAccountUser
Mint tokens for a service account roles/iam.serviceAccountTokenCreator
Use a service account through federation roles/iam.workloadIdentityUser

Prefer bindings on the specific bucket, dataset, secret, or service over project-wide bindings. Never give routine workloads roles/owner or roles/editor. A binding such as api@example-project.iam.gserviceaccount.com plus roles/storage.objectViewer grants only the permissions in that role, and conditions can further constrain it.

Comparison with AWS and Azure

Concept Google Cloud AWS Azure
Workload identity Service account IAM role Managed identity or service principal
Runtime attached identity Service account attached to a VM, GKE workload, or Cloud Run service IAM role for EC2, EKS, or Lambda System-assigned or user-assigned managed identity
External workload access Workload Identity Federation IAM OIDC or SAML identity provider with AssumeRoleWithWebIdentity Workload identity federation (federated credentials)
Downloadable credential Service account key IAM user access key Service principal secret or certificate

Code Example

This example creates a dedicated Cloud Run identity, grants access to one Secret Manager secret, and attaches the identity to the service.

export PROJECT_ID="example-project"
export REGION="us-central1"
export SERVICE_NAME="example-api"
export SA_NAME="example-api-production"
export SECRET_NAME="example-api-secret"
export IMAGE="us-docker.pkg.dev/example-project/example-services/example-api:latest"

gcloud config set project "${PROJECT_ID}"

gcloud iam service-accounts create "${SA_NAME}" \
  --display-name="Example API production identity"

export SA_EMAIL="${SA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com"

# Generate a demo secret; provision the real application credential securely.
openssl rand -base64 32 | \
  gcloud secrets create "${SECRET_NAME}" \
    --replication-policy=automatic \
    --data-file=-

# Grant access only to this secret.
gcloud secrets add-iam-policy-binding "${SECRET_NAME}" \
  --member="serviceAccount:${SA_EMAIL}" \
  --role="roles/secretmanager.secretAccessor"

gcloud run deploy "${SERVICE_NAME}" \
  --image="${IMAGE}" \
  --region="${REGION}" \
  --service-account="${SA_EMAIL}" \
  --set-secrets="APP_CREDENTIAL=${SECRET_NAME}:latest" \
  --no-allow-unauthenticated \
  --quiet

The deploying principal needs Cloud Run deployment permissions, permission to act as the service account, and access to the secret version being mounted. Cloud Run captures standard container stdout and stderr; a logging-writer role is not needed merely for that output.

Production Operations and Failure Modes

  • Use dedicated identities: Separate production, staging, deployment, and telemetry identities to improve attribution and limit blast radius.

  • Treat keys as exceptions: If unavoidable, store keys in a secrets manager, restrict access, rotate them, and revoke them promptly. Prefer attached identities or federation.

  • Monitor audit activity: Review Admin Activity and enabled Data Access audit logs. Alert on service-account changes, key creation, token minting, and repeated authorization failures.

  • Diagnose failures: 401 usually indicates missing, invalid, expired, or revoked credentials; 403 indicates authorization or policy denial. Disabled or deleted identities and failed federation conditions also cause immediate failures.

  • Plan for policy propagation: IAM and organization-policy changes can take several minutes to propagate, so avoid create-attach-delete workflows that require immediate effect.

  • Protect metadata access: Treat the metadata server as a privileged interface. Prevent SSRF vulnerabilities from allowing untrusted workloads to request arbitrary metadata-server paths.

7

What is Workload Identity Federation in Google Cloud?

Workload Identity Federation (WIF) lets external workloads authenticate to Google Cloud by exchanging credentials issued by an external identity provider for short-lived Google Cloud access tokens. It removes the need to distribute or store Google Cloud service account keys in CI/CD systems, hybrid environments, Kubernetes clusters, and other workloads outside Google Cloud.

Read full answer

Applies toGoogle Cloud IAM Workload Identity Federation (all current versions)

How It Works

  1. An external identity provider authenticates the workload and issues a signed, short-lived identity token or assertion.

  2. The workload presents that credential to Google Cloud’s Security Token Service for the configured workload identity pool and provider.

  3. Google Cloud validates the credential’s signature, issuer, audience, expiry, and any mapped attributes.

  4. The provider’s attribute mapping converts provider-specific claims into standard attributes such as google.subject. Its attribute condition can restrict which claims are accepted.

  5. Google Cloud evaluates the resulting principal against IAM policy bindings.

  6. The workload receives a short-lived access token, either by impersonating a service account or by using a federated principal directly on an IAM-enabled resource.

The trust relationship is therefore based on verifiable identity and contextual claims rather than a copied long-lived secret.

Core Components

  • Workload identity pool: A Google Cloud resource representing a trust boundary for external identities. Pools can be global or regional.

  • Provider: Defines how Google Cloud validates an external issuer, including its audience, attribute mappings, and attribute condition. WIF supports OIDC, SAML 2.0, and AWS identity sources.

  • Attribute mapping: Converts claims such as assertion.sub or a repository claim into attributes used in principal identifiers and conditions.

  • Attribute condition: A CEL expression that narrows acceptable identities, for example to one repository, AWS role, issuer, or audience.

  • Service account impersonation: The common pattern grants roles/iam.workloadIdentityUser on a service account to a matching federated principal. The workload can then request an access token for that service account.

  • Direct resource access: A federated principal can receive permissions directly on resources such as Cloud Storage buckets or BigQuery datasets without impersonating a service account.

  • Workforce Identity Federation: A separate Google Cloud mechanism for federating human users, such as employees, and is not the subject of WIF.

Comparison with AWS and Azure

Platform Federation mechanism Typical identity source Credential outcome
Google Cloud WIF Pool, provider, attribute mapping, condition, and IAM bindings OIDC, SAML 2.0, AWS, GitHub, Entra ID Service account impersonation token or direct resource access token
AWS IAM OIDC OIDC identity provider plus IAM role Kubernetes, GitHub, other OIDC providers Temporary AWS role session
AWS IAM Roles Anywhere Trust anchor, profile, and trust policy X.509 or AWS STS web identity Temporary AWS role session
Microsoft Entra Workload ID Federation Federated identity credential attached to a managed identity External OIDC issuer Microsoft Entra token for the managed identity

GCP WIF is structurally similar to AWS IAM OIDC because both exchange an external token for temporary cloud credentials. GCP uses pool and provider resources to define the trust relationship, while AWS represents it with an IAM identity provider and role trust policy. Azure attaches a federated credential to a managed identity.

Production Operations

  • Monitoring: Review available Cloud Audit Logs entries for IAM and token-exchange activity, together with the external provider’s authentication logs.

  • Failure diagnosis: Common causes include an incorrect issuer or audience, missing claims, an attribute condition that no longer matches, clock skew, an invalid principal identifier, or missing IAM bindings.

  • Token lifetime: External credentials and exchanged Google Cloud tokens should be short-lived. Service account impersonation is preferable to distributing static keys.

  • Revocation: Removing the service account IAM binding prevents further impersonation by that principal, although it does not necessarily prevent the underlying federated-token exchange. Deleting the provider prevents new exchanges using that provider.

  • Blast-radius control: Use separate pools or providers for unrelated trust domains and environments so a provider or condition change does not affect every consumer.

  • Incident response: Assume that already issued access tokens may remain usable until expiry. Disable the service account when necessary, revoke access through IAM policy changes, and use short token lifetimes to limit exposure.

  • Infrastructure as code: Manage pools, providers, conditions, service accounts, and IAM bindings through reviewed Terraform or equivalent automation.

Best Practices and Pitfalls

  • Require an exact issuer and explicit allowed audiences rather than trusting claims from a broad issuer.

  • Use narrow attribute conditions based on stable identifiers such as repository, AWS role, or workload identity.

  • Grant roles/iam.workloadIdentityUser only on the service account the principal must impersonate.

  • Give the impersonated service account only the permissions required by the workload; avoid roles such as roles/owner.

  • Prefer principal sets that match specific attributes over broad wildcard identities.

  • Test new repositories, roles, issuers, audiences, and conditions in a separate provider or pool before production rollout.

  • Never place identity tokens, credential configurations, or service account keys in logs or source control.

8

How do IAM conditions and organization policies work in Google Cloud?

IAM conditions make an IAM role binding conditional, while organization policies enforce guardrails on permitted operations and resource configurations. Both can restrict access, but conditions determine whether a particular request qualifies, whereas organization policies determine whether an action or configuration is allowed within a hierarchy scope.

Read full answer

Applies toGoogle Cloud IAM and Organization Policy Service (current stable); Google Cloud CLI (current stable)

Core Difference

  • IAM condition: Adds a Common Expression Language (CEL) expression to a role binding. The principal receives the binding’s permissions only when that expression evaluates to true.

  • Organization policy: Defines a constraint at an organization, folder, or project. It can restrict IAM permissions, disable risky capabilities, or enforce configuration rules.

  • Combined evaluation: IAM must allow an operation. If it does, an applicable organization policy can still prevent the operation or configuration.

  • Neither grants access by itself: IAM grants permissions; organization policies can only restrict what IAM permits.

IAM Conditions

IAM conditions are useful when the same role should have different effects depending on the resource, request, or time.

For example, an operations team might receive roles/compute.instanceAdmin.v1, but only for Compute Engine instances whose resource names begin with projects/example-project/zones/us-central1-a/instances/prod-. The condition could evaluate resource.type == "compute.googleapis.com/Instance" together with resource.name.startsWith("projects/example-project/zones/us-central1-a/instances/prod-").

Key CEL attributes include:

  • resource.type: The resource type being accessed.

  • resource.name: The resource’s full resource name.

  • request.time: The time of the API request.

  • request.auth.claims: Identity-provider claims when supplied and exposed by the relevant service.

Attribute availability depends on the target resource and service. A condition referring to an unsupported attribute cannot be used for that resource.

A condition affects only the binding that contains it. If the same principal has an unconditional binding for the same role at the organization, folder, project, or resource level, that broader grant can still provide access. Basic roles such as roles/owner, roles/editor, and roles/viewer do not support IAM conditions.

Organization Policies

Organization policies provide hierarchy-wide guardrails. Attach them at the organization, folder, or project level; applicable constraints are inherited by child folders and projects and their resources. The policies affect requests involving those resources rather than being inherited by principals as hierarchy objects.

Typical uses include:

  • Preventing user-managed service account key creation.

  • Restricting which identity domains may appear in IAM policy bindings.

  • Preventing public exposure of storage resources or public IP configuration for databases.

  • Restricting resource locations, external IP addresses, or other configurable properties.

For example, disabling service account key creation prevents the key-creation operation even for a principal whose IAM role would normally allow it. Google Cloud recommends Workload Identity Federation, service account impersonation, or other short-lived credential mechanisms instead of downloadable user-managed service account keys.

Organization policies do not grant permissions. Use them for platform-wide controls and IAM for identity-specific authorization. A lower-level configuration cannot simply relax a stricter ancestor constraint, so inspect the effective inherited policy.

Comparison With AWS and Azure

Concern Google Cloud AWS equivalent Azure equivalent
Conditional role grants IAM Conditions with CEL IAM condition keys and policy conditions Azure RBAC conditions and ABAC
Hierarchy-level permission limits Organization policy constraints Service Control Policies Azure Policy and management-group assignments
Resource configuration controls Organization policy constraints, including custom constraints AWS Config rules and control mechanisms Azure Policy
Main distinction IAM permits identities; organization policies restrict operations and configuration SCPs primarily cap IAM permissions Azure Policy primarily governs resource configuration and selected permissions

Production Best Practices and Failure Modes

  • Test conditions first: Validate CEL expressions against representative resources. A condition that matches resource names can be broader than expected if principals can create similarly named resources.

  • Check all applicable grants: Search organization, folder, project, and resource-level bindings. A conditional project grant does not override an unconditional inherited grant.

  • Roll out guardrails gradually: Test organization policies in a nonproduction folder or controlled project before enforcing them across critical workloads.

  • Preserve emergency access: Maintain tested break-glass identities and ensure their access remains compatible with each policy.

  • Audit changes: Use Cloud Audit Logs to detect IAM and organization policy modifications, and alert on unexpected changes.

  • Review effective permissions: IAM Recommender and Policy Intelligence can help identify overbroad roles and assess guardrail impact.

  • Back up IAM policies: Export policies before bulk changes so rollback does not depend on reconstructing them manually.

  • Use least privilege: Prefer groups over individual users, narrowly scoped roles, short-lived credentials, and Workload Identity Federation over long-lived user-managed service account keys.

9

How do you apply least privilege in Google Cloud?

Least privilege in Google Cloud means granting each identity only the permissions it needs, on the narrowest practical resource, and only for as long as necessary. Implement it with narrowly scoped IAM roles, groups, service identities, IAM Conditions, and deny policies.

Read full answer

Applies toGoogle Cloud IAM and gcloud CLI (current stable releases)

How Least Privilege Works

Google Cloud IAM combines who is requesting access, the role containing allowed permissions, and the resource on which the role is granted.

  • Resource hierarchy: Organizations contain folders, folders contain projects, and projects contain resources such as Cloud Run services, buckets, and Cloud SQL instances.

  • Inheritance: Roles granted at an organization, folder, or project level are inherited by its descendants. A broad parent-level grant can therefore expose many resources.

  • Narrow scope: Grant a role at the lowest resource where it is needed and where that role supports resource-level binding.

  • Time and attribute constraints: IAM Conditions can restrict access by time, resource attributes, request attributes, or identity attributes.

  • Explicit denial: IAM Deny policies override allow policies, providing defense in depth. They should supplement—not replace—narrowly scoped grants.

  • Separation of duties: Platform administrators, security administrators, developers, and workload identities should not share one highly privileged account.

Avoid roles/owner, roles/editor, and roles/viewer for routine access. These basic roles cover broad sets of GCP services and make privilege review and incident containment difficult.

Selecting Roles and Identities

Use predefined roles when they closely match the job requirement. Create custom roles only when predefined roles contain unnecessary permissions.

Task Prefer Appropriate scope
Deploy Cloud Run services roles/run.developer Specific service or project
Invoke Cloud Run services roles/run.invoker Specific service
Read Cloud Storage objects roles/storage.objectViewer Bucket or managed folder
Connect to Cloud SQL roles/cloudsql.client Project, narrowed with an IAM Condition on the instance name
Read BigQuery datasets roles/bigquery.dataViewer Dataset or table
Access a secret roles/secretmanager.secretAccessor Specific secret
View logs roles/logging.viewer Lowest supported scope

Identity practices are equally important:

  • Put human users in groups rather than granting individual user accounts.

  • Create a dedicated service account for each workload or environment.

  • Use Workload Identity Federation or workload identity mechanisms instead of downloadable service account keys.

  • Grant roles/iam.serviceAccountTokenCreator only to identities that must impersonate a service account.

  • Use separate deployment, runtime, and data-access service accounts.

  • Prefer IAM Conditions for temporary elevation, maintenance windows, or access limited to particular resources.

GCP, AWS, and Azure Equivalents

GCP AWS Azure
IAM role binding IAM policy containing role and principal Azure RBAC role assignment
Service account IAM role assumed through a trust policy Managed identity or service principal
IAM Condition IAM policy condition Azure role assignment condition
Resource hierarchy Organization, account, resource hierarchy Management group, subscription, resource group

The concepts are similar, but role names, condition attributes, inheritance behavior, and administrative APIs remain provider-specific.

Code Example

The following grants deployment permission only on one Cloud Run service:

gcloud config set project example-project

gcloud run services add-iam-policy-binding order-service \
  --region=us-central1 \
  --member="group:release-engineers@example.com" \
  --role="roles/run.developer"

# Allow a workload identity to invoke only this service.
gcloud run services add-iam-policy-binding order-service \
  --region=us-central1 \
  --member="serviceAccount:orders@example-project.iam.gserviceaccount.com" \
  --role="roles/run.invoker"

# Inspect recent IAM audit events.
gcloud logging read \
  'resource.type="project" AND protoPayload.serviceName="iam.googleapis.com"' \
  --project=example-project \
  --limit=20

If predefined roles are too broad, a project-level custom role can contain only the required permissions:

gcloud iam roles create orderDeployer \
  --project=example-project \
  --title="Order deployer" \
  --description="Deploy and inspect Cloud Run services" \
  --permissions=run.services.create,run.services.get,run.services.update \
  --stage=GA

Production Operations and Best Practices

  • Manage policies declaratively: Store IAM bindings in Terraform or another authoritative policy source. Detect and remove manual configuration drift.

  • Audit continuously: Enable relevant Cloud Audit Logs, export security-critical logs to a protected destination, and alert on policy changes, principal additions, and unusual privileged actions.

  • Review regularly: Re-evaluate permissions after role changes, service migrations, upgrades, and employee or contractor offboarding.

  • Test failure modes: Overly restrictive policies commonly cause HTTP 403 responses, Cloud Run invocation failures, database connection errors, or startup failures. Check audit logs and the effective inherited policy before broadening access.

  • Handle policy changes carefully: IAM updates can have a brief propagation delay. Avoid retry loops that repeatedly change policy or accidentally replace existing bindings.

  • Plan emergency access: Use controlled, monitored break-glass accounts rather than permanent broad grants. Time-bound IAM Conditions can reduce the exposure window.

  • Protect organization administration: Keep organization-level administration separate from day-to-day project access, and restrict billing and cost-management permissions so workloads cannot change billing configuration.

  • Review custom roles: Custom roles require explicit maintenance when APIs or required permissions change. Prefer predefined roles when their privilege level remains appropriate.

A strong least-privilege design is not simply the smallest possible role set; it is the smallest stable, auditable, and operationally reliable permission set that supports the workload.

10

What does Google Cloud Compute Engine provide?

Google Cloud Compute Engine is Google Cloud’s Infrastructure as a Service (IaaS) platform for running virtual machines with configurable CPU, memory, storage, networking, and operating systems. It provides the control of a traditional server while offloading facilities, hypervisors, and much of the hardware management to Google Cloud.

Read full answer

Applies toGoogle Cloud Compute Engine (current GA service)

Core Concepts

  • VM instances: Individual virtual machines that run Linux or Windows workloads. Each instance belongs to a specific Google Cloud zone, such as us-central1-a.

  • Machine types: Predefined or custom combinations of vCPU and memory. Burstable machine types can share physical cores and provide lower cost, while memory-optimized and compute-optimized types suit databases, analytics, and batch processing.

  • Storage: Instances can use Persistent Disk, Hyperdisk, or ephemeral local storage. Persistent disks outlive VM deletion when configured appropriately.

  • Images and snapshots: Boot from a Google-provided image, a custom image, or an existing disk. Snapshots provide point-in-time recovery and can be used to create images or new disks.

  • Networking: Compute Engine attaches instances to a VPC network through regional subnets. Instances can have internal and external IP addresses, firewall rules, Cloud NAT, and load-balancer backends.

  • Automation: Managed instance groups deploy identical VMs, perform health checks, replace unhealthy instances, and support rolling or canary upgrades.

Resource and Security Model

Google Cloud resources normally follow this hierarchy:

Organization → Folders → Projects

Compute Engine resources are created in a project. VPC networks are global, subnets are regional, and VM instances are zonal. Shared VPC lets one host project own shared networking while service projects contain their workloads.

Access is controlled through IAM and VPC firewall rules:

  • IAM controls who can administer or attach instances to projects.

  • Firewall rules control network traffic between instances and the internet.

  • Service accounts give applications a Google Cloud identity with narrowly scoped permissions.

  • OS Login manages SSH authentication through IAM rather than relying only on instance metadata.

  • Shielded VM provides secure boot, virtual trusted platform modules, and integrity monitoring where supported.

Avoid using the default service account for production applications. Instead, create a dedicated service account and grant only the permissions the workload needs.

High Availability and Production Operations

A single VM instance is a single point of failure. Common deployment patterns include:

  • Regional managed instance group: Spreads instances across multiple zones in one region and can automatically replace unhealthy VMs.

  • Global load balancing: Routes traffic to healthy backends in multiple regions for internet-facing HTTP(S) applications.

  • Multi-region deployment: Runs independent regional deployments for larger regional-failure coverage. A regional managed instance group does not survive the loss of its entire region.

Use Cloud Monitoring for instance metrics, uptime checks, dashboards, and alerting, and Cloud Logging for diagnostics. Alert on service-level indicators such as request errors, latency, and saturation rather than relying only on CPU utilization.

For recovery, schedule Persistent Disk snapshots and store them regionally or multi-regionally when required. Snapshots are block-level backups: quiesce databases or coordinate application-level backups when consistent recovery points matter.

Failure Modes and Trade-offs

  • Zone outage: Instances in that zone may become unavailable. Regional instance groups reduce this risk within a region.

  • Bad deployment: An image or startup script can make every newly created instance unhealthy. Use canary deployments and validate images before broad rollout.

  • Capacity or quota exhaustion: A zone may lack machine types, or the project may hit quotas for CPUs, disks, addresses, or load-balancer resources. Monitor quota usage and reserve capacity where justified.

  • Networking lockout: Incorrect firewall rules or IAM changes can block administration. Maintain controlled break-glass access.

  • Storage failure: Use appropriate disk types and replication, monitor disk health, and test restoration from snapshots.

  • Cost growth: Idle VMs, provisioned disks, snapshots, egress, and load balancers continue generating charges. Apply labels, budgets, and anomaly detection.

  • Maintenance: Eligible instances support live migration, but kernel updates, license changes, and some specialized configurations may still require restart or replacement. Managed rolling updates simplify fleet maintenance.

Comparison With AWS and Azure

Need Google Cloud AWS Azure
General-purpose VMs Compute Engine EC2 Azure Virtual Machines
Kubernetes GKE EKS AKS
Serverless containers Cloud Run ECS/Fargate or App Runner Azure Container Apps
Fully managed application platform App Engine Elastic Beanstalk App Service

Choose Compute Engine when the workload needs operating-system control, custom networking, specialized machine shapes, persistent VMs, or self-managed software. Prefer GKE for Kubernetes orchestration, Cloud Run for serverless containers, and App Engine when the platform should own most deployment and scaling decisions.

Best Practices

  • Deploy across multiple zones for production availability.

  • Use managed instance groups and rolling updates rather than managing every VM manually.

  • Apply least-privilege IAM, dedicated service accounts, and restricted firewall rules.

  • Enable OS Login or centrally managed SSH credentials.

  • Automate infrastructure with Terraform or Infrastructure Manager (Deployment Manager is deprecated) and version configurations.

  • Define monitoring, budgets, snapshot policies, and tested recovery procedures before launch.

  • Tag resources for ownership, environment, cost allocation, and automated policy enforcement.

11

What are Compute Engine machine families and custom machine types?

Machine families group Compute Engine machine types optimized for particular workloads, while custom machine types let you select an allowed combination of vCPUs and memory when no predefined configuration fits.

Read full answer

Applies toGoogle Cloud Compute Engine (current stable); Google Cloud CLI current stable; Terraform 1.x with the `google` provider 5.x+

Machine Family Concepts

A machine family represents a workload and hardware profile, such as general-purpose, compute-optimized, storage-optimized, or accelerator-optimized.

Family Typical use
e2 Cost-conscious, balanced general-purpose workloads
n2 High-performance general-purpose workloads
c3 Compute-heavy workloads requiring more vCPUs relative to memory
z3 High-throughput storage workloads
a3 GPU-accelerated workloads

A machine type is an exact configuration within a family. For example, n2-standard-4 specifies:

  • Family: n2

  • Shape: standard, representing a predefined vCPU-to-memory ratio

  • vCPUs: 4

  • Memory: The corresponding predefined amount of RAM

The machine type controls CPU and memory, but does not determine boot-disk type, disk throughput, VPC network, or external IP configuration. Those are separate instance settings.

Custom Machine Types

A custom machine type lets you specify:

  • The number of vCPUs

  • The amount of memory in MiB

For example, n2-custom-4-8192 requests 4 vCPUs and 8,192 MiB of memory. Custom configurations are not completely arbitrary: the vCPU count and memory amount must satisfy the selected family’s minimum, maximum, and allowed memory-per-vCPU limits.

Custom types are useful when a workload needs, for example, 4 vCPUs with 6 GiB of RAM when the nearest predefined type provides substantially more memory. They do not customize the underlying processor model, physical-core allocation, GPU configuration, or storage.

Choosing Between Options

Option When to use it Main trade-off
Predefined machine type Capacity and ratio already match the workload Simpler sizing and often predictable cost
Custom machine type A valid intermediate CPU-memory ratio is required More configuration work; unused RAM is still billed
Shared-core type Small, intermittent, or low-utilization services CPU sharing and throttling behavior
Accelerator-optimized type The workload requires attached GPUs Additional cost and capacity constraints

Use a custom type only after checking its effective hourly cost against the closest predefined type. Custom types are regular VM instances and are not shared-core or burstable machine types.

Code Example

Create and inspect a custom N2 instance:

# Create an N2 custom instance with 4 vCPUs and 8 GiB of RAM.
gcloud compute instances create gcp-custom-vm \
  --project="example-project-id" \
  --zone="us-central1-a" \
  --machine-type="n2-custom-4-8192" \
  --image-family="ubuntu-2404-lts-amd64" \
  --image-project="ubuntu-os-cloud" \
  --boot-disk-size="20GB" \
  --boot-disk-type="pd-balanced"

# Confirm the configured machine type.
gcloud compute instances describe gcp-custom-vm \
  --project="example-project-id" \
  --zone="us-central1-a" \
  --format="value(machineType)"

The equivalent Terraform configuration is:

resource "google_compute_instance" "custom_vm" {
  project      = "example-project-id"
  name         = "gcp-custom-vm"
  zone         = "us-central1-a"
  machine_type = "n2-custom-4-8192" # 4 vCPUs and 8192 MiB RAM

  boot_disk {
    initialize_params {
      image = "ubuntu-os-cloud/ubuntu-2404-lts-amd64"
      size  = 20
      type  = "pd-balanced"
    }
  }

  network_interface {
    network = "default"
  }
}

Production Behavior

Capacity is zonal: a custom configuration can be available in one zone but lack capacity in another. Before deployment, verify both CPU and memory quotas and account for possible zone-level capacity failures.

Use Cloud Monitoring to track sustained CPU utilization, memory pressure, disk throughput and latency, network throughput, and instance-level errors. Autoscaling should use measured application demand rather than vCPU count alone.

Changing an instance’s machine type requires stopping it and running gcloud compute instances set-machine-type; for repeatable production deployments, change the instance template and roll it out through a managed instance group instead. Back up mutable disks with snapshots or image-based backup procedures rather than relying on the VM lifecycle.

Best Practices

  • Benchmark with representative CPU, memory, and I/O workloads.

  • Compare custom sizing with predefined alternatives using actual billing data.

  • Reserve memory for workload growth instead of choosing the smallest possible custom shape.

  • Check regional quota and capacity before committing to a topology.

  • Use separate machine types for separate performance domains rather than relying on overcommitted shared instances.

  • Recreate instances through automation so machine-type upgrades are repeatable and auditable.

12

What are Compute Engine Spot VMs and preemption?

Preemptible VMs are discounted Compute Engine instances that Google Cloud may terminate to reclaim capacity. Spot VMs are the current Compute Engine name and provisioning model for this behavior, providing low-cost but interruptible compute capacity.

Read full answer

Applies toGoogle Cloud Compute Engine Spot VMs; current stable gcloud CLI and Terraform Google provider

How Preemption Works

A Spot VM runs on temporarily available capacity rather than capacity reserved specifically for the instance. Google Cloud may preempt it when that capacity is needed elsewhere.

The lifecycle is:

  1. The VM starts normally using standard machine types, disks, networking, and IAM identities.

  2. Google Cloud sends a preemption notice, normally allowing approximately 30 seconds before termination.

  3. The instance is terminated, and compute billing stops.

  4. Persistent disks are not deleted by the termination itself, but local SSD data is lost.

Treat the notice as best-effort. Applications must not depend on receiving every notification or completing cleanup successfully. “Preemptible” describes the termination behavior; “Spot” is the current product terminology and SPOT provisioning model.

Spot VMs Compared with Standard VMs

Property Spot VM Standard VM
Pricing Deeply discounted On-demand pricing
Capacity Uses spare capacity that may be reclaimed Not preempted for capacity reclamation
Interruption May receive a 30-second termination notice No capacity-driven preemption
SLA No Spot-specific SLA Eligible for the applicable Compute Engine SLA
Best suited for Batch jobs, CI workers, testing, rendering Critical services and predictable workloads

The corresponding services in other major clouds are AWS Spot Instances and Azure Spot VMs. Their economic model is similar: low prices, interruptibility, and no guarantee that capacity can be obtained.

Cloud Service Approximate interruption notice
Google Cloud Compute Engine Spot VMs 30 seconds
AWS EC2 Spot Instances About 2 minutes
Azure Azure Spot VMs At least 30 seconds (Scheduled Events)

Production Behavior

  • Use for interruptible work: Stateless APIs, asynchronous queues, CI runners, data processing, development environments, and rendering workloads are typical candidates.

  • Handle failure explicitly: Make jobs idempotent, checkpoint progress, and retry them on another instance after preemption.

  • Externalize state: Store durable state in Cloud SQL, Firestore, Bigtable, Cloud Storage, or another managed service instead of relying on the VM or local SSD.

  • Use instance groups for availability: A Managed Instance Group can replace preempted Spot VMs and combine Spot and Standard provisioning for graceful degradation.

  • Expect capacity failures: A Spot request can remain unscheduled even when standard capacity and project quota are available.

  • Monitor interruptions: Alert on preemption and termination events, Spot capacity failures, queue latency, and unsuccessful job retries.

  • Protect data: Configure disk retention and backups correctly, and use customer-managed encryption keys when required by policy.

Code Example

Create a standalone Spot VM with the current gcloud provisioning model:

gcloud compute instances create spot-worker \
  --project=example-project \
  --zone=us-central1-a \
  --machine-type=e2-medium \
  --provisioning-model=SPOT \
  --image-family=debian-12 \
  --image-project=debian-cloud

The equivalent Terraform resource uses the SPOT provisioning model:

resource "google_compute_instance" "spot_worker" {
  project      = "example-project"
  name         = "spot-worker"
  zone         = "us-central1-a"
  machine_type = "e2-medium"

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-12"
    }
  }

  network_interface {
    network = "default"
  }

  scheduling {
    provisioning_model = "SPOT"  # Allow capacity-driven preemption
    automatic_restart  = false
    on_host_maintenance = "TERMINATE"
  }
}

Best Practices and Security

  • Do not use Spot VMs as the only copy of important data. Configure persistent-disk retention, automated backups, and tested restoration procedures.

  • Design for abrupt termination. Graceful shutdown hooks are useful but cannot guarantee that cleanup completes.

  • Apply least privilege. Attach a dedicated service account with only the permissions the workload requires, and avoid broad access to production secrets.

  • Patch and scan images: Spot status does not reduce the need for vulnerability management, trusted images, and timely Google Cloud security updates.

  • Control costs: Set budgets and alerts, and monitor effective compute cost per completed job rather than only the hourly instance price.

  • Maintain fallback capacity: Use mixed or Standard-based instance groups when workload continuity is more important than minimizing compute cost.

13

What are managed instance groups in Google Cloud?

Managed instance groups (MIGs) are Compute Engine features that manage a collection of interchangeable virtual machines from an instance template. Google Cloud automatically creates, deletes, scales, health-checks, and replaces those instances according to the group’s configuration.

Read full answer

Applies toGoogle Cloud Compute Engine (current stable gcloud CLI)

Core Concepts

A MIG combines several Compute Engine resources:

  • Instance template: Defines the VM configuration, including machine type, boot image, network, service account, metadata, labels, firewall tags, and startup script.

  • Managed instance group: Controls the number and lifecycle of instances created from the template.

  • Autoscaler: Optionally changes the target size between configured minimum and maximum limits.

  • Health check: Determines whether instances are healthy and, when autohealing is enabled, causes unhealthy instances to be replaced.

  • Rolling update: Gradually replaces instances after a new instance template is applied.

The group manages instances individually, but they should behave as interchangeable members. A MIG does not itself distribute traffic; attach it as the backend of a load balancer or use it with a service that targets Compute Engine instances.

Zonal and Regional MIGs

Type Scope Availability characteristics Typical use
Zonal MIG One zone Can lose all instances in that zone Cost-sensitive or single-zone workloads
Regional MIG Multiple zones in one region Spreads instances across zones in the region Production services requiring zonal resilience

A regional MIG still operates in one Google Cloud region. Multi-region resilience requires separate regional MIGs, usually placed behind a global external Application Load Balancer.

Scaling, Health, and Deployments

  • Fixed-size groups: Maintain a specified number of instances and replace failed instances when the group detects problems.

  • Autoscaled groups: Adjust capacity using signals such as CPU utilization, Cloud Monitoring metrics, or a schedule.

  • Autohealing: A failed or unhealthy instance is deleted and recreated from the template. A MIG health check alone marks instances unhealthy; it does not necessarily replace them unless autohealing is configured.

  • Rolling updates: New instances are launched using an updated template before old instances are removed, subject to capacity and rollout policies.

Changing an instance template does not modify existing instances by itself. The updated template must first be applied to the managed instance group.

Service Comparison and Selection

Google Cloud AWS Azure Important distinction
Managed instance group Auto Scaling group Virtual Machine Scale Set GKE node pools use MIGs internally
GKE node pool EKS managed node group AKS node pool Choose when Kubernetes APIs and workload scheduling are required
Cloud Run ECS/Fargate Azure Container Apps Choose for serverless containers without VM management
Cloud Functions Lambda Azure Functions Choose for event-driven functions

Use a Compute Engine MIG when the team needs direct control over VMs, guest operating systems, networking, disks, or specialized instance configuration.

gcloud CLI Example

The following creates a service account, instance template, zonal MIG, autoscaler, and health check. The referenced subnet must already exist.

# Create a dedicated identity for the VM instances
gcloud iam service-accounts create example-mig

# Define the desired instance configuration
gcloud compute instance-templates create web-v1 \
  --machine-type=e2-standard-2 \
  --image-family=ubuntu-2404-lts-amd64 \
  --image-project=ubuntu-os-cloud \
  --region=us-central1 \
  --subnet=example-subnet \
  --tags=http-server,https-server \
  --service-account=example-mig@example-project.iam.gserviceaccount.com

# Create a managed group with two initial instances
gcloud compute instance-groups managed create web-mig \
  --zone=us-central1-a \
  --template=web-v1 \
  --size=2 \
  --base-instance-name=web

# Allow automatic scaling between two and ten instances
gcloud compute instance-groups managed set-autoscaling web-mig \
  --zone=us-central1-a \
  --min-num-replicas=2 \
  --max-num-replicas=10 \
  --target-cpu-utilization=0.6

# Configure an application-level health check and autohealing
gcloud compute health-checks create http web-health \
  --port=80 \
  --request-path=/healthz

gcloud compute instance-groups managed update web-mig \
  --zone=us-central1-a \
  --health-check=web-health \
  --initial-delay=300

After creating an updated template such as web-v2, apply it with a rolling update:

gcloud compute instance-groups managed rolling-action start-update web-mig \
  --zone=us-central1-a \
  --version=template=web-v2 \
  --max-surge=1 \
  --max-unavailable=0

Production Best Practices

  • Use regional MIGs for important services, and separate regional deployments for protection from a regional outage.

  • Keep instances stateless: replacement destroys local disks and local application state. Use Cloud SQL, Spanner, Firestore, Bigtable, or Filestore where durable state is required.

  • Make startup scripts idempotent: a replacement instance may rerun them and must recover cleanly.

  • Monitor capacity and health: alert on compute.googleapis.com/instance_group/managed/instance_count, autoscaler behavior, backend health, CPU, memory, disk pressure, and load-balancer latency.

  • Account for quota and scale-out delay: autoscaling cannot immediately bypass regional Compute Engine quotas or provisioning capacity limits.

  • Control rollout risk: use canary groups or weighted backend services when a change requires gradual validation.

  • Apply least privilege: use a dedicated service account, avoid instance-wide SSH keys and embedded credentials, retrieve secrets from Secret Manager, and restrict access with IAM and VPC firewall rules.

  • Plan upgrades deliberately: publish a new image or template, validate it, and use a rolling update rather than relying on in-place guest patching.

14

How does Compute Engine autoscaling work?

Managed instance group (MIG) autoscaling adjusts the number of Compute Engine instances in a managed instance group based on workload metrics. A Managed Autoscaler monitors Cloud Monitoring data, then creates or removes instances using the group’s instance template while keeping capacity within configured minimum and maximum limits.

Read full answer

Applies toGoogle Cloud Compute Engine (current stable)

How It Works

  1. A MIG manages interchangeable instances created from an instance template.

  2. An autoscaling policy defines the target metric, such as average CPU utilization or a custom Cloud Monitoring metric.

  3. The autoscaler periodically reads aggregated metrics for the group.

  4. If utilization exceeds the target, it increases the group’s desired size, subject to stabilization and cooldown rules.

  5. If utilization remains below the scale-in threshold, it decreases the desired size when the stabilization policy permits.

  6. New instances use the instance template, while removed instances are terminated and can be replaced if the group needs to return to its desired size.

The autoscaler manages MIG capacity rather than scaling standalone virtual machines. A regional MIG distributes capacity across zones and targets a configured distribution; a zonal MIG operates within one zone.

Scaling Policies and Cloud Equivalents

Capability Compute Engine AWS Azure
Managed workload group Regional or zonal MIG Auto Scaling group Virtual Machine Scale Set
Capacity definition Instance template Launch template or configuration VM image and configuration
Scaling signal CPU, Cloud Monitoring metrics, and other supported signals Target tracking, schedules, and predictive scaling Autoscale rules and schedules
Multi-zone placement Regional MIG Auto Scaling group spanning Availability Zones VMSS with zone spreading
Initial capacity target-size Desired capacity Default capacity

Common policies include:

  • CPU utilization target: Scales according to average CPU usage across the group. A lower target such as 50% reserves more headroom and normally triggers scale-out sooner than a 70% target; a higher target tolerates more utilization per instance but may react later.

  • Custom metric: Scales on a scalar application metric exposed to Cloud Monitoring, such as requests per second, queue depth, or Pub/Sub backlog.

  • Minimum size: Maintains baseline capacity for expected traffic and reduces cold-start latency.

  • Maximum size: Limits potential compute cost and runaway scaling, although quotas and regional capacity can impose additional limits.

  • Cooldown period: Prevents another scale-up for a defined period after a scale-out. The default is 60 seconds.

Production Behavior and Failure Modes

  • Cold starts matter: Autoscaling creates instances, but they become useful only after the guest OS, application startup, and readiness checks complete. Capacity may exist before it can serve traffic.

  • Scale-in can remove active instances: Keep sessions, jobs, and durable state outside the VM unless the application handles instance termination safely.

  • Health checks are separate from scaling: A MIG can replace unhealthy instances according to its health-checking configuration, while the autoscaler changes capacity according to load.

  • Thrashing is possible: An unrealistic metric target or long application startup time can cause repeated scale-out and scale-in cycles.

  • Regional quotas can block scale-out: Increasing max-size does not guarantee that Compute Engine can obtain the required CPU, memory, or IP addresses.

  • Rolling updates and scaling are separate: Changing an instance template affects future instances but does not automatically modify existing MIG instances. To adopt the change, update the MIG instances through a rolling-update or instance-update operation, validate application compatibility, and configure connection draining.

  • Monitor continuously: Alert on sustained maximum capacity, repeated instance recreation, failed health checks, excessive scale-out activity, and slow application readiness. Also track latency and error rates, not only CPU.

Security, Cost, and Best Practices

  • Put the application behind a regional load balancer that uses the MIG as a backend and checks application readiness.

  • Set min-size according to steady-state traffic and latency requirements. Every idle instance continues consuming compute and storage charges.

  • Use a dedicated service account with least-privileged IAM roles for the application. Do not place secret values in templates, startup scripts, or instance metadata.

  • Ensure Google-managed service agents retain the service-specific permissions required by the autoscaling and instance-group workflows.

  • Use custom metrics when CPU does not represent demand. Prefer a suitable scalar metric such as request rate or Pub/Sub backlog rather than relying only on CPU.

  • Test scale-out under realistic startup conditions and verify autoscaling behavior against quotas, regional capacity, and downstream dependency limits.

  • Prefer an autoscaler for variable production workloads, but retain capacity planning for predictable peaks, maintenance windows, and large launches.

15

What is a Google Compute Engine instance template?

A Compute Engine instance template is a reusable, regional definition of the configuration used to create Compute Engine virtual machines. It specifies settings such as the machine type, boot disk, network, identity, metadata, and scheduling options, but it does not run a VM itself.

Read full answer

Applies toGoogle Compute Engine API v1 and Google Cloud CLI (current stable)

How It Works

An instance template is a named regional resource with a URL such as projects/PROJECT_ID/regions/REGION/instanceTemplates/TEMPLATE_NAME. A single-instance creation can reference it through sourceInstanceTemplate, while a Managed Instance Group (MIG) version can reference its regional template URL. Compute Engine resolves that reference during provisioning and creates the instance using the stored configuration.

  • Reusable configuration: Multiple instances or MIGs can use the same template.

  • Regional scope: The template belongs to a region, while the instance or MIG selects an eligible zone within that region.

  • Persistent resource: The template remains available until explicitly deleted.

  • No automatic deployment: Creating or updating a template does not modify existing instances; instances adopt it only when newly created or replaced.

  • No direct runtime cost: The template is control-plane configuration, although the instances and other resources it provisions are billable.

Versioning templates by name, such as web-v1 and web-v2, is a common operational convention rather than a separate template-versioning API.

What a Template Defines

Area Common settings
Compute Machine type, CPU platform, scheduling, and Shielded VM configuration
Storage Boot disk, source image or snapshot, disk type, disk size, and additional disks
Networking Network, subnetwork, additional interfaces, and network tags
Identity Service account and OAuth scopes
Software Image selection, metadata, and startup scripts
Automation Labels, resource-manager tags, and settings shared by provisioned instances

An instance template is more than an OS image. An image supplies the boot filesystem, while the template supplies the complete VM definition. Pinning a specific image or a controlled custom image gives stronger reproducibility; using an image family can make provisioning depend on the current image selected for that family.

Service Comparisons

GCP concept Closest equivalent Main distinction
Compute Engine instance template AWS EC2 Launch Template Both define reusable VM configuration consumed by instance groups or auto-scaling groups
Compute Engine instance template Azure VM scale set model Azure normally incorporates image and VM configuration into the scale-set model rather than a standalone template resource
Cloud Run revision AWS Lambda function version, Azure Container Apps revision Intended for managed container serving rather than general-purpose VMs
GKE node pool EKS node group, AKS node pool Configures Kubernetes workers rather than standalone Compute Engine instances

Production Behavior and Failure Modes

  • Existing instances remain unchanged: A MIG template change does not itself perform a rollout; use a controlled rolling replacement or recreate instances.

  • Bad templates can repeat failures: Autohealing may repeatedly recreate unhealthy instances from the same defective definition.

  • Boot success is not readiness: A startup script can complete while its application is failing to start. Monitor application-level readiness separately.

  • Health checks need accurate targets: An incorrect port or path can cause unnecessary instance replacement and an incident.

  • Capacity and quotas can block rollouts: Replacement surges can exceed regional or zonal capacity and Compute Engine quotas.

  • Images and policies can expire: Deprecated images, organization policies, or policy denials can prevent new instances from being created.

  • Regional MIGs need spare capacity: Leaving headroom in multiple zones supports maintenance events and zone-level disruption.

  • Durable data belongs outside instances: Use persistent disks with scheduled snapshots for disk-backed workloads, or an appropriate managed database for critical state.

Production teams generally maintain canary and stable templates, observe rollout health, and retain the previous template until the replacement is proven. Templates should be managed through reviewed infrastructure-as-code, using Terraform’s google_compute_instance_template resource or Config Connector desired-state resources reconciled by Config Controller.

Security and Operational Practices

  • Use a dedicated service account with least-privilege IAM roles, and avoid downloadable service-account keys.

  • Limit OAuth scopes to those the workload actually requires.

  • Use OS Login and Identity-Aware Proxy TCP forwarding for administrative access rather than broad internet-facing SSH rules.

  • Never embed credentials in startup scripts, metadata, or templates; retrieve secrets through an approved secret-management path.

  • Treat startup scripts and custom images as privileged code because they run with administrative privileges and can alter the host.

  • Install the Ops Agent through the image or configuration pipeline so metrics and logs remain consistent across instances.

  • Configure the MIG health check against a meaningful readiness signal; keep application liveness detection separate so transient failures do not automatically trigger replacement.

  • Apply organization policies, retention controls, and change review to templates and the images they reference.

16

What does Google Kubernetes Engine (GKE) provide?

Google Kubernetes Engine (GKE) is Google Cloud’s managed Kubernetes service. Google operates the Kubernetes control plane, integrates clusters with Google Cloud IAM and networking, and provides options for automatically managing or self-managing worker nodes so teams can run containerized production workloads.

Read full answer

Applies toGoogle Kubernetes Engine (GKE), all currently supported versions

How It Works

GKE runs standard Kubernetes components, including the API server, scheduler, controller manager, and etcd. A regional cluster distributes the control plane across multiple Google Cloud zones, while a zonal cluster places it in one zone and is less expensive but has a smaller failure domain.

Worker nodes run as Compute Engine instances. With VPC-native alias IP ranges, Pods receive addresses from dedicated VPC subnets without relying on traditional Google Cloud instance routes. Kubernetes Services can expose applications through Google Cloud load balancers, while internal traffic remains inside the VPC.

Typical operational commands include:

  • gcloud container clusters describe CLUSTER_NAME --region REGION

  • gcloud container clusters get-credentials CLUSTER_NAME --region REGION

  • gcloud container node-pools list --cluster CLUSTER_NAME --region REGION

Deployment Options

Mode Node management Billing Best suited for
GKE Autopilot Google manages node pools, versions, scaling, and node configuration Primarily based on requested Pod resources, with additional charges for some resources Production workloads where teams want less cluster administration
GKE Standard Customers configure node pools, machine types, and upgrade or network settings; GKE manages the node image lifecycle Cluster fees plus Compute Engine, storage, and network costs Environments requiring custom node configurations, specialized hardware, or node-level control
GKE On-Prem Runs in an on-premises environment and is managed through Google Cloud tooling Separate GKE On-Prem pricing model Regulated, edge, or disconnected environments requiring Kubernetes with Google Cloud integration

Autopilot still permits standard Kubernetes manifests, Deployments, Services, and custom resources. It constrains node access and configuration because Google manages the node layer.

Core Platform Capabilities

  • Horizontal and vertical scaling: HPA adjusts replicas from observed demand; VPA recommends or applies resource-request changes.

  • Cluster autoscaling: Adds or removes nodes when Pods cannot be scheduled, subject to quota, capacity, and autoscaling-policy constraints.

  • Release channels: Rapid, Regular, Stable, and Extended channels provide different upgrade cadences. Extended supports longer-lived release lines.

  • Multi-cluster management: Fleet management organizes clusters and supports standardized configuration and policy enforcement.

  • Ingress and routing: GKE Ingress and the GKE Gateway controller integrate with Google Cloud load balancing.

  • Storage integration: Persistent disks and Cloud Storage integrations support stateful workloads, but application teams remain responsible for data protection.

  • Network policy: Kubernetes NetworkPolicy and Google Cloud firewall rules provide separate layers of network control.

Production Operations

GKE upgrades the control plane first and then the node pools, because nodes may not run a newer version than the control plane. In a zonal cluster the single control-plane replica is briefly unavailable during its upgrade; a regional cluster upgrades its replicas one at a time, so the Kubernetes API stays available.

In GKE Standard, teams should use maintenance windows, node-upgrade settings, and Pod disruption budgets rather than relying only on manual node replacement. Autopilot handles more of the lifecycle but does not eliminate the need to test application compatibility.

Production clusters should include:

  • Resource requests and limits: Required for scheduling, bin-packing, and capacity planning.

  • Health checks: Configure startup, readiness, and liveness probes so traffic reaches only healthy Pods.

  • Pod disruption budgets: Protect critical replicas during voluntary disruptions such as node maintenance.

  • Monitoring and logging: Use Cloud Monitoring and Cloud Logging, with alerts for saturation, crashes, rollout failures, and unavailable replicas.

  • Backup strategy: Kubernetes does not automatically back up application data. Use database-native backups, volume snapshots, or application-level replication according to recovery objectives.

  • Capacity planning: Monitor pending Pods, node utilization, quotas, and regional failure-domain distribution.

Security and Compliance

GKE combines Kubernetes RBAC with Google Cloud IAM. Standard Workload Identity for GKE maps a Kubernetes service account to a narrowly scoped IAM service account that the Pod can impersonate without downloadable service-account keys. Workload Identity Federation for GKE supports external identities through federation and can optionally use service-account impersonation.

Additional controls include:

  • Private clusters: Restrict control-plane access to authorized networks and a private endpoint.

  • Shielded GKE Nodes: Apply hardened node protections such as secure boot and integrity monitoring.

  • Least privilege: Grant only required IAM and Kubernetes permissions; avoid broad roles such as Owner or Editor.

  • Audit logging: Enable relevant Kubernetes and Google Cloud audit logs and monitor changes to RBAC, workloads, secrets, and network policy.

  • Secret management: Use Secret Manager instead of placing long-lived credentials in images or manifests.

  • Policy enforcement: GKE Policy Controller can detect insecure configurations before deployment.

GKE, EKS, and AKS

Platform Cloud Main advantage Main trade-off
GKE Google Cloud Integrated networking, IAM, load balancing, and operations Google-specific integrations can increase portability concerns
EKS AWS Strong AWS service and ecosystem integration Teams commonly manage more node and add-on infrastructure themselves
AKS Azure Strong Microsoft Entra ID and Azure integration Azure-specific identity and networking features can require platform-specific design

Choose GKE when workloads already use Google Cloud services such as BigQuery, Cloud SQL, Pub/Sub, or Cloud Storage, or when its managed node operations and networking reduce operational overhead. For a multi-cloud platform, standardize on Kubernetes manifests and isolate provider-specific APIs to preserve portability.

17

How do GKE Standard and GKE Autopilot differ?

GKE Standard gives teams direct control over Compute Engine node pools, machine types, and capacity, while GKE Autopilot manages the worker infrastructure using pod resource requests. Both run standard Kubernetes workloads on a Google-managed control plane; the primary difference is who manages the worker nodes and how capacity is provisioned and billed.

Read full answer

Applies toGoogle Kubernetes Engine (GKE) Standard and Autopilot, current supported GKE releases

Core Difference

  • GKE Standard: Teams create and manage node pools backed by Compute Engine instances. Node-pool size, autoscaling, machine types, labels, taints, disk configuration, and upgrade behavior are directly configurable.

  • GKE Autopilot: Google manages the complete worker-node lifecycle, including provisioning, configuration, bin-packing, and upgrades. Teams deploy Kubernetes resources and specify CPU, memory, and ephemeral-storage requests.

  • Shared behavior: Both expose Kubernetes APIs and support workloads such as Deployment, StatefulSet, and DaemonSet, with a Google-managed control plane.

  • Important distinction: Autopilot is not serverless Kubernetes. Applications still use Kubernetes manifests, APIs, RBAC, networking, and workload controllers.

Feature Comparison

Area GKE Standard GKE Autopilot
Worker nodes Customer-managed Compute Engine instances Google-managed and hidden from direct administration
Capacity planning Node count, node-pool shape, and machine type Pod resource requests and an optional compute class
Autoscaling Cluster autoscaler and optional node auto-provisioning Automatic capacity based on scheduled workloads
Billing Provisioned node resources, including unused capacity Requested CPU, memory, and ephemeral storage, subject to per-pod minimums
Node customization Broad control over images, labels, taints, disks, and hardware Opinionated configuration with workload restrictions
Upgrades Control plane managed by Google; node upgrade timing configurable Control plane and worker lifecycle handled by Google
Operational effort Higher Lower
Best fit Precise infrastructure or capacity control Conventional services seeking less node-management work

How Autopilot Works

Autopilot evaluates scheduled pods and groups them onto Google-provisioned workers. Capacity is added and removed as pod resource requests change.

  • Resource requests drive placement and cost: If CPU, memory, or ephemeral-storage requests are omitted, Autopilot applies defaults of 0.1 vCPU, 128 MiB memory, and 1.5 GiB ephemeral storage. Explicit, realistic requests are still important for scheduling and cost control.

  • Compute classes select capacity profiles: Workloads can use platform-defined general-purpose, balanced, memory-optimized, or accelerator-optimized classes, subject to regional and quota availability.

  • Node-level access is restricted: Privileged containers, host networking, hostPort, and host-path mounts are not supported. GKE sandbox can provide stronger workload isolation, but it does not expose unsupported node interfaces.

  • Workloads must remain node-agnostic: Applications should not depend on node names, manually installed host agents, arbitrary custom node labels, or a particular machine type.

Production Trade-offs

  • Standard advantages:

  • Precise control over machine types, local disks, custom node images, GPUs or TPUs, and node agents.

  • Dedicated node pools with taints, affinity, and labels for specialized workloads.

  • Reserved or preallocated capacity when predictable steady-state capacity is required.

  • Standard failure modes:

  • Node exhaustion, uneven rollout distribution, and autoscaler latency.

  • Configuration drift and stale images across node pools.

  • Disk pressure from logs, images, or poorly managed ephemeral storage.

  • Higher cost when allocated nodes remain underused.

  • Autopilot advantages:

  • Google handles node provisioning, maintenance, and replacement.

  • Lower operational effort and less directly provisioned idle capacity.

  • Faster onboarding for teams already familiar with Kubernetes.

  • Autopilot failure modes:

  • Overstated requests increase cost, while understated requests cause CPU throttling or OOM kills, because Autopilot sets limits equal to requests by default.

  • Sudden traffic increases can still encounter scale-out latency.

  • Node-dependent workloads may require redesign.

  • Requested resources and per-pod minimums remain billable even when utilization is low.

Selection and Operational Guidance

  • Choose Standard when workloads require specific machine shapes, local storage, custom node configuration, specialized agents, or tight control over capacity and scheduling.

  • Choose Autopilot for conventional services when reducing node-management overhead matters more than controlling exact node configuration. Autopilot supports selected accelerators, but its hardware and placement choices are more constrained than Standard.

  • Before migration, audit privileged containers, host networking, host paths, custom node labels, machine-type assumptions, storage drivers, and DaemonSets. DaemonSets are supported, but their resource requests and placement must fit the managed-node model.

  • Monitor CPU throttling, OOM kills, pending pods, scheduling latency, utilization, and requested-versus-used resources. Store manifests in version control and back up application data; Google manages the platform, not application-level recovery.

18

What is Google Cloud Run and when should you use it?

Cloud Run is Google Cloud’s serverless container platform. It runs container images behind managed HTTPS endpoints, automatically scales instances, and can scale to zero when idle. Google operates routing, certificates, and compute; teams own the application, image, configuration, and data dependencies.

Read full answer

Applies toGoogle Cloud Run (current stable service; no numbered releases) and Cloud Run functions (2nd gen)

Core Capabilities

  • Cloud Run services: Managed deployments of stateless containers that serve HTTP requests through managed ingress or receive events through Eventarc.

  • Cloud Run functions (2nd gen): Event-driven handlers built on Cloud Run and Eventarc, invoked by Pub/Sub, Cloud Storage, and other supported event sources.

  • Cloud Run jobs: Batch or administrative workloads that run to completion, with parallel tasks where appropriate; unlike services, they do not serve requests.

  • Revisions: Each deployment produces an immutable revision. Traffic can be split between revisions for canary testing or shifted back for rollback.

  • Container contract: The container must listen on the port supplied through the PORT environment variable and bind to 0.0.0.0. Logs written to stdout and stderr reach Cloud Logging.

When to Choose Cloud Run

Requirement Recommended option
Stateless HTTP API or microservice Cloud Run
Pub/Sub, Eventarc, or webhook processing Cloud Run functions (2nd gen) for new projects
Containerized background task Cloud Run Jobs
Predictable low latency with warm capacity GKE or Compute Engine
Persistent disks or local state Compute Engine or GKE with persistent storage
Privileged workloads or extensive cluster control GKE
Simple source-based PaaS deployment App Engine

Cloud Run is closely comparable to AWS App Runner and Azure Container Apps. AWS Lambda and Azure Functions are function-first alternatives, while GKE provides more operating-system and networking control at the cost of cluster administration.

Production Behavior

  • Autoscaling: Instances scale from zero to meet demand. Maximum-instance limits protect downstream systems and constrain resource consumption.

  • Cold starts: Image startup, runtime initialization, and application initialization add latency. A minimum of one instance keeps capacity warm but incurs idle cost.

  • Concurrency: Multiple requests can share an instance. Higher concurrency improves utilization but increases memory pressure and contention.

  • Resource allocation: By default, CPU is allocated during request processing, while memory remains allocated for the instance’s lifetime. Cloud Run Jobs allocate CPU throughout execution.

  • Timeouts: Service request timeouts can be configured up to 60 minutes. Long-running requests consume capacity and can trigger downstream timeouts.

  • Observability: Cloud Logging captures standard output and errors; Cloud Monitoring exposes service metrics for rate, errors, latency, and instance count; Cloud Trace and Error Reporting assist diagnosis.

Security and Failure Modes

  • IAM: Grant roles/run.invoker only to identities, groups, or service accounts that require invocation rather than exposing the service publicly.

  • Ingress: Use internal-only or controlled ingress when the API should not be directly reachable from the internet.

  • Service identity: Run each service with a dedicated, least-privilege service account rather than the broad default compute identity.

  • Secrets: Store credentials in Secret Manager and inject them at runtime; never bake them into container images.

  • Network controls: Use Direct VPC egress when the service requires controlled access to internal resources.

  • Supply chain: Scan images in CI and deploy by digest, such as image@sha256:..., because version tags can be overwritten.

  • Failure handling: Design handlers to tolerate cold starts, memory exhaustion, excessive concurrency, dependency failures, and unhandled termination signals. Make side effects idempotent because clients and event delivery may repeat work.

  • State and recovery: Container filesystems are ephemeral. Store durable state in Cloud SQL, Firestore, Spanner, Bigtable, or Cloud Storage and assume instances can disappear.

Deployment and Operations

A private deployment can use gcloud run deploy api --image=us-central1-docker.pkg.dev/example-project/apps/api@{{IMAGE_DIGEST}} --region=us-central1 --service-account=api@example-project.iam.gserviceaccount.com --no-allow-unauthenticated --ingress=internal-and-cloud-load-balancing --min-instances=1 --concurrency=80.

After testing a revision in CI, shift production traffic with gcloud run services update-traffic api --to-revisions {{REVISION_NAME}}=100 --region=us-central1.

For slow initialization, enable startup CPU boost where appropriate and design the application to become ready reliably. Handle SIGTERM gracefully, alert on user-visible symptoms such as p95 latency and error rate, and rehearse revision rollback before an incident requires it.

Best Practices and Cost Controls

  • Set explicit CPU, memory, concurrency, timeout, and maximum-instance limits.

  • Keep images small, use multi-stage builds, remove unnecessary components, and scan dependencies.

  • Use separate services and service accounts for separate trust boundaries.

  • Make handlers idempotent and bound retries with exponential backoff and jitter.

  • Use Cloud Run Jobs for batch work instead of keeping a service alive solely to process it.

  • Monitor costs with labels and budgets; budgets alert on spend but do not cap it. Minimum instances, sustained request processing, memory, and outbound networking all affect charges.

  • Avoid Cloud Run when the workload requires local disks, privileged host access, guaranteed ultra-low latency without warm capacity, or extensive cluster-level control.

19

How does Cloud Run auto-scale, and what causes cold starts?

Cloud Run uses request-based autoscaling to add or remove container instances as traffic changes. A cold start occurs when Cloud Run must create and initialize a new container instance, adding latency to the first request.

Read full answer

Applies toGoogle Cloud Run and Google Cloud CLI (current stable versions)

How Cloud Run Scales

  • Request routing: Cloud Run sends incoming requests to available instances of the active revision.

  • Concurrency: Each instance can handle multiple simultaneous requests. The default concurrency is 80, with a configurable maximum of 1000.

  • Scale-out: When request concurrency exceeds the capacity of the current warm pool, Cloud Run adds instances. Requests can briefly queue while capacity is provisioned, and scale-out stops at the service’s maximum instance limit.

  • Scale-in: When demand falls, Cloud Run removes unused instances.

  • Scale-to-zero: By default, a service can scale to zero instances after becoming idle. The next request triggers a cold start.

  • Minimum instances: Setting min-instances to a value greater than zero keeps that many instances warm, reducing cold starts but creating idle cost.

  • Revision independence: During a gradual rollout, each revision can receive different traffic and may require its own warm capacity.

The autoscaler is primarily driven by incoming request concurrency and traffic rather than by a traditional target-CPU metric.

What Causes a Cold Start

A cold start can include several stages:

  1. Provisioning: Cloud Run creates a new execution instance.

  2. Image retrieval: The container image is pulled, potentially from Artifact Registry or another registry.

  3. Container initialization: The container runtime starts the process and runs its entry point.

  4. Application initialization: The application loads configuration, establishes database connections, initializes SDK clients, and warms caches.

  5. Readiness: Once the container is ready, Cloud Run routes requests to it.

A large container image, slow registry access, heavy framework initialization, or unavailable downstream services can make initialization significantly slower.

Cold starts can occur both:

  • When a service scales from zero after inactivity.

  • When traffic causes the autoscaler to add instances beyond the already-warm pool.

min-instances=1 prevents scale-to-zero cold starts for that revision, but it does not prevent cold starts on newly added instances during a sudden burst.

Key Trade-offs

Setting Benefit Risk
Low concurrency Lower per-instance memory and latency risk More instances, higher cost, more scale-out activity
High concurrency Fewer instances for I/O-heavy workloads Longer requests, CPU contention, or out-of-memory failures
min-instances=0 Lowest idle cost Cold-start latency
min-instances > 0 Predictable latency and faster startup Continuous idle charges
Low max-instances Strong cost control Capacity exhaustion and possible HTTP 429 responses
High max-instances Better burst absorption Potentially very high cost if traffic or retries are uncontrolled

Cloud Run uses request-based CPU allocation by default, meaning CPU is generally allocated only while processing requests. Applications that perform continuous background work should evaluate instance-based CPU allocation instead.

Comparison with AWS and Azure

Platform Primary model Scale-to-zero Typical use case
Google Cloud Run Managed containers Supported Stateless APIs, web services, event processing
AWS Lambda Managed functions Supported Short event-driven functions
Azure Functions Managed functions Supported Event-driven and scheduled functions
AWS ECS/Fargate Containers Not natively request-based; desired count can be zero Long-running container workloads
Azure Container Apps Containers and revisions Supported Microservices with event-driven scaling

Choose Cloud Run when the workload is containerized and benefits from serverless request scaling without maintaining a server or cluster. Choose a service such as GKE or AWS ECS when long-running processes, specialized networking, or deep cluster control are more important.

Production Practices

  • Control scale-out cost: Set an appropriate max-instances value and configure budget alerts. Remember that retries and traffic spikes also create instances.

  • Avoid accidental public access: Use Cloud Run IAM and grant roles/run.invoker only to trusted identities or service accounts. Add --allow-unauthenticated only for intentionally public services.

  • Monitor capacity and latency: Track run.googleapis.com/request_count, run.googleapis.com/request_latencies, run.googleapis.com/container_instance_count, CPU utilization, memory utilization, and concurrency. Alert on elevated tail latency, sustained 429 responses, and abnormal instance growth.

  • Minimize startup work: Use small images, avoid unnecessary initialization, defer nonessential work, and cache dependencies where safe. Keep startup-time network calls to reliable, low-latency dependencies.

  • Warm new revisions carefully: During traffic splitting, direct a small amount of traffic to the new revision before increasing it so startup failures and latency are detected early.

  • Keep state external: Treat container filesystems as ephemeral. Store durable state in Cloud SQL, Spanner, Firestore, Bigtable, or Cloud Storage.

  • Plan for dependency failures: A database or API outage during initialization can cause startup failures, request errors, and repeated cold starts even when Cloud Run itself is healthy.

20

How do Cloud Run, Cloud Functions, and App Engine compare?

Cloud Run, Cloud Functions, and App Engine are Google Cloud’s main application-compute options, optimized for containers, event handlers, and managed applications respectively. The key distinction is the abstraction each platform exposes: an arbitrary container service, a function with a managed trigger, or an application-oriented PaaS.

Read full answer

Applies toGoogle Cloud Run (current stable), Cloud Functions (2nd gen), App Engine Standard and Flexible (current stable)

Service Comparison

Dimension Cloud Run Cloud Functions App Engine
Primary model Serverless container execution Event-driven function execution Managed application platform with Standard and Flexible variants
Invocation HTTP; Eventarc/CloudEvents; Pub/Sub push and Cloud Scheduler through HTTP or Eventarc HTTP, Google Cloud events, Pub/Sub, and schedules HTTP; Standard also supports cron and Task Queues, while Flexible is commonly called through HTTP or Cloud Scheduler
Runtime A container image supported by Cloud Run, listening on the port supplied through $PORT Supported language runtimes managed by Google Standard uses managed language runtimes; Flexible uses Compute Engine VMs and can run custom container runtimes
Scaling Automatic scaling, including scale-to-zero Automatic scaling, including scale-to-zero Standard automatic scaling can reach zero; Flexible uses VM-based automatic scaling and does not scale to zero
Deployment artifact Container image or source built through Cloud Build Function source package with its dependencies app.yaml; Flexible can additionally use a custom runtime container
Best fit APIs, microservices, custom containers, and event processors Small event handlers and integration glue code Existing PaaS applications or teams wanting application-level deployment and scaling management

Execution and Scaling

  • Cloud Run starts containers on Google-managed infrastructure. If no minimum instances are configured, a service can scale to zero, so the first request may incur a cold start. Revisions are immutable, which makes traffic splitting and rollback straightforward.

  • Cloud Functions 2nd gen is built on Cloud Run but adds a function-oriented programming model and managed event triggers. Google Cloud invokes a specific function and entry point rather than treating the deployment as an arbitrary HTTP container.

  • App Engine Standard provides managed language runtimes; its second-generation runtimes use Cloud Run technology while retaining App Engine’s application lifecycle, deployment, and scaling model. App Engine Flexible runs applications in Compute Engine VMs, offering more operating-system freedom but less aggressive scale-to-zero behavior.

  • Cloud Run and Cloud Functions are regional resources. An App Engine application’s location is selected when the application is created and can affect latency and regulatory placement.

Deployment and Traffic Management

  • Cloud Run supports immutable revisions, weighted traffic splitting, gradual rollouts, and traffic rollback.

  • Cloud Functions creates revisioned deployments, but it does not provide Cloud Run-style weighted traffic splitting between function revisions.

  • App Engine provides application versions and routing through service configuration and dispatch rules.

  • Cloud Run and Cloud Functions use IAM invoker permissions; public HTTP invocation requires an explicit grant to allUsers. App Engine access controls must be configured separately, and an HTTPS redirect does not itself make an application private.

AWS and Azure Equivalents

Google Cloud service Closest AWS equivalent Closest Azure equivalent Important distinction
Cloud Run AWS App Runner Azure Container Apps Cloud Run also resembles ECS/Fargate but adds managed request-based scaling and scale-to-zero
Cloud Functions AWS Lambda Azure Functions Google Cloud provides tightly integrated event sources and trigger types
App Engine AWS Elastic Beanstalk Azure App Service App Engine is more opinionated and provides an integrated application lifecycle

These mappings are not one-to-one. App Runner is primarily a managed web-service platform, while ECS/Fargate exposes more infrastructure-level control.

Production Guidance

  • Retries and idempotency: Event delivery can be repeated. Use deduplication keys, transactional updates, or idempotency records, and make retries safe. Returning an error from an event-triggered function commonly causes another delivery attempt.

  • State and recovery: These compute platforms are not durable database or backup systems. Keep authoritative state in services such as Cloud SQL, Firestore, Spanner, Bigtable, or Cloud Storage, and configure backups, point-in-time recovery, replication, or versioning as appropriate.

  • Observability: Monitor request latency, error rate, instance count, CPU utilization or throttling, and billing with Cloud Monitoring, Cloud Logging, Error Reporting, and Cloud Trace. Derive cold-start behavior from startup traces or logs rather than assuming a dedicated cold-start metric.

  • Capacity and cost: Review quotas, concurrency, and maximum-instance limits; exhaustion can appear as 429 responses or increased latency. Cloud Run, Cloud Functions, and App Engine Standard can scale to zero, whereas App Engine Flexible normally retains at least one instance. Minimum-instance settings reduce cold starts but create idle charges.

  • Security and upgrades: Use least-privilege runtime service accounts, grant invocation rights only to required identities, store secrets in Secret Manager, and restrict internal ingress where appropriate. Google patches managed language runtimes, while teams operating their own containers remain responsible for image maintenance, vulnerability scanning, and runtime upgrades.

21

How do App Engine standard and flexible environments differ?

App Engine standard is a tightly managed platform for supported language runtimes, while App Engine flexible runs applications in flexible compute instances and supports custom Docker-based runtimes. Standard is usually simpler and cheaper; flexible provides more operating-system, networking, and runtime control.

Read full answer

Applies toGoogle Cloud App Engine standard (second-generation runtimes) and flexible environments (current service versions)

Core Differences

Area Standard environment Flexible environment
Runtime model Google-managed runtimes for supported languages Managed instances with custom containers or supported runtimes
Deployment Source deployment using app.yaml and the configured runtime Deployment using app.yaml, commonly with env: flex and runtime: custom
Customization Restricted by the runtime sandbox Broader control through Dockerfiles, processes, libraries, and entrypoints
Filesystem Read-only except writable temporary storage such as /tmp Writable but ephemeral instance filesystem
Scaling Predefined instance classes and managed autoscaling Configurable instance resources, minimum instances, and autoscaling
Workloads HTTP APIs, web applications, and lightweight services Applications needing custom runtimes, more control, or longer-running processing
Cost profile Lower platform overhead and simpler instance pricing Greater instance overhead, but more control over allocated resources

Runtime and Request Model

Standard environment applications run inside Google-managed language and framework environments. This produces predictable startup and deployment behavior, but applications must comply with runtime restrictions. Unsupported native libraries, system packages, or custom processes usually require choosing a different runtime or flexible environment.

Flexible environment provides a conventional compute environment inside App Engine. Applications can use Docker, install operating-system packages, run multiple processes where supported, and define an entrypoint. The instance filesystem is still ephemeral, so neither environment should be treated as durable storage.

Standard is well suited to REST APIs, admin portals, and request/response applications. Flexible is preferable when the application requires a custom runtime, specialized native dependencies, or more control over the execution environment.

Scaling, Networking, and Deployment

Both environments integrate with App Engine’s global application routing and managed HTTPS. They support versioned deployments, traffic splitting, and rollback:

  • gcloud app deploy deploys a standard application from its configuration directory.

  • gcloud app deploy app.yaml deploys a flexible application whose app.yaml contains env: flex; runtime: custom also requires a valid Dockerfile and successful image build.

  • gcloud app versions list shows deployed versions.

  • gcloud app traffic set --splits=example-version=1.0 directs all traffic to the selected deployed version.

Autoscaling handles demand changes in both environments, but flexible applications have more explicit control over instance size and baseline capacity. Configure minimum instances only when cold starts are unacceptable and the resulting fixed cost is justified.

Networking differs materially: second-generation standard uses a Serverless VPC Access connector for private egress and does not support direct VPC egress, while flexible supports direct VPC egress. By default a connector carries only private-range traffic; if it is set to route all egress (egress_setting: all-traffic), internet-bound traffic then needs Cloud NAT. Verify private DNS, regional routing, and quotas as well.

Choosing a GCP Service

Requirement Recommended option
Simple supported runtime with minimal operations App Engine standard
Custom Docker image with App Engine-managed instances App Engine flexible
Event-driven HTTP containers or scale-to-zero Cloud Run
Managed job execution Cloud Run Jobs
Full Kubernetes orchestration GKE
Fully managed relational database Cloud SQL
Managed document database Firestore
Managed wide-column NoSQL database Cloud Bigtable

App Engine standard is conceptually closer to platforms such as Azure App Service or traditional Heroku-style PaaS. Flexible is closer to a managed container or VM platform, although it retains App Engine’s deployment and traffic-management model. For a new stateless container where scale-to-zero is important, Cloud Run often provides a better fit than flexible.

Production Operations and Security

  • Monitoring: Use Cloud Monitoring for request latency, error rate, instance utilization, and autoscaling behavior. Use gcloud app logs tail --service=default for rapid log inspection and Cloud Logging for retention and analysis.

  • Availability: Deploy a new version and shift traffic gradually rather than replacing all instances at once.

  • Backups: Application files are not a backup system. Use Cloud SQL automated backups or point-in-time recovery and, for Cloud Storage, versioning with suitable retention or replication; versioning alone is not an independent disaster-recovery copy.

  • Security: Use least-privilege IAM, a dedicated service account, Secret Manager, and narrowly scoped API permissions. The App Engine service account represents the application, not the signed-in end user.

  • Upgrades: Runtime deprecations require migration before the published deadline. For flexible, rebuild and test the container whenever the base image, dependencies, or runtime configuration changes.

Migration and Common Pitfalls

Moving from standard to flexible is an architectural migration, not just a configuration change. Validate filesystem assumptions, background processing, startup time, health checks, outbound networking, memory limits, and cost before shifting production traffic.

The most common failures are selecting flexible unnecessarily, assuming local files persist, ignoring runtime or regional quotas, and enabling minimum instances without accounting for their continuous cost. Test the exact deployment artifact, use gradual traffic migration, and retain rollback capacity during production releases.

22

What are Cloud Functions in GCP and what triggers do they support?

Cloud Run functions (formerly Cloud Functions, 2nd gen) are event-driven, single-purpose compute units. A deployed function runs in response to an HTTP request or a supported Google Cloud/Eventarc event, can scale to zero when idle, and is backed by a Cloud Run service.

Read full answer

Applies toCloud Run functions 2nd gen (Cloud Run API v1 and Cloud Functions API v2), current stable Google Cloud

How It Works

  • Build and runtime: Source is built into a container through Cloud Build and deployed as a Cloud Run service, inheriting Cloud Run’s IAM, revisions, networking, and observability model.

  • Event delivery: Non-HTTP provider events pass through Eventarc as CloudEvents 1.0. Eventarc filters can select attributes such as type, bucket, and name.

  • Scaling and limits: Functions scale to zero by default. min-instances reduces cold starts, max-instances limits blast radius, and concurrency defaults to 1 with a maximum of 1000. Supported memory ranges from 128 MiB to 32 GiB; timeout defaults to 60 seconds and can reach 3600 seconds.

  • Security and networking: Functions use a service account and require invoker authorization. HTTP functions are private by default. Default ingress allows internal and Cloud Load Balancing traffic; private API access can use a VPC connector or direct VPC egress.

Supported Trigger Types

Trigger Source Event or request type Operational notes
HTTP Function URL or load balancer HTTP request Use Cloud IAM or signed OIDC identity tokens for private endpoints.
Cloud Pub/Sub Topic google.cloud.pubsub.topic.v1.messagePublished At-least-once delivery, retries, and dead-letter topics are supported.
Cloud Storage Bucket Object finalized, deleted, archived, or metadata updated Filter by bucket and object-name pattern.
Firestore Database Document created, updated, or deleted Events have no ordering guarantee.
BigQuery Table google.cloud.bigquery.table.v1.rowInserted Delivered through Eventarc.
Other Eventarc events Audit Logs, Cloud Run, GKE, direct events, and supported partner sources such as AWS S3 and Azure Blob Storage Provider-specific CloudEvent types Not every arbitrary CloudEvent is supported.

Cloud Scheduler and Cloud Tasks are invocation patterns rather than separate native event-trigger types. They call an HTTP function, or Scheduler can publish to Pub/Sub for a Pub/Sub-triggered function. Cloud Tasks requires an authenticated HTTP target and provides controlled, rate-limited dispatch.

Firebase Functions is a separate Google-managed product with provider-specific triggers and the Firebase Functions SDK; those triggers should not be presented as native Cloud Run functions trigger types.

Production Considerations

  • Delivery: Assume at-least-once delivery. Make handlers idempotent using a business key or source message identifier, configure Pub/Sub dead-letter topics, and alert on dead-letter depth.

  • Failures: Treat exceptions and unsuccessful responses as retryable. Move long-running or high-memory work to Cloud Run Jobs or Batch, and use Workflows for multi-step orchestration.

  • Observability: Use structured logs, Error Reporting, and Cloud Monitoring metrics for invocations, failures, and latency. Trace requests across Pub/Sub, Eventarc, and downstream services.

  • Security: Use a dedicated least-privilege service account, keep secrets in Secret Manager, and restrict invoker permissions. For protected public endpoints, use an external HTTPS Load Balancer with Cloud Armor.

  • State and cost: Keep durable state in managed databases or storage with appropriate backups. min-instances reduces cold starts but bills continuously; memory allocation also determines allocated CPU, so monitor compute and logging costs.

Choosing the Right Service

Need GCP AWS Azure
Short event-driven function Cloud Run functions Lambda Functions
Custom container or long-lived service Cloud Run services ECS/Fargate Container Apps
Batch or long-running work Cloud Run Jobs ECS RunTask or Batch Container Apps jobs
Messaging and event routing Pub/Sub and Eventarc SNS/SQS and EventBridge Service Bus and Event Grid

Best Practices

  • Deploy only 2nd-gen functions; Cloud Functions 1st gen is legacy and no longer accepts new deployments.

  • Keep handlers small and focused, and move sustained workloads to Cloud Run services or jobs.

  • Manage deployments with Terraform’s google_cloudfunctions2_function resource or Cloud Build and Cloud Deploy, and pin supported runtimes.

  • Roll out revisions gradually, set max-instances, monitor quotas, and test retry, idempotency, and incident-alert paths before production traffic.

23

Which Google Cloud Storage classes fit different access patterns?

Google Cloud Storage (GCS) is Google Cloud’s managed object-storage service for unstructured data such as images, videos, backups, logs, and data-lake files. Its storage classes determine storage pricing, retrieval behavior, and minimum storage duration; they do not change the object’s location or GCS’s 11-nines durability target.

Read full answer

Applies toGoogle Cloud Storage (current stable service; Standard, Nearline, Coldline, Archive, and Autoclass)

Core Service Model

  • Bucket: A globally unique container belonging to a Google Cloud project.

  • Object: A file and its metadata, accessed through a path such as gs://example-org-data-12345/reports/2025.csv.

  • Locations: A bucket can use a single region, dual-region, or multi-region location. These affect latency, availability, and cost.

  • Consistency: GCS provides strong consistency for object and metadata operations, so applications can immediately read newly written data.

  • Interfaces: Objects are available through gcloud storage, JSON and XML APIs, client libraries, and other compatible command-line tools.

Storage Classes

Class or policy Typical access pattern Duration and behavior
Standard Frequent or unpredictable access No minimum duration; immediate access for active applications, websites, and analytics inputs
Nearline Accessed about monthly or less 30-day minimum; typically millisecond retrieval for backups and infrequently accessed reports
Coldline Accessed quarterly or less 90-day minimum; typically millisecond retrieval for long-lived, rarely accessed data
Archive Rarely accessed, such as legal, media, or disaster-recovery archives 365-day minimum; data is still available in milliseconds, but retrieval and operation charges are the highest
Autoclass Unpredictable or changing access patterns where object age is a poor proxy Automatically transitions objects among Standard, Nearline, Coldline, and (when the terminal class is Archive) Archive according to access; no minimum storage durations, early-deletion or retrieval fees, but a per-object management fee applies

Deleting an object before its minimum duration can incur an early-deletion charge. Standard is the default class, but an ordinary bucket can define another default and uploads can specify a class. An Autoclass-managed bucket controls transitions automatically.

Lifecycle and Cost Behavior

A lifecycle policy can automatically transition an object to a colder class as it ages. A transition changes the object’s storage class but does not move it to another bucket or geographic location.

  • Storage charges depend on bytes stored, storage class, and location.

  • Nearline, Coldline, and Archive generally charge less per GB stored than Standard but charge more for data retrieval.

  • Early deletion, repeated reads, bulk retrieval, operation charges, and network egress can materially change total cost.

  • Archive offers the lowest storage price, but its retrieval and operation charges make frequent reads expensive.

  • A lifecycle policy based only on age can be inefficient when different objects in the same bucket have very different access patterns; Autoclass can be more suitable in that case.

GCP and Cloud Comparisons

Google Cloud Closest AWS equivalent Closest Azure equivalent When to choose it
Cloud Storage Amazon S3 Azure Blob Storage Unstructured objects, event files, backups, media, and data-lake content
Cloud SQL Amazon RDS/Aurora Azure Database for PostgreSQL/MySQL Managed relational databases and SQL transactions
Firestore Amazon DynamoDB Azure Cosmos DB Application documents and real-time NoSQL workloads
Bigtable Amazon DynamoDB Azure Cosmos DB for NoSQL High-volume, wide-column data with predictable access patterns
BigQuery Amazon Redshift/Athena Azure Synapse/Fabric Analytical SQL over large datasets

These service mappings are conceptual. For storage classes, Amazon S3 Standard-IA and Azure Cool most closely resemble GCS Nearline, while S3 Glacier Deep Archive and Azure Archive most closely resemble GCS Archive, although pricing, minimum durations, and retrieval behavior differ.

Production Practices

  • Enable Uniform Bucket Level Access, public-access prevention, and least-privilege IAM. Use customer-managed encryption keys when regulatory requirements demand them.

  • Monitor request errors, latency, storage volume, lifecycle behavior, and costs with Cloud Monitoring, Cloud Logging, and budget alerts.

  • Use bucket versioning, soft delete, or retention policies according to recovery and compliance requirements; versioning alone is not a complete backup strategy.

  • Test Archive restoration procedures because retrieval charges and egress volume affect incident-recovery cost and duration.

  • Reclassify only after measuring access frequency. Incorrect lifecycle assumptions often cost more through retrieval and egress charges than the storage savings.

  • Audit access through Cloud Audit Logs, protect cross-region copies with appropriate IAM, and rehearse restoration before relying on backups during an incident.

24

How do Cloud Storage object lifecycle rules work?

Object lifecycle rules in Google Cloud Storage automatically transition eligible objects to a specified storage class, delete eligible object versions, abort incomplete multipart uploads, or apply holds. They reduce manual cleanup and storage costs, but their execution is asynchronous rather than an exact real-time deadline.

Read full answer

Applies toGoogle Cloud Storage (current stable) and Google Cloud CLI (current stable)

How Lifecycle Rules Work

Lifecycle rules belong to a bucket and can affect both existing and future objects. Each rule contains a condition and one action:

  • All conditions within a rule are combined with AND.

  • Separate rules are evaluated independently.

  • Cloud Storage evaluates rules asynchronously, so an object may remain eligible after its configured age or date.

  • Rules operate on object versions when versioning is enabled. age is measured from object creation; daysSinceNoncurrentTime measures time since a version became noncurrent.

Cloud Storage does not provide a separate lifecycle configuration on each object. For per-object timing, set the object’s customTime and use a bucket rule containing customTime or daysSinceCustomTime. Per-object temporaryHold and eventBasedHold fields provide separate hold controls.

Conditions and Actions

Common conditions include:

Category Conditions
Creation and age age, createdBefore
Custom time customTime, daysSinceCustomTime
Version status isLive, numNewerVersions
Noncurrent age noncurrentTimeBefore, daysSinceNoncurrentTime
Selection matchesPrefix, matchesSuffix, matchesStorageClass

Common actions are:

  • Delete: Deletes matching live or noncurrent object versions, subject to versioning, soft delete, retention, and holds.

  • SetStorageClass: Sets the specified target storage class. It is normally used to move objects into colder tiers such as NEARLINE, COLDLINE, or ARCHIVE.

  • AbortIncompleteMultipartUpload: Removes multipart uploads that were initiated but never completed.

  • SetTemporaryHold: Places a temporary hold that must be explicitly removed.

  • SetEventBasedHold: Places an event-based hold that is removed when the object is replaced.

Production Behavior and Trade-offs

  • Versioning: Deleting a live object in a versioned bucket normally makes it noncurrent instead of permanently removing it. Use isLive: false when targeting noncurrent versions.

  • Soft delete: With soft delete enabled, lifecycle deletion produces a recoverable soft-deleted object for the configured period.

  • Holds and retention: Existing holds can block lifecycle deletion. Temporary holds block deletion and replacement until removed; event-based holds are cleared by replacement. Retention policies provide stronger compliance protection.

  • Storage economics: Nearline, Coldline, and Archive transitions can incur minimum-duration charges and retrieval fees. Archive retrieval can also have substantial latency.

  • Recovery and monitoring: Lifecycle rules are not backups. Use versioning, replication, or a dedicated backup strategy, and monitor object counts, bytes by storage class, lifecycle-related activity, and bucket configuration changes in Audit Logs.

AWS and Azure Equivalents

Capability Google Cloud Storage Amazon S3 Azure Blob Storage
Age and date expiration Lifecycle age and date conditions Lifecycle expiration Lifecycle management policies
Prefix and tag filters Prefix and suffix filters Prefix and tag filters Prefix and blob index tag filters
Storage transitions SetStorageClass Storage-class transitions Access-tier and Blob-tier transitions
Per-object timing customTime No direct custom-timestamp equivalent No direct custom-timestamp equivalent

Code Example

This configuration transitions objects under logs/archive/ to Nearline after 30 days, deletes objects under logs/temp/ after 30 days, and aborts incomplete multipart uploads after one day.

{
  "rule": [
    {
      "action": {
        "type": "SetStorageClass",
        "storageClass": "NEARLINE"
      },
      "condition": {
        "age": 30,
        "matchesPrefix": "logs/archive/"
      }
    },
    {
      "action": {
        "type": "Delete"
      },
      "condition": {
        "age": 30,
        "matchesPrefix": "logs/temp/"
      }
    },
    {
      "action": {
        "type": "AbortIncompleteMultipartUpload"
      },
      "condition": {
        "age": 1
      }
    }
  ]
}

Apply and inspect the configuration with the gcloud CLI:

# Replace the lifecycle configuration on an existing bucket.
gcloud storage buckets update gs://example-bucket \
  --project=example-project \
  --lifecycle-file=./lifecycle.json

# Inspect the effective bucket configuration.
gcloud storage buckets get gs://example-bucket \
  --project=example-project \
  --format=json

Best Practices and Common Pitfalls

  • Test rules on a nonproduction bucket and use narrow prefixes or suffixes.

  • Treat the lifecycle file as the desired complete configuration because updating the bucket’s lifecycle configuration can replace existing rules.

  • Use versioning, soft delete, retention, holds, and replication as complementary controls rather than assuming lifecycle deletion guarantees immediate removal.

  • Grant only the required IAM permissions, including storage.buckets.update for lifecycle changes and storage.objects.update for per-object metadata or hold changes.

  • Monitor storage-class growth, retrieval charges, deleted-object counts, and unexpected lifecycle configuration changes.

25

What are signed URLs in Google Cloud Storage?

A signed URL in Google Cloud Storage is a time-limited HTTPS URL that grants temporary access to a specific object without exposing the service account private key or requiring an OAuth bearer token. The complete URL is itself a bearer credential; V4 is recommended, while V2 remains supported for compatibility.

Read full answer

Applies toGoogle Cloud Storage signed URLs (V4 recommended; V2 supported for compatibility), current Google Cloud; gcloud CLI (current stable)

How It Works

  • For V4 signing, a service account signs a canonical HTTP request with its signing key.

  • The URL contains X-Goog-Algorithm, X-Goog-Credential, X-Goog-Date, X-Goog-Expires, X-Goog-SignedHeaders, and X-Goog-Signature.

  • The signature covers the HTTP method, canonical object path, relevant query parameters, and designated headers.

  • Cloud Storage validates the signature and expiration and authorizes access according to the signing service account’s applicable IAM or ACL permissions.

  • V4 signed URLs can remain valid for up to seven days. Short expirations substantially reduce the impact of disclosure.

  • A signed URL does not make the bucket or object public and does not override applicable access controls, retention policies, legal holds, lifecycle behavior, or CMEK checks.

For example, a signed GET URL permits a browser to download reports/example.pdf. A signed PUT URL can authorize a direct upload when the signing service account has storage.objects.create, such as through roles/storage.objectCreator.

Operational Use

The current gcloud storage buckets sign-url command supports IAM-based V4 signing through --service-account; for example: gcloud storage buckets sign-url gs://example-bucket/reports/example.pdf --duration=15m --service-account=signer@example-project.iam.gserviceaccount.com --project=example-project --version=V4.

The principal running gcloud needs iam.serviceAccounts.signBlob on the target service account, commonly granted through roles/iam.serviceAccountTokenCreator on that specific service account. The signing identity also needs object access: scope storage.objects.get to the object for downloads, or grant storage.objects.create for uploads.

Signed URLs do not enable cross-origin browser access by themselves. Configure the bucket’s CORS policy separately when a browser uploads directly to Cloud Storage.

Cloud Comparisons

Cloud Equivalent Main difference
Google Cloud Storage Signed URL Query-string signature based on a Google service account
Amazon S3 Presigned URL Similar temporary capability generated through AWS credentials or SDKs
Azure Blob Storage Shared Access Signature SAS tokens can contain broader permissions and service policies

All three provide temporary delegated access. The principal operational concern is the same: minimize the lifetime, protect the complete URL, and monitor its use.

Security and Failure Modes

Symptom Likely cause Corrective action
403 Permission denied The signing identity lacks object access, or its applicable access has been revoked Grant only the required object permission and verify current access controls
400 Invalid signature The URL, HTTP method, path, query, or signed header was changed Send the URL exactly as generated and use the intended method
400 Request has expired The URL expired or system clocks are skewed Use a short lifetime and maintain accurate NTP synchronization
404 Not Found The object does not exist at the signed path Verify the bucket, object name, and any generation requirement
403 involving CMEK The signing identity lacks the required Cloud KMS permission on the key version Grant the appropriate KMS encrypt/decrypt role and verify key access

Treat a signed URL as a bearer credential. Anyone who obtains it can replay it until it expires, subject to the encoded method and access controls.

Production Practices

  • Prefer short expirations, commonly 5–15 minutes for downloads, rather than seven-day URLs by default.

  • Use IAM-based signBlob signing instead of storing service account private keys in CI systems, containers, or secret files.

  • Grant roles/iam.serviceAccountTokenCreator only on the service account being used for signing.

  • Never log complete signed URLs in application logs, traces, analytics payloads, support tickets, or browser telemetry.

  • Enable appropriate Cloud Audit Logs and monitor elevated 400 and 403 rates, unusual download volume, and repeated replay from unexpected networks.

  • Use narrowly scoped permissions such as storage.objects.get for downloads and storage.objects.create for uploads; avoid bucket-wide viewer grants unless listing is also required.

  • Prefer uniform bucket-level access and IAM, while accounting for legacy buckets that still use object ACLs.

  • After revoking access or handling an incident, verify that old URLs are rejected before issuing replacements.

  • Configure cache and object-generation behavior carefully because URL expiration and object replacement are separate concerns.

26

How do you secure a Google Cloud Storage bucket?

A secure Google Cloud Storage bucket combines uniform bucket-level access, public access prevention, least-privilege IAM, encryption, retention, and audit monitoring. Treat the bucket as a production data boundary rather than relying on individual object permissions or obscurity.

Read full answer

Applies toGoogle Cloud Storage (current stable) and Google Cloud CLI (current stable)

Core Security Controls

  • Uniform bucket-level access: Use bucket IAM for all access and disable per-object ACLs. This simplifies auditing and prevents inconsistent permissions.

  • Public access prevention: Block allUsers and allAuthenticatedUsers, even if someone adds a public IAM binding.

  • Organization policy: Enforce constraints/storage.publicAccessPrevention across the organization so project owners cannot weaken it.

  • TLS in transit: GCS uses TLS for HTTPS requests; do not permit unencrypted traffic to the storage.googleapis.com API.

  • Encryption at rest: Google-managed encryption is enabled by default. Use customer-managed encryption keys (CMEK) when key rotation, separation of duties, or key revocation must be centrally controlled.

  • Private sharing: Use short-lived signed URLs instead of making objects public.

Control Google Cloud Storage AWS S3 Azure Blob Storage
Authorization Bucket/object IAM Bucket policy and IAM Azure RBAC
Block public access Public Access Prevention and organization policy Block Public Access Disable Allow blob public access; keep container access private
Per-object ACLs Disabled with uniform bucket-level access Generally discouraged Available but generally discouraged
Customer-managed keys CMEK SSE-KMS Customer-managed keys

Least-Privilege IAM

  • Grant only the required predefined role, such as roles/storage.objectCreator or roles/storage.objectViewer.

  • Use a custom role when the separation between read, upload, overwrite, and delete permissions matters.

  • Prefer workload identities for GKE, Cloud Run, and other Google Cloud workloads. Avoid downloadable service-account keys.

  • Do not grant roles/storage.admin or roles/owner merely to let an application upload files.

  • Remember that bucket-level object roles can grant access to every object in the bucket. Use separate buckets for different trust boundaries when isolation is required.

Data Protection and Recovery

  • Enable object versioning to recover from accidental overwrite or deletion.

  • Configure lifecycle rules to delete obsolete noncurrent versions before they create unexpected storage costs.

  • Use retention policies or holds for regulated data. A bucket-level retention lock is irreversible and prevents shortening or removing the retention period.

  • Verify the bucket’s soft-delete configuration; soft delete and versioning serve different recovery purposes.

  • CMEK does not replace backups. For critical data, replicate to a separately administered bucket or region, protect the destination independently, and test restoration procedures.

  • Keep sensitive secrets in Secret Manager rather than storing them as ordinary bucket objects.

Monitoring and Incident Response

  • Enable Cloud Audit Logs, especially Data Access logs for Cloud Storage, and retain them in a protected log bucket.

  • Alert on IAM and bucket-policy changes, public-access attempts, unusual download volume, lifecycle changes, and retention-policy modifications.

  • Review denied requests and unusual service-account activity with Security Command Center or Cloud Logging.

  • Include bucket policies, IAM bindings, CMEK configuration, lifecycle rules, and retention settings in configuration-as-code and change reviews.

  • Back up bucket configuration separately; recreating a bucket does not necessarily recreate its IAM policies or retention controls.

Code Example

Create a private, versioned bucket and grant a workload service account narrowly scoped access:

# Create a private, versioned bucket; uniform access disables object ACLs.
gcloud storage buckets create gs://example-secure-bucket \
  --project=example-project \
  --location=us-central1 \
  --uniform-bucket-level-access \
  --public-access-prevention \
  --versioning

# Grant the workload service account object-creation access.
gcloud storage buckets add-iam-policy-binding \
  gs://example-secure-bucket \
  --project=example-project \
  --member=serviceAccount:uploader@example-project.iam.gserviceaccount.com \
  --role=roles/storage.objectCreator

# Inspect the effective bucket IAM bindings.
gcloud storage buckets get-iam-policy \
  gs://example-secure-bucket \
  --project=example-project

roles/storage.objectCreator permits object creation but does not grant object-read or explicit object-delete permission. It is not a write-once guarantee: a caller that can replace an existing object can change its contents. Use ifGenerationMatch=0 preconditions, retention or holds, or a separate bucket where existing content must not be replaced.

Production Checklist

  • Enforce public-access prevention at both bucket and organization levels.

  • Keep uniform bucket-level access enabled.

  • Grant access to identities, not broad groups, and review bindings regularly.

  • Use signed URLs with short expirations for controlled downloads.

  • Enable versioning, lifecycle management, CMEK where required, and tested recovery procedures.

  • Monitor audit logs and alert on configuration or access-pattern changes.

  • Never place real credentials in bucket names, policies, examples, or application configuration.

27

What is Google Cloud SQL managed relational database?

Cloud SQL is Google Cloud’s managed service for relational databases, supporting PostgreSQL, MySQL, and SQL Server. Google manages provisioning, patching, backups, monitoring integration, and high availability, while applications connect using standard database protocols.

Read full answer

Applies toGoogle Cloud SQL (current stable service; supported engine versions vary)

Core Capabilities

  • Managed database lifecycle: Google operates the operating system, database binaries, storage, and routine maintenance instead of requiring you to administer database hosts.

  • Supported engines: PostgreSQL, MySQL, and Microsoft SQL Server provide familiar SQL, transaction, indexing, and ecosystem support.

  • High availability: A regional instance maintains a synchronous standby in another zone and can fail over automatically within that region.

  • Read replicas: Additional instances can serve read-heavy workloads or place data closer to users. Cross-region replicas add latency and cost and do not provide a single automatic multi-region failover endpoint.

  • Backup and recovery: Automated backups support scheduled recovery, while point-in-time recovery uses database transaction logs to restore to a specific time.

  • Scaling: Instances can change compute tiers and storage capacity. storage-auto-increase lets disk grow without overwriting the existing disk size, within service limits.

Cloud SQL is primarily a transactional database. It is not a globally distributed database like Spanner, a document database like Firestore, or an analytics warehouse like BigQuery.

Production Architecture and Operations

A typical production deployment uses:

  • A regional high-availability instance for the primary database.

  • Private IP connectivity through VPC peering, Private Service Connect, or the legacy private-services-access model.

  • A connection pooler such as the Cloud SQL connectors to prevent excessive connections from exhausting the database’s connection limit.

  • Read replicas for reporting or geographically distributed reads, with applications checking replication lag.

  • Cloud Monitoring dashboards and alerts for CPU, memory, disk utilization, storage growth, replication lag, connection counts, and instance availability.

Google performs routine minor-version maintenance during a configurable maintenance window. Major engine-version upgrades generally require you to initiate and validate the upgrade. Before any major upgrade, test compatibility, take a backup, and rehearse the process in a nonproduction environment.

Security and Failure Modes

  • IAM controls who can administer instances through the Cloud SQL Admin API. Grant narrowly scoped roles such as roles/cloudsql.admin or roles/cloudsql.client rather than broad project roles.

  • Encryption at rest is provided by default, and customer-managed encryption keys are available for applicable configurations.

  • TLS protects connections when the client enforces certificate validation.

  • Private IP avoids exposing the database directly to the internet. Use it whenever the application and database share connected network resources.

  • Database authentication is separate from Google Cloud IAM; engine-specific IAM authentication should be enabled where supported.

Important failure modes include reaching a connection limit, running out of storage, excessive replication lag, backup failures, and application deployments that are incompatible with a schema change. Capacity alerts, tested restores, and connection pooling help prevent these issues from becoming incidents.

Service Selection

Requirement Recommended GCP service
Managed relational database with PostgreSQL, MySQL, or SQL Server Cloud SQL
Globally distributed, strongly consistent SQL with horizontal scaling Spanner
NoSQL document workloads Firestore
Wide-column NoSQL workloads Bigtable
Large-scale analytics and data warehousing BigQuery

The closest AWS equivalents are Amazon RDS and Aurora; Azure equivalents include Azure Database for PostgreSQL, Azure Database for MySQL, and Azure Database for SQL Server. Choose Cloud SQL when you want a conventional relational engine without server administration. Choose Spanner when global distribution and horizontal scale are primary requirements.

Best Practices and Common Pitfalls

  • Enable high availability for production workloads that cannot tolerate a zone failure.

  • Use private networking, TLS, and least-privilege IAM.

  • Keep application and database capacity aligned; monitor disk headroom and connection saturation.

  • Test point-in-time restoration regularly rather than assuming backups are usable.

  • Use read replicas intentionally, and ensure applications tolerate replication lag.

  • Include Cloud SQL charges for compute, storage, backups, replicas, and network egress in cost forecasts.

  • Avoid presenting a cross-region read replica as a transparent multi-region failover solution.

28

How do Cloud SQL high availability and read replicas work?

High availability in Google Cloud SQL uses a synchronous primary–standby pair within one region, while ordinary read replicas provide asynchronously replicated copies for read scaling or manual cross-region disaster recovery. Enterprise Plus cross-region replicas are a separate standby-style disaster-recovery feature that can support automated regional failover; these mechanisms solve different failure modes and do not replace backups.

Read full answer

Applies toGoogle Cloud SQL current stable (gcloud CLI current stable; PostgreSQL 16 examples)

Architecture Choices

Capability Regional high availability Ordinary read replica Backup
Replication Synchronous within a region Asynchronous; same or different region Scheduled backups plus transaction logs for PITR
Main purpose Survive an instance or zonal failure Scale reads or support manual regional DR Recover from deletion, corruption, or logical errors
Writes Primary is writable Read-only until promoted Restored copy is writable
Failover Automatic to the regional standby Manual promotion Manual restore
Recovery point Near-zero under normal HA operation Depends on replication lag Depends on backup retention and PITR window
Typical use Production transactional database Reporting, analytics, geographic DR Recovery and compliance

A Cloud SQL primary can have multiple ordinary read replicas. A same-region read replica remains asynchronous: it does not become an automatic failover pair with its source.

How It Works

Regional HA creates a primary and standby instance in the same region. The standby receives synchronous transaction-log updates, allowing Cloud SQL to promote it automatically when the primary becomes unavailable. This protects against instance and zonal failures, but not a complete regional outage.

Ordinary read replicas receive updates asynchronously. Applications connect to a replica’s separate endpoint and direct read-only traffic there. Because replication is asynchronous, the replica may lag behind the primary; even a measured lag of zero does not provide transaction-level point-in-time recovery.

For manual regional disaster recovery, create an ordinary cross-region read replica and promote it after a declared outage. Promotion makes the replica an independent writable instance. The original primary must then be fenced or otherwise prevented from accepting conflicting writes.

Enterprise Plus editions also offer cross-region replicas, a standby-style DR feature distinct from ordinary read replicas. They can be configured for automated regional failover. Replication remains asynchronous, so failover can lose transactions that have not yet reached the replica. Use ordinary read replicas when manual promotion and independent read endpoints are desired; use Enterprise Plus cross-region replicas when automatic DR failover is required.

Code Example

The following commands create a regional HA primary and an ordinary asynchronous cross-region read replica:

# Create a regional HA primary with scheduled backups.
gcloud sql instances create orders-primary \
  --project=example-project \
  --region=us-central1 \
  --database-version=POSTGRES_16 \
  --tier=db-custom-4-8192 \
  --availability-type=regional \
  --backup-start-time=03:00

# Create an ordinary asynchronous cross-region read replica.
gcloud sql instances create orders-read-replica \
  --project=example-project \
  --region=us-east1 \
  --database-version=POSTGRES_16 \
  --tier=db-custom-2-4096 \
  --master-instance-name=orders-primary

# Promote the ordinary read replica during a regional incident.
gcloud sql instances promote-replica orders-read-replica \
  --project=example-project

Promotion is a significant operational action. It is not the automated failover mechanism used by regional HA or configured Enterprise Plus cross-region replicas.

Production Operations

  • Monitor replication lag: Alert on the Cloud SQL replication-lag metric, CPU, memory, disk utilization, connection saturation, and instance or replica state.

  • Track operations: Use gcloud sql operations list --instance=orders-primary --project=example-project to follow failovers, promotions, backups, and configuration changes.

  • Plan failover behavior: Regional HA and automated cross-region failover interrupt existing sessions. Applications need connection retries with exponential backoff and jitter and must replace stale pooled connections.

  • Use replicas deliberately: Route repeatable, potentially expensive queries to ordinary read replicas, but ensure they tolerate slightly stale results.

  • Maintain backups: Read replicas do not protect against accidental deletion, corruption, or a harmful schema change. Use scheduled backups and, where supported, point-in-time recovery.

  • Test recovery: Regularly rehearse replica promotion and backup restoration, including application compatibility and DNS or connection-routing changes.

Security and Cost

  • Prefer private IP connectivity, restrict authorized networks, and avoid exposing database ports publicly.

  • Require encrypted database connections using the appropriate SSL/TLS settings for the selected engine.

  • Use least-privilege IAM identities for operators and automation. Grant the instance service account only permissions required by enabled integrations, such as customer-managed encryption keys.

  • Data is encrypted at rest by default; Cloud SQL also provides customer-managed key options where required.

  • Regional HA adds standby capacity, while read replicas add storage and compute resources. Monitor both budget and provider pricing before adding replicas.

  • Cross-region replicas improve disaster recovery, but they do not create a synchronous multi-region database and therefore do not provide zero-loss regional failover.

Failure Modes and Best Practices

  • Replica lag: Large writes, indexing, maintenance, or network problems can increase lag. Avoid placing latency-sensitive workflows behind replicas unless stale reads are acceptable.

  • Promotion conflict: After promoting an ordinary cross-region read replica, ensure the old primary cannot continue serving writes.

  • Connection routing: An ordinary read replica has its own endpoint. Use explicit routing rather than assuming replicas share the primary’s address.

  • Resource contention: Cap replicas appropriately and separate reporting workloads from OLTP workloads to avoid degrading the primary.

  • Schema changes: Apply changes through a controlled migration process and verify compatibility with every replica before completion.

  • Recovery ownership: Document who declares regional failure, promotes an ordinary replica or enables automated failover, updates routing, and reconciles the former primary.

29

How does Cloud Spanner differ from Cloud SQL?

Cloud Spanner is Google Cloud’s globally distributed relational database, designed for horizontal scaling and strong consistency across regions. Cloud SQL is a managed relational database service for MySQL, PostgreSQL, and SQL Server, offering familiar database engines with primarily vertical scaling and replica-based high availability.

Read full answer

Applies toGoogle Cloud Spanner and Google Cloud SQL (current versions)

How Cloud Spanner Works

  • Distributed storage: Data is automatically partitioned into tablets and distributed across machines. Applications address one logical database rather than manually managing shards.

  • Replication: Replicas provide durability and availability. Regional configurations span zones; multi-region or dual-region configurations spread replicas across locations.

  • Consistency: Spanner provides strongly consistent reads and serializable read-write transactions by default, including in multi-region deployments.

  • Transactions: Distributed transactions can encounter contention and may need application-level retries, especially when many transactions modify related rows.

  • Schemas and SQL: It supports relational schemas, indexes, joins, and transactions through the GoogleSQL and PostgreSQL dialects. It is not wire-compatible with MySQL or PostgreSQL servers.

  • Time and coordination: Internally, Spanner uses consensus protocols and TrueTime to coordinate replicated state and provide external consistency.

How Cloud SQL Works

Cloud SQL runs a managed MySQL, PostgreSQL, or SQL Server instance on Google-managed infrastructure.

  • Scaling: CPU and memory are increased by moving to larger machine types. Additional read replicas support read-heavy workloads, but a primary instance remains the write bottleneck.

  • High availability: Regional high-availability configurations maintain a standby in another zone within the same region. Cross-region replicas provide disaster recovery but normally replicate asynchronously.

  • Compatibility: Existing applications can often migrate with minimal SQL changes, including dialect-specific extensions and operational tools.

  • Management: Google handles infrastructure, patching, backups, monitoring, and routine maintenance while administrators retain database configuration and engine-level control.

Side-by-Side Comparison

Area Cloud Spanner Cloud SQL
Database model Globally distributed relational database Managed MySQL, PostgreSQL, or SQL Server
Main scale direction Horizontal, automatic table splitting Primarily vertical, with read replicas
Availability model Regional, multi-region, or dual-region Regional HA plus optional cross-region replicas
Consistency Strong consistency and serializable transactions by default Engine-specific transaction and replication behavior
SQL compatibility GoogleSQL or PostgreSQL dialect Native MySQL, PostgreSQL, or SQL Server
Global write latency Higher in multi-region configurations, but globally available Cross-region writes depend on asynchronous replicas
Typical workloads Global transactions, high-volume OLTP, low-latency services Existing enterprise applications, general OLTP, smaller or regional systems
Closest AWS equivalent Aurora is relational but not fully equivalent Amazon RDS or Amazon Aurora
Closest Azure equivalent Azure SQL Hyperscale is conceptually close Azure Database for PostgreSQL, Azure Database for MySQL, or Azure SQL Managed Instance

Production Operations

  • Monitoring: Monitor Spanner transaction latency, contention, CPU, storage, and regional performance. For Cloud SQL, track CPU, disk utilization and I/O, connections, replication lag, and backup status. Use Cloud Monitoring dashboards and alerts in both cases.

  • Backups: Both support backups and point-in-time recovery. Test restoration regularly; a successful backup job does not prove that the restore procedure meets its recovery-time objective.

  • Upgrades: Cloud SQL engine upgrades and maintenance windows require application testing, especially for major PostgreSQL, MySQL, or SQL Server versions. Spanner schema and query changes should also be tested for compatibility and performance.

  • Incidents: Spanner multi-region deployments improve regional resilience, but contention or hot partitions can still affect latency. Cloud SQL failovers can interrupt active connections, so applications need reconnect and retry logic.

  • Security: Use Cloud IAM with least-privilege roles, avoid static credentials, and use private connectivity where required. Cloud SQL IAM database authentication is available for PostgreSQL and MySQL, but not SQL Server.

  • Cost: Spanner costs depend on the instance configuration, processing capacity, storage, and network traffic. Cloud SQL costs include instance resources, storage, backups, network usage, and potentially SQL Server licensing.

Useful read-only inventory commands are gcloud spanner instances list --project=example-project and gcloud sql instances list --project=example-project. Backup state can be inspected with gcloud spanner backups list --instance=example-instance --project=example-project and gcloud sql backups list --instance=example-db --project=example-project.

When to Choose Each

Choose Cloud Spanner when global availability, elastic horizontal growth, strong consistency, and high write throughput are primary requirements. Choose Cloud SQL when compatibility with an existing database engine, predictable infrastructure-based sizing, or lower migration effort matters more.

Validate Spanner against the target workload using representative data volume, transaction contention, and multi-region latency. Do not select it solely from benchmark results: global writes, hot partitions, and retry behavior can materially change its operational profile.

30

When is Cloud Firestore the right database choice?

Google Cloud Firestore is a fully managed, NoSQL document database for storing and synchronizing application data at scale. It is a strong choice when applications need flexible document models, low-latency reads and writes, real-time updates, or offline-first mobile and web experiences.

Read full answer

Applies toGoogle Cloud Firestore (current managed service)

Core Concepts

  • Documents: Firestore stores records as JSON-like documents containing fields, nested maps, lists, and timestamps.

  • Collections and subcollections: Documents live in top-level collections and can contain subcollections. Subcollection IDs are unique only within their parent document.

  • Scalability: The service automatically distributes data and scales capacity, so teams do not manage servers, replicas, or database patching.

  • Client access:

  • Server applications typically access Firestore through client libraries.

  • Firebase client SDKs can connect directly to Firestore, providing real-time listeners and offline synchronization.

  • Modes:

  • Native mode is the standard choice for new document-oriented applications.

  • Datastore mode supports legacy Datastore applications and migration scenarios.

Firestore does not enforce joins, foreign keys, or arbitrary relational constraints. A document should generally represent a cohesive aggregate rather than a small portion of a heavily normalized schema.

Consistency and Querying

  • Reads of individual documents are strongly consistent.

  • Transactions provide serializable, strongly consistent access to multiple documents.

  • A regular multi-document query is not a database-wide snapshot, so applications must tolerate concurrent changes.

  • Firestore automatically indexes single fields, but many multi-field and collection-group queries require composite indexes.

  • Security rules are evaluated per document. A query must return only documents the caller is allowed to read; rules do not silently filter unauthorized documents from an otherwise authorized query.

Choosing the Right GCP Service

GCP service Best fit AWS/Azure approximation
Firestore Document data, mobile synchronization, real-time application state, serverless APIs Amazon DynamoDB or Azure Cosmos DB, though their data models differ
Cloud SQL Relational schemas, joins, mature SQL tooling, transactional reporting Amazon RDS/Aurora or Azure Database for PostgreSQL/MySQL
Spanner Large-scale relational data, global distribution, strong transactional guarantees No exact equivalent; DynamoDB has a different data model
Bigtable Massive key-value workloads and high-throughput ingestion Amazon DynamoDB or Azure Table Storage
BigQuery Analytics, large scans, historical reporting Amazon Redshift/Athena or Azure Synapse/Fabric

Choose Firestore for flexible application documents and user-facing synchronization. Choose Cloud SQL when relationships and SQL reporting dominate. Choose Spanner for globally distributed, strongly transactional relational workloads. Choose Bigtable for extreme key-value scale, and BigQuery for analytical processing.

Production Behavior

  • Locations: Firestore offers regional and multi-region locations. Select one deliberately because changing the location of an existing database is generally not supported. Multi-region provides wider geographic availability; regional locations can reduce latency and meet residency requirements.

  • Reliability: Firestore manages replication, failover, hardware maintenance, and service upgrades. Applications still need retry handling with exponential backoff for transient errors.

  • Backups: Configure and test the backup and point-in-time recovery features appropriate to the recovery objective. A backup is not useful until a restore has been exercised.

  • Monitoring: Track client latency, request errors, denied operations, quota consumption, database usage, and billing through Cloud Monitoring and Cloud Audit Logs.

  • Scaling limits: A single frequently updated document can become a contention point. Partition hot data across multiple documents or move the workload to Bigtable when its scale characteristics differ.

  • Diagnostics: Inspect the active database with gcloud firestore databases describe --database="(default)" --project="example-project".

Security, Reliability, and Cost Best Practices

  • Use IAM for service-to-service access and Firestore Security Rules for direct client access. Privileged server SDKs generally use IAM and bypass end-user rules.

  • Deny access by default, test rules against representative requests, and never store credentials or sensitive secrets inside documents.

  • Place sensitive fields behind server-side APIs when direct client access is unnecessary.

  • Keep documents reasonably sized and avoid unbounded lists. Firestore charges for reads, writes, storage, and network transfer, so document fan-out and chatty queries can increase cost sharply.

  • Use transactions for related invariants, but do not place large workloads inside long-running transactions.

  • Set alerts for elevated errors, latency, quota pressure, and unusual read or write volume. Review quota limits before traffic events or major launches.

31

What is Cloud Bigtable and which workloads suit it?

Cloud Bigtable is Google Cloud’s managed, NoSQL wide-column database for very large tables that require low-latency reads and high-throughput writes. It implements the original Google Bigtable data model and is suited to applications with a well-understood access pattern rather than ad hoc relational or document queries.

Read full answer

Applies toGoogle Cloud Bigtable current GA (Data API, HBase API, and emulator)

Architecture and Capabilities

  • Data model: A table contains rows identified by row keys, column families, column qualifiers, and timestamped cells. Cell versions remain until a garbage-collection policy removes them; scans expose row-key order, not insertion order.

  • Transactions: Bigtable provides strongly consistent reads and writes, single-row atomicity, and conditional mutations. It does not provide multi-row or cross-table transactions.

  • Scale and availability: Tables are split into tablets and automatically rebalanced. Three-way replication provides node-level availability, while multi-region instances can place replica clusters in different regions.

  • Interfaces: The primary Data API supports gRPC and REST. Bigtable also offers the HBase API, language-specific client libraries, Change Streams, authorized views, and a local emulator for development and tests.

  • Query limitations: There is no general-purpose SQL layer or arbitrary native secondary-index mechanism. Supported index-shaped access patterns can use materialized views; other cases require a deliberately designed reverse table.

Suitable and Unsuitable Workloads

  • Telemetry and time series: Device or monitoring data can be partitioned by entity and time, with wide rows and garbage-collection policies controlling retention.

  • Event pipelines: Pub/Sub can deliver events into Bigtable for durable serving or downstream processing. Ordering must be handled explicitly because Bigtable does not preserve Pub/Sub ordering by itself.

  • Feature serving: Recommendation, personalization, and ad-tech systems benefit from predictable key lookups and horizontal scale.

  • User activity and operational logs: High-cardinality activity data fits the wide-column model when retention and query patterns are planned.

  • Poor fit: Use Cloud SQL for relational OLTP, Spanner for globally distributed transactional workloads, BigQuery for analytical scans, and Firestore for flexible documents and synchronization. Use Cloud Storage lifecycle tiers for inexpensive long-term archival data.

Comparison With Other Cloud Data Stores

Primary need Google Cloud AWS Azure
Very large keyed tables Bigtable DynamoDB Cosmos DB Table API
Document store and mobile synchronization Firestore DynamoDB Cosmos DB
Relational OLTP Cloud SQL RDS or Aurora Azure Database for PostgreSQL
Serverless analytical scans BigQuery Redshift Serverless or Athena Fabric or Synapse serverless

Production Operations

  • Capacity: Use provisioned nodes for predictable load and autoscaling for bursty demand. Monitor sustained utilization rather than reacting to isolated spikes.

  • Key distribution: Avoid hot tablets caused by sequential or skewed keys. Prefixing or salting can spread writes, although it may sacrifice convenient range ordering and must be designed consistently.

  • Backups: Scheduled backups create point-in-time copies that can be restored to a new instance. Define the recovery point objective from the schedule, retain backups for the required period, and rehearse restores.

  • Monitoring: Track request latency, error rate, node utilization and count, backup status, and replication health. Alert on symptoms that affect users rather than relying only on infrastructure metrics.

  • Upgrades and incidents: Google manages the service’s infrastructure and rolling upgrades, but clients should still use bounded exponential backoff for transient UNAVAILABLE and RESOURCE_EXHAUSTED responses. A regional instance is vulnerable to a regional outage; use a multi-region instance with replica clusters and an appropriate routing policy, such as ANY_CLUSTER, when regional fault tolerance is required.

Security, Cost, and Pitfalls

  • IAM: Use the narrowest supported scope with predefined roles such as roles/bigtable.reader, roles/bigtable.user, and roles/bigtable.admin. Authorized views can expose only selected rows and columns.

  • Encryption and controls: Encryption at rest is enabled by default, with Cloud KMS customer-managed encryption keys available for applicable requirements. VPC Service Controls adds exfiltration protection; residency must still be enforced through instance, replica, and backup-location policies.

  • Cost: Provisioned node capacity and stored data are the principal cost drivers. Compare SSD and HDD capacity for the workload, use autoscaling carefully, and review idle tables, backups, and unused replicas.

  • Quotas and failure handling: Excessive load can produce throttling or increased latency rather than silent data loss. Apply backpressure, cap retries, and avoid full-table scans in latency-sensitive request paths.

32

What is Google Cloud Memorystore?

Google Cloud Memorystore is a family of managed, in-memory, Redis-compatible data stores for low-latency caching, session storage, leaderboards, and other application state. Google Cloud operates the infrastructure and control plane; applications connect with Redis or Valkey clients, depending on the Memorystore offering.

Read full answer

Applies toGoogle Cloud Memorystore current offerings: Memorystore for Redis and Redis Cluster (Redis 7.x), and Memorystore for Valkey (Valkey 8.x)

Core Capabilities

  • Compatibility: Supports standard Redis clients, common commands, and data structures such as strings, hashes, lists, sets, and sorted sets.

  • Separate offerings: Memorystore for Redis is a regional, single-primary service with an optional standby replica. Memorystore for Redis Cluster is a sharded, multi-primary service with replicas and automatic failover. Memorystore for Valkey is a separate Valkey-based offering with its own tier, capacity, and regional feature matrix. One instance cannot switch between Redis and Valkey.

  • High availability: Replicas can be placed across zones within a region, with managed failover when supported by the selected offering.

  • Scaling: Cluster-based offerings support scaling capacity and replicas, subject to quotas and regional availability.

  • Managed operations: Google provisions and monitors the service, performs infrastructure maintenance, and integrates with Cloud Monitoring.

  • Private networking: Applications connect through supported private VPC or Private Service Connect paths rather than a public internet endpoint.

Memorystore is usually a cache or ephemeral application tier. It should not automatically be treated as the authoritative system of record.

How It Works in Production

A common cache-aside request path is:

  1. The application reads from Memorystore.

  2. A cache hit returns the stored value.

  3. On a miss, the application queries Cloud SQL, Cloud Spanner, Firestore, or another source of truth.

  4. The application caches the result and returns it to the client.

Replication improves availability, but it is not equivalent to a backup. Some offerings provide snapshots or backup and restore, but persistence, retention, and recovery behavior differ by product and tier. Teams should verify recovery-point and recovery-time objectives and test restores rather than assuming zero data loss.

GCP and Cloud Comparisons

Requirement Google Cloud AWS equivalent Azure equivalent
Managed in-memory key-value store Memorystore ElastiCache Azure Managed Redis
Durable relational database Cloud SQL RDS or Aurora Azure Database for PostgreSQL or MySQL
Globally distributed transactional database Cloud Spanner DynamoDB or Global Aurora Cosmos DB
Document data model Firestore DynamoDB Cosmos DB
Wide-column NoSQL data Bigtable DynamoDB Cosmos DB for NoSQL

Choose Memorystore when predictable low-latency access is the priority. Choose Cloud SQL or Spanner for transactional durability, and Firestore or Bigtable when their document or wide-column data models fit the workload.

Failure Modes and Security

  • Memory exhaustion: Depending on the configured memory and eviction behavior, writes may be rejected or cached entries may be evicted.

  • Failover: Replication can be asynchronous, so recent writes may be lost during a failover. Clients also need bounded reconnect and backoff logic.

  • Regional failure: Replicas within one region do not by themselves provide regional disaster recovery; use backups and a separate recovery design where required.

  • Connection storms: A recovering service can be overwhelmed by simultaneous reconnects. Use connection pooling, jitter, and exponential backoff.

  • Administrative commands: Some administrative operations can block or change server behavior. Use only the controls documented for the selected offering; do not assume that upstream Redis ACL features are available.

  • Network exposure: Keep endpoints on private networking and restrict access to application workloads.

  • Identity and credentials: Use IAM for GCP resource administration and the offering’s documented authentication mechanism for data access. Keep credentials in Secret Manager rather than source code.

  • Encryption: Confirm the encryption and customer-managed key options applicable to the selected offering; they are not uniform across Memorystore products.

Best Practices

  • Use explicit TTLs for cache entries and select an eviction policy only where the offering supports it.

  • Monitor memory utilization, evictions or rejected writes, CPU, command latency, connections, replica health, and failover events in Cloud Monitoring.

  • Test failover, scaling, network interruption, restore, and regional recovery through failure drills.

  • Maintain capacity headroom for failover and traffic spikes.

  • Review quotas, cross-zone or data-transfer charges, replica costs, and budget alerts before provisioning.

  • Review managed release notes, maintenance windows, client compatibility, and major-version behavior before production upgrades.

33

How does BigQuery differ from a traditional data warehouse?

BigQuery is Google Cloud’s fully managed, serverless data warehouse and analytics platform. Unlike a traditional warehouse, it separates storage from compute and automatically allocates query resources, reducing cluster administration, capacity planning effort, and infrastructure maintenance.

Read full answer

Applies toGoogle Cloud BigQuery (current service version; Standard, Enterprise, and Enterprise Plus editions)

How BigQuery Works

BigQuery stores data in Google Cloud’s managed infrastructure using a columnar format optimized for analytical queries. It uses distributed query execution to scan large datasets in parallel.

Its main architectural characteristic is separated storage and compute:

  • Managed storage: Table data is durably stored and scaled by Google Cloud.

  • Independent compute: Query processing uses temporary compute resources that can scale independently of data volume.

  • Elastic execution: Large scans can run across many workers without provisioning warehouse nodes.

  • Slot-based compute: In capacity-based configurations, a slot represents approximately 1 vCPU and 1 GB of memory. Capacity can be purchased on demand or managed through reservations and autoscaling.

Unlike a cluster-based warehouse, a BigQuery job can still consume substantial compute while waiting for resources. Production teams therefore need workload management, query prioritization, and cost monitoring.

Comparison With Other Warehouses

Area BigQuery Traditional data warehouse Amazon Redshift Azure Synapse Analytics
Infrastructure Serverless; no cluster administration Nodes, storage, and software managed by the customer Provisioned clusters or managed serverless capacity Dedicated pools or serverless SQL
Storage and compute Independently managed Usually closely coupled Usually cluster-oriented, with serverless options Dedicated or serverless compute models
Scaling Automatically scales query execution Commonly requires adding nodes or reconfiguring a cluster Requires scaling clusters or serverless capacity Requires managing pools or serverless capacity
Pricing Query processing, storage, or reserved capacity Primarily provisioned infrastructure and storage Cluster capacity, serverless usage, or reservations Compute, storage, and licensing by workload
Administration Little infrastructure administration Significant operational responsibility Moderate to low, depending on the model Moderate to low, depending on the model
Query characteristics Large scans, SQL analytics, streaming, and data lake workloads Broad SQL analytics and reporting SQL analytics and data lake integration SQL analytics, data lake, and Spark workloads

BigQuery is conceptually closer to a managed cloud warehouse such as Redshift Serverless or the serverless SQL pool in Azure Synapse than to a self-hosted database. Its differentiator is the depth of serverless integration and automatic separation of compute from storage.

Data Ingestion and Ecosystem

BigQuery supports several ingestion patterns:

  • Batch loads: Data is imported from Google Cloud Storage, other cloud storage, or supported files.

  • Streaming inserts: New rows can be inserted continuously, commonly through the BigQuery Storage Write API.

  • Pub/Sub integration: A Pub/Sub subscription can write records into BigQuery through the Storage Write API.

  • Dataflow pipelines: Dataflow provides managed batch and streaming pipelines for transformations and BigQuery writes.

  • External and federated queries: BigQuery can query supported data in Cloud Storage and perform cross-cloud or external-data federation with appropriate configuration.

For high-volume ingestion, the Storage Write API is generally more operationally appropriate than issuing individual streaming-insert requests. A common production architecture uses Pub/Sub for event delivery, Dataflow for validation and transformation, and BigQuery for analytical storage.

Production Operations

BigQuery manages infrastructure concerns such as node failures, storage expansion, and software upgrades, but the underlying service can still experience outages and capacity constraints. Application and workload failures also remain possible.

  • Query failures: Queries can fail because of invalid SQL, resource contention, exceeded limits, quota violations, or upstream data problems.

  • Long-running queries: Monitor elapsed time, bytes processed, slot usage, and reservation contention.

  • Observability: Use Cloud Monitoring, Cloud Logging, BigQuery audit logs, INFORMATION_SCHEMA, and job metadata.

  • Cost control: Partition large tables, use clustering selectively, limit unnecessary columns, and apply table or partition expiration where retention permits.

  • Recovery: Time travel supports recovery of recently changed or deleted table data. Table snapshots provide explicit point-in-time recovery.

  • Regional resilience: Choose an appropriate region or multi-region, and design critical analytical workloads around service availability and recovery objectives.

  • Data lifecycle: Expiration policies reduce cost, but deleting expired data can also remove access needed for incident investigation or compliance.

Storage and compute are separated, but this does not eliminate capacity planning. Heavy concurrency can still require reserved capacity to provide predictable latency and cost.

Security and Governance

BigQuery supports Google Cloud IAM, dataset-level access, and predefined or custom roles. Use the least-privilege role that grants the required table and job permissions; avoid assigning broad project-level owner or editor roles to analytics users.

Key controls include:

  • IAM authorization: Grant roles/bigquery.dataViewer or more specific permissions only where required.

  • Column and row protection: Use authorized views, row-level security policies, policy tags, and dynamic data masking where supported.

  • Encryption: Google Cloud encrypts data at rest and in transit; customer-managed encryption keys can be used where regulatory requirements justify them.

  • VPC Service Controls: Helps prevent data exfiltration around Google Cloud resources and APIs.

  • Auditing: BigQuery audit logs support investigations into data access and administrative activity.

  • Workload identity: Prefer attached service accounts over downloadable service-account keys.

For sensitive datasets, apply classification, retention, deletion, and access-review processes. IAM controls who may query data, but it does not replace data-quality validation or governance of downstream exports.

When to Choose BigQuery

Choose BigQuery when the workload involves large-scale SQL analytics, variable demand, ad hoc exploration, reporting, streaming ingestion, or querying data in Cloud Storage.

Do not treat it as an OLTP database: high-frequency transactional reads and writes usually fit Cloud SQL, AlloyDB, or Spanner better. Choose a traditional or provisioned cloud warehouse when sustained workloads require fixed low-latency compute, highly specialized tuning controls, or contractual capacity predictability. On Google Cloud, BigQuery is usually the strongest default for analytical workloads unless those operational requirements point elsewhere.

34

How does BigQuery pricing work?

BigQuery pricing combines query-processing charges with storage and optional feature charges. Standard and Enterprise editions primarily charge per TiB scanned, while Capacity editions charge for baseline and autoscaled slot capacity.

Read full answer

Applies toGoogle BigQuery Standard, Enterprise, and Capacity editions (current pricing)

Pricing Models

Model Primary billing meter Best suited for
Standard on-demand Bytes processed; at current published rates, the first 1 TiB per month is free per billing account per region, then $6.25 per TiB Intermittent or unpredictable analytics
Enterprise on-demand Bytes processed at $10 per TiB, with no monthly free query-processing allowance Workloads requiring Enterprise features
Capacity Baseline capacity and autoscaled slots, priced by slot time rather than bytes processed Stable, sustained, or high-concurrency analytics

On-demand queries are charged for the data examined, not the number of rows returned. A query that returns no rows can still incur charges if it examines billable data. Selecting fewer columns and applying partition filters usually reduces the bytes processed.

Capacity editions provide committed baseline slots and additional autoscaling for temporary demand. They avoid per-TiB on-demand query charges, but baseline overprovisioning wastes money and excessive autoscaling can materially increase the bill.

Other Charge Dimensions

  • Storage: Billed per GiB-month. Active, Long-term, and Archival logical storage have different rates and minimum-duration implications; lifecycle policies can reduce storage cost.

  • Streaming ingestion: BigQuery Storage Write API and streaming inserts are charged separately from query processing, based on the applicable row and volume meters.

  • Data transfer: Transfers outside Google Cloud can incur standard network charges. Separately metered transfer and analytics products may also charge.

  • Additional features: BigQuery ML operations and BI Engine capacity can incur charges beyond ordinary query processing.

  • Scheduled and materialized work: Scheduled queries and materialized-view refreshes consume the same underlying query-processing resources as interactive queries. Materialized-view storage may also incur storage charges.

Reducing Query Costs

  1. Filter partitions: Use a valid partition filter so BigQuery can prune irrelevant partitions.

  2. Use clustering selectively: Cluster on frequently filtered or joined columns so BigQuery can skip unnecessary blocks. Clustering complements rather than replaces partition pruning.

  3. Select required columns: Avoid SELECT *; BigQuery’s columnar storage means unreferenced columns generally need not be read.

  4. Reuse results: Materialized views, destination tables, and scheduled aggregates avoid repeatedly computing the same result, although their refreshes and storage are billable.

  5. Use approximate functions: APPROX_COUNT_DISTINCT is substantially cheaper than exact COUNT(DISTINCT ...), but returns an approximate result.

  6. Set a byte ceiling: maximum_bytes_billed prevents an on-demand job from processing beyond a configured limit.

  7. Control wildcard tables: Explicitly constrain wildcard ranges and use dry-run estimates to detect unexpectedly broad scans. A dry run estimates processed bytes but does not display a resolved partition list.

  8. Match capacity to demand: Use on-demand pricing for spiky workloads and Capacity editions for sustained demand. Monitor utilization before adding baseline slots.

AWS and Azure Equivalents

Requirement Google Cloud AWS Azure
Serverless, pay-per-query analytics BigQuery on-demand Amazon Athena Fabric Warehouse or Synapse serverless
Managed reserved analytics compute BigQuery Capacity Amazon Redshift Fabric capacity or Synapse dedicated pools

Choose BigQuery for a fully managed warehouse spanning Google Cloud data, Athena for S3-native ad hoc analysis, Redshift for more provisioned warehouse control, and Fabric or Synapse where Microsoft-oriented tooling and licensing are priorities.

Production Guardrails and Pitfalls

  • Monitor total_bytes_processed, total_bytes_billed, query duration, and total_slot_ms in INFORMATION_SCHEMA.JOBS.

  • Export detailed Cloud Billing data to BigQuery and attribute charges using project, service, and label dimensions. Queries against that export can themselves incur BigQuery processing charges.

  • Configure billing budgets and alerts, but do not treat alerts as automatic spending controls.

  • Apply least-privilege IAM and separate projects or teams where stronger cost and data boundaries are required.

  • Do not assume logical views reduce cost because queries against views can still examine the underlying tables.

  • Use dry runs and EXPLAIN to validate pruning before production rollout, while retaining byte caps because estimates can differ from actual processing.

35

How do partitioned and clustered tables work in BigQuery?

Partitioned tables divide data into prunable units, while clustered tables organize data by frequently filtered columns. BigQuery supports both independently and can combine them to narrow a scan first by partition and then by data block.

Read full answer

Applies toGoogle Cloud BigQuery (current service; GoogleSQL)

Partitioning in BigQuery

Partitioning splits a logical table into separate partitions. When a query includes a filter on the partition key, BigQuery can use partition pruning to avoid reading irrelevant partitions.

The supported partitioning approaches include:

  • Date/time column partitioning: Partitions by a DATE, TIMESTAMP, or DATETIME column, or by a supported expression derived from one. The available partition granularity depends on the source type.

  • Ingestion-time partitioning: Uses the time when data was ingested. Daily ingestion-time tables expose _PARTITIONDATE, while _PARTITIONTIME identifies the exact ingestion-time partition boundary.

  • Integer-range partitioning: Assigns integer values to user-defined ranges and is useful for numeric bucketing.

Pruning is most effective when a query filters directly on the partitioning column or on a filter that BigQuery can map to the partition boundary. Applying an unsuitable wrapper function can reduce pruning effectiveness. Partition granularity also affects storage: excessive granularity creates many small partitions and metadata overhead.

For eligible time-based partitions, require_partition_filter = TRUE prevents queries that omit a partition filter. It enforces the presence of a filter but does not by itself guarantee that the filter prunes data efficiently. The partition_expiration_days option can delete old partition data automatically, subject to retention and recovery requirements.

Clustering in BigQuery

Clustering organizes table data according to one to four clustering columns. The first column has the highest precedence, followed by the remaining columns in their declared order.

BigQuery stores clustering metadata with its data blocks. When query predicates apply to clustering columns, BigQuery can skip blocks whose stored value ranges cannot satisfy the predicate. Clustering therefore works best for common equality or range filters, such as customer_id, tenant_id, or status.

Clustering has several important limitations:

  • It is not a traditional B-tree index.

  • It does not enforce uniqueness.

  • It does not guarantee a global or query-result row order.

  • Its effectiveness depends on query patterns, data distribution, and block pruning.

  • It does not replace partitioning when the primary requirement is time-based filtering, pruning, or retention.

Partitioning Compared With Clustering

Aspect Partitioning Clustering
Purpose Divide a table into independently prunable units Organize data by frequently filtered columns
Data model Logical partitions based on time, ingestion time, or integer ranges Column-oriented sort order represented through data blocks and metadata
Primary benefit Time-based pruning and lifecycle management Block pruning within the data selected by other filters
Typical filters Partition dates, timestamp ranges, or integer-range boundaries Equality and range predicates on business columns
Retention Supports partition expiration for eligible time-based partitions Does not provide an independent retention mechanism
Relationship to other feature Can be clustered within each partition Can be combined with partitioning
Main limitation Poor granularity or filters that do not prune Unhelpful columns or highly skewed query patterns

When both are used, BigQuery can first exclude whole partitions and then apply clustering metadata to the remaining data. A partition by event date combined with clustering by customer and event type is a common design, although the clustering columns should be validated against actual query statistics.

Production and Operational Considerations

  • Cost: Query charges depend on bytes processed or capacity consumption, while storage has separate charges. Effective pruning reduces unnecessary scanning, but excessively granular partitions can increase metadata and management overhead.

  • Ingestion: Clustering can add data-organization work during writes, so representative batch and streaming loads should be benchmarked. Clustering does not impose a separate clustering charge.

  • Monitoring: Track bytes scanned, slot time, execution latency, ingestion failures, partition counts, and pruning behavior through Cloud Monitoring and BigQuery metadata or system views.

  • Schema and workload changes: Reassess partition granularity and clustering columns as data volume, retention needs, and common predicates change. Removing clustering columns can make subsequent writes cheaper because the table no longer needs that organization.

  • Security: Apply IAM controls at the project and dataset levels. Partitioning and clustering are storage-organization features, not access-control boundaries.

  • Recovery: Partition expiration is not a backup mechanism. Use time travel or table snapshots for point-in-time recovery, copy data to durable storage for longer-term retention, and configure cross-region disaster recovery when regional failover is required.

Best Practices and Common Pitfalls

  • Choose a partition column that nearly every production query filters on.

  • Match partition granularity to the workload; avoid creating tiny partitions unless the operational benefit justifies them.

  • Prefer direct predicates on the partition column instead of filtering through wrapper functions.

  • Select clustering columns from observed query filters rather than assumptions.

  • Keep clustering-column data reasonably distributed and avoid columns with severe skew.

  • Do not rely on clustering for uniqueness or result ordering.

  • Verify pruning with a dry run (estimated bytes processed) and the job’s execution details; BigQuery has no EXPLAIN statement.

  • Re-evaluate the design periodically as data volume, retention requirements, and query patterns change.

36

How does BigQuery ML enable machine learning?

BigQuery ML is Google Cloud’s SQL-native machine-learning service. It trains models from BigQuery data and performs inference through SQL, allowing analytical models to be built without first exporting warehouse data to a separate training platform.

Read full answer

Applies toGoogle Cloud BigQuery ML (all versions)

How It Works

  1. Train with SQL: CREATE MODEL selects an algorithm, identifies the label column, and reads training data from a table or query.

  2. Process the data: BigQuery plans and executes the training query using its managed compute infrastructure.

  3. Store the model: The resulting model and its metadata become BigQuery resources in a dataset.

  4. Evaluate and predict: Functions such as ML.EVALUATE, ML.PREDICT, and ML.EXPLAIN_PREDICTIONS expose evaluation and scoring through SQL.

For models trained directly from BigQuery tables, views, joins, or aggregates, data does not need to be exported to another machine-learning platform. This is particularly effective when the source is already structured as large warehouse tables.

Supported Modeling Approaches

BigQuery ML covers common analytical and tabular workloads:

  • Supervised learning: Linear regression, logistic regression, boosted trees, and deep neural networks.

  • Unsupervised learning: K-means clustering.

  • Recommendation systems: Matrix factorization and factorization machines.

  • Forecasting: Time-series models.

  • Evaluation: Metrics such as accuracy, precision, recall, F1 score, RMSE, and algorithm-specific measures, depending on the model type.

Typical uses include churn prediction, demand forecasting, fraud scoring, customer segmentation, and operational classification. Vertex AI is generally preferable when a workload requires custom training code, distributed deep learning, a feature store, a model registry, or a managed real-time endpoint.

Production Behavior

  • Retraining: BigQuery ML does not automatically retrain a model when its input data changes. Schedule controlled retraining with Cloud Scheduler, Workflows, or another orchestration layer.

  • Evaluation: Evaluate a candidate model on held-out data before promoting it. A newly trained model is not automatically better merely because it is newer.

  • Monitoring: Use Cloud Monitoring for BigQuery operational metrics and scheduled queries or dashboards for evaluation results, training cost, data freshness, drift, and prediction distributions.

  • Scoring: Native BigQuery ML predictions execute as part of a query. BigQuery ML itself does not expose a continuously running prediction endpoint.

  • Real-time serving: Supported BigQuery ML models can be exported to Cloud Storage and deployed through Vertex AI. That creates a separate Vertex AI serving path with its own scaling, latency, monitoring, and cost implications.

  • Cost control: Monitor bytes scanned, slot usage, training frequency, and inference-query size. A prediction over a large table can be expensive because it may scan substantial data.

  • Incident diagnosis: Unexpected predictions can result from stale training data, schema or concept drift, data leakage, changed feature transformations, or model degradation—not only from an infrastructure failure.

Service Comparison

Capability BigQuery ML Vertex AI Amazon SageMaker AI Azure Machine Learning
Primary strength SQL-native analytical ML Full ML and MLOps platform Broad AWS ML platform Enterprise Azure ML platform
Typical data source BigQuery tables and queries BigQuery, local, cloud, or streaming data S3 and other AWS sources Data Lake, Blob Storage, and Azure sources
Training interface SQL Python, notebooks, APIs, and SDKs Python, notebooks, and SDKs Python, notebooks, and SDKs
Real-time serving Not provided directly by BigQuery ML Managed endpoints and other serving options Managed endpoints Managed endpoints
Best fit Embedded warehouse predictions Custom models, deep learning, and MLOps Existing AWS-centric teams Existing Microsoft-centric teams

Production Best Practices

  • Prevent leakage: Exclude post-outcome fields and train only on observations available before the prediction period.

  • Validate feature types: Keep training and scoring columns compatible; schema changes can cause query failures or materially alter results.

  • Compare before replacement: Evaluate candidate models against the current model on the same validation population.

  • Version the pipeline: Store model-creation SQL, feature definitions, evaluation queries, and the training-data cutoff in source control. Use separate model names or reproducible model artifacts for rollback.

  • Apply least privilege: Grant training and inference identities only the required BigQuery dataset, model, and table permissions.

  • Automate promotion gates: Schedule retraining with evaluation thresholds, logging, failure alerts, and an explicit promotion or rollback step.

  • Monitor drift: Track input distributions, label rates, missing values, and prediction distributions even when model execution itself remains healthy.

37

How does Google Cloud Pub/Sub deliver messages?

Google Cloud Pub/Sub delivers messages asynchronously through topics and subscriptions: publishers send messages to a topic, and each attached subscription receives its own copy for independent pull or push processing. Delivery is generally at least once, so consumers must handle duplicates and acknowledge messages only after successful processing.

Read full answer

Applies toGoogle Cloud Pub/Sub (current service)

How It Works

  1. A producer publishes a message to a topic. The publisher client may batch messages before sending them to the Pub/Sub service.

  2. Pub/Sub makes the message available to every subscription attached to that topic. Each subscription tracks its own acknowledgement state, while multiple consumers attached to one subscription share the messages as competing consumers.

  3. A subscriber receives messages through pull by calling Pub/Sub, or through push when Pub/Sub sends HTTPS POST requests to a configured endpoint.

  4. After successful processing, the subscriber acknowledges the message. A pull subscriber does this with an acknowledgement request; for push delivery, Pub/Sub treats HTTP 102 and 2xx responses as success.

  5. If a message is negatively acknowledged or its acknowledgement deadline expires, Pub/Sub redelivers it. Unacknowledged messages are retained for a bounded period, currently up to seven days, rather than indefinitely.

A message can contain payload data, attributes, a server-assigned message ID, a publish timestamp, and an optional ordering key. A subscription can also have a dead-letter topic. After the subscription’s configured maximum number of failed delivery attempts, the message is forwarded to that topic for investigation and controlled replay; without a dead-letter policy, it is dropped.

Delivery Semantics

  • At-least-once delivery: A message can be delivered more than once if processing succeeds but the acknowledgement is lost, or if the subscriber fails before acknowledging it.

  • Idempotent processing: Consumers should make repeated handling safe, commonly by enforcing a unique constraint on the subscription ID plus message ID or an application event ID.

  • Exactly-once delivery: Pub/Sub supports this only for pull subscriptions with the required publisher and subscription opt-in. It prevents Pub/Sub from redelivering a successfully acknowledged message, but it does not make external side effects atomic with the acknowledgement, so idempotency remains necessary.

  • Ordered delivery: When the publisher client enables message ordering, messages with the same ordering key are delivered in publish order within a subscription. Different keys may be processed concurrently, and duplicates can still occur.

  • Pull delivery: The subscriber controls polling and concurrency and should acknowledge or negatively acknowledge each message.

  • Push delivery: Pub/Sub retries unsuccessful requests with exponential backoff until the delivery-attempt limit is reached. Authenticated push can use OIDC tokens to authenticate the Pub/Sub service to the endpoint.

  • Payload limit: A message can be up to 10 MB. Larger data should normally be stored in Cloud Storage and referenced from the message.

Service Comparisons

Delivery pattern Google Cloud AWS Azure
Event fan-out Pub/Sub topic and subscriptions SNS topics with SQS, Lambda, or HTTPS consumers Event Grid topics
Queue-style decoupling Pub/Sub subscription with competing consumers SQS standard or FIFO queues Service Bus queues
High-throughput streaming Pub/Sub Kinesis Data Streams or MSK Event Hubs
Serverless consumption Cloud Run, Cloud Functions, or GKE Lambda, ECS, or EKS Functions, Container Apps, or AKS

Pub/Sub is particularly suitable when one event must reach several independent consumers or when producers and consumers operate on different schedules.

Security and Operations

  • Grant narrowly scoped predefined roles such as roles/pubsub.publisher and roles/pubsub.subscriber, and keep publisher and subscriber identities separate.

  • A dead-letter topic requires the Pub/Sub service agent to have permission to publish to it, commonly through roles/pubsub.publisher.

  • Use authenticated push with narrowly scoped audiences, and validate incoming webhook requests at the receiving service.

  • Pub/Sub encrypts data in transit and at rest by default; use customer-managed encryption keys when organizational policy requires them.

  • Monitor pubsub/subscription/num_undelivered_messages, pubsub/subscription/oldest_unacked_message_age, and pubsub/subscription/num_push_requests_dropped. Instrument publisher-side errors separately and use logs for endpoint failures.

  • Review the topic’s message-storage policy, regional requirements, quotas, and expected traffic spikes. Pub/Sub scales automatically, but publishers and subscribers can still encounter throttling under extreme load.

Production Best Practices

  • Acknowledge only after the required durable processing has succeeded, and use lease management for long-running handlers.

  • Make handlers idempotent because redelivery and duplicate processing are normal operational realities.

  • Configure bounded retries and dead-letter handling for poison messages; inspect and validate dead-letter messages before replaying them.

  • Version event schemas, publish schema versions as attributes, and reject incompatible payloads without crashing the entire consumer.

  • Keep payloads small and store larger objects in Cloud Storage.

  • Monitor backlog age, delivery failures, and dead-letter volume rather than throughput alone.

  • Test dependency outages, expired acknowledgements, malformed messages, throttling, and dead-letter replay procedures before production incidents occur.

38

What is the difference between push and pull Pub/Sub subscriptions?

A push subscription lets Pub/Sub deliver messages to an HTTPS endpoint, while a pull subscription requires the subscriber to retrieve messages from Pub/Sub. By default, both provide at-least-once delivery, so subscribers must tolerate duplicates and acknowledge messages only after successful processing.

Read full answer

Applies toGoogle Cloud Pub/Sub (all current versions)

How Delivery Works

With a push subscription:

  1. Pub/Sub sends an HTTPS POST request to the configured push endpoint.

  2. The request contains the message data, attributes, and delivery metadata.

  3. The endpoint processes the message and returns a successful HTTP status.

  4. Pub/Sub treats a 2xx response as an acknowledgment; failed or timed-out requests are retried.

With a pull subscription:

  1. The subscriber connects to Pub/Sub using the Pull or StreamingPull API.

  2. Pub/Sub returns available messages and assigns them an acknowledgment deadline.

  3. The subscriber processes each message and explicitly acknowledges it.

  4. Unacknowledged messages become eligible for redelivery when their lease expires.

Push Versus Pull

Area Push subscription Pull subscription
Delivery initiation Pub/Sub initiates the request Subscriber initiates the request
Typical latency Low latency without waiting for the subscriber to poll StreamingPull provides low latency; synchronous Pull depends on client behavior
Scale control Pub/Sub initiates and retries requests; the endpoint must scale for received and redelivered traffic The subscriber controls concurrency and worker count, subject to subscription and project quotas
Network exposure Requires a reachable HTTPS endpoint, usually protected with OIDC authentication Requires outbound access to Pub/Sub; no public application endpoint is needed
Processing control The endpoint controls acknowledgment through its HTTP response The client controls acknowledgment and lease extension
Infrastructure Can use Cloud Run, GKE, or another HTTPS service Usually uses continuously running workers on GKE, Compute Engine, or Cloud Run
Cost profile Can reduce idle compute with serverless scaling, but delivery attempts and endpoint traffic still cost money Keeps subscriber infrastructure running, even when the backlog is empty

Delivery Semantics and Failure Modes

  • At-least-once delivery by default: Messages can be delivered more than once. A duplicate can occur when processing succeeds but the acknowledgment is lost, or when a subscriber fails after performing side effects.

  • Exactly-once delivery: Pull subscriptions can enable Pub/Sub exactly-once delivery, which prevents duplicate delivery by the service during the subscription’s message-retention period. It does not make arbitrary external side effects exactly-once, so handlers should remain idempotent.

  • Acknowledgment deadline: Both subscription types use an acknowledgment deadline. Pull subscribers can extend it with ModifyAckDeadline; push endpoints must return 2xx, and non-success responses or timeouts cause redelivery under the retry policy.

  • Redelivery: Repeated processing failures increase redelivery attempts. Configure a dead-letter topic when messages repeatedly fail and need separate investigation.

  • Ordering: When ordering is enabled, messages published with the same ordering key are delivered in order. Duplicate or retried messages must still be handled idempotently.

  • External side effects: A database transaction can make a database write idempotent, but it does not automatically make an entire external workflow exactly-once.

Security and Operational Trade-offs

  • Push security: Use authenticated push with an OIDC token. The receiving service must validate the token’s signature, issuer, audience, and expected service account. Do not expose an unauthenticated public endpoint or treat URL secrecy as authentication.

  • Pull security: Grant the workload’s identity the roles/pubsub.subscriber role. Workload Identity on GKE or attached service identities are preferable to downloaded service-account keys.

  • Monitoring: Track undelivered messages, oldest unacknowledged age, redeliveries, acknowledgment-deadline expirations, push response codes, and endpoint latency.

  • Backpressure: A slow push endpoint can accumulate retries, while a slow pull consumer can accumulate an unacknowledged backlog. Set retention, dead-letter, and alerting policies based on business recovery requirements.

  • Capacity planning: Test autoscaling and Pub/Sub quotas with realistic burst traffic rather than steady-state averages.

Best Practices

  • Make handlers idempotent, using the Pub/Sub message ID or an application-level deduplication key.

  • Acknowledge only after the message has been durably processed.

  • Extend the acknowledgment deadline for long-running work.

  • Return a 2xx response only when the push endpoint considers processing successful.

  • Use separate dead-letter topics and alerts for poison messages.

  • Choose push for serverless endpoints and naturally bursty workloads; choose pull when the subscriber needs explicit concurrency control, private networking, or long-lived processing.

39

What delivery and ordering guarantees does Pub/Sub provide?

Google Cloud Pub/Sub provides at-least-once delivery by default and offers best-effort ordering within an ordering key for subscriptions with message ordering enabled. Exactly-once delivery is also available for pull subscribers, but it does not replace idempotent business logic or provide a global ordering guarantee.

Read full answer

Applies toGoogle Cloud Pub/Sub (current service; the guarantees described apply to all versions)

Delivery Semantics

When a subscriber receives a message, Pub/Sub starts an acknowledgement deadline. The subscriber must acknowledge the message after successfully processing it.

  • Acknowledged: Pub/Sub considers delivery complete.

  • Not acknowledged: The acknowledgement deadline expires and Pub/Sub redelivers the message.

  • Duplicate delivery: Possible by default because a subscriber may finish processing but crash before sending the acknowledgement.

  • Message retention: Unacknowledged messages are retained according to the subscription’s message-retention configuration, subject to Pub/Sub quotas and service availability.

Exactly-once delivery can reduce redeliveries after a successful acknowledgement, but it does not guarantee exactly-once effects in an external system. For example, a subscriber could update a database and then crash before acknowledging the message. The message might be processed again, so the database update should still use an idempotency key or a transactional design.

Ordering Semantics

Pub/Sub ordering depends on an ordering key assigned by the publisher. Messages published with the same key are intended to be delivered in order to a subscription with message ordering enabled.

Important limitations include:

  • Ordering applies within an ordering key, not across different keys.

  • Messages without an ordering key have no meaningful ordering guarantee.

  • Ordering is regional; messages published or consumed in different regions should not be treated as globally ordered.

  • A subscriber must process messages for the same key sequentially. Concurrent processing can defeat application-level ordering.

  • If a subscriber fails or an acknowledgement deadline expires, redelivery and retries can complicate the sequence. Therefore the guarantee is best effort rather than an absolute, failure-free ordering contract.

  • Exactly-once delivery and ordering are independent features that can be enabled together: exactly-once is supported by pull subscribers only, and because successfully acknowledged messages are not redelivered, it reinforces rather than weakens the per-key order.

A common design is to use a customer ID, account ID, or entity ID as the ordering key. This keeps events for one entity ordered while allowing different entities to be processed in parallel.

Production Failure Modes

The most important operational failure mode is choosing the wrong acknowledgement timing:

  • Acknowledge too early: The message may be removed from the subscription before processing completes, causing data loss.

  • Acknowledge too late: The message may be delivered again, causing duplicate processing.

  • Poison messages: A message that repeatedly fails can block progress for its ordering key. Configure a dead-letter topic and maximum delivery attempts where appropriate.

  • Consumer outages: Messages accumulate in the subscription and are delivered after consumers recover, within retention and quota limits.

  • Incorrect scaling: Adding consumers does not mean events for the same ordering key are processed concurrently. Use key-aware partitioning or a single logical consumer per key.

Comparison With AWS and Azure

Requirement Google Cloud Pub/Sub AWS Azure
Default delivery At least once SNS/SQS Standard at least once Service Bus at least once
Keyed ordered processing Ordering keys with ordering enabled SQS FIFO MessageGroupId Service Bus sessions
Exactly-once processing option Available for pull subscribers; still requires idempotency Usually implemented with deduplication or idempotent consumers Usually implemented with idempotent or transactional processing

Choose Pub/Sub when the application needs managed publish/subscribe routing, multiple independent subscriptions, key-based serialization, and regional Google Cloud integration. Use SQS FIFO or Service Bus sessions when a platform team specifically needs their queue semantics or stronger operational integration with AWS or Azure.

Best Practices and Monitoring

  • Enable ordering per subscription with gcloud pubsub subscriptions create --enable-message-ordering, and publish keyed messages with gcloud pubsub topics publish --ordering-key.

  • Use idempotent handlers even when exactly-once delivery is enabled.

  • Acknowledge only after the business transaction has committed.

  • Monitor undelivered messages, oldest unacknowledged-message age, redeliveries, acknowledgement-deadline expirations, and dead-letter counts.

  • Alert on sustained backlog growth or increasing message age rather than only on message count.

  • Grant publishers only permission to publish, and grant subscribers only the permissions required to consume and acknowledge.

  • Use a dead-letter topic for poison-message handling, and test recovery procedures before production traffic reaches the failure path.

40

How does Google Cloud Dataflow relate to Apache Beam?

Google Cloud Dataflow is Google Cloud’s managed service for running large-scale batch and streaming data-processing pipelines. Apache Beam is the open-source, multi-language SDK and programming model used to define those pipelines; Dataflow is one of Beam’s execution runners.

Read full answer

Applies toGoogle Cloud Dataflow (current service) and Apache Beam 2.x SDKs

Core Relationship

The relationship is similar to using portable SQL with different execution engines:

  • Apache Beam defines transformation logic using constructs such as PCollection, transforms, windows, triggers, and side inputs.

  • Beam runner executes the pipeline on a processing platform.

  • Dataflow Runner is Google Cloud’s implementation of that runner.

  • The same Beam pipeline can potentially run on other runners, including Flink, Spark, or the local runner, although production features and operational behavior differ.

Beam supports Java, Python, and Go. A typical production pipeline therefore consists of Beam SDK code, runner-specific options, and deployment through gcloud or CI/CD.

How Dataflow Executes a Pipeline

  1. The launcher submits the Beam pipeline and its options to Dataflow.

  2. Beam constructs a logical processing graph from the SDK operations.

  3. Dataflow Runner translates that graph into an executable Dataflow job.

  4. Dataflow provisions workers, distributes work, checkpoints streaming state, and restarts failed work when necessary.

  5. Sources such as Pub/Sub, BigQuery, and Cloud Storage provide input; sinks such as BigQuery, Cloud Storage, and Bigtable receive output.

Dataflow supports both models:

  • Batch: Processes bounded input and completes the job.

  • Streaming: Processes unbounded input continuously, using windows, triggers, and state to aggregate events over time.

Dataflow is a managed execution service: Google provisions and operates worker compute, while customers configure pipeline code, worker capacity, and autoscaling. Costs can still arise from worker processing and connected resources such as Pub/Sub, BigQuery, and Cloud Storage usage.

Production Behavior and Failure Modes

Dataflow retries failed work and restores streaming state from checkpoints. Nevertheless, an end-to-end pipeline is not automatically exactly-once. Retries, source redelivery, and sink writes can produce duplicates, so sinks should support idempotency where possible.

Important operational concerns include:

  • Poison messages: Invalid records can repeatedly fail and block work. Validate records early and route persistent failures to a dead-letter topic.

  • Skewed work: A hot source partition or key can create stragglers that aggregate autoscaling metrics do not reveal.

  • Dependency outages: A temporarily unavailable sink causes backlogs, worker growth, and higher cost.

  • State growth: Unbounded streaming state increases checkpoint time and recovery duration.

  • Job failures: Monitor failed tasks, worker restarts, backlog age, throughput, and checkpoint health in Cloud Monitoring and Cloud Logging.

Use a dedicated worker service account with only the required Pub/Sub, BigQuery, and storage permissions. Avoid user credentials or downloaded service-account keys. For sensitive workloads, evaluate private connectivity, VPC Service Controls, and customer-managed encryption where supported.

Cloud Service Comparison

Cloud Closest managed alternatives Key distinction
Google Cloud Dataflow Managed Beam execution with integrated autoscaling, monitoring, and batch/streaming support
AWS Amazon Managed Service for Apache Flink; Glue and EMR for batch Does not provide a direct one-to-one equivalent to the Beam-managed experience
Azure Microsoft Fabric Real-Time Analytics; Azure Synapse, Databricks, or HDInsight for other workloads No direct one-to-one Beam-managed equivalent; Azure Stream Analytics is scheduled for retirement on 30 November 2027

Choose Dataflow when the team values a portable Beam model, unified batch and streaming development, and managed execution without operating Dataflow worker infrastructure or a Flink or Spark cluster.

Best Practices and Common Pitfalls

  • Test locally with the Beam Direct Runner, then validate representative scale and failure cases on Dataflow.

  • Make sink writes idempotent, for example by deduplicating on event_id.

  • Partition and cluster BigQuery output tables for the expected query patterns.

  • Monitor backlog age and error rate, not only worker utilization.

  • Keep the Beam SDK version compatible with the Dataflow runtime and test streaming updates before rollout.

  • Define source, sink, and dead-letter retention policies; Dataflow itself does not back up application data.

41

What does Google Cloud Dataproc provide?

Google Cloud Dataproc is a managed service for running Apache Spark and Hadoop workloads. It provides autoscaling Compute Engine-based clusters and Dataproc Serverless for managed Spark batch processing and interactive sessions, with integration into Google Cloud storage, IAM, networking, logging, and monitoring.

Read full answer

Applies toGoogle Cloud Dataproc 2.x cluster images and Dataproc Serverless 2.x runtimes

How It Works

Dataproc abstracts much of the operational work required to run Spark and Hadoop:

  • Cluster mode: Dataproc creates Compute Engine instances in your VPC. A master node coordinates the workload, while worker nodes run Spark executors or other processing tasks.

  • Serverless mode: Dataproc Serverless runs Spark workloads without requiring users to configure or manage persistent virtual machines. Interactive Spark sessions are available in supported regions and runtimes.

  • Storage: Jobs commonly read and write Cloud Storage data. Cluster images can also use Hadoop Distributed File System and other connectors.

  • Networking: Cluster nodes communicate through the VPC network and can use only private IP addresses.

  • Lifecycle: Batch clusters are normally disposable. Teams recreate them from versioned configuration rather than maintaining long-lived compute for every workload.

Spark still executes an application driver, partitions input into tasks, and schedules those tasks on executors. Dataproc manages the compute and supporting services, but application-level concerns such as data skew, memory pressure, retries, and output correctness remain the job developer’s responsibility.

Core Capabilities

  • Apache Spark: Supports Spark SQL, PySpark, Scala and Java applications, Structured Streaming, MLlib, and custom Spark processing.

  • Hadoop ecosystem: Cluster images can provide components such as HDFS and Hive, depending on the selected image version and optional components.

  • Autoscaling: Cluster-mode workers can scale between configured minimum and maximum worker counts.

  • Job submission: Workloads can be submitted through the gcloud CLI and the Dataproc job or batch APIs.

  • Configuration: Clusters support custom machine types, image versions, optional components, labels, networking, encryption, and autoscaling.

  • Service integration: Dataproc works with Cloud Storage, BigQuery, Cloud Monitoring, Cloud Logging, Artifact Registry, IAM, and VPC networking.

Production Operations

Dataproc is designed primarily for ephemeral, data-intensive batch processing.

  • Monitoring: Use Cloud Monitoring for cluster and job metrics and Cloud Logging for driver, executor, and system logs. Consistent labels support dashboards, incident analysis, and charge attribution.

  • Failure recovery: Failed executors can normally be replaced, but retrying a failed job can repeat partially completed work. Make writes idempotent and use checkpointing where appropriate.

  • State: Do not treat local cluster disks as durable storage. Keep code and configuration in source control, dependencies in versioned artifact repositories, and input and output data in Cloud Storage or another durable service.

  • Upgrades: Test new image versions against representative jobs. For critical pipelines, validate results on a canary cluster before replacing production clusters.

  • Capacity: Compute Engine quotas, subnet IP capacity, and zonal resource availability can delay cluster creation or autoscaling.

  • Cleanup: Use external automation such as Cloud Scheduler and Workflows to delete disposable clusters after successful pipeline completion.

Dataproc does not itself guarantee exactly-once processing. End-to-end correctness depends on Spark retries, checkpointing, and the semantics of the destination system.

Security and Cost

  • Run cluster nodes in a private subnet without public IP addresses. Enable Private Google Access for required Google APIs and use Cloud NAT when general outbound internet access is necessary; restrict permitted destinations with network controls where possible.

  • Grant users and service accounts only the IAM permissions needed to submit jobs and access data.

  • Use VPC Service Controls when required by data-protection policy.

  • Keep Cloud Storage buckets private and use narrowly scoped IAM access.

  • Use Google Cloud’s default encryption or customer-managed encryption keys when required.

  • Use Cloud Audit Logs for administrative activity and enable Data Access logs where the required audit coverage includes storage access.

Dataproc pricing includes cluster-management charges, Compute Engine usage, persistent disks, networking, and storage. Configured minimum workers continue incurring charges while a cluster exists. Serverless removes idle cluster costs, although BigQuery may be less expensive for SQL-only analytics.

Choosing the Right Service

Requirement Preferred GCP service Reason
Spark and Hadoop batch processing Dataproc Managed Spark and Hadoop execution with cluster or Serverless modes
Managed SQL warehouse analytics BigQuery Serverless, pay-per-query analytics without cluster management
Apache Beam batch and streaming pipelines Dataflow Managed execution for Beam pipelines
Custom containerized workloads on Kubernetes GKE Kubernetes scheduling and broader deployment flexibility
Interactive Spark development Dataproc Serverless or cluster mode Serverless interactive sessions where offered; cluster mode provides a dedicated environment with direct node control

The closest AWS equivalents are Amazon EMR and EMR Serverless. Current Azure managed Spark counterparts include Azure Databricks, Microsoft Fabric Spark, and Synapse Spark; Azure HDInsight is being retired and is not appropriate for new deployments. Dataproc is especially attractive when the workload already uses Cloud Storage, BigQuery, and Google Cloud IAM.

Best Practices

  • Use version-controlled Terraform for repeatable cluster definitions.

  • Pin image and runtime versions, then test upgrades before production rollout.

  • Set autoscaling limits from measured workload requirements.

  • Configure alerts for failed jobs, executor failures, resource exhaustion, and abnormal cluster growth.

  • Separate development, staging, and production through projects, networks, IAM policies, and labels.

  • Delete disposable clusters automatically after successful pipeline completion.

42

What is Google Cloud Composer?

Google Cloud Composer is Google Cloud’s managed service for Apache Airflow, used to define, schedule, and operate data-engineering workflows as DAGs. Google manages the Airflow runtime and supporting infrastructure, while teams manage DAGs, dependencies, access, and operational policy.

Read full answer

Applies toGoogle Cloud Composer 2 and 3 with Apache Airflow 2.10.5 or Airflow 3.x image versions

How It Works

A Composer environment runs managed Airflow infrastructure on Google Kubernetes Engine, with the Airflow metadata database in Cloud SQL. Google operates the control plane, Kubernetes infrastructure, database service, and maintenance.

  1. DAG source is stored in an associated Cloud Storage bucket and made available to Airflow for parsing.

  2. The Airflow scheduler evaluates schedules and dependencies and creates task instances.

  3. Worker pods run operators that execute the actual work.

  4. Task state is stored in the Airflow metadata database, logs are written to Cloud Logging, and operational metrics are available in Cloud Monitoring.

Composer orchestrates work in other Google Cloud services through operators. For example, a DAG can query BigQuery, launch a Cloud Run Job, create a GKE workload, publish to Pub/Sub, or submit a Dataflow job.

Core Capabilities

  • Scheduling: Cron, date, interval, and timedelta schedules are supported. Asset, event, and calendar-trigger availability depends on the Composer generation and Airflow image version. UTC is the default, but DAGs may define another timezone; catchup controls whether missed scheduled intervals are backfilled.

  • Dependency management: Tasks form a directed graph and wait for their upstream dependencies to succeed.

  • Retries and timeouts: Operators can define retry limits, delays, and execution timeouts.

  • Dynamic task mapping: Airflow 2.3 and later can expand one task into multiple task instances from a runtime-sized collection.

  • Custom logic: Python tasks and supported third-party operators cover workflows that cannot be expressed with simple service calls.

  • Environment isolation: Each environment has its own configuration, Airflow metadata database, and DAG bucket. It also has an assigned service account, which can be shared but should normally be dedicated and minimally privileged.

Production Operations

Use gcloud to inspect environments:

  • gcloud composer environments list --project=example-project --location=us-central1 lists environments.

  • gcloud composer environments describe example-environment --project=example-project --location=us-central1 shows configuration and runtime state.

  • gcloud composer environments create example-environment --project=example-project --location=us-central1 --environment-size=ENVIRONMENT_SIZE_SMALL creates an environment using the default image.

Monitor scheduler delay, task duration, failures, worker saturation, Cloud SQL load, DAG parsing errors, and service quotas. Configure Cloud Logging sinks and retention explicitly where compliance or incident investigation requires it.

Composer does not back up application data processed by tasks. Keep DAG source and dependency manifests in version control, use Cloud Storage versioning or retention controls where appropriate, and use the backup and restore facilities supported by the selected Composer generation.

Image-version upgrades require compatibility testing because Airflow upgrades can change operator behavior, deprecated parameters, dependency resolution, and task semantics. Test DAGs in a non-production environment and verify restoration procedures before production use.

Failure Modes and Security

Common incidents include:

  • Scheduler delay: metadata-database pressure or expensive DAG parsing delays task creation.

  • Worker saturation: concurrent tasks create queues despite healthy individual workers.

  • External-service failures: quotas, authentication errors, network restrictions, and service outages fail otherwise valid tasks.

  • Dependency failures: an unsuccessful upstream task prevents its downstream chain from progressing.

  • DAG deployment errors: syntax errors, missing packages, and import-time exceptions prevent new runs.

Run tasks under a dedicated service account with only the permissions required by its operators. Prefer attached service-account identities over downloaded keys, use Workload Identity Federation where the target supports it, and restrict Airflow UI and operator access with IAM. For sensitive workloads, evaluate private networking, VPC Service Controls, and customer-managed encryption keys where supported.

AWS and Azure Comparison

Capability Google Cloud Composer Amazon Managed Workflows for Apache Airflow (MWAA) Azure Data Factory
Core model Managed Apache Airflow Managed Apache Airflow Managed data-integration workflows
Workflow language Python DAGs Python DAGs Pipelines and activities, not Airflow DAGs
Best fit GCP-native, code-driven orchestration AWS-native Airflow workflows Data ingestion, transformation, and integration
Key distinction Managed GKE-based Airflow runtime Closest direct Airflow equivalent Not an Apache Airflow service

Choose Composer for flexible Python-based orchestration across GCP services. For simple scheduled jobs, Cloud Scheduler with Cloud Run or Workflows may be simpler. Composer is less suitable for sub-second event processing, continuously running streaming applications, or workflows dominated by simple API calls where Airflow’s scheduling overhead is unnecessary.

43

What is a GCP VPC and how is it global?

A Virtual Private Cloud (VPC) network is a logically isolated private network in Google Cloud. It is a global resource that can contain regional subnets across multiple regions and zones, allowing supported workloads in different locations to communicate privately within one network boundary.

Read full answer

Applies toGoogle Cloud VPC networking (all current versions)

Core Model

  • VPC network: A software-defined network managed by Google that defines IP address space, routing, and private connectivity. It is not a physical network or a single data center.

  • Resource hierarchy: A VPC belongs to a project, which can be placed beneath folders and an organization. With Shared VPC, a host project owns the network while service projects attach supported workloads.

  • Subnet: Allocates an IPv4 range for Compute Engine instances and other supported resources. A subnet is regional and can provide secondary ranges for services such as GKE.

  • IAM and firewall separation: IAM controls who can administer network resources, while VPC firewall rules control which network traffic is permitted. Production environments need both.

  • Internal communication: Resources connected to the same VPC can communicate using internal IP addresses, subject to routing, firewall, and product-specific configuration requirements.

Global and Regional Scope

Resource Scope Meaning
VPC network Global One network can support workloads across multiple regions and zones
Subnet Regional Allocates IP ranges within one Google Cloud region
Compute Engine instance Zonal Uses an interface in a subnet in the instance’s region
Global external IPv4 address Global Attaches only to resources that support global external addresses
Regional external IPv4 address Regional Attaches only to supported resources in its region
VPC firewall rule VPC-wide by default Applies across the network; narrow it with targets such as tags or service accounts
Internal/external load balancer Regional or global Depends on the specific load-balancing product
Cloud NAT Regional Provides outbound internet connectivity for resources in its region

“Global” describes the VPC’s logical control and addressing scope; it does not eliminate regional latency, capacity limits, or failures. A multi-region deployment still requires regional subnets, regional dependencies, regional capacity, and per-location monitoring.

Connectivity, Routing, and Security

Google manages the VPC as a software-defined network. Its network definition and routing configuration are global resources, while packet forwarding remains tied to the regional infrastructure serving each workload.

  • Internet egress: Instances without external IP addresses generally need Cloud NAT for outbound internet connectivity. Cloud NAT does not accept unsolicited inbound connections.

  • Private access to Google services: Private Google Access sends traffic for supported Google APIs and services through private addresses.

  • Managed private connectivity: Private Service Connect and Private Service Access support private access to managed services through different architectural patterns.

  • Hybrid connectivity: Cloud VPN and Cloud Interconnect connect a VPC to on-premises environments; another cloud can be reached through VPN, peering, or an Interconnect service provider.

  • Default firewall behavior: VPCs include implied rules that deny incoming traffic and permit outgoing traffic. Explicit rules should restrict sensitive ports and administrative access.

  • Routing: Custom routes participate in route selection alongside system-generated and default routes. The most specific matching prefix is selected, with route attributes determining selection when prefixes tie. The default internet route is only the fallback.

Comparison With AWS and Azure

Capability Google Cloud AWS Azure
Primary network resource VPC network VPC Virtual network
Default network scope Global Regional Regional
Subnet scope Regional Availability Zone Virtual network, with zone-aware deployment
Static public IP scope Global or regional Regional Regional
Managed private access Private Service Connect, Private Service Access, and Private Google Access for supported Google APIs VPC endpoints and AWS PrivateLink Private endpoints and Azure Private Link

GCP’s global VPC simplifies multi-region addressing, but AWS and Azure can also connect regional networks. AWS commonly uses Transit Gateway; Azure commonly uses Virtual WAN or hub-and-spoke VNet peering.

Production Practices and Failure Modes

  • Plan address space: Use non-overlapping CIDR ranges across VPC peering, VPN, Interconnect, and Shared VPC relationships.

  • Monitor consumption: Track subnet address usage and relevant quotas before deploying large GKE clusters or autoscaling groups.

  • Apply least privilege: Lower firewall-rule priorities are evaluated first. Restrict SSH, RDP, database, and management ports to trusted sources.

  • Design for regional failures: Cloud NAT, subnet capacity, and regional load balancers remain regional even when the VPC is global.

  • Automate and audit changes: Manage networking through reviewed Terraform or configuration pipelines, enable supported firewall logging, and detect configuration drift.

  • Watch common failures: IP exhaustion, subnet quota limits, asymmetric routes, missing NAT egress, incorrect firewall priorities, and unavailable regional capacity are more likely than failure of the global VPC definition itself.

44

How do VPC subnets and firewall rules work in GCP?

VPC subnets define regional IP address space in a Google Cloud VPC network, while firewall rules are VPC-wide, stateful controls that allow or deny traffic to and from Compute Engine instances, including the instances backing GKE nodes and managed instance groups.

Read full answer

Applies toGoogle Cloud VPC networking and Compute Engine (current stable versions)

Core Concepts

  • VPC network: A global, private logical network owned by a project or a Shared VPC host project. Resources communicate through internal IP addresses.

  • Subnet: A regional IP range within a VPC network. Its address space is available to resources in every availability zone of that region, unlike an AWS subnet, which belongs to one Availability Zone.

  • Instance interface: Each Compute Engine network interface is attached to a subnet in the same VPC network.

  • Secondary IP ranges: Additional ranges configured on a subnet for alias IP addresses, commonly used for GKE pod address space. GKE can use another secondary range for Service address space.

  • Private Service Connect addresses: Allocated from internal address ranges reserved for Private Service Connect rather than from a subnet’s primary range.

  • Firewall rule: A global resource within a VPC that filters ingress or egress traffic by protocol, port, source or destination, and target.

  • Network tags: Labels attached to instances or interfaces. They identify workloads in firewall rules without hard-coding IP addresses.

  • Resource hierarchy: VPCs belong to projects, which can be organized under folders and an organization for centralized IAM and governance.

How Traffic Is Evaluated

Firewall rules are evaluated against the relevant Compute Engine virtual network interface before traffic reaches the instance.

  • Rules have priorities from 0 to 65535; a lower number is evaluated first.

  • The first matching rule determines whether traffic is allowed or denied. If no custom rule matches, the implied rules apply.

  • An ingress rule can match source ranges, source tags, source service accounts, protocols and ports, and target tags or service accounts.

  • An egress rule can match destination ranges, destination service accounts, protocols and ports, and the source workload through target tags or service accounts.

  • Processing is stateful: reply traffic for an allowed connection is automatically permitted.

  • The implied rules deny all ingress and allow all egress. Custom allow rules therefore do not restrict egress unless a higher-priority deny rule is configured.

  • Assigning a public IP address does not bypass firewall rules; the applicable traffic must still be allowed.

Auto-mode VPC networks create default SSH and RDP rules from 0.0.0.0/0. Custom-mode networks avoid these legacy defaults and are generally preferable for production networks.

Production Design

  • Use a custom-mode VPC network and create only the subnets required by each region.

  • Plan address space carefully. Primary and secondary ranges cannot overlap within one VPC. Primary subnet ranges can be expanded but not reduced.

  • Use a controlled administrative bastion reached through IAP TCP forwarding rather than exposing port 22 globally. For an IAP-forwarded bastion, allow port 22 from 35.235.240.0/20 and enforce strong IAM or OS Login controls.

  • Use Cloud NAT when private instances require outbound internet access without public IP addresses. Apply explicit egress firewall policy and monitor NAT port exhaustion.

  • Enable VPC Flow Logs and firewall-rule logging. VPC Flow Logs and firewall logs are separate signals; use firewall log entries in Cloud Logging to investigate denied traffic and create log-based alerts for unexpected denials.

  • Use Cloud Armor on supported HTTP(S) proxy and load-balancer frontends for layer-7 protection. VPC firewall rules remain primarily network- and transport-layer controls.

  • Use Shared VPC when service projects need centrally administered networking. Use VPC peering or Private Service Connect when isolated networks require controlled private reachability; neither extends firewall-policy ownership automatically.

Cloud Comparison

Capability Google Cloud AWS Azure
Network scope Global VPC with regional subnets Global VPC with Availability Zone-specific subnets Regional VNet with subnets
Subnet reach Spans all zones in a region One Availability Zone per subnet Can span availability zones
Network filtering Stateful firewall rules with allow and deny Stateful security groups and stateless network ACLs Stateful Network Security Groups and Azure Firewall
Rule-targeting dimensions Tags and instance service accounts Security groups and tags Security-group membership and service tags

Common Pitfalls

  • Opening administrative ports to 0.0.0.0/0 exposes workloads directly to the internet.

  • Overlapping or poorly partitioned ranges cause routing and migration problems, while conflicting firewall priorities make policy difficult to reason about.

  • Firewall rules do not provide application authentication, TLS termination, rate limiting, or HTTP request filtering.

  • GKE firewall rules act on node interfaces rather than directly on pod interfaces; use Kubernetes NetworkPolicy for pod-level controls.

  • Cloud Run and Cloud Functions are not directly targeted by VPC firewall rules in the same way as Compute Engine instances; use their ingress, IAM, and VPC connectivity controls.

  • Source and target tags identify eligible instances within the same VPC network, not arbitrary external traffic.

45

What is Shared VPC in Google Cloud?

Shared VPC is a Google Cloud networking model in which an organization’s host project owns the VPC network and subnets, while attached service projects run their own compute resources. This centralizes network configuration without moving workloads such as virtual machines into the host project.

Read full answer

Applies toGoogle Cloud Shared VPC (current)

How It Works

  • Host project: Owns the VPC network, subnets, routes, firewall rules, Cloud Router, and other shared network resources.

  • Service projects: Remain responsible for their own Compute Engine instances, managed instance groups, Cloud Run services, GKE nodes, and other resources.

  • Shared subnets: Are defined once in the host project and selected when creating resources in an attached service project.

  • Resource ownership: A virtual machine remains in its service project, including its IAM policies, logs, quotas, and billing, even though its network interface uses a subnet owned by the host project.

  • Organization requirement: The host and service projects generally belong to the same Google Cloud organization. An organization administrator designates the host project and controls which projects can participate.

Shared VPC centralizes network addressing and policy, but it does not share virtual machines, disks, databases, or other compute resources.

IAM and Security

IAM combines permissions from both projects:

  • Service project permissions: Determine who can create resources in that project.

  • Host project subnet permissions: Determine who can attach resources to a shared subnet.

  • Network administration: Host project network administrators manage central routes, firewall rules, and subnets.

  • Least privilege: Grant roles/compute.networkUser on only the required shared subnet rather than granting host-project network administration.

The host project’s firewall rules and routes affect every attached service project. A bad route, overly permissive ingress rule, or exhausted shared IP address can therefore create a multi-project incident.

Shared VPC Compared with Other Models

Model Ownership Multi-project use Closest external equivalent
Google Cloud Shared VPC One host project owns the network Service projects use shared subnets AWS VPC sharing
Google Cloud VPC peering Each side owns a separate VPC Peer-to-peer connection between two VPCs AWS VPC peering
Azure hub-and-spoke A hub VNet owns central connectivity Spoke VNets peer with the hub Azure hub-and-spoke
AWS Transit Gateway A gateway connects multiple VPCs Central transit without sharing subnets Google Cloud VPC peering plus Cloud Router

Choose Shared VPC when multiple projects need centrally managed subnets and consistent network controls. Use VPC peering when separate VPC ownership is more important than sharing a common address space.

Production and Operational Behavior

  • Address management: Allocate non-overlapping CIDR ranges centrally, such as 10.10.0.0/16, and divide them into regional subnets.

  • Connectivity: Use Cloud Router and dynamic routes for on-premises or partner connectivity. Shared VPC does not itself create a VPN or Cloud Interconnect connection.

  • Egress: A centrally managed Cloud NAT can provide outbound connectivity, but its configuration and address capacity affect all attached workloads that use that NAT and subnet.

  • Monitoring: Monitor host-project firewall metrics, router health, NAT utilization, and subnet address exhaustion. Service-project dashboards should correlate Compute Engine failures with host-network health.

  • Logging: Enable Firewall Rules Logging and retain audit logs according to the organization’s incident and compliance requirements.

  • Quotas and billing: Network configuration is administered centrally, while compute resource quotas remain associated with the owning service project. Network charges may be attributed to the host project.

  • Failure isolation: Centralization improves consistency but increases blast radius. Test route and firewall changes against representative service projects before broad rollout.

Best Practices and Common Pitfalls

  • Manage the host project with Terraform: Treat host-project VPCs, subnets, routes, firewall rules, and IAM bindings as reviewed infrastructure code.

  • Use separate subnets by environment and region: For example, use 10.10.10.0/24 for production and 10.20.10.0/24 for staging.

  • Do not grant broad host-project roles: Use roles/compute.networkUser on individual subnets wherever possible.

  • Plan capacity centrally: Monitor consumed IP addresses and Cloud NAT ports before expansion.

  • Control project attachment: Review organization policy and audit logs to prevent unauthorized service projects from joining.

  • Test detach and recovery procedures: Attached resources and subnet dependencies can prevent service-project detachment or host-project deletion.

  • Avoid overlapping CIDR ranges: Shared VPC does not make overlapping subnet ranges routable.

46

VPC Peering vs Shared VPC in Google Cloud

VPC peering creates a private, direct connection between two independent VPC networks, while Shared VPC allows multiple projects to use one centrally managed VPC network. Peering is a connectivity relationship between networks; Shared VPC is an ownership and IAM model for sharing a network across projects.

Read full answer

Applies toGoogle Cloud VPC networking and Shared VPC (current stable)

Core Difference

Aspect VPC peering Shared VPC
Main purpose Connect two separate VPC networks Let multiple projects use one VPC network
Network ownership Each project owns its VPC The host project owns the VPC and subnets
Administrative model Each side manages its own network The host project centrally controls networking
Connectivity Private-IP communication between the peered networks Service-project resources use the host project’s network directly
Transitivity Non-transitive Not a peering model; resources are already on the shared network
Typical use case Connecting isolated networks or environments Sharing a platform network across an organization

A common misconception is that these features are alternatives. They solve different problems and can be combined: service projects can use a Shared VPC network, and that shared network can peer with another VPC.

How VPC Peering Works

A peering connects two VPC networks directly, either within one project or across projects. Google Cloud exchanges routes between the peered networks automatically, so instances with compatible internal IP addresses can communicate over private IP.

Important constraints include:

  • Non-transitivity: If VPC A peers with VPC B and VPC B peers with VPC C, VPC A cannot reach VPC C through those peerings.

  • No overlapping subnet ranges: The peered networks must not use the same IP address space.

  • Separate firewall policies: Each VPC controls its own firewall rules. Each side must allow the initial ingress traffic, while stateful firewall behavior permits return traffic for an established connection without a separate return rule. Independent traffic initiated from the other direction requires its own ingress rule.

  • No central network administrator: Peering does not transfer ownership of either network.

  • Private connectivity: Peering provides internal-network connectivity; it is not an internet gateway or an IAM-sharing mechanism.

Peering is useful when teams or environments need controlled, explicit links without giving one project administrative ownership of the other network.

How Shared VPC Works

With Shared VPC, a host project owns the VPC network, subnets, routes, and firewall configuration. One or more service projects attach to the host and create resources using its shared subnets.

For example, a company might have:

  • A production host project owning shared subnets such as 10.20.0.0/16.

  • A payments service project running Compute Engine instances in 10.20.10.0/24.

  • A billing service project running GKE clusters whose nodes use 10.20.20.0/24.

There is no additional VPC peering between the host and service projects because their resources already use the same VPC network.

Shared VPC is useful for enforcing central network controls while allowing product teams to manage their own compute, Kubernetes, and application resources. Attachment requires appropriate IAM permissions and may be constrained by organization policies.

IAM and Security

Peering and Shared VPC both work alongside IAM, but they enforce access differently:

  • Peering: IAM permissions remain project-specific. Network reachability does not automatically grant access to VMs, disks, APIs, or data.

  • Shared VPC: Service-project principals need an appropriate role, commonly roles/compute.networkUser, on each shared subnet they use. roles/compute.networkAdmin provides broader administrative authority.

  • Least privilege: Grant service teams access only to the shared subnets and resources they require.

  • Auditability: Review host-project IAM changes and firewall changes centrally, while retaining service-level resource logs and ownership.

Shared VPC introduces a larger blast radius because one team controls connectivity for several service projects. Apply change management, infrastructure-as-code approvals, and monitoring to host-project changes.

Operational Trade-offs

  • Peering failures: Common causes include overlapping IP ranges, missing firewall rules, incorrect peering configuration, route or quota problems, and incorrect assumptions about transitive routing.

  • Shared VPC failures: Common causes include incomplete service-project attachment, missing roles/compute.networkUser, shared-subnet address exhaustion, host-project networking limits, or an organization policy preventing the attachment.

  • Monitoring: Monitor reachability, firewall-rule logs, relevant service SLIs, and peering state. For Shared VPC, monitor shared-subnet capacity and host-project networking because subnet exhaustion or a bad centralized change can affect every attached project.

  • Recovery: Keep network definitions in Terraform or another declarative system, and validate firewall, IAM, and subnet changes before applying them.

  • Backups: Back up network configuration and relevant IAM policies, but do not treat a configuration snapshot as a substitute for rebuilding network resources from source control.

Choose VPC peering when independent networks need a direct bilateral connection. Choose Shared VPC when multiple projects should operate on one centrally governed network.

47

What is Google Cloud NAT?

Google Cloud NAT is a managed, regional service that provides outbound internet connectivity to Compute Engine instances that do not have external IP addresses. It translates private source addresses and ports to a shared public NAT IP address without providing inbound connectivity.

Read full answer

Applies toGoogle Cloud NAT (current stable service)

How It Works

A Cloud NAT gateway is created on a Cloud Router and serves selected subnets in one VPC network and region.

  1. A VM without an external IP initiates an outbound connection.

  2. The VPC route directs eligible traffic through the Cloud NAT gateway.

  3. Cloud NAT translates the VM’s private IP and source port into a public NAT IP and allocated port.

  4. The external service sees the Cloud NAT IP, not the VM’s private IP.

  5. Return traffic is translated back to the originating VM.

Cloud NAT supports outbound TCP, UDP, and ICMP traffic. It supports both manually configured static NAT IP addresses and automatic allocation of ephemeral NAT IP addresses as demand grows.

  • Private VM egress: Instances can access internet destinations without exposing themselves through public IP addresses.

  • Regional scope: One gateway can serve multiple subnets in the same region; resources in other regions require their own gateway or another egress design.

  • Outbound only: Cloud NAT cannot accept unsolicited inbound connections, perform port forwarding, or replace an external application load balancer.

  • Traffic forwarding: It supports outbound connectivity to internet destinations and, where routing permits, peered VPC networks.

Key Components and Design Choices

  • Cloud Router: Hosts the Cloud NAT configuration and participates in VPC route management.

  • Subnet selection: The gateway can serve specific subnets or all subnets in the region.

  • NAT IP addresses: Manual mode gives predictable public egress addresses for firewall allowlists; automatic mode simplifies capacity management.

  • IP version: Cloud NAT provides IPv4 egress. IPv6-only architectures need a separate IPv6 networking design.

  • Logging: Optional Cloud NAT logs provide visibility into translated flows, including source VM, destination, protocol, port, and traffic volume.

Production Operations

Cloud NAT is commonly used when workloads need controlled egress without directly exposing individual VMs.

  • Monitoring: Track connection counts, port utilization, allocation failures, processed bytes, and NAT IP consumption in Cloud Monitoring.

  • Logging: Enable Cloud NAT logging during validation and incidents, then evaluate the additional log volume and cost for production.

  • Capacity: Investigate high port utilization and allocation failures, especially with many short-lived connections or substantial UDP traffic.

  • Quotas: Monitor project and regional quotas for Cloud NAT gateways, NAT IP addresses, and related networking resources.

  • Backups and upgrades: There are no customer-managed gateway instances, snapshots, or software upgrades. Google operates the data plane, but the surrounding routing, IAM, firewall, and logging configuration remains your responsibility.

  • Incident diagnosis: Check whether the subnet route points to the correct Cloud Router, whether the gateway has available NAT IP capacity, and whether VPC firewall rules permit the required traffic.

Useful commands include:

  • gcloud compute routers nats list --router=example-router --region=us-central1 --project=example-project

  • gcloud compute routers nats describe example-nat --router=example-router --region=us-central1 --project=example-project

Security and Cost Trade-offs

Cloud NAT centralizes outbound public addresses, making provider-side IP allowlisting practical. However, it is an address-translation service, not an application-layer firewall: it does not inspect URLs, payloads, or application behavior. Use VPC firewall policies, DNS controls, secure web proxies, and workload-aware egress controls when stronger policy enforcement is required.

The main charges are for allocated NAT IP addresses and processed data, in addition to any applicable network, logging, and observability charges. Manual mode can improve address predictability but requires capacity planning; automatic mode reduces allocation work but may make the exact public address less predictable unless carefully configured.

AWS and Azure Equivalents

Cloud Equivalent Scope and notable behavior
Google Cloud Cloud NAT Regional managed egress for private VMs; no inbound access
AWS NAT Gateway Zonal resource; production designs generally use one per Availability Zone
Azure NAT Gateway Associated with subnets and backed by configured public IP resources

Choose Cloud NAT when private Compute Engine workloads need centralized outbound access. Use Cloud Load Balancing, Cloud Run, GKE, or another ingress-capable service when external users must initiate connections.

48

What is Private Google Access in Google Cloud?

Private Google Access provides private connectivity from Compute Engine instances and supported hybrid workloads to Google APIs and supported Google Cloud services without sending service traffic over the public internet. It provides network routing, not authorization; IAM permissions are still required.

Read full answer

Applies toGoogle Cloud Private Google Access (current)

How It Works

For Compute Engine, Private Google Access is enabled per subnet. Google Cloud automatically creates and manages the appropriate routes for eligible VM network interfaces, including existing instances; no manually created route is required.

  • The VM does not need an external IP address.

  • Cloud NAT is not required to reach supported Google APIs and services.

  • Service traffic follows Google’s internal network rather than the public internet.

  • DNS generally resolves the service’s public endpoint, but the Private Google Access route directs the connection over Google’s private infrastructure.

  • The workload still needs the appropriate IAM permissions, such as roles/storage.objectViewer when reading Cloud Storage objects.

  • Private Google Access does not provide general internet access. Arbitrary internet destinations still require Cloud NAT, a proxy, or another approved egress design.

Supported Workloads

  • Compute Engine instances: Access Cloud Storage, BigQuery, Pub/Sub, and supported Google APIs without external addresses.

  • GKE private clusters: Nodes and node-local workloads can reach required Google services through Private Google Access. The managed GKE control plane does not rely on the customer VPC’s Private Google Access route.

  • Cloud Batch and data-processing jobs: Temporary workers can retrieve inputs or publish results without public IP addresses.

  • Hybrid workloads: On-premises hosts connected to a VPC through HA VPN or Cloud Interconnect can access supported Google APIs over private connectivity.

Only supported Google destinations use Private Google Access. A service that instead requires a PSC endpoint, a service-owned private IP, or another interface must be connected using that mechanism.

Differences from Other Private Connectivity Options

Option Primary purpose Endpoint or address requirement Outbound internet path
Private Google Access Access Google APIs and supported services No No
Private Service Connect Privately consume supported managed services PSC endpoint No
Private Service Access Allocate supported services’ private IPs through VPC peering Service-owned private IP range No
Cloud NAT Provide outbound internet connectivity for VMs No Yes

For Cloud SQL, PSC connectivity uses a PSC endpoint, while direct private-IP connectivity uses Private Service Access and VPC peering. Private Google Access is not the mechanism that allocates Cloud SQL private addresses.

Private Google Access is also distinct from VPC Service Controls. VPC Service Controls adds perimeter-based protections against unauthorized access and data exfiltration; it does not replace the routing supplied by Private Google Access.

Configuration and Validation

Private Google Access is a subnet setting; there is no network-wide switch. Enable it on each subnet that needs it:

gcloud compute networks subnets update SUBNET_NAME --region=REGION --enable-private-ip-google-access

Inspect the setting with:

gcloud compute networks subnets describe SUBNET_NAME --region=REGION --format="get(privateIpGoogleAccess)"

Administrators should verify that:

  1. The VM or node has no external IP address when private-only connectivity is required.

  2. DNS resolution is working.

  3. The workload’s service account has the required IAM roles.

  4. Private Google Access is enabled on the subnet the workload uses.

  5. The destination service supports Private Google Access.

  6. Organization policies or VPC Service Controls do not block the operation.

Production and Security Considerations

  • Least privilege: Private routing does not grant API access; grant only the IAM permissions the workload requires.

  • Egress control: Cloud NAT egress rules control which Cloud NAT external addresses are selected, including for particular destinations. Configure VPC firewall egress rules when destination restrictions are required; ingress-only firewall rules do not provide egress filtering.

  • Monitoring: Use Cloud Monitoring and Cloud Logging to track API errors, authorization failures, latency, quota errors, and throttling.

  • Service limitations: Verify Private Google Access support for each service and region; some managed services require PSC or Private Service Access.

  • Failure domains: Private Google Access avoids public-internet transit, but it does not eliminate service outages, quota limits, DNS failures, IAM errors, or VPC Service Controls denials.

  • Combined designs: A VM can use Private Google Access for supported Google services while using Cloud NAT or another approved path for general internet egress.

49

Which GCP load balancer types are available?

GCP load balancers belong to two current families—Application Load Balancers for Layer 7 traffic and Network Load Balancers for Layer 4 traffic—with seven supported non-legacy configurations. The correct choice depends on protocol, geographic scope, private versus public access, routing capabilities, and source-IP preservation.

Read full answer

Applies toGoogle Cloud Platform current stable services

Available Load Balancer Types

GCP load balancer Traffic layer Scope Important characteristics
Global external Application Load Balancer HTTP(S), gRPC Global Uses a global anycast IP; supports global backend services, URL maps, Cloud CDN, and Cloud Armor
Regional external Application Load Balancer HTTP(S), gRPC Regional Uses a regional external IP and regional backend services; useful when regional isolation is required
Global internal Application Load Balancer HTTP(S), gRPC Global Provides private Layer 7 routing across regions for internal applications
Regional internal Application Load Balancer HTTP(S), gRPC Regional Provides private Layer 7 balancing within one region
External passthrough Network Load Balancer TCP, UDP Regional Passes traffic directly to backends and preserves the original client source IP
Internal passthrough Network Load Balancer TCP, UDP Regional Provides private Layer 4 balancing within a VPC and preserves the original client source IP
External TCP/UDP Network Load Balancer TCP, UDP Regional Designed for high-throughput external traffic; does not preserve the original client source IP

For completeness, the Classic Application Load Balancer remains a documented legacy type for existing deployments. New deployments should generally use the global external Application Load Balancer instead.

How They Work

A load balancer generally consists of a forwarding rule and IP address, a backend service, health checks, and—for Application Load Balancers—a URL map and TLS configuration.

  • Application Load Balancers inspect HTTP or gRPC properties and route requests using host names, paths, headers, and other attributes. They support TLS termination.

  • Network Load Balancers operate at Layer 4 and route by IP address, port, and protocol without inspecting the application payload. They do not terminate TLS.

  • Supported backend types vary by product and can include managed or zonal instance groups, serverless NEGs, and individual endpoints.

Choosing the Right Type

  • Use the global external Application Load Balancer for public HTTP(S) applications with global users, CDN requirements, or multi-region backends.

  • Use the regional external Application Load Balancer when traffic and backends must remain within one region.

  • Use the global internal Application Load Balancer for private Layer 7 applications requiring global routing across regions.

  • Use the regional internal Application Load Balancer for private HTTP(S) services confined to one region.

  • Use an external passthrough Network Load Balancer when public TCP traffic must retain the original client IP.

  • Use an internal passthrough Network Load Balancer for private TCP or UDP traffic that requires source-IP preservation.

  • Use an external TCP/UDP Network Load Balancer for high-volume external TCP or UDP workloads where source-IP preservation is unnecessary.

AWS and Azure Equivalents

GCP type Closest AWS equivalent Closest Azure equivalent
Global external Application Load Balancer Application Load Balancer combined with Global Accelerator Front Door
Regional external Application Load Balancer Internet-facing Application Load Balancer Application Gateway
Global internal Application Load Balancer No direct single-service equivalent Front Door Premium with Private Link, depending on design
Regional internal Application Load Balancer Internal Application Load Balancer Internal Application Gateway
Passthrough Network Load Balancer Network Load Balancer Load Balancer
External TCP/UDP Network Load Balancer Network Load Balancer Load Balancer

These are functional comparisons rather than exact feature-for-feature equivalents.

Production Considerations

  • Configure health checks with the correct protocol, port, and path. A faulty check can mark every backend unhealthy and cause an outage.

  • Allow health-check traffic through firewall rules and restrict direct backend access where required, using proxy-only subnets where applicable.

  • Use managed certificates, modern SSL policies, and automated certificate renewal for HTTPS load balancers.

  • Monitor backend health, latency, error rates, dropped connections, utilization, and quotas in Cloud Monitoring; analyze available load-balancer logs in Cloud Logging.

  • Use connection draining and staged instance-group replacements during maintenance.

  • Plan regional capacity and inter-region data-transfer costs for globally routed backends. Global routing improves availability but does not eliminate regional capacity requirements.

  • Where backends need outbound internet access, plan egress through Cloud NAT or private access paths rather than assigning public IP addresses unnecessarily.

50

What is Cloud CDN in Google Cloud?

Cloud CDN is Google Cloud’s global HTTP content-caching service. It stores eligible responses at Google’s edge locations so users receive cached content closer to them, reducing latency and origin load.

Read full answer

Applies toGoogle Cloud CDN (current service)

How It Works

Cloud CDN is enabled on a backend service used by a global external Application Load Balancer. Common origins include Compute Engine instance groups, Cloud Run services, and Cloud Storage backend buckets.

For each eligible GET or HEAD request:

  1. The load balancer routes the request to a nearby Cloud CDN point of presence.

  2. Cloud CDN checks its cache using a cache key derived from factors such as protocol, host, path, and query parameters.

  3. A cache hit is returned directly from the edge.

  4. A cache miss is forwarded to the origin. The response may then be cached according to its HTTP headers and the configured cache mode.

The principal cache modes are:

  • USE_ORIGIN_HEADERS: Honors origin directives such as Cache-Control, Expires, and s-maxage.

  • CACHE_ALL_STATIC: Caches static content according to Cloud CDN’s static-content rules.

  • FORCE_CACHE_ALL: Caches all eligible responses even when the origin indicates otherwise; use cautiously.

Cache-Control and related response headers control how long content remains cached. Responses marked private or no-store should not be placed in a shared cache unless the design deliberately overrides that policy.

Production Behavior

  • Origin overload: A sudden cache miss burst can overwhelm a small origin. Origin Shield can consolidate cache fills through an intermediate location to reduce origin traffic.

  • Stale content: Changes remain visible until the TTL expires or an invalidation propagates. Invalidation is not instantaneous.

  • Cache fragmentation: Excessive query parameters or Vary headers can create many variants of the same object, reducing hit rates and increasing storage use.

  • Origin outage: Cached objects may continue to be served, while cache misses and expired objects fail until the origin recovers.

  • Eviction: Edge cache capacity is finite, so less-used objects can be evicted before their TTL. Cloud CDN is a performance layer, not the source of truth.

For mutable objects, retain the authoritative copy in Cloud Storage or another durable origin. Deploy versioned URLs or use controlled invalidation rather than deleting and immediately recreating files under the same path.

Security and Service Selection

Cloud CDN accelerates delivery but is not a replacement for a web application firewall, authentication, or origin security.

  • Terminate TLS at the load balancer and redirect HTTP to HTTPS.

  • Use Cloud Armor where supported for rate limiting, WAF rules, and DDoS defenses.

  • Use signed URLs or signed cookies for paid, private, or expiring content.

  • Avoid caching personalized responses, authorization results, or responses containing user-specific headers.

  • Restrict direct origin access where practical so users cannot bypass CDN controls.

Requirement Google Cloud AWS equivalent Azure equivalent
General web and API caching Cloud CDN CloudFront Azure Front Door
Compute-based origin Compute Engine EC2/ALB Azure VM/Application Gateway
Serverless/API origin Cloud Run behind external Application Load Balancer Lambda function URL behind CloudFront Azure Functions behind Azure Front Door
Object-storage origin Cloud Storage backend bucket S3 origin Azure Blob Storage origin
Large video delivery Media CDN CloudFront Azure Front Door

Use Cloud CDN for ordinary HTTP delivery. Consider Media CDN for large-scale video workflows that require specialized caching, logging, or media-oriented controls.

Monitoring and Operations

Cloud Monitoring provides load-balancer and cache metrics. Important signals include:

  • loadbalancer/https/backend/cdn/cache/lookup_count: Cache lookups.

  • loadbalancer/https/backend/cdn/cache/response_count: Responses served from cache.

  • loadbalancer/https/backend/cdn/cache/eviction_count: Objects evicted from the edge.

  • loadbalancer/https/request_count: Total request volume.

A practical cache-hit ratio is cache response_count / cache lookup_count. Alert on a sustained decline, origin latency, elevated 5xx responses, and abnormal eviction rates. Enable load-balancer request logging so incident analysis can distinguish hits, misses, origin failures, and invalidation-related traffic.

Best Practices and Pitfalls

  • Set explicit TTLs with Cache-Control; do not rely on implicit expiration behavior.

  • Use immutable, versioned asset names for long-lived static files.

  • Keep cache keys aligned with actual representation differences.

  • Load-test cache misses and origin-failure scenarios, not only steady-state traffic.

  • Do not combine FORCE_CACHE_ALL with authenticated or personalized responses.

  • Monitor costs for cache fills, requests, and bandwidth separately from load-balancer charges.

  • Keep origin capacity sufficient for cold caches, traffic spikes, and recovery after an outage.

51

How does Google Cloud DNS manage authoritative DNS?

Google Cloud DNS is Google Cloud’s managed authoritative DNS service. It serves public zones through Google’s global DNS infrastructure and private zones through the VPC DNS resolver, so users and workloads can resolve names without operating authoritative name servers or maintaining zone files.

Read full answer

Applies toGoogle Cloud DNS (current stable managed service)

How It Works

  • Managed zone: A namespace such as example.com. containing resource record sets and their metadata. A public zone is visible through Google’s public DNS infrastructure; a private zone is visible only through attached VPC networks and eligible peered or shared-VPC networks.

  • Record sets: Entries mapping names to addresses or targets, including A, AAAA, CNAME, MX, TXT, and SRV. A zone apex cannot use a CNAME; use A, AAAA, or another supported design.

  • TTL: Controls how long recursive resolvers may cache an answer. A lower TTL allows changes to propagate more quickly but increases query volume because clients and resolvers refresh records more often.

  • Delegation: Creating a public zone does not automatically delegate it. The domain registrar must contain the Cloud DNS name-server delegation; until it does, authoritative queries do not reach the new zone.

  • Forwarding zones: Private forwarding zones send otherwise unmatched queries to configured internal forwarding targets, such as an on-premises DNS resolver, rather than storing those answers in Cloud DNS.

  • DNSSEC: Cloud DNS can sign public zones and manage the signing keys. The correct DS record must be installed at the delegation parent, normally through the registrar. Without a DS record, the delegation is treated as insecure; a stale or invalid DS record can cause validating resolvers to return SERVFAIL.

  • Routing policies: Public zones can use zone-level routing policies for weighted round robin, geolocation, failover, or latency-based selection. These policies are separate from DNS record types.

Cloud DNS is authoritative and is distinct from Google Public DNS, which is a recursive resolver. Cloud DNS answers from published zone data; it does not ordinarily perform recursive resolution on behalf of public-zone clients.

Production Behavior

  • Traffic management: Static records always return the same answer. Use Cloud DNS routing policies when selection must depend on weights, location, endpoint health, or latency. A global external Application Load Balancer is a separate mechanism: DNS resolves to its anycast IP, and the load balancer selects a backend after the connection reaches it.

  • Caching and rollout: DNS changes are not instantaneous because recursive resolvers can retain records for their TTL. Lower TTLs before a migration, retain old records during the transition, and restore them only after the old TTL has expired.

  • Monitoring: Use Cloud Monitoring for query, record-count, and other service metrics. Run external DNS probes from multiple regions to measure availability and resolution latency from the client’s perspective.

  • Auditability: Enable Cloud Audit Logs and retain management-operation logs for IAM, zone, record, and DNSSEC changes. Data Access logging can be enabled where read operations must also be audited. DNS query contents are not necessarily captured by configuration audit logs.

  • Backups and recovery: Treat zones as infrastructure as code. Store reviewed Terraform or equivalent configuration in version control, use CI validation, and retain enough historical state to recreate records after accidental deletion or configuration errors.

  • Security: Grant least-privilege IAM roles, scope zone-level access where appropriate, and use a dedicated service account for automation. Private DNS controls name visibility but is not an authentication boundary; applications still need IAM, mTLS, service authorization, and secrets management.

  • Cost: Charges can include managed zones, queries, and records. Use a zone for multiple related names, establish budgets and alerts, and avoid unnecessary zones or high query volumes.

Service Comparison

Capability Google Cloud DNS AWS Route 53 Azure DNS
Public authoritative DNS Cloud DNS public managed zone Route 53 public hosted zone Azure DNS public zone
Private DNS Private zone attached to VPC networks Private hosted zone associated with VPCs Private DNS zone linked to VNets
Health and routing DNS routing policies Health checks and routing policies Traffic Manager plus Azure DNS
DNSSEC Supported for public zones Supported Supported for public zones
Cloud integration Compute Engine, GKE, Cloud Run, and multi-cloud workloads Broad AWS and multi-cloud integration Broad Azure and hybrid integration
Best fit GCP-centric workloads or portable public DNS AWS-centric estates with mature routing features Azure-centric estates with Microsoft integrations

Common Pitfalls

  • Missing delegation: The zone exists, but the registrar still points to other name servers, producing NXDOMAIN or unexpected answers.

  • Private-zone visibility: The VPC is not attached or does not have eligible peering, or hybrid callers lack a path to the relevant VPC DNS service. Per-client IAM permission is not required for ordinary private-name resolution.

  • Stale caches: Records remain resolvable until cached TTLs expire; a successful API update does not erase every resolver’s cached answer.

  • Invalid apex CNAME: Cloud DNS does not permit a CNAME at a zone apex.

  • DNSSEC validation failure: A DS record exists but is stale or inconsistent with the signed zone, or a key transition was not completed correctly.

  • DNS treated as authentication: Restricting a name to a private zone does not replace application authorization, mTLS, or secrets management.

52

How do Cloud Interconnect and Cloud VPN provide GCP connectivity?

Cloud Interconnect provides private connectivity between a Google Cloud VPC network and an on-premises network through a physical connection to Google’s network. Cloud VPN creates an encrypted IPsec tunnel between a VPC and an external VPN gateway over the public internet.

Read full answer

Applies toGoogle Cloud current stable services (Cloud Interconnect and Cloud VPN)

How They Work

Both services extend a VPC network to on-premises or another cloud environment, but use different transports and routing models.

  • Cloud Interconnect

  • Uses a private physical connection rather than the public internet.

  • Dedicated Cloud Interconnect connects directly to a Google edge point of presence.

  • Partner Interconnect connects through a supported network provider.

  • A VLAN attachment connects the physical connection to a VPC.

  • BGP commonly exchanges routes between the VPC’s Cloud Router and the on-premises router. Static routing is also possible.

  • Cloud VPN

  • Uses IPsec with IKEv2 to encrypt traffic between the VPC and an external VPN gateway.

  • HA VPN creates two redundant tunnels within a region.

  • Dynamic routing uses BGP through Cloud Router; static routes are also supported.

  • HA VPN protects against a single tunnel or gateway failure, but does not automatically protect against an ISP or physical-site failure.

Service Details

Cloud Interconnect is designed for predictable throughput, low latency, and large data transfers. It avoids routing sensitive traffic over the public internet, although Cloud Interconnect does not by itself provide IPsec-style encryption. Use Cloud VPN over Interconnect or an additional encryption mechanism such as MACsec where appropriate.

A typical deployment includes:

  1. A physical circuit or partner connection.

  2. A VLAN attachment in a supported Google Cloud region.

  3. A Cloud Router with a BGP peer.

  4. Firewall rules permitting only required traffic.

  5. Monitoring for attachment, BGP, packet, and latency conditions.

Provisioning generally requires capacity planning and lead time, especially for Dedicated Cloud Interconnect. Production installations normally use redundant attachments and diverse paths, potentially through different providers or metropolitan areas.

Cloud VPN is faster to provision and useful for moderate traffic, temporary connectivity, backup links, or locations where physical connectivity is unavailable. The external gateway must be configured with matching IKE/IPsec settings, public IP addresses, and a pre-shared key.

For new deployments, use HA VPN rather than the older single-tunnel Classic VPN design.

Comparison

Capability Cloud Interconnect Cloud VPN
Transport Private physical connection Public internet
Encryption Not inherently provided by the interconnect IPsec/IKEv2 encryption
Typical bandwidth High and predictable Moderate, dependent on internet capacity
Provisioning time Often weeks or longer Minutes to hours
Routing BGP or static routes BGP or static routes
High availability Multiple attachments and diverse paths Two HA VPN tunnels; additional paths for site/region failures
Common AWS equivalent AWS Direct Connect AWS Site-to-Site VPN
Common Azure equivalent Azure ExpressRoute Azure VPN Gateway

Choose Cloud Interconnect when latency, throughput, and predictable private connectivity are primary requirements. Choose Cloud VPN when rapid deployment, encrypted connectivity, or a secondary link is more important.

Production Operations

  • Monitor link utilization, packet loss, latency, tunnel state, and BGP session health using Cloud Monitoring and Cloud Logging.

  • Configure alerts for a tunnel or attachment going down, sustained route flapping, and abnormal traffic changes.

  • Use BGP timers and metrics appropriate to the application’s failover tolerance; overly aggressive timers can cause unnecessary route churn.

  • Keep routing and firewall configuration in Terraform or another infrastructure-as-code system.

  • Test failure scenarios deliberately by disabling a tunnel, withdrawing a route, or failing the primary ISP.

  • Remember that two redundant HA VPN tunnels can still share one ISP. Similarly, redundant Interconnect attachments can share the same provider or facility.

  • Treat VPN gateways and firewall rules as production network infrastructure: apply least-privilege IAM, change control, and regular configuration reviews.

Best Practices and Failure Modes

  • Use HA VPN for new designs: Two tunnels protect against tunnel and gateway failures, but use multiple regions or ISPs for broader disaster recovery.

  • Encrypt sensitive Interconnect traffic: A private physical path is not a substitute for application-layer or cryptographic protection.

  • Protect shared secrets: Store the VPN pre-shared key in Secret Manager or an equivalent secrets system, and inject it through a sensitive IaC variable rather than shell history.

  • Avoid overly permissive firewall rules: Permit required source ranges, protocols, and ports explicitly.

  • Plan redundancy by failure domain: Check whether paths share power, fiber, providers, routers, or locations.

  • Use controlled BGP changes: Coordinate route advertisements and withdrawals to prevent accidental blackholing or traffic loops.

53

What is Google Cloud Armor and how does it protect applications?

Cloud Armor is Google Cloud’s edge security service for protecting internet-facing HTTP and HTTPS applications from Layer 3/4 and Layer 7 distributed denial-of-service attacks, web application attacks, excessive requests, and bots. It combines WAF controls, rate limiting, adaptive protection, and security policy enforcement with Google’s global load-balancing infrastructure.

Read full answer

Applies toGoogle Cloud Armor, all current versions (Standard and Enterprise tiers)

How It Works

Cloud Armor security policies attach to backend services and backend buckets behind supported external Application Load Balancers. Backend targets can include Compute Engine instance groups, serverless NEGs for Cloud Run or App Engine, and Cloud Storage backend buckets.

A typical request path is:

  1. The client connects through a Google global anycast address.

  2. Google’s edge infrastructure absorbs large-scale network-layer attacks.

  3. Cloud Armor evaluates the request against the configured security policy.

  4. A matching rule allows, denies, rate-limits, or redirects the request. Logging can be enabled for the rule.

  5. Allowed traffic is forwarded to the application backend.

Global security policies protect the global external Application Load Balancer. Backend security policies provide a separate backend-enforcement option, particularly for regional deployments and Cloud Armor Enterprise configurations.

Cloud Armor does not replace application controls. It cannot validate business authorization, sanitize every form of input, or protect private east-west traffic that never passes through the load balancer.

Security Capabilities

  • Preconfigured WAF rules: Protect against common vulnerabilities using rules based on the OWASP Core Rule Set and Google-managed attack signatures.

  • Custom rules: Match on properties such as URL path, host, query string, HTTP method, headers, and source IP range.

  • Rate limiting: Restrict abusive request volume per client or another configured aggregation key, reducing application exhaustion.

  • Adaptive Protection: Detects abnormal Layer 7 traffic patterns and automatically applies mitigations for likely DDoS attacks; it can also recommend WAF or rate-limiting changes.

  • Bot management: Cloud Armor Enterprise provides advanced bot detection and management for scenarios such as credential-stuffing attacks and resource scraping, and can integrate with reCAPTCHA Enterprise to challenge suspicious clients.

  • DDoS protection: Combines Google’s network infrastructure with Layer 7 controls designed to prevent application overload.

Rules are evaluated in policy order. Administrators should place narrowly targeted deny and rate-limit rules before broad allows while keeping logging and default behavior understandable.

Production Operations

Cloud Armor observability comes from Cloud Logging and Cloud Monitoring. When request logging is enabled, security-policy logs help identify denied traffic, matched rules, client IP addresses, user agents, and other request attributes. Useful operational signals include:

  • Sudden increases in denied requests

  • Unexpected changes in allowed request volume

  • Backend latency or error-rate increases

  • Repeated rate-limit matches

  • WAF matches associated with a known vulnerability

  • Legitimate users being blocked after a rule or application change

Policy changes should follow the same controls as other production changes: test in a non-production policy, review the rule scope, monitor immediately after rollout, and maintain a rollback path. Broad exclusions can silently weaken WAF protection and should be removed once the underlying issue is fixed.

Failure Modes and Security Boundaries

  • Bypassing the load balancer: If attackers can reach a backend directly, Cloud Armor cannot inspect that traffic. Use private backends, restrictive VPC firewall rules, and no unnecessary public IP addresses.

  • Overly broad allow rules: An early allow rule can bypass later WAF or rate-limit controls. Understand rule precedence before deployment.

  • Poor rate-limit keys: Limiting by a shared NAT address may block many legitimate users, while easily spoofed or overly specific keys may provide little protection.

  • WAF false positives: Business applications using unusual parameters or workflows can trigger legitimate traffic. Add narrow, documented exclusions rather than disabling protection.

  • Application-layer abuse: Cloud Armor reduces harmful traffic, but it does not fix insecure authorization, exposed secrets, or application vulnerabilities.

  • Availability dependency: Internet protection depends on traffic using the protected load-balancing path. Test failover, health checks, and incident runbooks rather than assuming the edge eliminates all availability risks.

AWS, Azure, and GCP Comparison

Capability Google Cloud AWS Azure
Managed WAF and DDoS protection Cloud Armor AWS WAF + AWS Shield Azure Front Door WAF + DDoS Protection
Primary protected entry point External Application Load Balancer CloudFront, ALB, or regional services Front Door, Application Gateway, and related services
Bot management Cloud Armor Enterprise AWS WAF Bot Control Azure Bot Protection
Private east-west traffic Cloud NGFW and VPC firewall controls Network Firewall, Security Groups, and VPC controls Azure Firewall and NSGs
Quotas and pricing Tiered feature and policy limits Per-rule, request, and Shield pricing Plan- and rule-based pricing

Choose Cloud Armor when a GCP application is exposed through an external load balancer and needs integrated WAF, DDoS mitigation, rate limiting, and edge policy enforcement. Use Cloud NGFW or VPC firewall controls for private network segmentation, and implement authorization and validation inside the application itself.

54

What is Google Cloud Identity-Aware Proxy (IAP)?

Identity-Aware Proxy (IAP) is a Google Cloud service that provides identity-based access control for HTTP(S) applications and IAM-authorized TCP tunneling for SSH access to Compute Engine instances. For web applications, IAP performs authentication and authorization before the request reaches the backend, while the application can validate IAP’s signed identity assertion.

Read full answer

Applies toGoogle Cloud IAP (all currently supported versions)

How It Works

IAP commonly sits between a browser or client and a backend application, often behind an external Application Load Balancer:

  1. A user requests an IAP-protected HTTPS application.

  2. If the user has not authenticated, IAP redirects the browser to the configured identity flow, which can use Google sign-in or a federated external identity provider.

  3. IAP evaluates the user’s Cloud IAM permissions for the protected resource.

  4. If access is allowed, IAP forwards the request and adds identity headers, including the signed X-Goog-IAP-JWT-Assertion header.

  5. If access is denied, IAP blocks the request before it reaches the backend.

For SSH, IAP TCP forwarding creates an encrypted tunnel from the client to a Compute Engine instance. IAM controls who may establish the tunnel through the roles/iap.tunnelResourceAccessor role, but IAP does not replace SSH authentication: the client still performs normal SSH host and user authentication after the tunnel is established. This allows SSH to work without exposing port 22 directly to the internet.

IAM and Identity

IAP separates identity verification from resource authorization:

  • Authentication: The identity provider verifies the user, using Google Workspace, Cloud Identity, or a configured federated identity provider.

  • Web authorization: roles/iap.httpsResourceAccessor permits a principal to access IAP-protected web applications.

  • Tunnel authorization: roles/iap.tunnelResourceAccessor permits a principal to establish an IAP TCP tunnel.

  • Service agent: roles/iap.serviceAgent gives the IAP service agent the permissions it needs to operate on protected resources.

Applications exposed to untrusted networks should validate the signed IAP JWT assertion, including its signature, expected audience, expiry, and relevant identity claims. The simpler authenticated-user email header must not be trusted unless the backend can guarantee that requests cannot bypass IAP.

Supported Scenarios

Capability Google Cloud AWS equivalent Azure equivalent
Application access control IAP with Cloud IAM ALB authentication with Cognito or another OIDC provider App Service Authentication with Microsoft Entra ID
Browser-based sign-in Google Workspace, Cloud Identity, or federated identity provider Cognito or an external OIDC provider Microsoft Entra ID or an external OAuth/OIDC provider
SSH access IAP TCP forwarding plus normal SSH authentication SSM Session Manager Azure Bastion
Authorization source Cloud IAM policies IAM for AWS resources; application authorizers or application logic for protected apps Azure RBAC; application authentication and authorization for application data

These products are not exact one-to-one equivalents. In particular, Cognito and Entra ID primarily provide identity, while an application or authorizer usually decides what an authenticated user can access.

Security and Production Considerations

  • Prevent bypass: Do not give users a direct route to the backend. Use private backends where possible. On Compute Engine, permit only the applicable external Application Load Balancer and health-check paths; for IAP SSH forwarding, allow TCP port 22 from IAP’s 35.235.240.0/20 range.

  • Do not trust arbitrary headers: A client can forge X-Goog-Authenticated-User-Email. Block direct access or validate X-Goog-IAP-JWT-Assertion.

  • IAP is not a WAF: Use Cloud Armor for edge web-application and DDoS controls, plus application-level validation, rate limiting, and abuse prevention.

  • Machine clients need a different pattern: Use IAM-authenticated service accounts, Workload Identity Federation, or service-specific authentication rather than treating IAP as a machine-to-machine gateway.

  • Availability dependency: IAP depends on Google Cloud and the configured identity provider. Monitor load-balancer health, backend latency, relevant logs, and identity-provider failures.

  • Audit access: Enable appropriate Cloud Audit Logs and review changes to IAP settings and IAM bindings, especially changes to roles/iap.httpsResourceAccessor and roles/iap.tunnelResourceAccessor.

Best Practices and Failure Modes

  • Grant roles/iap.httpsResourceAccessor at the narrowest practical resource scope rather than project-wide.

  • Use groups or workforce identities for onboarding and offboarding instead of individual user bindings.

  • Test successful authentication, denied IAM access, expired sessions, and direct-backend bypass attempts.

  • Validate the signed IAP JWT in sensitive applications rather than relying only on email headers.

  • Monitor load-balancer logs and metrics, audit IAM changes, and correlate authentication failures with backend health.

  • For IAP-forwarded SSH, continue managing sshd, key access, patching, and OS-level logging; IAP protects the network path but does not secure the guest operating system.

55

What is Google Cloud Secret Manager?

Google Cloud Secret Manager is a managed service for securely storing, controlling, and retrieving sensitive values such as passwords, API keys, and certificates. It provides encrypted storage, versioning, IAM-based access control, replication, and audit logging without putting secrets directly in application code or container images.

Read full answer

Applies toGoogle Cloud Secret Manager (all currently supported versions)

How It Works

  • Secrets and versions: A secret is a logical resource, while each stored value creates an immutable version. A secret can have up to 100 versions.

  • Replication policy: Secret resources use a global API, but their payloads are replicated according to the secret’s policy:

  • automatic replicates data across Google Cloud regions for higher availability.

  • user-managed selects specific replication locations.

  • Version states: Versions can be enabled, disabled, or destroyed. A disabled version cannot be accessed normally but can be re-enabled; a destroyed version cannot be restored. Audit records remain available.

  • Payload limit: A single secret version supports up to 64 KiB of secret data.

  • Encryption: Data is encrypted at rest by default. Customer-managed encryption keys through Cloud KMS can be configured when additional key-control requirements exist.

  • Size and format: Secret Manager stores arbitrary values, commonly text, JSON, certificates, or short configuration blobs.

Security and IAM

Secret Manager uses Google Cloud IAM at the organization, folder, project, or individual secret level. Access to secret data and administrative operations are separate permissions.

The commonly used role is:

  • roles/secretmanager.secretAccessor: Allows the principal to access secret payload versions.

A production application should normally receive only roles/secretmanager.secretAccessor on the specific secrets it needs. Avoid granting broad administrative roles to runtime identities.

Service accounts are recommended for Compute Engine, GKE, Cloud Run, and other hosted workloads. Avoid downloading the secret manually and embedding it in deployment configuration. In GKE, the Secrets Store CSI Driver can mount Secret Manager secrets into workloads.

Secret Manager supports TLS in transit, and retrieving metadata does not reveal the secret payload. Secret values remain sensitive if copied into logs, shell history, CI output, Terraform state, or crash reports.

Production Behavior

  • Version updates: Adding a version does not update consumers that already read or cached its value. An explicitly referenced version remains fixed, while a consumer that reads the latest alias on each request receives the newest enabled version at the next successful read.

  • Controlled rotation: Add and validate a new version before consumers adopt it. If production consumers use latest, adding the version changes that alias immediately, so use pinned versions or a deployment-controlled configuration when a staged rollout is required. For irreversible rollback, add the previous value again as a new version rather than assuming a destroyed version can be restored.

  • Availability: Treat Secret Manager access as a startup dependency. Applications should handle transient failures with bounded retries, timeouts, and clear readiness checks.

  • Monitoring: Enable and retain Secret Manager Data Access audit logs where access visibility is required. Monitor version creation, IAM changes, denied requests, and unusual access patterns through Cloud Logging and Security Command Center.

  • Incident response: If a value is compromised, add a replacement version, update consumers, and then disable or destroy the exposed version. Removing a secret alone does not update already-running applications.

Service Comparisons

Need Google Cloud AWS Azure
Application secrets Secret Manager Secrets Manager Key Vault
Encryption-key management Cloud KMS KMS Key Vault or managed HSM
General secret types Primarily secret values Secrets, binary values, and related metadata Secrets, keys, and certificates
Typical GCP choice Secret Manager for credentials; Cloud KMS for keys — —

Secret Manager is appropriate when an application needs a managed secret payload with IAM and versioning. Cloud KMS is different: it manages cryptographic keys used for encryption, signing, and envelope encryption rather than application passwords or API tokens.

Best Practices and Failure Modes

  • Use least-privilege IAM and separate deployment identities from runtime identities.

  • Use secret references, not literal values, in Cloud Run configuration, Kubernetes manifests, and CI variables.

  • Never print payloads during routine diagnostics; verify access through metadata and audit logs instead.

  • Require rotation procedures that add, validate, activate, and retire versions without causing coordinated downtime.

  • Handle 403 errors by checking IAM bindings and workload identity, and 404 errors by checking the project, secret name, and replication location.

  • Handle disabled or destroyed versions explicitly; restarting the application does not make them accessible again.

  • Keep audit logs in a protected log sink, because application logs and audit records can be deleted or altered independently.

56

What are Cloud KMS and customer-managed encryption keys in GCP?

Cloud KMS is Google Cloud’s centralized service for creating, protecting, rotating, and using cryptographic keys without exposing their raw key material. A customer-managed encryption key (CMEK) is an encryption key whose use and lifecycle are controlled by the customer rather than through Google-managed encryption keys.

Read full answer

Applies toGoogle Cloud current stable services (Cloud KMS API v1)

Core Concepts

Cloud KMS organizes keys into a hierarchy:

  • Key ring: A logical container for related cryptographic keys and an administrative boundary. A key ring has a fixed location.

  • Crypto key: A named key resource containing metadata, an IAM policy, and one or more cryptographic key versions.

  • Key version: An immutable set of cryptographic key material. Rotation creates a new version instead of changing an existing one.

  • Location: Regional, multi-region, and global locations are supported. A key cannot be moved after creation, and its location must be compatible with the resource using it.

  • Purpose: Restricts allowed operations, including symmetric encryption, asymmetric encryption, digital signatures, and HMAC operations.

  • Protection level: Determines where cryptographic operations are performed, including SOFTWARE, HSM, EXTERNAL, and EXTERNAL_VPC protection where supported.

Cloud KMS is comparable to AWS KMS and Azure Key Vault. The closest AWS concept is a customer-managed key in AWS KMS; Azure customer-managed keys are commonly backed by Azure Key Vault.

How CMEK Encryption Works

Managed Google Cloud services typically use envelope encryption:

  1. The service generates a random data encryption key (DEK).

  2. Cloud KMS encrypts the DEK with the CMEK.

  3. The service stores the encrypted DEK with the encrypted resource or its metadata.

  4. When the resource is read, the service requests decryption of the DEK through Cloud KMS and then decrypts the data.

Cloud KMS authorizes operations through IAM and returns cryptographic results rather than exporting the underlying key material. For standard symmetric keys stored in Cloud KMS, key material is not exportable.

CMEK and Other Key Models

Model Management and control Operational trade-off
Google-managed encryption keys Google manages key protection and rotation; customers have little direct lifecycle control Simplest, but customers cannot independently rotate or disable the key
CMEKs in Cloud KMS Customers control key IAM, rotation, disabling, destruction, and location Greater control and separation of duties, with additional IAM, quota, and availability dependencies
CMEKs backed by Cloud HSM or Cloud External Key Manager Customers retain control in dedicated HSMs or supported external key systems Stronger custody or sovereignty requirements, but service and key-type support can be narrower
Customer-supplied encryption keys (CSEK), where supported Customers generate, protect, and supply key material to the consuming service Avoids reliance on Cloud KMS for cryptographic operations but creates key-delivery, storage, recovery, and rotation burdens

A CMEK changes control over the key lifecycle, but it does not remove dependence on Google Cloud. IAM errors, key-location restrictions, KMS quotas, and service outages can still make protected data unavailable.

Security and Operational Behavior

  • IAM is the authorization boundary: The predefined role roles/cloudkms.cryptoKeyEncrypterDecrypter grants symmetric encryption and decryption operations. Production grants should be limited to the identities and service agents required by each service.

  • Managed-service identities matter: For service-encrypted CMEK integrations, grant the exact service agent or service identity specified in that service’s documentation. Depending on the service, the calling principal may also require KMS permissions; a runtime identity’s direct grant does not automatically grant a managed service permission to use the key.

  • Rotation is forward-looking: A new primary version handles new encryption. Existing ciphertext remains associated with the key version that protected its DEK and is not automatically re-encrypted.

  • Disabling can affect existing data: Disabling a key version prevents cryptographic operations with that version, including decryption of data that still depends on it. Consumers may receive authorization or availability errors.

  • Destruction is hazardous: A disabled key version can be scheduled for destruction. Destruction can be canceled while the version is scheduled (30 days by default, configurable from 24 hours to 120 days), but a destroyed version cannot be restored. Do not assume Cloud KMS will prevent destruction because another Google Cloud resource still references the key.

  • Audit usage carefully: Cloud Audit Logs records KMS administration. Data Access logging must be enabled separately if decrypt-operation visibility is required. Monitor key administration, IAM changes, disabled keys, authorization failures, latency, and quota consumption without logging plaintext or key material.

Production Best Practices

  • Use separate keys for production, non-production, and distinct security or compliance boundaries.

  • Manage key rings, key versions, IAM bindings, and locations through reviewed infrastructure as code.

  • Require approval for key or version destruction and use lifecycle safeguards such as Terraform prevent_destroy.

  • Re-encrypt stored data deliberately before retiring older key versions.

  • Validate that service identities, key locations, IAM conditions, and KMS quotas match each consumer’s requirements.

  • Maintain tested recovery and incident runbooks for disabled keys, mistaken IAM changes, pending destruction, quota exhaustion, and regional failures.

57

What is VPC Service Controls in Google Cloud?

VPC Service Controls is a Google Cloud security perimeter that limits how supported Google Cloud resources can be accessed and where their data can be sent. It works alongside IAM, protecting organizations from unauthorized access and data exfiltration across project or organizational boundaries.

Read full answer

Applies toGoogle Cloud VPC Service Controls (current)

Core Concepts

A service perimeter defines a protected boundary for Google Cloud resources.

  • Access policy: The organizational policy container that owns service perimeters, access levels, and perimeter bridges.

  • Service perimeter: A boundary associated with an access policy and the projects it protects.

  • Protected resources: Resources inside the perimeter, such as a project containing Cloud Storage buckets or BigQuery datasets.

  • Restricted services: Google Cloud services protected by the perimeter. Support varies by service, so verify that each resource type is covered.

  • Access level: Grants specified users or groups permission to cross the boundary under configured conditions. It is an explicit exception, not a general public-access setting.

  • Perimeter bridge: Allows resources in one perimeter to communicate with resources in another perimeter.

VPC Service Controls is not a replacement for IAM. IAM still determines what an identity is authorized to do, while VPC Service Controls determines whether the request is allowed to cross the security boundary at all.

How Enforcement Works

For a request involving a protected service, Google Cloud evaluates the request’s identity and available access context against the perimeter’s policies.

  1. Ingress evaluation: Checks whether the requester is inside the perimeter or has an applicable access level or other explicit exception.

  2. Identity authorization: IAM still evaluates the caller’s roles and permissions.

  3. Egress evaluation: Checks where a resource inside the perimeter is permitted to send data.

  4. Logging: Access Context Manager audit logs and Cloud Audit Logs can show denied or allowed perimeter activity.

Ingress and egress rules should normally be restrictive. For example, a perimeter can allow workloads in a production project to read a specific BigQuery dataset while preventing arbitrary external access. Egress rules are especially important because unrestricted egress can allow sensitive data to leave the boundary.

VPC Service Controls applies primarily to supported Google Cloud service APIs and resources. It is not a general-purpose firewall, does not inspect arbitrary network traffic, and does not prevent an authorized identity from reading data that it is already permitted to access.

GCP and Cloud Comparisons

Capability Google Cloud AWS Azure
Perimeter or policy boundary VPC Service Controls Organizations SCPs and related controls Azure Policy and Management Groups
Network path restriction Service perimeter and related networking controls VPC endpoints, PrivateLink, and security groups Private Link, private endpoints, NSGs, and Azure Firewall
Main purpose Prevent unauthorized Google Cloud access and data exfiltration Control API actions and network access Control resource behavior and network access
Closest equivalent No exact one-to-one equivalent SCPs plus PrivateLink or VPC controls Azure Policy plus Private Link or Firewall

Choose VPC Service Controls when the primary requirement is a Google Cloud data-access boundary, such as isolating production analytics data from development identities or external contractors. Use IAM, VPC firewall rules, private connectivity, and organization policies alongside it.

Production Best Practices and Failure Modes

  • Start in a non-production environment: Test every client, integration, CI/CD job, and support workflow before applying the perimeter broadly.

  • Use narrow access levels: Grant access to specific identities or groups and use conditions where possible. Treat access levels as production change objects.

  • Avoid wildcard egress rules: Allow only required destinations and services, then monitor unexpected access-denied events.

  • Monitor continuously: Review Access Context Manager audit logs, Cloud Audit Logs, Security Command Center findings, and application errors for perimeter violations.

  • Plan for failures: A missing access level, incorrect resource membership, unsupported service, or overly restrictive egress rule can interrupt production workloads.

  • Protect the boundary itself: Limit who can modify access policies, perimeters, access levels, and bridges. VPC Service Controls does not replace encryption, DLP, least-privilege IAM, or data-loss prevention controls.

  • Plan recovery and continuity: Keep tested break-glass access for authorized responders, but audit its use. VPC Service Controls provides no backup or data-recovery capability.

58

What is Google Cloud Security Command Center?

Google Cloud Security Command Center (SCC) is Google Cloud’s centralized security posture and threat-detection service. It brings asset inventory, vulnerability findings, misconfiguration detection, threat detection, sensitive-data discovery, and compliance capabilities into one view.

Read full answer

Applies toGoogle Cloud Security Command Center (Standard, Premium, and Enterprise editions; current service)

Core Capabilities

SCC helps security and operations teams identify and prioritize risk across Google Cloud resources.

  • Asset visibility: Discovers Google Cloud assets and organizes them by organization, folder, project, and resource type.

  • Security posture: Identifies misconfigurations involving IAM, public access, encryption, networking, and other supported controls.

  • Vulnerability management: Reports vulnerable software, container images, and deployed workloads for supported resources.

  • Threat detection: Uses security telemetry and logs to identify suspicious behavior, including anomalous service-account activity, malware, and cryptocurrency mining.

  • Sensitive-data discovery: Identifies sensitive information in supported data stores. Access Transparency Analyzer separately provides access-risk insights where available.

  • Compliance management: Evaluates resources against supported benchmarks and frameworks and provides posture findings and compliance reports.

  • Attack-path intelligence: Shows how supported weaknesses and excessive permissions might form a reachable attack path.

Availability depends on the configured edition, enabled modules, resource coverage, and regional support.

How It Works

SCC combines Google Cloud resource metadata with security telemetry generated by Google Cloud services.

  1. Discovery uses Google Cloud Asset Inventory and other resource information to identify assets and their configurations.

  2. Detection modules inspect configurations, vulnerability information, identity activity, network behavior, and service logs.

  3. Findings identify the affected resource, risk category, severity, evidence, and remediation guidance.

  4. Notification and export sends findings through Pub/Sub and supported integrations such as SIEM or ticketing workflows.

  5. Attack-path analysis, where supported, relates weaknesses across resources to reveal exploitable chains.

A finding is a risk signal, not proof of exploitation. For example, a public-bucket finding still requires investigation to determine whether the bucket contains sensitive data and was intentionally exposed.

Production Operations

A production deployment should integrate SCC with existing SRE, security, and incident-response workflows.

  • Alert on new high-severity findings affecting production, internet-facing, identity, or regulated resources.

  • Group findings by project, resource, category, and business owner.

  • Create tickets containing the affected resource, evidence, remediation guidance, owner, and escalation deadline.

  • Measure finding age, recurrence, and closure quality rather than tracking only alert volume.

  • Export relevant findings through Pub/Sub and enrich them with asset ownership.

  • Apply deduplication, severity normalization, suppression, and documented risk-acceptance rules.

Active organization-level findings can be reviewed with:

gcloud scc findings list --organization=ORGANIZATION_ID --state=ACTIVE

SCC does not replace Cloud Audit Logs, Cloud Logging, or a full SIEM. It adds security-focused findings and context to those sources.

IAM and Organizational Scope

SCC is normally activated at the organization level to provide centralized coverage across folders and projects.

  • Grant least-privilege access for finding triage, finding management, and security administration.

  • Restrict broad viewer access because findings can expose infrastructure and vulnerability details.

  • Separate users who investigate findings from users who administer organization-wide security settings.

  • Align access with the organization’s folder and project ownership model.

  • Monitor SCC configuration changes through Cloud Audit Logs.

  • Keep authoritative remediation in the underlying resource rather than relying on finding state as proof that the risk is fixed.

Comparison With AWS and Azure

Platform Central security platform Native threat detection Vulnerability management Sensitive-data discovery
Google Cloud Security Command Center SCC threat modules SCC vulnerability findings Edition- and resource-dependent
AWS AWS Security Hub AWS GuardDuty Amazon Inspector AWS Macie
Azure Microsoft Defender for Cloud Defender for Cloud plans Defender integrations Defender or Purview, depending on plan and service

Choose SCC when a team wants a unified Google Cloud view spanning IAM posture, vulnerabilities, threats, compliance, and security operations. The AWS and Azure comparisons also combine several related products rather than relying on one service for every capability.

Best Practices and Common Pitfalls

  • Validate coverage: Compare SCC’s asset inventory with expected projects, regions, and services.

  • Prioritize actionable findings: Focus first on internet-facing, production, identity, and data-bearing resources.

  • Track remediation quality: Monitor finding age, recurrence, overdue remediation, and reopened issues.

  • Tune carefully: Validate intentional configurations and document accepted exceptions before suppressing alerts.

  • Protect finding data: Restrict access because findings reveal resource names, vulnerabilities, and relationships.

  • Check boundaries: Confirm detector availability, edition requirements, resource support, and regional behavior before relying on a finding type operationally.

59

How do Google Cloud Logging and log sinks work?

Google Cloud Logging is Google Cloud’s managed service for ingesting, storing, querying, and monitoring logs from GCP resources, applications, and supported agents. A log sink is a filtered routing rule that copies matching log entries to another destination while leaving the original entries in Cloud Logging.

Read full answer

Applies toGoogle Cloud Logging (current supported service) and gcloud CLI (current stable)

Core Architecture

  • Ingestion: Logs arrive from services such as Compute Engine, GKE, Cloud Run, and Cloud Storage, or from the Ops Agent, Cloud Logging API, and application stdout/stderr.

  • Organization: Each entry contains a logName, resource.type, severity, timestamps, labels, and text or structured payload fields.

  • Querying: Logs Explorer and the Logging API use filters based on fields such as severity, resource.type, and logName.

  • Retention: The automatically created _Default bucket normally retains logs for 30 days, while required logs in _Required are retained for 400 days. Retention can be changed where permitted.

  • Export: Sinks create independent copies; exporting does not remove or shorten the source log’s retention.

How Sink Routing Works

  1. A sink defines a destination, optional filter, and dedicated writer identity.

  2. Incoming entries are evaluated against each sink’s filter.

  3. Every matching sink receives a copy of the entry.

  4. The writer identity authenticates to the destination and writes the exported records.

Sinks can be owned by an organization, folder, or project. A project sink includes that project’s child resources by default but does not capture logs from other projects. Routing is asynchronous and should not be treated as an exactly-once or ordered messaging system. Destination failures can delay exports, and organizations should not rely on sink delivery for transactional correctness.

Destination Choices

Destination Typical use Sink writer access AWS/Azure equivalent
Cloud Storage Low-cost, long-term archives roles/storage.objectCreator Amazon S3 / Azure Blob Storage
Pub/Sub Event streaming and custom processing roles/pubsub.publisher Amazon Kinesis or Firehose / Azure Event Hubs
BigQuery SQL-based log analytics roles/bigquery.dataOwner for sink-managed dataset/table creation; scoped roles/bigquery.dataEditor for an existing target Athena / Azure Log Analytics
Cloud Logging bucket Separation of retention or access boundaries roles/logging.bucketWriter on the destination bucket CloudWatch log groups / Log Analytics workspaces

Cloud Storage is usually best for retention, Pub/Sub for real-time processing, and BigQuery for structured analysis. AWS CloudWatch Logs and Azure Monitor Logs provide similar central logging, but their export mechanisms are subscription filters and diagnostic settings respectively.

Code Example

The following creates a bucket, exports Admin Activity audit logs, and grants the sink permission to write objects:

PROJECT_ID="example-project"
BUCKET_NAME="example-log-archive-unique-suffix"
SINK_NAME="audit-archive"

gcloud storage buckets create "gs://${BUCKET_NAME}" \
  --project="${PROJECT_ID}" \
  --location="us-central1" \
  --uniform-bucket-level-access

gcloud logging sinks create "${SINK_NAME}" \
  "storage.googleapis.com/${BUCKET_NAME}" \
  --project="${PROJECT_ID}" \
  --log-filter="logName:\"projects/${PROJECT_ID}/logs/cloudaudit.googleapis.com%2Factivity_log\""

WRITER_IDENTITY="$(gcloud logging sinks describe "${SINK_NAME}" \
  --project="${PROJECT_ID}" \
  --format="value(writerIdentity)")"

gcloud storage buckets add-iam-policy-binding "gs://${BUCKET_NAME}" \
  --project="${PROJECT_ID}" \
  --member="${WRITER_IDENTITY}" \
  --role="roles/storage.objectCreator"

The gcloud logging sinks create command creates a dedicated service account automatically when one is not supplied. For Pub/Sub, grant that identity roles/pubsub.publisher on the topic. For BigQuery, use roles/bigquery.dataOwner when the sink must create the dataset and table; for a pre-existing target, grant the narrowest applicable dataset or table permissions, commonly roles/bigquery.dataEditor.

Production, Security, and Failure Modes

  • Monitor delivery: Alert on sink errors and monitor destination-specific signals such as Pub/Sub backlog, failed object writes, and BigQuery ingestion failures.

  • Control permissions: roles/logging.configWriter can redirect project logs, so grant it narrowly. Review sink changes as security-sensitive configuration.

  • Protect archives: For Cloud Storage, consider retention policies, bucket versioning, object lock, customer-managed encryption keys, and lifecycle rules.

  • Avoid sensitive data: Logs commonly contain tokens, request bodies, and user data. Redact secrets before ingestion rather than relying on downstream filtering.

  • Plan capacity and cost: Storage objects, BigQuery streaming ingestion, network transfer, and longer retention all increase cost. Use targeted filters and lifecycle controls.

  • Design for failure: A sink is a secondary export path, not a guaranteed backup. Keep primary logs available through Cloud Logging and validate archived records periodically.

60

How does Google Cloud Monitoring work, and how are alerts created?

Cloud Monitoring is Google Cloud’s time-series observability service. It ingests metrics from Google Cloud services, agents, and custom applications, stores them for analysis, powers dashboards, and evaluates alert policies that notify operators when conditions are breached.

Read full answer

Applies toGoogle Cloud Monitoring (current) and Google Cloud CLI (current stable)

How Monitoring Works

Google Cloud services automatically publish metrics such as CPU utilization, request latency, database saturation, and error counts. Agents and applications can publish custom business and application metrics.

  • Metric: A measured value, such as compute.googleapis.com/instance/cpu/utilization.

  • Time series: The values of one metric for one monitored resource and label combination.

  • Monitored resource: The object producing the metric, such as a Compute Engine instance or Cloud Run revision.

  • Dashboard: A visualization combining charts, metrics, and layout information.

  • Alerting policy: A rule containing one or more conditions and zero or more notification channels. Conditions can evaluate thresholds, missing data, or matched logs.

Monitoring policies are evaluated periodically against recently received data. Delayed or missing telemetry can therefore delay detection or prevent a normal threshold condition from firing. Cloud Logging log-based metrics also appear as Monitoring time series, but Cloud Monitoring is primarily a metrics service rather than a full log-management system.

How Alert Evaluation Works

A threshold condition selects time series with a filter, aggregates their data, and compares the result with a threshold. For example:

metric.type = "compute.googleapis.com/instance/cpu/utilization"
AND resource.type = "gce_instance"

A condition can specify:

  • Comparison: Such as COMPARISON_GT for greater than.

  • Threshold: The value that triggers the condition.

  • Duration: How long the condition must remain true.

  • Aggregation: An alignment period and function such as ALIGN_MEAN.

  • Absence condition: Fires when matching time series stop reporting data.

Each matching time series is evaluated independently unless a cross-series reduction is configured. An absence condition can detect missing data for a series previously observed, but it cannot detect a resource that never produced any time series.

Multiple conditions can be combined with AND or OR. This supports multi-window, multi-burn-rate alerting for SLOs, which is less noisy than a single low threshold.

Code Example

First, identify a verified notification channel:

gcloud monitoring channels list --project=example-project

Then create an alerting policy that alerts when the 60-second mean CPU utilization exceeds 80% for five minutes:

gcloud monitoring alerts create \
  --project=example-project \
  --display-name='GCE CPU above 80 percent' \
  --criteria='metric.type="compute.googleapis.com/instance/cpu/utilization" AND resource.type="gce_instance" AND resource.labels.zone="us-central1-a"' \
  --aggregation='alignmentPeriod=60s,perSeriesAligner=ALIGN_MEAN' \
  --conditions='condition=conditionThreshold,comparison=COMPARISON_GT,thresholdValue=0.8,duration=300s' \
  --notification-channels='projects/example-project/notificationChannels/1234567890' \
  --enabled

CPU utilization is expressed as a fraction from 0 to 1, so 0.8 means 80%. The filter covers every matching instance in the specified zone; restrict it with resource.labels.instance_id to target one instance.

List and inspect the resulting policies with:

gcloud monitoring alerts list --project=example-project

gcloud monitoring alerts describe ALERT_POLICY_ID \
  --project=example-project

Production Practices

  • Use service-level signals: Prefer user-visible symptoms such as latency and error rate over isolated resource metrics.

  • Add a duration: A five-minute threshold is generally less noisy than an instantaneous spike.

  • Avoid alert fatigue: Give every alert an owner, severity, runbook, and clear response action.

  • Monitor the alert path: Track notification-channel failures, disabled policies, missing metrics, and pending channel verification.

  • Use aggregation carefully: Cross-series reduction can combine related instances, but it can also conceal failures affecting only a subset.

  • Prefer burn-rate alerts for SLOs: They balance fast detection against sustained error-budget consumption.

  • Control cardinality: Excess labels can consume the active time-series quota and increase costs for billable custom metrics.

  • Use complementary checks: Metric alerts detect abnormal measurements, absence alerts detect missing expected telemetry, and uptime checks test configured synthetic endpoints.

Security and Failure Modes

  • Grant roles/monitoring.viewer for read access and roles/monitoring.editor only where alert policies must be changed; use custom roles for tighter privilege control.

  • Scope permissions to the narrowest practical folder, project, or monitored resource.

  • Verify email and SMS recipients, and protect webhook channels with appropriate authentication.

  • Store webhook credentials in an approved secret-management system rather than scripts or source control.

  • Remember that healthy monitoring infrastructure does not guarantee healthy applications: agents, permissions, quotas, and metric pipelines can all fail.

  • Combine alerts with dashboards, logs, traces, and incident-management processes. A notification alone does not establish severity, diagnosis, or ownership.

61

How do Cloud Trace and Cloud Profiler work in Google Cloud?

Cloud Trace provides distributed tracing for request lifecycles across services, while Cloud Profiler samples execution stacks to identify CPU, memory, allocation, and contention hotspots. They are complementary: tracing identifies a slow request path, and profiling shows where a workload consumes resources around the same time, although the two datasets are not directly linked at the individual-span level.

Read full answer

Applies toGoogle Cloud Trace API v2 and current supported Cloud Profiler agents; Cloud Profiler’s managed agent has no public semantic version

Core Difference

  • Cloud Trace: Records parent and child operations as spans, including timestamps, durations, attributes, status, and identifiers such as trace_id and span_id.

  • Cloud Profiler: Periodically captures stack samples and aggregates them into CPU, heap, allocation, contention, or thread profiles.

  • Combined use: A trace may show that POST /checkout consistently takes 800 ms. A CPU profile collected from the same service and version may then reveal CPU-heavy serialization or encryption. It will not expose time spent waiting inside a remote database unless that wait is separately visible through contention or other telemetry.

Cloud Trace is primarily a request-oriented latency-analysis tool. Cloud Profiler is a continuous, workload-oriented performance-analysis tool whose profiles represent aggregated activity rather than one request.

How They Work

Cloud Trace combines automatic and instrumented telemetry:

  • Some supported Google Cloud services provide trace data automatically.

  • Google Cloud client libraries and supported OpenTelemetry instrumentation create application spans.

  • W3C Trace Context, commonly carried in the traceparent header, propagates trace context between services.

  • Exported spans are grouped into traces by trace ID. With OpenTelemetry, an OpenTelemetry Collector can receive OTLP spans and export selected traces to Google Cloud.

  • Span attributes can identify the service, operation, RPC method, HTTP route, status code, location, and associated resource.

Cloud Profiler uses an agent running with the workload:

  • The agent periodically samples stacks and uploads aggregated profiles.

  • Profiles are associated with the service, service version, monitored resource, and profile type.

  • CPU profiles identify sampled CPU time and its call paths.

  • Heap profiles show allocations made since the previous heap-profile collection, while allocation profiles show allocation activity over their configured interval.

  • Contention profiles identify time spent waiting for locks and other synchronization primitives.

  • Thread profiles provide a view of activity across an application’s threads.

Neither product replaces structured logging or metrics. Traces provide request-level causality, profiles provide workload-level code evidence, and Cloud Monitoring supplies fleet-wide signals and SLO measurements.

Service Comparisons

Google Cloud service AWS equivalent Azure equivalent Important distinction
Cloud Trace AWS X-Ray Azure Application Insights Distributed request tracing and dependency analysis
Cloud Profiler AWS CodeGuru Profiler Azure Monitor Profiler Cloud Profiler supports CPU, heap, allocation, contention, and thread profiling; the cloud products differ in instrumentation, analysis interfaces, sampling, retention, and pricing

Migration should preserve instrumentation concepts rather than assume that configuration, permissions, or sampling behavior is identical across providers.

Production Practices

  • Sampling: Use parent-based trace sampling to control volume. Head sampling can discard rare errors before they are known, so use collector-side tail sampling to retain errors, high-latency requests, and other interesting traces when collector resources permit.

  • Security: Treat trace attributes and profile labels as potentially sensitive. Never record tokens, credentials, personal data, raw request bodies, or unrestricted query strings. Use application-level allowlists and redaction.

  • Overhead: Benchmark trace-export and profiler-agent overhead under peak load. Monitor CPU usage, collector memory pressure, dropped spans, upload failures, and ingestion volume.

  • Observability: Correlate trace IDs with structured logs. Alert on failed exports, unhealthy profiler agents, abnormal trace volume, and service-level objective regressions.

  • Incident response: Capture profiles while the symptom is active, then compare profiles across failing and healthy versions or regions. Keep trace sampling and deployment changes controlled so incident evidence is not obscured.

  • Cost control: Remove unused attributes, tune trace sampling, and monitor billing separately for trace ingestion or retention and Cloud Profiler processing.

  • Retention: Treat traces and profiles as diagnostic data with managed quotas and retention periods, not as permanent audit or compliance records.

62

How does Google Cloud Build automate CI/CD workflows?

Google Cloud Build is Google Cloud’s managed CI/CD service. It fetches source code, executes build steps in isolated environments, publishes artifacts such as container images, and records build status and logs for deployment workflows.

Read full answer

Applies toGoogle Cloud Build (current stable service)

How It Works

A build begins through a repository trigger or a manual command such as gcloud builds submit:

  1. Cloud Build retrieves the requested source revision.

  2. It provisions the required build environment.

  3. Each step runs in an isolated container. Steps share the build workspace, allowing later steps to consume files produced by earlier ones.

  4. The build pushes images or other artifacts to services such as Artifact Registry or Cloud Storage.

  5. Cloud Build publishes the final status and logs to Cloud Logging.

A typical pipeline compiles an application, runs tests, builds a container image, scans it, and publishes it. A nonzero exit code fails the build, and subsequent dependent steps do not run.

Capabilities and Design Choices

  • cloudbuild.yaml: Defines steps, images, substitutions, artifacts, and build options.

  • Build steps: Run shell commands, Docker commands, Cloud SDK commands, or custom tooling.

  • Triggers: Start builds from repository events, branch or tag patterns, or manual submissions.

  • Substitutions: Parameterize image names, regions, and versions. Built-in values such as COMMIT_SHA and BUILD_ID are available for supported triggers.

  • Service accounts: Control what the build can access. The build service account should have only the permissions required by the pipeline.

  • Caching: Reuse dependencies or build outputs to reduce execution time and cost.

  • Artifacts: Container images commonly go to Artifact Registry; build outputs and logs can also use Cloud Storage.

  • Promotion: Cloud Build can publish artifacts, while Cloud Deploy is generally preferable for controlled promotion through environments such as development, staging, and production.

Cloud Build Compared With AWS and Azure

Capability Google Cloud AWS Azure
Managed build execution Cloud Build CodeBuild Azure Pipelines
Primary configuration cloudbuild.yaml buildspec.yml azure-pipelines.yml
Separate orchestration service Cloud Deploy and related services CodePipeline Usually jobs within Azure Pipelines
Artifact container registry Artifact Registry Elastic Container Registry Azure Container Registry

Cloud Build is usually a strong choice when the workload already runs on GCP and needs integration with IAM, Artifact Registry, Cloud Build triggers, and Google Cloud operational tooling. Azure Pipelines provides a broader workflow suite, while CodeBuild separates build execution from pipeline orchestration more explicitly.

Production Operations

  • Monitor builds: Alert on failure rate, queue delay, build duration, and infrastructure-related errors using Cloud Monitoring and Cloud Logging.

  • Control promotion: Do not deploy production directly from an untrusted pull request. Require protected branches, approval gates, tests, and provenance checks.

  • Manage cost: Cloud Build consumes compute resources according to the selected worker configuration. Reduce unnecessary parallelism, use appropriate machine types, and enable caching.

  • Handle capacity limits: Organization quotas can delay or reject builds. Monitor quota consumption and avoid scheduling large builds without capacity planning.

  • Retain evidence: Configure log retention according to audit and incident-response requirements.

  • Test pipeline changes: Treat builder-image updates and base-image changes like production changes by validating them in a non-production pipeline.

Security and Failure Modes

  • Use a least-privilege service account instead of broad project-wide roles.

  • Retrieve credentials from Secret Manager rather than storing them in source, substitutions, or image layers.

  • Avoid Docker socket mounting or privileged containers unless the pipeline has a documented, reviewed need.

  • Pin builder images and dependencies by immutable version or digest where practical.

  • Treat network failures, registry outages, and flaky tests as distinct incidents. Scripts should implement safe, idempotent retries because Cloud Build does not generally retry an arbitrary failed step.

  • Record the source commit, builder configuration, artifact digest, and deployment approval together so an artifact can be traced back to the code that produced it.

63

How does Google Cloud Artifact Registry work?

Artifact Registry is Google Cloud’s managed repository for storing and distributing container images and software packages. It gives teams a private, IAM-controlled alternative to public registries such as Docker Hub while integrating directly with GKE, Cloud Run, Cloud Build, and other Google Cloud services.

Read full answer

Applies toGoogle Cloud Artifact Registry (current service; no numbered release versions)

Core Concept

Artifact Registry organizes artifacts into repositories scoped to a Google Cloud project and a location, such as us-central1 or europe-west1.

  • Container images: Docker and OCI images used by GKE, Cloud Run, and Compute Engine.

  • Language packages: Formats such as Maven, npm, Gradle, Python, and Go.

  • Versioned artifacts: Each artifact version is stored by digest, while tags provide movable names such as latest or v2.4.1.

  • Managed service: Google Cloud operates the registry, including storage, scaling, and service maintenance; customers do not run registry containers themselves.

The repository format determines which artifact types it accepts. For example, a Docker repository should not be used to store Maven artifacts.

How It Works

A typical container-image workflow is:

  1. A CI pipeline builds an image and pushes it to a repository.

  2. Artifact Registry authenticates the publisher and stores the image manifest and layers.

  3. A tag such as v2.4.1 points to the immutable image digest.

  4. GKE, Cloud Run, or another runtime authenticates and pulls the required manifest and layers.

  5. The runtime starts the container using the resolved image.

A repository can be created with the gcloud CLI:

gcloud artifacts repositories create app-images --repository-format=docker --location=us-central1 --description="Application images"

Images in that repository are addressed using a hostname such as us-central1-docker.pkg.dev/example-project/app-images/image-name:tag.

Production Operations

  • Region selection: Repository location affects latency, data residency, and resilience. Choose the location close to the workloads that pull the artifacts.

  • Multi-region: Dual-region and multi-region repository locations can improve availability, but they introduce replication behavior and should be selected according to recovery requirements.

  • Monitoring: Track push failures, pull errors, latency, storage growth, and quota-related issues through Cloud Monitoring and Cloud Audit Logs.

  • Availability: A regional repository should not be treated as a cross-region backup. For critical release artifacts, replicate or promote digests to another repository and location.

  • Retention: Deleting a version does not merely remove a tag; it removes the referenced artifact. Define retention and cleanup rules that match release and rollback requirements.

  • Upgrades: The service is server-side and requires no customer-managed infrastructure upgrades, but consumers should still test new application and platform releases before production rollout.

Production deployments should prefer immutable digests over tags. Otherwise, moving a tag such as latest can silently cause different instances to run different code.

Security and Failure Modes

Use IAM to grant only the required repository permissions:

  • roles/artifactregistry.reader: Pull artifacts.

  • roles/artifactregistry.writer: Push and pull artifacts.

  • roles/artifactregistry.admin: Manage repository configuration; avoid routine use.

For example, grant a Cloud Run runtime service account read access:

gcloud artifacts repositories add-iam-policy-binding app-images --location=us-central1 --member=serviceAccount:runtime@example-project.iam.gserviceaccount.com --role=roles/artifactregistry.reader

Common operational failures include:

Failure Typical cause Mitigation
404 or artifact not found Wrong project, region, repository, tag, or digest Verify the full image URI and repository location
403 permission denied Missing IAM role or incorrect service account Check the principal and repository-level IAM binding
Pull timeouts Network policy, VPC Service Controls, or regional service issue Review network paths and Google Cloud service status
Unexpected image version A mutable tag was moved Deploy by digest and control who can push tags
Storage or quota pressure Excess versions or insufficient quotas Apply retention rules and review quotas and billing

For external or user identities, prefer Workload Identity Federation over long-lived service-account keys. Sensitive repositories should also use least-privilege IAM, audit logging, vulnerability scanning through services such as Container Analysis, and encryption controls appropriate to organizational policy.

AWS and Azure Comparison

Capability Google Cloud AWS Azure
Managed artifact registry Artifact Registry Amazon ECR Azure Container Registry
Container images Supported Supported Supported
Language package hosting Maven, npm, Gradle, Python, Go, and others Primarily container images; CodeArtifact for packages Primarily container images; Azure Artifacts for packages
Private IAM integration Google Cloud IAM AWS IAM Azure RBAC and Entra ID
Common consumption GKE and Cloud Run EKS and ECS AKS and Container Apps
Best fit GCP-native CI/CD and container platforms AWS-centric workloads Azure-centric workloads

Choose Artifact Registry for private container and package distribution on Google Cloud, especially when CI/CD, GKE, and Cloud Run already use Google Cloud IAM and networking.

64

How does Google Cloud Deploy work?

Google Cloud Deploy is a managed release-orchestration service for delivering containerized applications to GKE, Cloud Run, and Cloud Run jobs. It builds a versioned release once, then progressively promotes that same release through development, staging, and production targets.

Read full answer

Applies toGoogle Cloud Deploy (managed service, 2024) and gcloud CLI 480+

Architecture and Responsibilities

Cloud Deploy coordinates the path from a source revision to a running application:

  1. Cloud Build or another CI system builds and tests the application.

  2. Skaffold defines how the application is built, packaged, and deployed.

  3. Cloud Deploy creates a release and manages rollouts across one or more targets.

  4. The target environment performs the actual deployment, such as updating a GKE workload or a Cloud Run service.

Cloud Deploy is not a complete CI/CD platform by itself. It focuses on controlled delivery, promotion, and rollout verification.

Core Concepts

  • Delivery pipeline: Defines the ordered sequence of targets through which a release moves.

  • Target: Represents an environment or deployment destination, such as a GKE cluster, Cloud Run service, or Cloud Run job.

  • Release: A versioned deployment unit containing the application image and deployment configuration.

  • Rollout: One attempt to deliver a release to a particular target.

  • Execution configuration: Controls target-specific behavior, including service accounts, permissions, GKE namespaces, Cloud Run services, rollout strategy, and approval requirements.

  • Rollout strategy: Supports standard deployment and canary delivery, where a configured percentage of traffic receives the new version before full rollout.

A release can be promoted without rebuilding it. This reduces differences between what passed testing and what reaches production.

Production Behavior and Failure Handling

Cloud Deploy supports manual promotion, approval gates, canary rollouts, and verification through the rollout strategy. Verification commonly relies on health checks and Cloud Monitoring signals.

Operational behavior is important:

  • A failed rollout is recorded against the release and target. It does not automatically prove that the previous production version remains healthy.

  • A failed canary can affect a subset of production traffic, so error rates, latency, saturation, and business metrics should be evaluated before further promotion.

  • Inspect release and rollout status using commands such as gcloud deploy releases describe and gcloud deploy rollouts describe.

  • Cloud Deploy does not replace database migration tooling, feature-flag management, or application-level rollback procedures.

  • Teams should define who can approve, retry, promote, and roll back a release before an incident occurs.

Cloud Logging and Cloud Monitoring should capture rollout events and correlate them with application telemetry. High-risk releases should use canaries, small traffic increments, explicit acceptance criteria, and a documented rollback decision owner.

Security and Operational Trade-offs

Use separate identities for pipeline execution, release creation, promotion, and runtime workloads. Grant only the permissions each target requires, and avoid embedding credentials in Skaffold files or release metadata.

For sensitive workloads:

  • Use private GKE clusters where appropriate.

  • Store application secrets in Secret Manager rather than manifests.

  • Use dedicated service accounts and avoid broad primitive roles.

  • Restrict promotion permissions to authorized release operators.

  • Review Cloud Audit Logs for deployment and approval activity.

  • Evaluate VPC Service Controls and customer-managed encryption keys for workloads with strict isolation or compliance requirements.

Cloud Deploy adds little direct infrastructure cost, but deployments still incur charges for Cloud Build, GKE or Cloud Run, logs, storage, and network egress.

AWS and Azure Equivalents

Capability Google Cloud AWS Azure
Release orchestration Cloud Deploy CodePipeline with CodeDeploy Azure DevOps Pipelines with Environments
Build and test Cloud Build CodeBuild Azure Pipelines
Container deployment GKE or Cloud Run EKS or ECS AKS or Azure Container Apps
Controlled promotion Targets and rollouts Deployment groups and actions Environments and approvals

These products are not identical. AWS CodePipeline is a broader workflow orchestrator, while Azure DevOps provides pipeline and environment controls as part of a larger application-lifecycle platform.

Operational Best Practices

  • Keep pipeline, target, and release names consistent across regions and environments.

  • Promote immutable, uniquely tagged images rather than mutable tags.

  • Require automated tests and security scans before production promotion.

  • Use approval gates for sensitive targets and canary delivery for high-traffic services.

  • Monitor deployment health with service-level indicators, not only container readiness.

  • Test promotion and rollback procedures regularly under production-like conditions.

  • Record the release identifier in change tickets and incident timelines for traceability.

65

How should I manage GCP infrastructure with Terraform?

Use Terraform’s Google provider to declare GCP resources as code, review the resulting plan, and apply changes through Google Cloud APIs. Store Terraform state in a private Google Cloud Storage bucket so teams share a consistent view of production infrastructure.

Read full answer

Applies toTerraform >= 1.6, < 2.0 and hashicorp/google provider >= 6.0, < 8.0

How It Works

Terraform uses a declarative workflow:

  1. Run terraform init to install providers and configure the GCS backend.

  2. Run terraform plan; by default, Terraform refreshes remote objects before calculating changes.

  3. Review the plan for replacements, IAM changes, exposed resources, and quota impact.

  4. Run terraform apply to execute the approved changes through GCP APIs.

  5. Persist resource attributes in state for subsequent operations.

GCP’s hierarchy is normally organization → folders → projects. Permissions, organization policies, and service quotas can prevent otherwise valid plans from being applied.

GCP area Example Terraform resources Typical choice
Applications google_cloud_run_v2_service, google_container_cluster, google_app_engine_application, google_cloudfunctions2_function Cloud Run for stateless HTTP services; GKE when Kubernetes is required; App Engine for its managed application platform; Cloud Functions for event-driven functions
Compute google_compute_instance, google_compute_instance_group, google_compute_region_instance_group_manager Compute Engine for custom VMs, operating systems, or machine-level control
Data google_storage_bucket, google_sql_database_instance, google_spanner_instance, google_firestore_database, google_bigtable_instance, google_bigquery_dataset Cloud SQL for relational workloads, Spanner for globally distributed transactions, Firestore for documents, Bigtable for wide-column data, and BigQuery for analytics
Messaging google_pubsub_topic, google_pubsub_subscription Durable event delivery between services
Networking google_compute_network, google_compute_subnetwork, google_compute_global_address, google_compute_backend_service VPC networking and global load balancing

Code Example

This creates a versioned GCS bucket, Cloud Run service, runtime service account, and invoker binding.

main.tf:

terraform {
  required_version = ">= 1.6.0, < 2.0.0"

  required_providers {
    google = {
      source  = "hashicorp/google"
      version = ">= 6.0, < 8.0"
    }
  }

  backend "gcs" {}
}

provider "google" {
  project = var.project_id
}

variable "project_id" {
  type = string
}

variable "region" {
  type    = string
  default = "us-central1"
}

variable "invoker_member" {
  type        = string
  description = "IAM member allowed to invoke Cloud Run"
}

resource "google_project_service" "required" {
  for_each = toset([
    "iam.googleapis.com",
    "run.googleapis.com",
    "storage.googleapis.com",
  ])

  project            = var.project_id
  disable_on_destroy = false
}

resource "google_service_account" "runtime" {
  account_id = "example-run-runtime"

  depends_on = [google_project_service.required["iam.googleapis.com"]]
}

resource "google_storage_bucket" "application" {
  name                        = "example-app-data-${var.project_id}"
  location                    = var.region
  uniform_bucket_level_access = true
  force_destroy               = false

  versioning {
    enabled = true
  }

  depends_on = [google_project_service.required["storage.googleapis.com"]]
}

resource "google_cloud_run_v2_service" "application" {
  name     = "example-api"
  location = var.region
  ingress  = "INGRESS_TRAFFIC_ALL"

  template {
    service_account = google_service_account.runtime.email

    containers {
      image = "us-docker.pkg.dev/cloudrun/container/hello"

      ports {
        container_port = 8080
      }

      resources {
        limits = {
          cpu    = "1000m"
          memory = "512Mi"
        }
      }
    }
  }

  depends_on = [google_project_service.required["run.googleapis.com"]]
}

resource "google_cloud_run_v2_service_iam_member" "invoker" {
  project  = var.project_id
  location = var.region
  name     = google_cloud_run_v2_service.application.name
  role     = "roles/run.invoker"
  member   = var.invoker_member
}

output "service_url" {
  value = google_cloud_run_v2_service.application.uri
}

backend.hcl:

bucket = "example-terraform-state"
prefix = "gcp/prod"

Create and protect the backend bucket before initializing Terraform:

gcloud services enable storage.googleapis.com --project=example-project

gcloud storage buckets create gs://example-terraform-state \
  --project=example-project \
  --location=us-central1 \
  --uniform-bucket-level-access

gcloud storage buckets update gs://example-terraform-state \
  --versioning

gcloud auth application-default login
gcloud config set project example-project

Initialize and apply with all required values:

terraform init -backend-config=backend.hcl
terraform fmt
terraform validate
terraform plan -out=tfplan \
  -var="project_id=example-project" \
  -var="invoker_member=group:platform@example.com"
terraform apply tfplan

Production Workflow

  • Remote state: Keep the GCS backend private, enable versioning and an appropriate retention or recovery policy, restrict access to the execution identity, and regularly test recovery.

  • Authentication: Use Application Default Credentials locally. In CI/CD, use Workload Identity Federation or workload identity rather than stored service-account keys.

  • Change control: Commit reviewed configuration and terraform.lock.hcl, require plan approval, save plans securely, and apply the exact reviewed plan.

  • Concurrency: Serialize pipelines that target the same state prefix because Terraform uses the backend lock.

  • Operations: Monitor availability, latency, errors, and saturation with Cloud Monitoring and Cloud Logging; use labels, budgets, and billing exports to control cost.

Failure Modes and Trade-offs

  • State loss: Losing state makes resource ownership difficult to reconstruct. Test restoration from a prior bucket version.

  • Partial apply: A resource may be created before Terraform records it in state. Find it by stable GCP name or ID and import it instead of creating a duplicate.

  • Replacement risks: Changes such as a bucket name or region can force replacement. Use prevent_destroy for critical resources and review destructive plans carefully.

  • API failures: IAM errors, organization policies, quota exhaustion, and transient API failures can stop an apply. Retry only operations known to be safe and verify final GCP state.

  • Backup limitations: Terraform manages infrastructure, not application data. Configure Cloud SQL backups, Firestore recovery, GKE volume backups or snapshots, and other service-specific recovery procedures.

Security and Operational Practices

  • Grant only the permissions required by each workflow; avoid broad roles such as roles/owner and roles/editor.

  • Treat state and saved plans as sensitive. Marking an output sensitive = true only hides it from CLI output; it does not remove the value from state.

  • Store secrets in Secret Manager and reference them from workloads instead of placing secret values directly in Terraform variables.

  • Use organization policies, VPC Service Controls where appropriate, and narrowly scoped IAM bindings to constrain production changes.

  • Upgrade providers in a separate change, review the provider changelog, commit the lock file, and validate upgrades in a non-production environment before promotion.

66

What is Vertex AI in Google Cloud?

Vertex AI is Google Cloud’s unified machine learning platform for building, training, deploying, and operating ML models and generative AI applications. It combines managed training, online and batch prediction, model governance, workflow orchestration, and access to Google’s foundation models such as Gemini through a shared control plane, console, and set of service APIs.

Read full answer

Applies toGoogle Cloud Vertex AI (current stable service)

What It Provides

  • A single control plane for the ML lifecycle: data preparation support, training, evaluation, deployment, monitoring, and governance.

  • Managed infrastructure for custom training jobs, including distributed training, hyperparameter tuning, and containerized workloads.

  • Managed serving through endpoints with autoscaling, traffic splitting, and separate batch prediction.

  • MLOps capabilities such as Vertex AI Pipelines, Model Registry, Vertex AI Experiments, and Model Monitoring.

  • Generative AI access through supported Vertex AI APIs, including Gemini models, embeddings, and tuning and evaluation capabilities.

Core Components

Component Purpose
Training Runs custom or AutoML training jobs on managed compute
Endpoints Serves models for online prediction with scaling and versioning
Batch Prediction Runs large-scale offline inference
Model Registry Manages model versions, artifact references, and metadata
Pipelines Orchestrates reproducible ML workflows
Feature Store Manages and serves ML features consistently
Workbench Provides managed notebook environments for development
Model Monitoring Detects skew and drift for supported model types and analyzes prediction logs; prediction-quality analysis requires suitable ground-truth data and configuration

When to Use It

  • Use AutoML or managed training when you want Google Cloud to handle much of the model development and infrastructure.

  • Use custom training when you need control over frameworks, containers, distributed strategies, or specialized hardware such as GPUs and TPUs.

  • Use Vertex AI endpoints for low-latency online inference and batch prediction for large offline scoring jobs.

  • Use supported generative AI APIs for workloads that need enterprise controls such as IAM, VPC Service Controls, audit logging, and—where offered—regional data residency.

Operational Considerations

  • Cost is driven by training compute, endpoint minimum replicas, accelerator usage, storage, and prediction traffic. Choose minimum replica counts deliberately; scale-to-zero configurations can reduce cost but must satisfy cold-start and availability requirements.

  • Not every Vertex AI feature is available in every region. Validate service, model, and feature availability before designing multi-region or regulated workloads.

  • Secure access with least-privilege IAM, appropriately scoped service accounts, VPC Service Controls, and Cloud Audit Logs. Apply Private Service Connect or customer-managed encryption keys only where supported by the specific service and region.

  • Monitor training jobs, endpoint latency, error rates, quota usage, and model drift; configure alerts before production traffic depends on an endpoint.

  • Plan model versioning, rollback, and traffic splitting so failed deployments can be reversed without downtime.

Comparison with Other Clouds

Capability Google Cloud AWS Azure
Managed ML platform Vertex AI Amazon SageMaker Azure Machine Learning
Managed generative AI models Gemini via Vertex AI Amazon Bedrock Azure OpenAI Service
ML pipelines Vertex AI Pipelines SageMaker Pipelines Azure Machine Learning Pipelines
Online model serving Vertex AI Endpoints SageMaker Endpoints Azure ML managed endpoints

Best Practices

  • Keep training and serving containers reproducible and versioned.

  • Register production model versions in Model Registry and retain evaluation evidence and lineage in Vertex AI Model Registry or Vertex ML Metadata as appropriate.

  • Separate development, staging, and production projects or environments.

  • Use pipelines for repeatable retraining rather than manual notebook execution.

  • Test endpoint scaling, rollback, and monitoring before exposing a model to production traffic.

67

How can a machine learning model be deployed on Vertex AI?

Deploying a model on Vertex AI means registering a trained model artifact in the Model Registry, creating an Endpoint, and deploying the model to that endpoint with a chosen machine type and autoscaling policy. Vertex AI then serves online predictions over HTTPS and can also run batch prediction jobs against Cloud Storage or BigQuery data.

Read full answer

Applies toGoogle Cloud Vertex AI (current stable)

How It Works

The deployment flow has three stages:

  1. Upload the model to the Model Registry, pointing at a container image (pre-built or custom) and the artifact location in Cloud Storage.

  2. Create an endpoint, which is the serving resource that exposes a prediction URL.

  3. Deploy the model to the endpoint, specifying machine type, replica counts, and traffic split.

Vertex AI handles load balancing, autoscaling, health checks, and logging to Cloud Monitoring automatically. A single endpoint can host multiple model versions with traffic splitting for canary rollouts.

Deployment Options

Approach When to use Container
Pre-built prediction container Standard frameworks (scikit-learn, XGBoost, TensorFlow, PyTorch) Google-provided image
Custom container Custom inference code, preprocessing, or unsupported frameworks Your own image in Artifact Registry
Batch prediction Offline scoring of large datasets No endpoint needed

Online prediction suits low-latency, request/response workloads. Batch prediction is cheaper for bulk scoring and writes results to Cloud Storage or BigQuery.

Key Configuration

  • Machine type: choose based on model size and latency needs (e.g., n1-standard-4, n1-standard-8, or accelerator-backed types with GPUs).

  • Autoscaling: set --min-replica-count and --max-replica-count; Vertex AI scales between them based on request load.

  • Traffic split: deploy a new version with a percentage of traffic for safe rollouts, then promote to 100%.

  • Explainability: enable Vertex AI Explainable AI for feature attributions when supported by the framework.

  • Monitoring: model monitoring detects training-serving skew and prediction drift; alerts route through Cloud Monitoring.

Comparison with Other Clouds

Capability Vertex AI AWS SageMaker Azure ML
Managed endpoint Vertex AI Endpoint SageMaker Endpoint Azure ML Managed Endpoint
Model registry Vertex AI Model Registry SageMaker Model Registry Azure ML Model Registry
Batch scoring Batch prediction job Batch Transform Batch endpoint
Autoscaling Built-in replica scaling Auto Scaling policies Managed endpoint scaling

Code Example

# 1. Upload model to the registry
gcloud ai models upload \
  --region=us-central1 \
  --display-name=my-sklearn-model \
  --container-image-uri=us-docker.pkg.dev/vertex-ai/prediction/sklearn-cpu.1-0:latest \
  --artifact-uri=gs://my-bucket/models/sklearn-model/

# 2. Create an endpoint
gcloud ai endpoints create \
  --region=us-central1 \
  --display-name=my-prediction-endpoint

# 3. Deploy the model to the endpoint (replace IDs from previous output)
gcloud ai endpoints deploy-model ENDPOINT_ID \
  --region=us-central1 \
  --model=MODEL_ID \
  --display-name=sklearn-v1 \
  --machine-type=n1-standard-4 \
  --min-replica-count=1 \
  --max-replica-count=4 \
  --traffic-split=0=100

# 4. Send an online prediction request
gcloud ai endpoints predict ENDPOINT_ID \
  --region=us-central1 \
  --json-request=request.json

For batch scoring without an endpoint:

gcloud ai batch-prediction-jobs create \
  --region=us-central1 \
  --model=MODEL_ID \
  --input-uri=gs://my-bucket/input/data.jsonl \
  --output-uri=gs://my-bucket/output/ \
  --machine-type=n1-standard-4

Best Practices and Common Pitfalls

  • Use a dedicated service account for the endpoint with least-privilege access to Cloud Storage artifacts; avoid the default compute service account in production.

  • Pin container image digests rather than mutable tags to make deployments reproducible.

  • Set --min-replica-count=1 for latency-sensitive services to avoid cold starts; scale to zero only for non-critical workloads.

  • Enable model monitoring from day one to catch drift; define alerting thresholds before launch.

  • Test with a traffic split before full promotion; roll back by redeploying the previous version at 100%.

  • Watch costs: accelerator-backed machines and always-on replicas are the main cost drivers; right-size and use batch prediction where latency allows.

  • Common pitfall: deploying a model whose container image and artifact format mismatch (e.g., scikit-learn version skew) causes runtime load failures; validate locally with the same container first.

68

How does GCP billing work and how do you set budget alerts?

GCP billing centers on a billing account that pays for one or more projects and meters usage by SKU. Budgets are alarm thresholds rather than hard spending caps, and can notify operators or publish to Pub/Sub for automated response.

Read full answer

Applies toGoogle Cloud current stable, Google Cloud CLI current stable, and Terraform Google Provider 7.x

How GCP Billing Works

  • A billing account is the payment entity. It can pay for multiple projects, while each project has only one billing account at a time.

  • Usage is metered per SKU and then priced according to service, region, resource type, pricing model, and applicable discounts. Eligible credits—such as free-trial, promotional, spend-based, commitment, or negotiated credits—are applied automatically to the resulting charges.

  • Compute Engine may receive Sustained Use Discounts, while eligible Compute Engine, memory, and related usage may receive Committed Use Discounts for one- or three-year commitments.

  • The console provides billing reports and invoice views. BigQuery billing export is useful for repeatable SQL analysis, but it is not authoritative: export data can be delayed or differ from the invoice because of credits, adjustments, and late-arriving usage, so finance reconciliation must use the invoice.

  • Labels are useful for allocating costs, but not every GCP resource supports labels. Keep their keys consistent and include project, environment, service, and owner where applicable.

Budgets and Alerts

  • A budget has an amount, scope, and threshold rules. It can cover a billing account or selected projects and can filter eligible services, labels, annotations, and other billing dimensions.

  • Thresholds use actual or forecast spend. Forecast thresholds depend on sufficient usage history and Cloud Billing forecast data, so actual-spend thresholds are a safer minimum.

  • Notifications can go to email recipients or a Pub/Sub topic. Pub/Sub only publishes an event; a separate subscriber must implement actions such as scaling down non-production capacity.

  • A budget does not stop consumption or create a spending ceiling. Quotas can constrain particular resources or request rates, but they are not a general spend cap.

  • Cloud Billing must have permission to publish to the selected topic. Allow for IAM propagation and monitor both the budget service and the subscriber.

Cost Management and Comparison

Aspect GCP AWS Azure
Detailed cost export BigQuery Cost and Usage Report in S3, queried with Athena Cost Management exports
Budget behavior Alarm and notification only Alarm and notification only Alarm and notification only
Universal spend cap None None None
Usage guardrails Quotas and organization policies Service quotas Quotas and Azure Policy
Discounts SUDs and CUDs Savings Plans and Reserved Instances Savings Plans and Reservations
  • Use the Recommender to identify idle VMs, unattached disks, and rightsizing opportunities.

  • Use CUDs for stable eligible workloads and Spot VMs for interruptible batch processing.

  • Use quotas and organization policies as resource guardrails, but monitor billable services such as egress, which may not have a meaningful quota.

Code Example

# Create the notification topic.
gcloud pubsub topics create billing-alerts --project=my-project

# Allow Cloud Billing to publish budget notifications.
gcloud pubsub topics add-iam-policy-binding billing-alerts \
  --project=my-project \
  --member=serviceAccount:cloud-billing@system.gserviceaccount.com \
  --role=roles/pubsub.publisher

# Create a monthly budget with email and Pub/Sub notifications.
gcloud billing budgets create \
  --billing-account=012345-6789AB-CDEF01 \
  --display-name="prod-monthly-budget" \
  --budget-amount=5000USD \
  --threshold-rule=percent=0.5,basis=forecasted-spend \
  --threshold-rule=percent=0.9,basis=forecasted-spend \
  --threshold-rule=percent=1.0,basis=actual-spend \
  --notifications-rule-email-addresses=oncall@example.com \
  --notifications-rule-pubsub-topic=projects/my-project/topics/billing-alerts
terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 7.0"
    }
  }
}

provider "google" {
  project = "my-project"
}

resource "google_pubsub_topic" "billing_alerts" {
  name    = "billing-alerts"
  project = "my-project"
}

resource "google_pubsub_topic_iam_member" "billing_publisher" {
  topic  = google_pubsub_topic.billing_alerts.name
  role   = "roles/pubsub.publisher"
  member = "serviceAccount:cloud-billing@system.gserviceaccount.com"
}

resource "google_billing_budget" "prod" {
  billing_account = "012345-6789AB-CDEF01"
  display_name    = "prod-monthly-budget"

  budget_filter {
    projects = ["projects/my-project"]
  }

  amount {
    specified_amount {
      currency_code = "USD"
      units         = "5000"
    }
  }

  threshold_rules {
    threshold_percent = 0.5
    spend_basis       = "FORECASTED_SPEND"
  }

  threshold_rules {
    threshold_percent = 0.9
    spend_basis       = "FORECASTED_SPEND"
  }

  threshold_rules {
    threshold_percent = 1.0
    spend_basis       = "CURRENT_SPEND"
  }

  all_updates_rule {
    pubsub_topic = google_pubsub_topic.billing_alerts.id
  }

  depends_on = [google_pubsub_topic_iam_member.billing_publisher]
}

Best Practices and Common Pitfalls

  • Enable BigQuery billing export early, retain the export configuration in Terraform, and reconcile anomalies against the invoice.

  • Enforce label conventions through provisioning policy and CI checks; use Resource Manager tag policies when managing supported tags.

  • Treat budgets as alarms. Use quotas to limit eligible usage and make any billing-disablement or scaling automation idempotent, tested, and protected by approvals.

  • Give the Pub/Sub consumer only the permissions required for its response, and monitor undelivered messages, failed alerts, and forecast accuracy.

  • Review CUD commitments and Spot VM interruption settings regularly.

  • Track egress, cross-region traffic, data processing, and license charges, which are common sources of unexpected cost.

  • Separate unrelated business units into billing accounts when chargeback, access control, or ownership differs.

69

What are Google Cloud committed use and sustained use discounts?

Committed Use Discounts (CUDs) are contractual discounts for predictable consumption of eligible Google Cloud resources over a one- or three-year term. Sustained Use Discounts (SUDs) are automatic discounts for eligible Compute Engine resources that remain running for a substantial portion of the billing month.

Read full answer

Applies toGoogle Cloud Platform (current stable billing and pricing model; eligibility and rates vary)

Core Difference

Characteristic Committed Use Discount Sustained Use Discount
Purchase model You purchase a commitment No purchase is required
Commitment period One or three years None
Eligible resources Eligible Compute Engine resources and other supported services, depending on commitment type Certain Compute Engine machine types in eligible regions
Discount trigger Usage is covered by an active commitment Eligible resources run continuously for a sufficient portion of the month
Payment risk You pay for the commitment even if usage is lower than expected No committed spend; stopping usage reduces the discount
Capacity guarantee Generally none None

How CUDs Work

CUDs are purchased through a Google Cloud billing account and commonly take these forms:

  • Resource-based CUDs: Apply to specific Compute Engine resources, such as vCPU, memory, GPUs, or local SSD, and usually have regional scope.

  • Spend-based CUDs: Provide discounts against eligible spend across supported services without requiring the same resource-level configuration in every project.

The typical workflow is:

  1. Identify a stable baseline workload and its billing account.

  2. Select the eligible resource or spend scope, region where applicable, commitment duration, and discount level.

  3. Purchase the commitment.

  4. Match the commitment against eligible usage in associated projects.

  5. Monitor utilization and remaining commitment before the commitment expires.

A commitment is primarily a billing commitment, not a capacity reservation. It does not normally guarantee that Compute Engine capacity will be available. Quotas, reservations, and regional capacity must be managed separately.

How SUDs Work

SUDs apply automatically to eligible Compute Engine usage. There is no contract or prepayment, but the discount depends on how continuously the resource runs during the billing month.

For example, a long-lived internal VM that runs throughout the month can receive a SUD. If the VM is repeatedly stopped and started, the accumulated usage pattern may not qualify for the same discount. This makes SUDs most useful for stable services, development environments that remain online, and autoscaled fleets whose underlying nodes remain active.

SUDs should not be assumed to apply to every Compute Engine product. Spot VMs, preemptible workloads, some machine families, and some managed services may not qualify.

AWS and Azure Equivalents

Google Cloud AWS Azure
CUD EC2 Reserved Instances or Savings Plans Reservations or Savings Plans
SUD No broadly equivalent automatic VM-use discount No broadly equivalent automatic VM-use discount

AWS and Azure generally achieve sustained-use savings through reservations, savings plans, or product-specific discount programs rather than a directly equivalent automatic mechanism.

Production Considerations

  • Budget monitoring: Configure budgets and alerts in Cloud Billing, and export detailed billing data to BigQuery for commitment-utilization analysis.

  • Underutilization: A CUD can become expensive if workloads are migrated, deleted, or resized. Compare the discount against realistic steady-state consumption before purchasing.

  • Expiry planning: Track commitment end dates. If a commitment is not renewed, covered usage can return to standard pricing.

  • Regional scope: Resource-based commitments may not cover the same workload after migration to another region.

  • Capacity and quotas: A discount does not remove quota limits or guarantee regional capacity.

  • Security and governance: Restrict commitment purchases with appropriate billing-account IAM permissions. Financial discounts do not replace workload identity, least-privilege access, audit logging, or data-protection controls.

  • Workload selection: Use CUDs for stable, committed baselines and planned multi-year capacity. Use SUDs for continuously running eligible VMs, and consider Spot VMs for fault-tolerant batch work where interruption is acceptable.

For billing-account administration, gcloud billing accounts list and gcloud billing accounts describe BILLING_ACCOUNT_ID help identify the account that owns the commitment and associated billing scope.

70

How can you reduce Google Cloud Platform costs?

The most effective GCP cost reduction combines rightsizing, removing idle resources, matching pricing and capacity to demand, and continuously controlling spend. Use cost controls that preserve reliability, security, and recovery objectives rather than optimizing the invoice in isolation.

Read full answer

Applies toGoogle Cloud current stable services, Google Cloud CLI current stable release, and BigQuery CLI current stable release

Start with Cost Attribution

Before changing capacity, attribute spending and ownership:

  • Resource hierarchy and IAM: Use folders and projects to separate environments and ownership boundaries, and grant least privilege at the narrowest practical scope.

  • Labels and billing export: Label supported resources and export detailed charges to BigQuery; analyze spend by project, service, SKU, label, and day.

  • Idle resources: Review idle VMs, disks, IP addresses, snapshots, Cloud SQL instances, GKE node pools, and database capacity.

  • Shared costs: Attribute shared networking and logging to a platform project or cost center so teams are not discouraged from using required controls.

Match Capacity and Pricing to Demand

Lever Best use Operational trade-off
Autoscaling GKE, Cloud Run, Cloud Functions, and App Engine Requires useful metrics, safe scale-out latency, and maximum-capacity limits
Spot VMs Batch, CI, and fault-tolerant jobs Workloads must checkpoint and retry after preemption
Committed use discounts Predictable Compute Engine and eligible GKE node usage Commitments must match resource and region; unused commitments still incur charges
Rightsizing CPU, memory, disks, and database capacity Insufficient headroom can increase latency or cause outages
Scale-to-zero Cloud Run, Cloud Functions, and App Engine standard Cold starts affect latency

Prefer flexible machine families and current-generation hardware when benchmarks show better price/performance. Test migrations against representative workloads rather than assuming a newer instance type is cheaper.

Control Data, Service, and Network Costs

  • BigQuery: Partition large tables by date, cluster by frequently filtered columns, and avoid SELECT *. Use --maximum_bytes_billed to reject unexpectedly large queries. Compare total on-demand cost before buying slot reservations.

  • Cloud Storage: Use lifecycle rules to delete temporary data and transition infrequently accessed objects to Nearline, Coldline, or Archive. Include noncurrent object versions.

  • Cloud SQL: Stop nonproduction instances, remove replicas not required by recovery objectives, and right-size CPU, memory, and storage. Backups and retained logs also consume storage.

  • GKE: Enable node autoscaling, remove permanently idle pools, and monitor managed-cluster fees, load balancers, and persistent disks.

  • Cloud Run, Cloud Functions, and App Engine: Use zero-minimum instances where supported, cap maximum instances, tune concurrency, and select suitable instance classes. App Engine flexible always keeps at least one instance running, so it cannot scale to zero.

  • Spanner and Bigtable: Right-size Spanner nodes and replicas; review Bigtable cluster nodes, replication, and retained backups. Both may continue charging after traffic falls.

  • Firestore and Pub/Sub: Minimize document operations, retry amplification, subscriber fan-out, and duplicate subscriptions. Use TTL for expired Firestore documents and account for Pub/Sub deliveries and snapshot storage.

  • VPC networking and load balancing: Review egress, NAT, inter-region traffic, and load-balancer footprint. Use regional or internal load balancing where architectural requirements permit.

  • Logging and monitoring: Exclude unnecessary verbose logs, bound retention, and control metric cardinality. Preserve security and incident-response evidence.

Budgets are alerts, not automatic spending caps. Combine them with quotas, least-privilege IAM, and organization policies to constrain runaway workloads.

Production Guardrails

A production cost program should include:

  1. Daily billing analysis and anomaly detection.

  2. Budget alerts with explicitly configured recipients.

  3. Ownership labels enforced through infrastructure as code.

  4. Automated shutdown for explicitly nonproduction environments.

  5. Gradual rollout, SLO comparison, and rollback procedures for rightsizing and upgrades.

Protect required backups, snapshots, replicas, and recovery processes; lower spending must not weaken service objectives.

Code Example

These commands inspect labels, create an autoscaled zonal GKE node pool, create budget thresholds, and limit billable BigQuery bytes:

# Show Compute Engine instances and their attribution labels.
gcloud compute instances list \
  --project=PROJECT_ID \
  --format="table(name,zone,machineType,status,labels)"

# Add or update labels without removing unrelated existing labels.
gcloud compute instances update INSTANCE_NAME \
  --project=PROJECT_ID \
  --zone=ZONE \
  --update-labels=env=prod,team=platform,cost-center=123

# Start with one node and scale according to workload demand.
gcloud container node-pools create platform-pool \
  --project=PROJECT_ID \
  --cluster=CLUSTER_NAME \
  --zone=ZONE \
  --machine-type=e2-standard-4 \
  --num-nodes=1 \
  --enable-autoscaling \
  --min-nodes=1 \
  --max-nodes=10

# Create budget thresholds at 50%, 90%, and 100%.
gcloud billing budgets create \
  --billing-account=BILLING_ACCOUNT_ID \
  --display-name=platform-budget \
  --budget-amount=1000USD \
  --threshold-rule=percent=0.5 \
  --threshold-rule=percent=0.9 \
  --threshold-rule=percent=1.0

# Fail rather than bill more than 1 GB of scanned data.
bq query \
  --project=example-project \
  --use_legacy_sql=false \
  --maximum_bytes_billed=1000000000 \
  'SELECT COUNT(*) FROM `example-project.example_dataset.events`'

Best Practices and Common Pitfalls

  • Optimize total cost: A cheaper configuration that causes throttling or retries may increase overall cost.

  • Model commitments carefully: Include growth, shutdowns, migrations, and unused-commitment charges.

  • Automate recurring waste: Prefer lifecycle policies, scheduled shutdowns, and autoscaling over manual cleanup.

  • Find hidden charges: Review egress, storage operations, inter-region traffic, managed-service fees, snapshots, and observability ingestion.

  • Preserve recovery capabilities: Do not remove backups, replicas, or logging required by recovery objectives.

  • Validate every saving: Compare cost, latency, error, and saturation metrics before and after each change.

71

How do GCP services compare to AWS equivalents?

GCP and AWS offer largely parallel cloud platforms, but their regional defaults, IAM structures, billing models, and operational trade-offs differ enough that services should be mapped by capability rather than by name alone.

Read full answer

Applies toGoogle Cloud Platform and Amazon Web Services current stable service catalogs (2025)

Core Service Mapping

Category GCP AWS Equivalent Operational distinction
Virtual machines Compute Engine EC2 Compute Engine meters per second with a one-minute minimum and offers eligible sustained-use discounts; current-generation EC2 also meters per second and supports On-Demand, Savings Plans, Reserved Instances, and Spot.
Managed Kubernetes GKE Standard / Autopilot EKS / Fargate Autopilot provides an opinionated managed-node configuration; Fargate supplies serverless node capacity for EKS.
Serverless containers Cloud Run App Runner / Fargate Cloud Run can scale to zero when minimum instances are zero; it bills for requests and allocated CPU/memory time, rounding each request’s billable time up to the nearest 100 ms.
Managed application PaaS App Engine Elastic Beanstalk / App Runner App Engine standard manages deployment and scaling closely; Beanstalk gives more infrastructure control, while App Runner is comparatively simple.
Serverless functions Cloud Functions, 2nd gen Lambda 2nd-gen Cloud Functions are built on Cloud Run infrastructure.
Object storage Cloud Storage S3 Both use globally unique bucket names. An S3 bucket has one home Region, and its objects are not globally replicated by default.
Relational databases Cloud SQL RDS / Aurora Both support managed MySQL and PostgreSQL deployments, private connectivity, backups, high availability, and automated maintenance.
Globally distributed SQL Spanner Aurora DSQL / Aurora Global Database Aurora DSQL is the closest current AWS analogue, but its PostgreSQL interface and architecture differ from Spanner.
Document NoSQL Firestore DynamoDB Firestore databases default to a multi-region location unless a regional location is selected; DynamoDB multi-region replication uses Global Tables.
Wide-column NoSQL Bigtable Keyspaces / DynamoDB use cases Bigtable supports large operational and analytical workloads; Keyspaces is Cassandra-compatible, while DynamoDB has a different data model.
Data warehouse BigQuery Redshift / Athena BigQuery is serverless and separates storage from query-compute charges.
Messaging and streaming Pub/Sub SNS + SQS / Kinesis Pub/Sub combines topic distribution with pull/push subscriptions; AWS separates notification, queueing, and streaming into distinct services.
Networking VPC VPC A GCP VPC is a global resource with regional subnets; an AWS VPC is regional.
Load balancing Cloud Load Balancing ALB / NLB A GCP global external Application Load Balancer uses anycast frontend IPs; an AWS ALB is regional.
Monitoring Cloud Monitoring + Cloud Logging + Cloud Trace CloudWatch + X-Ray CloudWatch covers metrics and logs; on GCP, metrics come from Cloud Monitoring and logs from the separate Cloud Logging service. Tracing is provided by Cloud Trace and X-Ray.
Secrets and keys Secret Manager + Cloud KMS Secrets Manager / Parameter Store + KMS All support centrally managed secrets and envelope encryption; use IAM, rotation, audit logs, and customer-managed keys where required.

Key Production Differences

  • Resource hierarchy: GCP IAM policies can inherit through organizations, folders, and projects. AWS identity policies remain account-scoped, while Organizations, organizational units, SCPs, and StackSets provide multi-account governance.

  • Global versus regional defaults: GCP networks span regions, but subnets, Cloud NAT instances, and many regional backends remain regional. AWS VPCs and load balancers are regional unless separate global services are used.

  • Billing: GCP sustained-use discounts apply automatically only to eligible machine types; AWS commitment discounts generally require Savings Plans or Reserved Instances. Both also support discounted interruptible capacity through Spot Instances.

  • Workload identity: GCP Workload Identity Federation issues short-lived credentials for service accounts through federated external identity providers. AWS uses IAM roles and OIDC or SAML identity providers.

  • Network egress: Both charge for internet egress and cross-region or cross-zone transfers in many configurations. Model internal Google traffic, AWS service endpoints, private-service access, and NAT paths before estimating cost.

  • Control versus convenience: GCP generally offers opinionated managed modes such as GKE Autopilot and Cloud Run. AWS often provides more control at the infrastructure or orchestration layer, but that can increase node, networking, and upgrade work.

When to Choose Each

  • Choose GCP for BigQuery analytics, Spanner’s globally distributed strongly consistent SQL, a global VPC resource, or GKE Autopilot’s reduced node-management burden.

  • Choose AWS when a required AWS service, regional footprint, compliance commitment, or procurement relationship makes it the better operational fit.

  • For hybrid or multi-cloud: use Terraform, Kubernetes, PostgreSQL, and Kafka where practical, while retaining explicit exit plans for proprietary data and messaging APIs.

Operational Trade-offs

  • Observability: Standardize collection on OpenTelemetry, but configure metric, log, and trace retention and sampling explicitly rather than relying on provider-specific defaults.

  • Disaster recovery: Regional Cloud SQL deployments and regional GKE clusters can span zones; RDS Multi-AZ and EC2 Auto Scaling likewise use multiple Availability Zones. Multi-region recovery still requires an explicit, tested design.

  • Cost management: Use GCP Recommender, Cloud Billing budgets, and AWS Cost Explorer, Cost Optimization Hub, and AWS Budgets. Labels and tags are required for useful allocation and chargeback.

  • Backups: Managed backups do not replace restoration tests. Validate point-in-time recovery, backup retention, cross-region copies, and actual recovery-time and recovery-point objectives.

Best Practices

  • Compare service capabilities, SLAs, quotas, networking behavior, and operational burden before selecting an equivalent.

  • Apply least privilege with groups or service accounts, use short-lived credentials, and avoid long-lived access keys.

  • Manage infrastructure as code, isolate Terraform state by environment and cloud boundary, and review changes through version control.

  • Test failover, regional evacuation, restore procedures, quota exhaustion, and incident runbooks before production launch.

72

How should you architect high availability across GCP zones and regions?

Use a layered high-availability (HA) design that removes single points of failure at each failure domain: spread replicas across zones, isolate regional stacks, and introduce global traffic routing where required. In GCP, multi-zone availability protects against a zone failure, while multi-region availability protects against a broader regional outage; neither alone provides complete disaster recovery.

Read full answer

Applies toGoogle Cloud Platform (current stable), gcloud CLI (current stable)

Design by Failure Domain

  • Zone: Place replicas in different zones within a region. A zone can fail because of hardware, power, or a deployment issue while the rest of the region remains available.

  • Region: Deploy independent regional stacks when an entire region must be survivable. A regional outage can affect control planes, network paths, and regional managed-service endpoints simultaneously.

  • Global layer: Use a global external Application Load Balancer for stateless HTTP(S) workloads that must survive regional failures. For GKE or Compute Engine backends, the load balancer uses health checks to route traffic only to healthy endpoints.

  • Resource hierarchy: Organize resources under the organization, folders, and projects according to environment, ownership, and lifecycle. Apply IAM and policy controls at the highest practical level; use labels for ownership, cost, and regional deployment metadata.

Service Selection

Architecture pattern GCP AWS equivalent Azure equivalent
Global HTTP(S) ingress Global external Application Load Balancer CloudFront with ALB Front Door
Zone-resilient containers Regional GKE cluster Multi-AZ EKS Zone-spread AKS
Regional managed relational database Cloud SQL regional HA RDS Multi-AZ Zone-redundant Azure SQL
Multi-region transactional data Spanner multi-region with global writes Aurora Global Database with a regional writer Cosmos DB with multi-region writes
Globally replicated document data Firestore multi-region DynamoDB global tables Cosmos DB with multi-region replication
Durable object storage Cloud Storage dual-region or multi-region S3 with cross-region replication Storage GRS/RA-GRS with paired-region replication

Choose Cloud Run for independently deployed, event-driven services; choose GKE when teams need Kubernetes APIs, custom scheduling, or platform extensibility. Use Cloud Storage for objects, Cloud SQL for relational transactions, Spanner for globally distributed transactions, and BigQuery multi-region for analytics—not low-latency transactional writes.

How the Architecture Behaves

  • A regional GKE cluster runs its control plane across three zones in that region. It survives individual zone failures but does not survive loss of the entire region.

  • Cloud SQL regional HA maintains a standby in another zone and can perform automatic failover. It does not provide active regional failover; cross-region read replicas are not a substitute for a tested write-failover design.

  • Cloud Run services deployed in multiple regions provide independent regional capacity. A global load balancer can route requests across regional serverless NEGs, although deployment and data-consistency policies must be defined separately.

  • Spanner multi-region replicates data across regions and coordinates writes globally. Its consistency, latency, and cost trade-offs differ from independently replicated databases.

Failure Modes and Trade-offs

  • Bad release: A deployment can break every healthy replica. Canary one region first, monitor error rates and latency, and retain a rapid rollback path.

  • Data-region outage: Stateless capacity can fail over, but a regional database may still block writes. Replicate data, define a promotion process, and document the accepted RPO and RTO.

  • Retry storm: A dependency outage can overwhelm recovering backends. Use bounded exponential backoff, jitter, circuit breakers, and idempotent operations.

  • Partial failure: A dependency may become slow rather than unavailable. Load-balancer health checks and timeouts must reflect user-visible symptoms.

  • Cost and latency: Duplicate regional capacity and cross-region traffic add cost. Evaluate data residency, egress charges, and whether every request truly requires global routing.

Security and Operational Controls

  • Use per-service service accounts, least-privilege IAM, Secret Manager, VPC Service Controls, and private connectivity where supported.

  • Restrict Cloud Run ingress and grant invocation access only to the required identity; avoid allUsers unless public access is intentional.

  • Use Admin Activity audit logs, which are always enabled, enable Data Access audit logs where required, and use Cloud Logging, Monitoring, Cloud Trace, and Error Reporting.

  • Protect databases with backups and point-in-time recovery where supported. Store recovery artifacts separately and perform scheduled restore tests.

  • Apply budgets, anomaly detection, and cost labels to duplicated regional infrastructure and inter-region transfer.

Production Checklist

  • Define failure scenarios and explicit RPO, RTO, and error-budget targets.

  • Verify capacity for losing an entire zone or region.

  • Exercise regional failover and database restoration regularly.

  • Confirm that retries are bounded and write operations are idempotent.

  • Monitor regional saturation, replication lag, failed requests, and failover events.

  • Rehearse bad-deployment rollback, certificate rotation, secret rotation, and regional dependency loss.

73

How do you migrate on-premises workloads to Google Cloud?

A successful on-premises migration uses a migration wave: discover and classify the workload, build a secure landing zone, transfer data, validate it, and then cut over traffic with a tested rollback path. The appropriate GCP service depends on whether the workload is stateless, containerized, relational, distributed, or event-driven.

Read full answer

Applies toGoogle Cloud CLI current stable and current Google Cloud managed services

Migration Strategy

Start with discovery rather than immediately copying servers:

  • Inventory: Record CPU, memory, disks, operating systems, licenses, dependencies, data stores, traffic patterns, and recovery objectives.

  • Classify: Mark each application as rehost, replatform, refactor, retire, or retain.

  • Wave dependencies: Migrate shared identity, DNS, network connectivity, databases, and secrets before dependent applications.

  • Success criteria: Set measurable targets for downtime, data loss, latency, error rate, and recovery time.

  • Pilot: Build a nonproduction migration and automate deployment before scaling across production waves.

Use separate projects or folders for landing, migration tooling, shared networking, and production. Centralize audit logs and billing, but grant access through groups and narrowly scoped IAM roles.

Choosing the Target Service

Workload Recommended GCP service Key consideration
Existing VM Compute Engine Fastest rehost path; guest customization and licensing still apply
Containerized application GKE Managed control plane, autoscaling, and workload portability
Stateless HTTP container Cloud Run Automatic scaling and revision-based deployments
Event-driven function Cloud Functions Trigger-driven execution without server management
Relational database Cloud SQL Managed PostgreSQL, MySQL, or SQL Server
Globally distributed transactional data Spanner Strong consistency and horizontal scale across regions
Application key-value state Firestore or Bigtable Firestore suits document state; Bigtable suits large operational workloads
Analytical data BigQuery Serverless SQL analytics and federated data exploration
Object data Cloud Storage Durable storage with lifecycle, versioning, and retention controls
VMware-heavy environment VMware Engine Extends the VMware operating model into GCP

App Engine remains appropriate for applications built around its platform model, but Cloud Run generally offers a more container-focused operational model for new HTTP services.

Connectivity and Data Transfer

Requirement Approach
Moderate initial traffic or temporary hybrid setup HA VPN with Cloud Router and BGP
Sustained high throughput or low-latency hybrid access Cloud Interconnect, typically with a redundant topology
Large file transfers Storage Transfer Service with on-premises agents
Supported relational databases Database Migration Service for Cloud SQL targets
Incremental database synchronization Database Migration Service or carefully tested change-data-capture tooling

Maintain dual-running paths until validation completes. Connectivity alone does not synchronize application data, and successful ping tests do not prove application correctness. Encrypt transit, restrict routes, monitor VPN or Interconnect health, and document the fallback when the private link fails.

Code Example

The following creates a minimal private network, imports a prepared VMware virtual disk, and starts a Compute Engine instance without a public IP.

set -euo pipefail

gcloud config set project example-project
gcloud services enable \
  compute.googleapis.com \
  storage.googleapis.com \
  sqladmin.googleapis.com \
  container.googleapis.com \
  run.googleapis.com \
  logging.googleapis.com \
  monitoring.googleapis.com

gcloud compute networks create example-vpc --subnet-mode=custom

gcloud compute networks subnets create example-subnet \
  --network=example-vpc \
  --region=us-central1 \
  --range=10.0.0.0/24 \
  --enable-private-ip-google-access \
  --enable-flow-logs

gcloud compute firewall-rules create allow-https-from-vpc \
  --network=example-vpc \
  --direction=INGRESS \
  --action=ALLOW \
  --rules=tcp:443 \
  --source-ranges=10.0.0.0/24 \
  --target-tags=migrated-app

# Prerequisite: onprem-app.vmdk has been uploaded to the bucket.
gcloud compute images create example-onprem-image \
  --source-uri=gs://example-migration-bucket/onprem-app.vmdk \
  --guest-os-features=UEFI_COMPATIBLE

gcloud compute instances create app-01 \
  --zone=us-central1-a \
  --machine-type=e2-standard-2 \
  --image=example-onprem-image \
  --subnet=example-subnet \
  --no-address \
  --tags=migrated-app

Use IAP tunneling for administrative access rather than public SSH. Ensure the subnet has controlled private egress, such as Cloud NAT, when the application requires outbound Internet access.

For GKE, deploy the migrated container through a reviewed Kubernetes manifest rather than copying VM-specific configuration:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: migrated-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: migrated-app
  template:
    metadata:
      labels:
        app: migrated-app
    spec:
      containers:
        - name: app
          image: us-central1-docker.pkg.dev/example-project/migration/app:v1
          ports:
            - name: http
              containerPort: 8080
          readinessProbe:
            httpGet:
              path: /readyz
              port: http
          livenessProbe:
            httpGet:
              path: /healthz
              port: http
          resources:
            requests:
              cpu: 250m
              memory: 256Mi
            limits:
              cpu: "1"
              memory: 512Mi
---
apiVersion: v1
kind: Service
metadata:
  name: migrated-app
spec:
  selector:
    app: migrated-app
  ports:
    - port: 80
      targetPort: http

The application must expose /readyz and /healthz on port 8080. Configure GKE node autoscaling and disruption budgets for production availability.

Cutover and Validation

  1. Reduce DNS TTL and confirm every dependency is available in GCP.

  2. Perform the final database synchronization while writes are controlled.

  3. Compare checksums, row counts, transaction outcomes, and business-level reports.

  4. Run performance, failover, security, and restore tests.

  5. Shift a small percentage of traffic, observe metrics, and increase gradually.

  6. Retain the on-premises environment and a rollback plan through the agreed stabilization period.

Use Cloud Monitoring uptime checks, latency and error dashboards, and alerting on logs and metrics. If cutover fails, route traffic back, preserve diagnostic evidence, and reconcile any writes that occurred only on GCP.

Production Readiness

  • Security: Use IAM groups, service accounts, Secret Manager, VPC Service Controls, private service access, and least-privilege roles. Avoid long-lived service account keys.

  • Backups: Configure Compute Engine snapshots, Cloud SQL automated backups and point-in-time recovery, and tested GCS versioning or retention policies.

  • Upgrades: Use GKE release channels, Cloud Run revision rollouts, and staged database maintenance windows.

  • Incidents: Maintain runbooks, dependency maps, ownership information, and rehearsed rollback procedures.

  • Cost: Apply labels, budgets, quota monitoring, rightsizing, and committed-use discounts where justified. Include network egress and idle provisioned capacity in migration estimates.

  • Common failure: Do not migrate databases independently of their applications; incompatible time zones, DNS behavior, certificates, firewall rules, and vendor licensing are frequent causes of late cutover failures.

74

How do you connect a Cloud Run service to Cloud SQL securely?

Connect Cloud Run to Cloud SQL over the VPC network using Cloud SQL’s private IP, and authenticate with a least-privilege service account (either via the Cloud SQL Auth Proxy sidecar with IAM authentication, or via a password pulled from Secret Manager). Never expose the database on a public IP, and never bake credentials into the image.

Read full answer

Applies toCloud Run (fully managed, second-generation runtimes) with Cloud SQL for PostgreSQL 16 / MySQL 8.0 (current GCP)

How It Works

Cloud Run is serverless, so “connecting to a database” means three things must line up:

  1. Network path: Cloud Run must be able to reach the Cloud SQL private IP. Outbound traffic from Cloud Run to a VPC requires either a Serverless VPC Access connector or direct VPC egress; with either one, the PRIVATE_RANGES egress setting sends only private-address traffic into the VPC, while ALL_TRAFFIC sends everything. Without one, Cloud Run only has public internet egress and cannot reach a private-IP instance.

  2. TLS: The Auth Proxy always encrypts its connection, but a direct private-IP connection is unencrypted unless the instance enforces SSL (--ssl-mode=ENCRYPTED_ONLY) and the client uses it. PostgreSQL clients should use sslmode=verify-full (with sslrootcert from the server CA) rather than disable.

  3. Identity: the Cloud Run service account authenticates. Grant roles/cloudsql.client so the Cloud SQL Auth Proxy or connector can connect, plus roles/cloudsql.instanceUser only if the service account logs in with IAM database authentication.

Option Connectivity Credentials Notes
Cloud SQL Auth Proxy sidecar 127.0.0.1:5432 IAM (--auto-iam-authn) or password TLS terminated at the proxy; proxy is not a connection pooler
Direct private-IP connection 10.10.x.x:5432 Password from Secret Manager Fewest moving parts; you handle TLS verification yourself
Public IP Internet IAM proxy or password Avoid unless required; larger attack surface

Cloud Equivalents

GCP AWS Azure
Cloud Run Lambda / ECS Fargate Azure Container Apps
Cloud SQL for PostgreSQL RDS for PostgreSQL Azure Database for PostgreSQL
Secret Manager Secrets Manager Key Vault
Serverless VPC Access / direct VPC egress Lambda VPC configuration / ECS awsvpc networking VNet integration

Wiring It Up (gcloud)

export PROJECT_ID=example-project
export REGION=us-central1
gcloud config set project "$PROJECT_ID"

# 1. VPC with a subnet for Cloud Run and a range reserved for Cloud SQL
gcloud compute networks create app-vpc --subnet-mode=custom
gcloud compute networks subnets create app-subnet \
  --network=app-vpc --region=$REGION --range=10.10.0.0/24 \
  --enable-private-ip-google-access

gcloud compute addresses create cloudsql-range --global \
  --purpose=VPC_PEERING --prefix-length=16 --network=app-vpc
gcloud services vpc-peerings connect --service=servicenetworking.googleapis.com \
  --ranges=cloudsql-range --network=app-vpc

# 2. Cloud SQL instance: private IP only, no public IP, regional HA
gcloud sql instances create app-db \
  --database-version=POSTGRES_16 --availability-type=REGIONAL \
  --tier=db-custom-2-7680 --region=$REGION \
  --network=projects/$PROJECT_ID/global/networks/app-vpc \
  --allocated-ip-range-name=cloudsql-range \
  --no-assign-ip --backup-start-time=03:00

gcloud sql databases create app --instance=app-db
DB_PASSWORD="$(openssl rand -base64 24)"
gcloud sql users create appuser --instance=app-db \
  --password="$DB_PASSWORD"

# 3. Service account + least-privilege IAM (prefer secret-level binding)
gcloud iam service-accounts create run-sql-sa
gcloud projects add-iam-policy-binding "$PROJECT_ID" \
  --member="serviceAccount:run-sql-sa@$PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/cloudsql.client"
gcloud projects add-iam-policy-binding "$PROJECT_ID" \
  --member="serviceAccount:run-sql-sa@$PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/cloudsql.instanceUser"

printf '%s' "$DB_PASSWORD" | gcloud secrets create db-password \
  --replication-policy=automatic --data-file=-
gcloud secrets add-iam-policy-binding db-password \
  --member="serviceAccount:run-sql-sa@$PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/secretmanager.secretAccessor"

# 4. VPC connector that can route to the Cloud SQL private range
gcloud compute networks vpc-access connectors create sql-connector \
  --network=app-vpc --region=$REGION --ip-cidr-range=10.8.0.0/28 \
  --min-throughput=200 --max-throughput=1000

Deploy the service with the Auth Proxy as a sidecar from the manifest below:

gcloud run services replace service.yaml --region=$REGION

Code Example

# service.yaml — Cloud Run with the Cloud SQL Auth Proxy sidecar
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: app
  namespace: example-project
spec:
  template:
    metadata:
      annotations:
        run.googleapis.com/vpc-access-connector: sql-connector
        run.googleapis.com/vpc-access-egress: private-ranges-only
    spec:
      serviceAccountName: run-sql-sa@example-project.iam.gserviceaccount.com
      containers:
        - name: cloud-sql-proxy
          image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.11.4  # pin a version/digest
          args:
            - "--instance-connection-name=example-project:us-central1:app-db"
            - "--private-ip"          # instance has no public IP
            - "--port=5432"
            - "--structured-logs"
          resources:
            limits: { cpu: "1", memory: 512Mi }
        - name: app
          image: us-docker.pkg.dev/example-project/app/app:latest
          env:
            - name: DB_HOST
              value: "127.0.0.1"
            - name: DB_PORT
              value: "5432"
            - name: DB_USER
              value: "appuser"
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: db-password
                  key: latest
          ports:
            - containerPort: 8080
          resources:
            limits: { cpu: "1", memory: 512Mi }

To use IAM database authentication instead of a password, set the cloudsql.iam_authentication=on database flag, add --auto-iam-authn to the proxy arguments, and register the service account as a database user (for PostgreSQL the user name is the email without .gserviceaccount.com). Then grant only the privileges the application needs:

gcloud sql users create run-sql-sa@example-project.iam \
  --instance=app-db --type=cloud_iam_service_account
GRANT CONNECT ON DATABASE app TO "run-sql-sa@example-project.iam";

The application then connects as run-sql-sa@example-project.iam with no password.

Production Failure Modes

  • Connection storms: Cloud Run scales out and each instance opens its own pool. Cap it with --max-instances and --concurrency, and use PgBouncer or a managed pooler in front of Cloud SQL. The Auth Proxy does not pool connections.

  • Connector throughput: a VPC connector saturates at its max throughput (1000 Mbps). Raise --max-throughput for large payloads.

  • Cold starts: scale-to-zero adds connection latency; keep --min-instances=1 for latency-sensitive APIs.

  • IAM propagation delay: a new role binding can take up to a minute or two to take effect — a fresh deploy may briefly see 403 from the Admin API.

  • Timeouts: set driver connect timeouts and statement_timeout; Cloud Run request timeout tops out at 60 minutes, so long queries must be async.

  • Backups/DR: enable automated backups and PITR (shown in Terraform below). Cloud SQL does not fail over across regions automatically — use a cross-region replica for DR.

  • Monitoring: alert on cloudsql.googleapis.com/database/cpu/utilization, database/network/connections, database/disks/utilization, plus Cloud Run latency/error metrics; enable Cloud SQL Insights and VPC/Cloud SQL admin activity audit logs.

Best Practices

  • Private IP only (--no-assign-ip), TLS enforced, IAM-based auth, secrets in Secret Manager.

  • Grant roles/cloudsql.client and roles/secretmanager.secretAccessor — not roles/cloudsql.admin.

  • Set --max-instances and use a connection pooler; scale the Cloud SQL tier with the expected concurrency, not the average.

  • Pin the proxy image tag/digest and redeploy deliberately — the proxy is part of your data path.

  • For strict isolation, consider Private Service Connect for Cloud SQL (--enable-private-service-connect with --allocated-psc-range) instead of the peered range.

Terraform Skeleton

resource "google_sql_database_instance" "db" {
  name             = "app-db"
  region           = "us-central1"
  database_version = "POSTGRES_16"
  deletion_protection = true
  settings {
    tier              = "db-custom-2-7680"
    availability_type = "REGIONAL"
    ip_configuration {
      ipv4_enabled    = false
      private_network = google_compute_network.app_vpc.id
    }
    backup_configuration {
      enabled                  = true
      point_in_time_recovery_enabled = true
      start_time               = "03:00"
    }
  }
}

resource "google_cloud_run_v2_service" "app" {
  name     = "app"
  location = "us-central1"
  template {
    service_account = google_service_account.run.email
    vpc_access {
      network_interface {
        network    = google_compute_network.app_vpc.id
        subnetwork = google_compute_subnetwork.app.id
      }
      egress = "ALL_TRAFFIC"   # direct VPC egress: no connector needed
    }
    containers {
      image = "us-docker.pkg.dev/example-project/app/app:latest"
      env {
        name  = "DB_HOST"
        value = google_sql_database_instance.db.private_ip_address
      }
      env {
        name  = "DB_PORT"
        value = "5432"
      }
      env {
        name = "DB_PASSWORD"
        value_source {
          secret_key_ref {
            secret  = "db-password"
            version = "latest"
          }
        }
      }
    }
  }
}
75

How do you design a scalable web application on Google Cloud Platform?

A scalable GCP web application is typically a stateless tier (Cloud Run or GKE) behind a global HTTP(S) load balancer, with managed data stores (Cloud SQL, Spanner, Firestore), Memorystore for caching, Cloud Storage for static assets, and Pub/Sub for asynchronous work, all secured with least-privilege IAM and observed via Cloud Monitoring.

Read full answer

Applies toGoogle Cloud Platform (current stable APIs and gcloud CLI)

Architecture Overview

  • Global external HTTP(S) load balancer terminates TLS, routes by host/path, and fronts Cloud CDN for cacheable content.

  • Stateless compute holds no local state; sessions live in Memorystore Redis or signed cookies.

  • Managed data layer handles durability, replication, and backups; the app tier scales horizontally.

  • Async processing via Pub/Sub decouples slow work (email, image processing, webhooks) from request latency.

Compute Choice

Service Best for Scaling model Notes
Cloud Run Stateless HTTP containers, event-driven Request-based, scale to zero Fastest to ship; cold starts possible
GKE (Autopilot) Complex microservices, custom networking HPA + cluster autoscaler More control, higher ops cost
App Engine (standard) Simple web apps, legacy PaaS Automatic Less flexible than Cloud Run
Compute Engine Stateful/legacy, custom OS needs MIG autoscaling Most control, most ops burden

Choose Cloud Run for most new HTTP services; move to GKE when you need sidecars, custom ingress, or fine-grained networking.

Data Layer

  • Cloud SQL (PostgreSQL/MySQL): regional relational data with automated backups and read replicas.

  • Cloud Spanner: globally distributed, strongly consistent relational data at higher cost.

  • Firestore: document store for flexible schemas and real-time sync.

  • Bigtable: wide-column store for high-throughput time-series/IoT workloads.

  • Memorystore (Redis): caching, sessions, rate limiting.

  • Cloud Storage: static assets, uploads, backups; serve via Cloud CDN.

Networking and Load Balancing

  • Place compute in a VPC with private IPs; use Serverless VPC Access connectors so Cloud Run reaches Cloud SQL/Memorystore privately.

  • Attach Cloud Armor policies for WAF, rate limiting, and DDoS protection.

  • Use Cloud CDN in front of Cloud Storage and cacheable endpoints.

  • Prefer private service access for Cloud SQL instead of public IPs.

Scaling and Autoscaling

  • Cloud Run: set --concurrency, --min-instances (to avoid cold starts on critical paths), and --max-instances (cost guardrail); autoscaling is request/concurrency-based.

  • GKE: Horizontal Pod Autoscaler on CPU/memory/custom metrics plus cluster autoscaler; use Pod Disruption Budgets.

  • Database: scale reads via replicas; scale writes via sharding or Spanner; watch connection limits (use connection pooling, e.g., Cloud SQL Auth Proxy or PgBouncer).

Security

  • One service account per service with least-privilege IAM roles; avoid the default Compute service account.

  • Use Workload Identity on GKE and service account impersonation for CI/CD.

  • Store secrets in Secret Manager, mounted as env vars or files; never bake secrets into images.

  • Enable VPC Service Controls for sensitive data perimeters; enforce org policies (e.g., disable public IPs).

  • Sign and scan container images; deploy only from Artifact Registry.

Observability

  • Cloud Logging (structured JSON), Cloud Monitoring (metrics, dashboards, alerting), Cloud Trace (latency), Error Reporting.

  • Define SLOs and burn-rate alerts; use uptime checks on the load balancer.

  • Propagate trace context across services; log request IDs.

Cost Optimization

  • Scale to zero where latency allows; set max instances to cap spend.

  • Use committed use discounts for steady GKE/Compute baseline.

  • Cache aggressively at CDN and Redis; right-size Cloud SQL.

  • Avoid cross-region egress; keep compute and data in the same region.

Code Example

# Deploy a containerized service to Cloud Run with scaling and VPC access
gcloud run deploy web-api \
  --image=us-central1-docker.pkg.dev/PROJECT_ID/repo/web-api:latest \
  --region=us-central1 \
  --allow-unauthenticated \
  --concurrency=80 \
  --min-instances=2 \
  --max-instances=50 \
  --cpu=1 --memory=512Mi \
  --vpc-connector=projects/PROJECT_ID/locations/us-central1/connectors/run-connector \
  --set-secrets=DB_PASSWORD=db-password:latest

Best Practices and Pitfalls

  • Do not store session state on instances; it breaks horizontal scaling.

  • Watch Cloud SQL connection limits; use pooling and the Cloud SQL Auth Proxy.

  • Test cold-start latency before relying on scale-to-zero for user-facing paths.

  • Keep regions consistent to avoid egress cost and latency; use multi-region only when justified.

  • Automate everything with Terraform; treat infra as code and review changes via CI.

76

How do you design a production data analytics pipeline on GCP?

A production analytics pipeline on Google Cloud typically uses Pub/Sub for event ingestion, Dataflow for stream and batch processing, Cloud Storage for immutable raw data, and BigQuery as the analytical warehouse. This architecture supports both real-time dashboards and large-scale batch reporting while remaining independently scalable.

Read full answer

Applies toGoogle Cloud CLI 500.0.0+ with current stable Google Cloud managed services

End-to-End Architecture

  1. Sources publish events: Applications, APIs, databases, and operational systems publish to a versioned Pub/Sub topic. For example, an order service publishes order.created events containing an event_id, event timestamp, and business payload.

  2. Pub/Sub buffers ingestion: Pub/Sub provides durable, at-least-once delivery and absorbs temporary traffic spikes. It decouples producers from consumers.

  3. Dataflow validates and transforms: A Dataflow streaming pipeline consumes events, validates schemas, removes invalid records, enriches data, and writes curated results.

  4. Cloud Storage preserves raw data: Dataflow or an ingestion service writes immutable raw objects to Cloud Storage. These objects provide a replay source for backfills and recovery.

  5. BigQuery serves analytics: Curated events and dimensional tables are stored in partitioned BigQuery tables. Looker, Looker Studio, or internal APIs run dashboards, reports, and ad hoc analysis.

  6. Batch processing uses the same model: Scheduled sources land in Cloud Storage. Cloud Scheduler and Workflows trigger a Dataflow batch job that produces additional BigQuery tables.

Service Selection

Capability Google Cloud AWS equivalent Azure equivalent
Event ingestion Pub/Sub SNS/SQS or EventBridge Event Grid or Service Bus
Stream processing Dataflow Kinesis Data Analytics, Glue, EMR Stream Analytics or Fabric
Raw object storage Cloud Storage S3 Blob Storage
Analytical warehouse BigQuery Redshift or Athena Synapse or Fabric
Semantic layer and BI Looker QuickSight Power BI

Choose BigQuery for SQL analytics, especially when elasticity and large scans matter. Choose Bigtable for high-throughput, low-latency operational key-value access, not as a general-purpose SQL warehouse. Use Cloud Storage plus BigQuery external tables when retaining Parquet files is more important than materializing every dataset.

Data Model and Processing

  • Define an explicit data contract, preferably using a registered Avro or Protobuf schema. Reject incompatible changes before they reach downstream consumers.

  • Include an immutable event_id, an occurrence timestamp, and an ingestion timestamp. Use event_id for deduplication because Pub/Sub delivery is at least once.

  • Partition BigQuery tables by date and cluster by frequently filtered columns such as customer_id or event_type.

  • Keep raw, validated, and curated datasets separate. This prevents source alterations from silently changing analytical results.

  • Route poison messages to a Pub/Sub dead-letter topic. Retain enough metadata to identify the schema version, failure reason, and source.

Reliability, Security, and Cost

  • Reliability: Configure Dataflow checkpointing, bounded retries, and autoscaling. Acknowledge Pub/Sub messages only after successful downstream processing. Test recovery by replaying raw Cloud Storage data.

  • Availability: Deploy pipelines in the required region and, when justified, maintain an independently deployable failover job in another region. Do not assume automatic cross-region Dataflow failover.

  • Monitoring: Alert on Pub/Sub backlog age, undelivered message count, Dataflow job failures and stalled workers, BigQuery failures, and data-freshness SLOs. Also monitor row counts, null rates, duplicate rates, and late events.

  • Security: Use separate service accounts for ingestion and processing. Grant narrowly scoped roles such as roles/pubsub.publisher, roles/pubsub.subscriber, roles/storage.objectAdmin, roles/bigquery.dataEditor, and roles/dataflow.worker. Avoid primitive project-wide roles.

  • Data protection: Use CMEK where required, VPC Service Controls for sensitive workloads, private networking for Dataflow workers, and organization policies restricting public access.

  • Cost control: Partition and cluster BigQuery tables, limit repeated full-table scans, control Dataflow worker autoscaling, and apply Cloud Storage lifecycle rules. Use BigQuery slots when predictable consumption justifies committed capacity.

Code Example

The following creates the main managed resources and an initial partitioned BigQuery table. The project must already exist, have billing enabled, and the caller must have permission to enable APIs and create these resources.

set -euo pipefail

PROJECT_ID="example-analytics-project"
REGION="us-central1"
BUCKET="gs://${PROJECT_ID}-raw-analytics"

gcloud services enable \
  pubsub.googleapis.com \
  bigquery.googleapis.com \
  storage.googleapis.com \
  dataflow.googleapis.com \
  --project="$PROJECT_ID"

gcloud pubsub topics create analytics-events \
  --project="$PROJECT_ID"

gcloud pubsub subscriptions create analytics-events-sub \
  --topic=analytics-events \
  --project="$PROJECT_ID"

gcloud storage buckets create "$BUCKET" \
  --location=US \
  --project="$PROJECT_ID"

gcloud bq datasets create analytics \
  --location=US \
  --project="$PROJECT_ID"

gcloud bq query \
  --project="$PROJECT_ID" \
  --use_legacy_sql=false \
  '
  CREATE OR REPLACE TABLE analytics.events (
    event_id STRING NOT NULL,
    event_ts TIMESTAMP NOT NULL,
    customer_id STRING,
    event_type STRING,
    payload JSON
  )
  PARTITION BY DATE(event_ts)
  CLUSTER BY customer_id, event_type;
  '

Production Practices

  • Define freshness, completeness, latency, and availability SLOs before launch.

  • Test schema evolution, duplicate delivery, poison messages, regional failures, and partial Dataflow failures.

  • Version schemas and pipelines; deploy schema changes through backward-compatible migrations.

  • Use object versioning or retention policies for critical raw data, and regularly test restoration procedures.

  • Keep secrets in Secret Manager and avoid placing credentials in Pub/Sub messages, BigQuery rows, or Cloud Storage objects.

  • Maintain a replayable raw layer and documented backfill process. During an incident, preserve dead-letter messages rather than deleting them.

77

How do you secure a new Google Cloud organization?

Securing a new GCP organization means locking down the resource hierarchy, IAM, and organization policies before any workload is deployed, then enabling centralized logging and guardrails that apply to every project created afterward.

Read full answer

Applies toGoogle Cloud Platform (gcloud CLI 4xx, current stable)

Resource Hierarchy and IAM

  • Create the organization under a verified domain via Cloud Identity or Google Workspace; the organization is the root of all resources.

  • Use folders to separate environments (prod, staging, dev) and business units; apply IAM at the folder level so child projects inherit policy.

  • Grant roles/resourcemanager.organizationAdmin to a small break-glass group only; use custom roles or predefined roles scoped to folders/projects for everyone else.

  • Avoid primitive roles (Owner, Editor, Viewer) on the organization; prefer fine-grained predefined roles.

  • Enable Workload Identity Federation for CI/CD and external workloads instead of long-lived service account keys.

Organization Policies

  • Set constraints at the organization level so they inherit to all projects: disable service account key creation (constraints/iam.disableServiceAccountKeyCreation), require OS Login (constraints/compute.requireOsLogin), restrict VM external IPs, and require Shielded VMs.

  • Restrict resource locations with constraints/gcp.resourceLocations to keep data in approved regions.

  • Disable creation of default networks and require VPC flow logs via policy where supported.

  • Use Policy Intelligence (IAM Recommender) to remove unused permissions over time.

Identity and Access

  • Enforce MFA and SSO through Cloud Identity/Workspace; disable consumer Gmail accounts for org resources.

  • Use groups, not individual users, in IAM bindings; rotate membership via your IdP.

  • Create a dedicated break-glass project and account with Organization Administrator, monitored and alerted on use.

Logging, Monitoring, and Detection

  • Admin Activity audit logs are always on; explicitly enable Data Access audit logs for sensitive services (Cloud Storage, BigQuery, Secret Manager).

  • Create an organization-level log sink to a central, tamper-resistant project or BigQuery dataset for long-term retention and SIEM ingestion.

  • Enable Security Command Center (Premium where budget allows) for misconfiguration and threat detection.

  • Set up alerting on IAM policy changes, new service account keys, and firewall rule changes.

Network and Data Protection

  • Use VPC Service Controls perimeters around sensitive projects to prevent data exfiltration.

  • Default-deny firewall rules; allow only required ports and sources.

  • Encrypt data at rest with CMEK via Cloud KMS for regulated workloads; manage keys in a separate project.

  • Use Secret Manager for credentials; never store secrets in source code or environment variables in plaintext.

Code Example

# List organizations
gcloud organizations list

# Disable service account key creation org-wide
gcloud resource-manager org-policies enable-enforce \
  constraints/iam.disableServiceAccountKeyCreation \
  --organization=ORG_ID

# Require OS Login on Compute instances
gcloud resource-manager org-policies enable-enforce \
  constraints/compute.requireOsLogin \
  --organization=ORG_ID

# Restrict resource locations
gcloud resource-manager org-policies set-policy policy.yaml --organization=ORG_ID

# Organization-level audit log sink to a central GCS bucket
gcloud logging sinks create org-audit-sink \
  storage.googleapis.com/central-audit-bucket \
  --organization=ORG_ID \
  --include-children \
  --log-filter='logName:"cloudaudit.googleapis.com"'

policy.yaml:

name: organizations/ORG_ID/policies/gcp.resourceLocations
spec:
  rules:
  - values:
      allowedValues:
      - in:us-locations

Best Practices and Common Pitfalls

  • Apply guardrails at the organization/folder level, not per project, so new projects inherit them automatically.

  • Do not grant roles/owner to developers; use project-level roles/editor only where justified and time-boxed.

  • Test organization policies in a non-prod folder first; some constraints break existing automation.

  • Grant the log sink’s service account the required writer role on the destination bucket or dataset.

  • Review SCC findings and IAM Recommender suggestions on a recurring schedule, not just at setup.

  • Keep a documented break-glass procedure and test it quarterly.

AWS/Azure Equivalents

GCP AWS Azure
Organization + Folders AWS Organizations + OUs Management Groups
Organization Policies SCPs Azure Policy
Cloud Identity / Workspace IAM Identity Center Entra ID
VPC Service Controls (no direct equivalent) Private Link + NSGs
Security Command Center Security Hub / GuardDuty Microsoft Defender for Cloud

Choose GCP organization policies when you want declarative, inheritable constraints; combine with folders for environment isolation and SCC for continuous posture monitoring.

78

How do I troubleshoot 503 errors on a Google Cloud Run service?

A Cloud Run 503 service-unavailable error usually means either no healthy container can serve the request or the application itself returned 503. Troubleshoot by separating platform availability failures from application, dependency, and rollout failures.

Read full answer

Applies toGoogle Cloud Run managed services (current stable) and gcloud CLI 2.x

Triage Workflow

  1. Confirm the URL and status code

  2. A wrong URL normally returns 404.

  3. Authentication or IAM failures normally return 403, not 503.

  4. Test both the service root and an application-owned health endpoint.

  5. Inspect service and revision state

  6. Check the active image, environment configuration, service account, resource limits, traffic assignments, and status conditions.

  7. Compare the failing revision with the last known-good revision.

  8. Inspect the service IAM policy if callers receive 403 responses.

  9. Read logs

  10. Inspect container logs for crashes, missing environment variables, failed dependency calls, port-binding errors, startup-probe failures, and explicit 503 responses.

  11. Check Cloud Logging for errors from Cloud Run revisions.

  12. Check production metrics

  13. In Cloud Monitoring, review request count by status code, request latency, container instance count, CPU, and memory.

  14. A sudden drop in ready instances, repeated restarts, or memory saturation explains many platform-generated 503s.

What 503 Means Under the Hood

Cloud Run routes incoming requests through the Google Front End to an available revision instance. A container must listen on the port supplied through the PORT environment variable and bind to 0.0.0.0, not only a loopback address. If a startup probe is configured, it must also succeed before the instance receives traffic.

A platform-generated 503 commonly indicates:

  • Every instance has crashed, failed its startup checks, or otherwise never became ready.

  • The container is listening on the wrong address or port.

  • Memory exhaustion is causing out-of-memory restarts.

  • A revision has no ready instance.

  • The service configuration points to an image that cannot start.

The application can also deliberately return 503 when a database, cache, external API, or other required dependency is unavailable. Application-generated 503s normally have corresponding application logs. Exceeding the CPU limit normally causes throttling and increased latency rather than an automatic restart.

Cloud Run’s default request timeout is 300 seconds. Exceeding the request timeout normally produces a 504, although an intermediary or application may translate dependency timeouts into 503.

Common Failure Modes

Evidence Likely cause Corrective action
Container logs end with a startup exception Invalid configuration, missing secret, failed initialization Correct the startup dependency and create a new revision
Startup probe never succeeds Incorrect probe path, dependency still unavailable, or unhealthy startup Fix the probe or startup dependency
Error mentions binding or PORT Incorrect listen address or port Listen on 0.0.0.0:$PORT
Process exits immediately Bad entrypoint, command, image, or incompatible architecture Verify the container command and image locally
Application logs show a 503 response Dependency or business-health failure Check database, VPC, DNS, quota, and dependency latency
Errors affect only the newest revision Bad deployment or configuration change Shift traffic back to the known-good revision
Instances restart with out-of-memory errors Memory pressure or leak Profile memory, adjust limits, and fix the leak

Code Example

#!/usr/bin/env bash
set -euo pipefail

PROJECT="example-project"
REGION="us-central1"
SERVICE="example-service"
HEALTH_PATH="/healthz"

# Show configuration, conditions, traffic, and the active image.
gcloud run services describe "$SERVICE" \
  --region="$REGION" \
  --project="$PROJECT" \
  --format=yaml

# Inspect the service IAM policy.
gcloud run services get-iam-policy "$SERVICE" \
  --region="$REGION" \
  --project="$PROJECT"

# List revisions and their traffic state.
gcloud run revisions list \
  --service="$SERVICE" \
  --region="$REGION" \
  --project="$PROJECT"

# Read recent logs and recent errors for this service's revisions.
LOG_FILTER="resource.type=\"cloud_run_revision\" AND resource.labels.service_name=\"${SERVICE}\""

gcloud logging read "$LOG_FILTER" \
  --project="$PROJECT" \
  --freshness=1h \
  --limit=200

gcloud logging read "${LOG_FILTER} AND severity>=ERROR" \
  --project="$PROJECT" \
  --freshness=1h \
  --limit=200

URL="$(gcloud run services describe "$SERVICE" \
  --region="$REGION" \
  --project="$PROJECT" \
  --format='value(status.url)')"

# The identity token works for a private service and is also accepted by a public one.
TOKEN="$(gcloud auth print-identity-token)"

for request_path in "/" "$HEALTH_PATH"; do
  curl --max-time 15 -i \
    -H "Authorization: Bearer ${TOKEN}" \
    "${URL}${request_path}"
done

If the latest revision is broken, identify an exact healthy revision from gcloud run revisions list and restore traffic to it:

PROJECT="example-project"
REGION="us-central1"
SERVICE="example-service"
REVISION="example-service-00041-abc" # Replace with an exact revision ID from the list.

gcloud run services update-traffic "$SERVICE" \
  --to-revisions "${REVISION}=100" \
  --region="$REGION" \
  --project="$PROJECT"

Traffic shifting is an immediate mitigation, not a repair. Diagnose the failed revision before redeploying.

Production Readiness

  • Configure Cloud Monitoring alerts for 5xx rates, latency, instance count, CPU, and memory.

  • Use Cloud Trace and structured application logs to identify slow dependencies.

  • Keep the runtime service account least-privileged; startup failures may result from missing IAM bindings.

  • Grant access to Secret Manager resources only when required, and never print secret values in incident logs.

  • Validate private connectivity and Cloud SQL or VPC connectivity from the service’s environment.

  • Treat successful deployment as insufficient: readiness proves that the container startup contract succeeded, including any configured startup probe, not that business endpoints are healthy.

  • Increasing minimum instances can reduce cold-start latency, but it does not fix crashing containers and increases cost.

79

How do you set up CI/CD for a containerized application on GCP?

A typical GCP CI/CD flow for containers uses Cloud Build to build and test images, Artifact Registry to store them, and Cloud Deploy to promote releases across environments to Cloud Run or GKE. Keeping builds, artifacts, and deployments inside GCP lets you enforce IAM and audit everything in one place.

Read full answer

Applies toGoogle Cloud (Cloud Build, Artifact Registry, Cloud Deploy; gcloud CLI 470+, Terraform 1.x)

How It Works

  • A push to your source repository (GitHub, GitLab, or Bitbucket; Cloud Source Repositories is closed to new customers) triggers a Cloud Build trigger.

  • Cloud Build runs tests, builds the container image, and pushes it to Artifact Registry.

  • Cloud Deploy creates a release and rolls it out through configured targets (dev, staging, prod), with optional manual approvals between stages.

  • The target runtime (Cloud Run or GKE) pulls the new image and runs the container.

Key Components

  • Cloud Build: managed build service driven by a cloudbuild.yaml file.

  • Artifact Registry: regional repository for Docker images (replaces Container Registry).

  • Cloud Deploy: managed continuous delivery with promotion, approvals, and rollback.

  • Runtime: Cloud Run for serverless containers, GKE for Kubernetes workloads.

Pipeline Stages

  1. Trigger on push to main.

  2. Run unit and integration tests.

  3. Build image tagged with the commit SHA.

  4. Push image to Artifact Registry.

  5. Create a Cloud Deploy release.

  6. Auto-deploy to dev; require approval to promote to prod.

Runtime Comparison

Factor Cloud Run GKE
Ops overhead None (serverless) Cluster management
Scaling Automatic, to zero HPA / cluster autoscaler
Best for Stateless HTTP services Complex, stateful, or multi-service workloads
Networking Managed ingress Full VPC control

Security and Best Practices

  • Use Workload Identity Federation for GitHub/GitLab; avoid long-lived service account keys.

  • Enable Artifact Analysis (container scanning) and block vulnerable images.

  • Apply least-privilege IAM: separate build, deploy, and runtime service accounts.

  • Use distroless or minimal base images and pin versions.

  • Enable Binary Authorization to enforce attestation before deploy.

  • Tag images with commit SHA and never overwrite latest in production.

Code Example

Create the repository and submit a build:

gcloud artifacts repositories create my-repo \
  --repository-format=docker --location=us-central1

# COMMIT_SHA is populated automatically only for trigger-based builds;
# a manual submit leaves it empty, so run this file through a trigger.
gcloud builds submit --config=cloudbuild.yaml

cloudbuild.yaml:

steps:
  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t',
           'us-central1-docker.pkg.dev/${PROJECT_ID}/my-repo/my-app:${COMMIT_SHA}', '.']
  - name: 'gcr.io/cloud-builders/docker'
    args: ['push',
           'us-central1-docker.pkg.dev/${PROJECT_ID}/my-repo/my-app:${COMMIT_SHA}']
images:
  - 'us-central1-docker.pkg.dev/${PROJECT_ID}/my-repo/my-app:${COMMIT_SHA}'

Cloud Deploy pipeline (clouddeploy.yaml):

apiVersion: deploy.cloud.google.com/v1
kind: DeliveryPipeline
metadata:
  name: my-pipeline
serialPipeline:
  stages:
    - targetId: dev
    - targetId: prod

Create a release:

gcloud deploy releases create rel-001 \
  --delivery-pipeline=my-pipeline --region=us-central1 \
  --images=my-app=us-central1-docker.pkg.dev/PROJECT_ID/my-repo/my-app:SHA

Common Pitfalls

  • Building images with latest tags breaks rollback and auditability.

  • Skipping image scanning lets vulnerable base images reach production.

  • Coupling build and deploy in one step removes the approval gate between environments.

  • Forgetting to set --region consistently causes release lookup failures.

Sign in to keep reading

You've opened 2 full answers. Sign in to read all 79 Google Cloud (GCP) answers, mark what you know and pick up where you left off.

Report a mistake

What's wrong?

All Cloud, DevOps & Infrastructure subjects