Apache Hop
Download
User manual 2.16.0

User manual

Development Documentation

Edit this page

Google Cloud Platform Service Account Setup

To connect to Google Cloud services, you need to set up a service account in your Google Cloud Platform (GCP) project and download the authentication credentials.

Creating a Service Account

  1. Navigate to the Google Cloud Console:

  2. Create a Service Account:

    • Navigate to "IAM & Admin" > "Service Accounts"

    • Click "Create Service Account"

    • Provide a name and description for your service account

    • Click "Create and Continue"

  3. Assign Roles:

    • Assign the appropriate roles to your service account based on the service you’re connecting to:

      • For BigQuery:

        • BigQuery Data Viewer (roles/bigquery.dataViewer) - for read access

        • BigQuery Job User (roles/bigquery.jobUser) - for running queries (required)

        • BigQuery Data Editor (roles/bigquery.dataEditor) - if write access needed

        • BigQuery Data Owner (roles/bigquery.dataOwner) - if full control needed

      • For Cloud Storage: Storage Object Viewer, Storage Object Admin (if write access needed)

      • For other services: Consult the specific service documentation for required roles

    • Click "Continue" and then "Done"

  4. Generate and Download Key:

    • Click on the created service account

    • Go to the "Keys" tab

    • Click "Add Key" > "Create new key"

    • Select "JSON" format

    • Click "Create" to download the key file

  5. Secure the Key File:

    • Store the downloaded JSON key file in a secure location

    • Note the full path to this file - you’ll need it for authentication configuration

    • Ensure the file has appropriate permissions (readable only by the user running Hop)

Alternative: Using Application Default Credentials

Apache Hop can also use Google Cloud’s Application Default Credentials (ADC) if you’re running Hop on Google Cloud Platform or have configured the Google Cloud SDK locally.

To use ADC: 1. Install and configure the Google Cloud SDK 2. Run gcloud auth application-default login to set up default credentials 3. In your Hop connection, you can omit service account key file parameters

This method is particularly useful for development environments or when running Hop on Google Cloud Platform services.

Security Best Practices

  • Principle of Least Privilege: Only assign the minimum roles necessary for your use case

  • Key Rotation: Regularly rotate service account keys (recommended every 90 days)

  • Environment Variables: Consider using environment variables to store key file paths instead of hardcoding them

  • Access Control: Restrict access to service account key files using appropriate file system permissions

  • Monitoring: Enable audit logging to monitor service account usage