Skip to main content

Documentation Index

Fetch the complete documentation index at: https://openmetadata-feat-feat-2mbfixdeploy.mintlify.app/llms.txt

Use this file to discover all available pages before exploring further.

​
EKS on Amazon Web Services Deployment

OpenMetadata supports the Installation and Running of Application on Elastic Kubernetes Services (EKS) through Helm Charts. However, there are some additional configurations which needs to be done as prerequisites for the same.
All the code snippets in this section assume the default namespace for kubernetes. This guide presumes you have AWS EKS Cluster already available.

​
Prerequisites

​
AWS Services for Database as RDS and Search Engine as ElasticSearch

It is recommended to use Amazon RDS and Amazon OpenSearch Service for Production Deployments. We support
  • Amazon RDS (MySQL) engine version 8 or higher
  • Amazon RDS (PostgreSQL) engine version 12 or higher
  • Amazon OpenSearch engine version 3.x (minimum 3.0.0, recommended 3.3.0)
When using AWS Services the SearchType Configuration for elastic search should be opensearch, for both cases ElasticSearch and OpenSearch, as you can see in the ElasticSearch configuration example below.
We recommend
  • Amazon RDS to be in Multiple Availability Zones.
  • Amazon OpenSearch (or ElasticSearch) Service with Multiple Availability Zones with minimum 2 Nodes.
Make sure to increase sort_buffer_size (for MySQL) or work_mem (for PostgreSQL) to the recommended value of 20MB or more using the database parameter group setting. This is especially important when running migrations to prevent Out of Sort Memory Error. You can revert the setting once the migrations are complete.
Starting with OpenMetadata 1.12, we recommend using the Kubernetes native orchestrator for running ingestion pipelines. This eliminates the need for Apache Airflow and simplifies your deployment.
The Kubernetes orchestrator runs ingestion pipelines as native K8s Jobs and CronJobs. For full documentation on features, configuration options, and troubleshooting, see the Kubernetes Orchestrator Guide.
The recommended OMJob Operator approach requires installing Custom Resource Definitions (CRDs), which needs elevated cluster permissions. If your cluster policies don’t allow CRDs, you can disable the operator by setting useOMJobOperator: false and omjobOperator.enabled: false in your values file to use native K8s Jobs instead.

​
OpenMetadata Values Configuration

Create your openmetadata-values.yaml with the following configuration: 
# openmetadata-values.yaml
openmetadata:
  config:
    # Database configuration
    elasticsearch:
      host: <AMAZON_OPENSEARCH_SERVICE_ENDPOINT_WITHOUT_HTTPS>
      searchType: opensearch
      port: 443
      scheme: https
      connectionTimeoutSecs: 5
      socketTimeoutSecs: 60
      keepAliveTimeoutSecs: 600
      batchSize: 10
      auth:
        enabled: true
        username: <AMAZON_OPENSEARCH_USERNAME>
        password:
          secretRef: elasticsearch-secrets
          secretKey: openmetadata-elasticsearch-password
    database:
      host: <AMAZON_RDS_ENDPOINT>
      port: 3306
      driverClass: com.mysql.cj.jdbc.Driver
      dbScheme: mysql
      dbUseSSL: true
      databaseName: <RDS_DATABASE_NAME>
      auth:
        username: <RDS_DATABASE_USERNAME>
        password:
          secretRef: mysql-secrets
          secretKey: openmetadata-mysql-password

    # Kubernetes Orchestrator configuration
    pipelineServiceClientConfig:
      enabled: true
      type: "k8s"
      metadataApiEndpoint: http://openmetadata:8585/api

      k8s:
        ingestionImage: "docker.getcollate.io/openmetadata/ingestion-base:1.12.0"
        useOMJobOperator: true

# Enable the OMJob Operator (recommended for production)
omjobOperator:
  enabled: true
  image:
    repository: docker.getcollate.io/openmetadata/omjob-operator
    tag: "1.12.0"
For advanced configuration options such as resource limits, job lifecycle settings, failure diagnostics, RBAC, and security contexts, see the Kubernetes Orchestrator Guide.

​
Create Kubernetes Secrets

Create the required secrets for RDS and OpenSearch: 
# Database secret
kubectl create secret generic mysql-secrets \
  --from-literal=openmetadata-mysql-password=<YOUR_RDS_PASSWORD>

# OpenSearch secret
kubectl create secret generic elasticsearch-secrets \
  --from-literal=openmetadata-elasticsearch-password=<YOUR_OPENSEARCH_PASSWORD>

​
Deploy OpenMetadata

# Add the OpenMetadata Helm repository
helm repo add open-metadata https://helm.open-metadata.org/
helm repo update

# Install OpenMetadata (no dependencies chart needed with K8s orchestrator)
helm install openmetadata open-metadata/openmetadata \
  --values openmetadata-values.yaml
With the Kubernetes orchestrator, you don’t need to deploy the openmetadata-dependencies chart that includes Airflow. This significantly simplifies your deployment.

​
Verify the Deployment

# Check pods are running
kubectl get pods

# Check the K8s orchestrator health in OpenMetadata UI
# Navigate to Settings → Preferences → Health
If you prefer to use Apache Airflow as the orchestrator for your EKS deployment, see the Airflow on EKS guide.