Docker Advanced Configuration & Troubleshooting

Configure port mapping, load balancing, AWS services, volumes, and troubleshoot common Docker deployment issues for OpenMetadata.

Docker Advanced Configuration & Troubleshooting

Port Mapping / Port Forwarding

We are shipping the OpenMetadata server and UI at container port and host port 8585. You can change the host port number according to your requirement. As an example, You could update the ports to serve OpenMetadata Server and UI at port 80

To achieve this -

• You just have to update the ports mapping of the openmetadata-server in the docker-compose.yml file under openmetadata-server docker service section.

---
ports:
  - "80:8585"

• Once the port is updated if there are any containers running remove them first using docker compose down command and then recreate the containers once again by below command

docker compose up --detach

Run OpenMetadata with a load balancer

You may put one or more OpenMetadata instances behind a load balancer for reverse proxying. To do this you will need to add one or more entries to the configuration file for your reverse proxy.

Nginx

To use OpenMetadata behind Nginx reverse proxy, add an entry resembling the following in the http context of your Nginx configuration file for each OpenMetadata instance.

server {
    access_log /var/log/nginx/stage-reverse-access.log;
    error_log /var/log/nginx/stage-reverse-error.log;
    server_name stage.open-metadata.org;
    location / {
        proxy_pass http://127.0.0.1:8585;
    }
}

Run OpenMetadata with AWS Services

If you are running OpenMetadata in AWS, it is recommended to use Amazon RDS and Amazon OpenSearch Service.

We support

• Amazon RDS (MySQL) engine version 8 or higher
• Amazon OpenSearch (ElasticSearch) engine version 9.x (minimum 9.0.0, recommended 9.3.0) or Amazon OpenSearch engine version 3.x (minimum 3.0.0, recommended 3.3.0)
• Amazon RDS (PostgreSQL) engine version 12 or higher

Note:- When using AWS Services the SearchType Configuration for elastic search should be opensearch, for both cases ElasticSearch and OpenSearch, as you can see in the ElasticSearch configuration example.

For Production Systems, we recommend Amazon RDS to be in Multiple Availability Zones. For Amazon OpenSearch (or ElasticSearch) Service, we recommend Multiple Availability Zones with minimum 3 Master Nodes.

Once you have the RDS and OpenSearch Services Setup, you can update the environment variables below for OpenMetadata Docker Compose backed systems to connect with Database and ElasticSearch.

# MySQL Environment Variables
DB_DRIVER_CLASS='com.mysql.cj.jdbc.Driver'
DB_SCHEME='mysql'
DB_PARAMS='allowPublicKeyRetrieval=true&useSSL=true&serverTimezone=UTC'
DB_USER_PASSWORD='<DATABASE_USER_PASSWORD>'
DB_HOST='<DATABASE_HOST_NAME>'
DB_USER='<DATABASE_USER_NAME>'
OM_DATABASE='<DATABASE_NAME>'
DB_PORT='<DATABASE_PORT>'
# ElasticSearch Environment Variables
SEARCH_TYPE='opensearch'
ELASTICSEARCH_SOCKET_TIMEOUT_SECS='60'
ELASTICSEARCH_USER='<ELASTICSEARCH_USERNAME>'
ELASTICSEARCH_CONNECTION_TIMEOUT_SECS='5'
ELASTICSEARCH_PORT='443'
ELASTICSEARCH_SCHEME='https'
ELASTICSEARCH_BATCH_SIZE='10'
ELASTICSEARCH_HOST='<ELASTICSEARCH_HOST_URL>'
ELASTICSEARCH_PASSWORD='<ELASTICSEARCH_PASSWORD>'
ELASTICSEARCH_CLUSTER_ALIAS='<clusterAlias>'

Replace the environment variables values with the RDS and OpenSearch Service ones and then provide this environment variable file as part of docker compose command.

docker compose --env-file ./env-mysql up --detach

Advanced

Add Docker Volumes for OpenMetadata Server Compose Service

There are many scenarios where you would want to provide additional files to the OpenMetadata Server and serve while running the application. In such scenarios, it is recommended to provision docker volumes for OpenMetadata Application.

If you are not familiar with Docker Volumes with Docker Compose Services, Please refer to official documentation for more information.

For example, we would like to provide custom JWT Configuration Keys to be served to OpenMetadata Application. This requires the OpenMetadata Containers to have docker volumes sharing the private and public keys. Let’s assume you have the keys available in jwtkeys directory in the same directory where your docker-compose file is available in the host machine.

In scenarios where you need to provide a custom openmetadata.yaml configuration file to the OpenMetadata application, you can do so by mounting the file as a volume in the Docker container. This is especially useful for configurations that cannot be controlled through environment variables.

We add the volumes section to mount the keys or openmetadata.yaml onto the docker containers create with docker compose as follows -

services:
    openmetadata-server:
        ...
        volumes:
            - ./jwtkeys:/etc/openmetadata/jwtkeys
            - ./openmetadata.yaml:/opt/openmetadata/conf/openmetadata.yaml
        ...

The above example uses bind mounts to share files and directories between host machine and openmetadata container.

Next, in your environment file, update the jwt configurations to use the right path from inside the container.

...
# JWT Configuration
RSA_PUBLIC_KEY_FILE_PATH="/etc/openmetadata/jwtkeys/public_key.der"
RSA_PRIVATE_KEY_FILE_PATH="/etc/openmetadata/jwtkeys/private_key.der"
...

Ensure that the default environment variables are set appropriately to complement the settings in your openmetadata.yaml.

Once the changes are updated, if there are any containers running remove them first using docker compose down command and then recreate the containers once again by below command

docker compose up --detach

Troubleshooting

Java Memory Heap Issue

If your openmetadata Docker Compose logs speaks about the below issue -

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "AsyncAppender-Worker-async-file-appender"
Exception in thread "pool-5-thread-1" java.lang.OutOfMemoryError: Java heap space
Exception in thread "AsyncAppender-Worker-async-file-appender" java.lang.OutOfMemoryError: Java heap space
Exception in thread "dw-46" java.lang.OutOfMemoryError: Java heap space
Exception in thread "AsyncAppender-Worker-async-console-appender" java.lang.OutOfMemoryError: Java heap space

This is due to the default JVM Heap Space configuration (1 GiB) being not enough for your workloads. In order to resolve this issue, head over to your custom openmetadata environment variable file and append the below environment variable

#environment variable file
OPENMETADATA_HEAP_OPTS="-Xmx2G -Xms2G"

The flag Xmx specifies the maximum memory allocation pool for a Java virtual machine (JVM), while Xms specifies the initial memory allocation pool.

Restart the OpenMetadata Docker Compose Application using docker compose --env-file <my-env-file> -f docker-compose.yml up --detach which will recreate the containers with new environment variable values you have provided.

PostgreSQL Issue permission denied to create extension “pgcrypto”

If you are facing the below issue with PostgreSQL as Database Backend for OpenMetadata Application,

Message: ERROR: permission denied to create extension "pgcrypto"
Hint: Must be superuser to create this extension.

It seems the Database User does not have sufficient privileges. In order to resolve the above issue, grant usage permissions to the PSQL User.

GRANT USAGE ON SCHEMA schema_name TO <openmetadata_psql_user>;
GRANT CREATE ON EXTENSION pgcrypto TO <openmetadata_psql_user>;

In the above command, replace <openmetadata_psql_user> with the sql user used by OpenMetadata Application to connect to PostgreSQL Database.

In the above command, replace <openmetadata_psql_user> with the sql user used by OpenMetadata Application to connect to PostgreSQL Database.

Security

Please follow our Enable Security Guide to configure security for your OpenMetadata installation.

Next Steps

1. Refer the How-to Guides for an overview of all the features in OpenMetadata.
2. Visit the Connectors documentation to see what services you can integrate with OpenMetadata.
3. Visit the API documentation and explore the rich set of OpenMetadata APIs.