Release History
Help

YMatrix

Quick Start

Deployment

Data Usage

Manage Clusters

Security

Best Practice

Technical Principles

Advanced Features

Graph Database

Guide

SQL Reference

FAQ

MatrixGate Kafka Encrypted Connection (SSL) Configuration

This document describes how to enable SSL encrypted connections when MatrixGate connects to Kafka. The SSL configuration described here applies to MatrixGate v6.8.2 and later.

1 Feature Overview

SSL connections and existing SASL authentication are independent configuration items. They can be enabled separately or together.

MatrixGate currently supports one-way TLS: the Kafka connection is encrypted, and MatrixGate verifies the Kafka broker certificate.

After SSL-ca-cert is configured, MatrixGate enables SSL for Kafka connections.

Note!

SSL-ca-cert must be the path of the CA certificate corresponding to the Kafka broker certificate. The certificate file must be readable on the machine where the MatrixGate process runs.

2 Prerequisites

Before configuration, confirm that:

  • MatrixGate is v6.8.2 or later.
  • Kafka brokers have SSL listeners enabled.
  • The MatrixGate node can access the Kafka brokers.
  • The CA certificate file is saved on the MatrixGate node and is readable by the MatrixGate process.

3 Configuration Examples

The following examples show only key Kafka source configuration items related to SSL and SASL. Adjust broker addresses, certificate paths, credentials, topics, partitions, and jobs for your environment.

3.1 SSL Only

When only SSL is enabled, configure the Kafka broker addresses and CA certificate path.

[source]
  source = "kafka"

[source.kafka]
  # Kafka broker addresses. Separate multiple brokers with commas.
  kafka-broker = "broker1:9094,broker2:9094"

  # CA certificate path used to verify Kafka broker certificates. SSL is enabled after this parameter is configured.
  SSL-ca-cert = "/path/to/ca.pem"

  [[source.kafka.topic]]
    topic = "topic_name"
    partition = "*"
    job = "job_name"

3.2 SASL_SSL_PLAIN

When SSL and SASL/PLAIN are enabled together, configure SSL parameters and the SASL username and password.

[source]
  source = "kafka"

[source.kafka]
  kafka-broker = "broker1:9093,broker2:9093"
  SSL-ca-cert = "/path/to/ca.pem"

  # Enable SASL authentication.
  enable-SASL = true

  # Enable Kafka SASL handshake. Standard Kafka SASL connections usually set this to true.
  enable-SASL-handshake = true

  # SASL mechanism. PLAIN means SASL/PLAIN username and password authentication.
  SASL-mechanism = "PLAIN"
  SASL-user = "user"
  SASL-password = "password"

  [[source.kafka.topic]]
    topic = "topic_name"
    partition = "*"
    job = "job_name"

3.3 SASL_SSL_SCRAM

When SSL and SASL/SCRAM are enabled together, configure SSL parameters, the SASL username and password, and the SCRAM hash algorithm.

[source]
  source = "kafka"

[source.kafka]
  kafka-broker = "broker1:9093,broker2:9093"
  SSL-ca-cert = "/path/to/ca.pem"

  enable-SASL = true
  enable-SASL-handshake = true

  # SCRAM means SASL/SCRAM authentication.
  SASL-mechanism = "SCRAM"
  SASL-user = "user"
  SASL-password = "password"

  # SCRAM hash algorithm. Supported values are SHA-256 and SHA-512.
  SCRAM-algorithm = "SHA-512"

  [[source.kafka.topic]]
    topic = "topic_name"
    partition = "*"
    job = "job_name"

3.4 SASL_SSL_GSSAPI

When SSL and SASL/GSSAPI are enabled together, configure SSL parameters and SASL/GSSAPI parameters.

3.4.1 Authenticate with a keytab File

[source]
  source = "kafka"

[source.kafka]
  kafka-broker = "broker1:9093,broker2:9093"
  SSL-ca-cert = "/path/to/ca.pem"

  enable-SASL = true
  enable-SASL-handshake = true
  SASL-mechanism = "GSSAPI"

  # KEYTAB means keytab-file authentication.
  GSSAPI-auth-type = "KEYTAB"
  GSSAPI-kerberos-config-path = "/etc/krb5.conf"
  GSSAPI-keytab-path = "/etc/security/keytabs/kafka.service.keytab"
  GSSAPI-user = "kafka/host.example.com"
  GSSAPI-realm = "EXAMPLE.COM"
  GSSAPI-service-name = "kafka"

  [[source.kafka.topic]]
    topic = "topic_name"
    partition = "*"
    job = "job_name"

3.4.2 Authenticate with Username and Password

[source]
  source = "kafka"

[source.kafka]
  kafka-broker = "broker1:9093,broker2:9093"
  SSL-ca-cert = "/path/to/ca.pem"

  enable-SASL = true
  enable-SASL-handshake = true
  SASL-mechanism = "GSSAPI"

  # USER means username and password authentication.
  GSSAPI-auth-type = "USER"
  GSSAPI-kerberos-config-path = "/etc/krb5.conf"
  GSSAPI-user = "user"
  GSSAPI-password = "password"
  GSSAPI-realm = "EXAMPLE.COM"
  GSSAPI-service-name = "kafka"

  [[source.kafka.topic]]
    topic = "topic_name"
    partition = "*"
    job = "job_name"
← Previous
Kafka
Next →
Spark