This document offers a brief introduction to utilizing the Flink Kubernetes Operator. By following this guide, readers will learn how to deploy both the Flink operator and a sample Flink job on a local Kubernetes setup.
Prerequisites
Ensure that you have the following installed on your local system:
Make sure that the kubectl and helm commands are available on your system.
For Docker, we recommend installing Docker Desktop and configuring it with at least 8GB of RAM. For Kubernetes, you should have a running Kubernetes instance on Docker.
Deploying the Operator
To deploy the Flink Kubernetes Operator, follow these steps:
Install the certificate manager on your Kubernetes cluster to enable the webhook component (needed only once per cluster):
kubectl create -f https://github.com/jetstack/cert-manager/releases/download/v1.8.2/cert-manager.yaml
If the cert manager installation fails, you can disable the webhook by passing --set webhook.create=false to the Helm install command for the operator.
Deploy the Flink Kubernetes Operator using Helm:
helm repo add flink-operator-repo https://downloads.apache.org/flink/flink-kubernetes-operator-1.9.0/
helm install flink-kubernetes-operator flink-operator-repo/flink-kubernetes-operator
To find the list of stable versions, visit Apache Flink Downloads.
The Helm chart defaults to the
ghcr.io/apache/flink-kubernetes-operatorimage repository. If you encounter connectivity issues or prefer Dockerhub, use--set image.repository=apache/flink-kubernetes-operatorduring installation.
Verify your installationusingkubectlandhelm:
kubectl get pods
NAME READY STATUS RESTARTS AGE
flink-kubernetes-operator-fb5d46f94-ghd8b 2/2 Running 0 4m21s
helm list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
flink-kubernetes-operator default 1 2022-03-09 17 (tel:12022030917):39:55.461359 +0100 CET deployed flink-kubernetes-operator-{{< version >}} {{< version >}}
Submitting a Flink Job
With the operator running, you can now submit a Flink job:
kubectl create -f https://raw.githubusercontent.com/apache/flink-kubernetes-operator/release-1.9/examples/basic.yaml
You can follow the job logs to monitor its progress. After a successful startup (which may take a minute initially), view the logs with:
kubectl logs -f deploy/basic-example
To expose the Flink Dashboard, add a port-forward rule:
kubectl port-forward svc/basic-example-rest 8081
The Flink Dashboard will then be accessible at localhost:8081.
To stop your job and delete the FlinkDeployment, use:
kubectl delete flinkdeployment/basic-example
This guide helps you get started with deploying the Flink Kubernetes Operator and running Flink jobs on a local Kubernetes installation using Docker. For other deployment methods, please refer to the Apache Flink website.