[Série][AKS] Provisionando Bancos de Dados no Azure Kubernetes Service – Vitess

Fala minha gente! Espero que estejam todos bem!

Voltando com a série onde utilizamos nosso AKS para provisionar e escalar bancos de dados em containers falaremos desta vez de um dos bancos distribuídos que mais escalam horizontalmente: Vitess!

O Vitess é um projeto graduado da CNCF que entrega um cluster banco de dados MySQL distribuído por meio de uma camada de replicação que permite que você replique dados entre diversas instancias de MySQL, transformando o próprio MySQL em storage distribuído para seus dados. Além disso o Vitess tem a capacidade de utilizar Shardings que é a divisão de dados de uma determinada tabela ou conjunto de tabelas entre clusters diferentes com o intuito de isolar dados focados em uma determinada região do mundo, aumento a performance de escrita e leitura.

Para quem quiser conhecer mais a fundo como o Vitess funciona e sua arquitetura eu apresento aqui nesta palestra que fiz para o KCD Brasil de 2022:

Vamos começar, para isso precisamos criar uma pasta e baixar os arquivos que utilizaremos em nosso lab:

mkdir vitess
cd vitess
git clone https://github.com/vitessio/vitess
cd vitess/examples/operator

Vamos criar nosso namespace para encapsular os recursos que iremos provisionar:

kubectl create namespace vitess

Agora vamos criar o Operator para depois provisionarmos nosso cluster:

kubectl apply -f operator.yaml -n vitess

Agora vamos provisionar nosso cluster Vitess, para isso precisamos primeiro criar a configuração que será utilizada pelos nós do Cluster, crie um arquivo chamado vitess-cluster-config.yaml com o conteúdo:

apiVersion: v1
kind: Secret
metadata:
  name: vitess-cluster-config
type: Opaque
stringData:
  users.json: |
    {
      "user": [{
        "UserData": "user",
        "Password": ""
      }]
    }
  init_db.sql: |
    # This file is executed immediately after mysql_install_db,
    # to initialize a fresh data directory.

    ###############################################################################
    # Equivalent of mysql_secure_installation
    ###############################################################################
    # We need to ensure that super_read_only is disabled so that we can execute
    # these commands. Note that disabling it does NOT disable read_only.
    # We save the current value so that we only re-enable it at the end if it was
    # enabled before.
    SET @original_super_read_only=IF(@@global.super_read_only=1, 'ON', 'OFF');
    SET GLOBAL super_read_only='OFF';

    # Changes during the init db should not make it to the binlog.
    # They could potentially create errant transactions on replicas.
    SET sql_log_bin = 0;
    # Remove anonymous users.
    DELETE FROM mysql.user WHERE User = '';

    # Disable remote root access (only allow UNIX socket).
    DELETE FROM mysql.user WHERE User = 'root' AND Host != 'localhost';

    # Remove test database.
    DROP DATABASE IF EXISTS test;

    ###############################################################################
    # Vitess defaults
    ###############################################################################

    # Vitess-internal database.
    CREATE DATABASE IF NOT EXISTS _vt;
    # Note that definitions of local_metadata and shard_metadata should be the same
    # as in production which is defined in go/vt/mysqlctl/metadata_tables.go.
    CREATE TABLE IF NOT EXISTS _vt.local_metadata (
      name VARCHAR(255) NOT NULL,
      value VARCHAR(255) NOT NULL,
      db_name VARBINARY(255) NOT NULL,
      PRIMARY KEY (db_name, name)
      ) ENGINE=InnoDB;
    CREATE TABLE IF NOT EXISTS _vt.shard_metadata (
      name VARCHAR(255) NOT NULL,
      value MEDIUMBLOB NOT NULL,
      db_name VARBINARY(255) NOT NULL,
      PRIMARY KEY (db_name, name)
      ) ENGINE=InnoDB;

    # Admin user with all privileges.
    CREATE USER 'vt_dba'@'localhost';
    GRANT ALL ON *.* TO 'vt_dba'@'localhost';
    GRANT GRANT OPTION ON *.* TO 'vt_dba'@'localhost';

    # User for app traffic, with global read-write access.
    CREATE USER 'vt_app'@'localhost';
    GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, FILE,
      REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES,
      LOCK TABLES, EXECUTE, REPLICATION CLIENT, CREATE VIEW,
      SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER
      ON *.* TO 'vt_app'@'localhost';

    # User for app debug traffic, with global read access.
    CREATE USER 'vt_appdebug'@'localhost';
    GRANT SELECT, SHOW DATABASES, PROCESS ON *.* TO 'vt_appdebug'@'localhost';

    # User for administrative operations that need to be executed as non-SUPER.
    # Same permissions as vt_app here.
    CREATE USER 'vt_allprivs'@'localhost';
    GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, FILE,
      REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES,
      LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW,
      SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER
      ON *.* TO 'vt_allprivs'@'localhost';

    # User for slave replication connections.
    # TODO: Should we set a password on this since it allows remote connections?
    CREATE USER 'vt_repl'@'%';
    GRANT REPLICATION SLAVE ON *.* TO 'vt_repl'@'%';

    # User for Vitess filtered replication (binlog player).
    # Same permissions as vt_app.
    CREATE USER 'vt_filtered'@'localhost';
    GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, FILE,
      REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES,
      LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW,
      SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER
      ON *.* TO 'vt_filtered'@'localhost';

    FLUSH PRIVILEGES;

    RESET SLAVE ALL;
    RESET MASTER;

    # custom sql is used to add custom scripts like creating users/passwords. We use it in our tests
    # {{custom_sql}}

    # We need to set super_read_only back to what it was before
    SET GLOBAL super_read_only=IFNULL(@original_super_read_only, 'ON');
  rbac.yaml: |
    rules:
    - resource: "*"
      actions:
        - "get"
        - "create"
        - "put"
        - "ping"
      subjects: ["*"]
      clusters: ["*"]
    - resource: "Shard"
      actions:
        - "emergency_failover_shard"
        - "planned_failover_shard"
      subjects: ["*"]
      clusters: ["*"]

Agora vamos criar o arquivo que vai provisionar o cluster propriamente dito, chame-o de vitess-cluster.yaml e insira:

apiVersion: planetscale.com/v2
kind: VitessCluster
metadata:
  name: vitess
  namespace: vitess
spec:
  images:
    vtctld: vitess/lite:latest
    vtadmin: vitess/vtadmin:latest
    vtgate: vitess/lite:latest
    vttablet: vitess/lite:latest
    vtbackup: vitess/lite:latest
    vtorc: vitess/lite:latest
    mysqld:
      mysql80Compatible: vitess/lite:latest
    mysqldExporter: prom/mysqld-exporter:v0.11.0
  cells:
  - name: zone1
    gateway:
      authentication:
        static:
          secret:
            name: vitess-cluster-config
            key: users.json
      replicas: 1
      resources:
        requests:
          cpu: 100m
          memory: 256Mi
        limits:
          memory: 256Mi
  vitessDashboard:
    cells:
    - zone1
    extraFlags:
      security_policy: read-only
    replicas: 1
    resources:
      limits:
        memory: 128Mi
      requests:
        cpu: 100m
        memory: 128Mi
  vtadmin:
    rbac:
      name: vitess-cluster-config
      key: rbac.yaml
    cells:
      - zone1
    apiAddresses:
      - http://localhost:14001
    replicas: 1
    readOnly: false
    apiResources:
      limits:
        memory: 128Mi
      requests:
        cpu: 100m
        memory: 128Mi
    webResources:
      limits:
        memory: 128Mi
      requests:
        cpu: 100m
        memory: 128Mi

  keyspaces:
  - name: commerce
    durabilityPolicy: none
    turndownPolicy: Immediate
    vitessOrchestrator:
      resources:
        limits:
          memory: 128Mi
        requests:
          cpu: 100m
          memory: 128Mi
      extraFlags:
        recovery-period-block-duration: 5s
    partitionings:
    - equal:
        parts: 1
        shardTemplate:
          databaseInitScriptSecret:
            name: vitess-cluster-config
            key: init_db.sql
          tabletPools:
          - cell: zone1
            type: replica
            replicas: 3
            vttablet:
              extraFlags:
                db_charset: utf8mb4
              resources:
                limits:
                  memory: 256Mi
                requests:
                  cpu: 100m
                  memory: 256Mi
            mysqld:
              resources:
                limits:
                  memory: 512Mi
                requests:
                  cpu: 100m
                  memory: 512Mi
            dataVolumeClaimTemplate:
              accessModes: ["ReadWriteOnce"]
              resources:
                requests:
                  storage: 10Gi
  updateStrategy:
    type: Immediate

Agora basta executar ambos:

kubectl create -f vitess-cluster-config.yaml -n vitess
kubectl create -f vitess-cluster.yaml -n vitess

Vamos ver agora como ficou toda nossa estrutura e componentes:

kubectl get all -n vitess
kubectl get pvc -n vitess
kubectl get pv -n vitess

 Após provisionamento dos recursos vamos agora instalar algumas coisas necessárias para operar neste banco de dados, começando pelo mysql-client:

brew install mysql-client

Agora vamos instalar o Vitess localmente para que possamos ter o client vtctlclient disponível para importamos os dados em nossos bancos de dados:

brew install vitess

Para podermos acessar no cluster vamos precisar fazer alguns Port Forwards, altere o arquivo pf.sh deixando assim:

#!/bin/sh

kubectl -n vitess port-forward --address localhost "$(kubectl -n vitess get service --selector="planetscale.com/component=vtctld" -o name | head -n1)" 15000 15999 & process_id1=$!
kubectl -n vitess port-forward --address localhost "$(kubectl -n vitess get service --selector="planetscale.com/component=vtgate,!planetscale.com/cell" -o name | head -n1)" 15306:3306 & process_id2=$!
kubectl -n vitess port-forward --address localhost "$(kubectl -n vitess get service --selector="planetscale.com/component=vtadmin" -o name | head -n1)" 14000:15000 14001:15001 & process_id3=$!
sleep 2
echo "You may point your browser to http://localhost:15000, use the following aliases as shortcuts:"
echo 'alias vtctlclient="vtctlclient --server=localhost:15999 --logtostderr"'
echo 'alias vtctldclient="vtctldclient --server=localhost:15999 --logtostderr"'
echo 'alias mysql="mysql -h 127.0.0.1 -P 15306 -u user"'
echo "Hit Ctrl-C to stop the port forwards"
wait $process_id1
wait $process_id2
wait $process_id3

Feito isso execute:

./pf.sh &
alias vtctlclient="vtctlclient --server=localhost:15999"
alias mysql="mysql -h 127.0.0.1 -P 15306 -u user"

No comando acima também criamos dois alias para comandos que vamos executar para acessar o cluster Vitess, sendo assim, vamos executar alguns scripts no Vitess, o primeiro cria o Schema\Banco de Dados Commerce, o segundo cria o VSchema:

vtctlclient ApplySchema -- --sql="$(cat create_commerce_schema.sql)" commerce
vtctlclient ApplyVSchema -- --vschema="$(cat vschema_commerce_initial.json)" commerce

Agora vamos logar em nosso cluster Vitess com o comando mysql, depois vamos executar:

mysql
show databases;
use commerce;
show tables;

É isso gente! Devo fazer outro post falando sobre Sharding exclusivamente para o Vitess, onde iremos estender este lab para mostrar o quão escalável ele é!

Até a próxima!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: