Usage

Requirements

At least three nodes in the Kubernetes cluster with 16 CPU/32 GB RAM configuration.
850 GB of Persistent Storage per execution node.
300 GB of Persistent Storage per consensus node.
Helm 3.5+ - This is the earliest version of Helm tested. Charts may work with earlier versions but it is untested.
Kubernetes 1.22+ - This is the earliest version of Kubernetes tested. Charts may work with earlier versions but it is untested.
PV provisioner support in the underlying infrastructure

Installation

Monitoring System

If you already have Prometheus installed in your cluster, you can skip this step.

Every chart we support contains the ability to enable monitoring and alerting out of the box. A combination of Prometheus + Grafana + Alertmanager is used for monitoring.

Add the prometheus-community helm repository and check that you have access to the chart:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

Install Prometheus + Grafana + Alertmanager:

helm upgrade --install kube-prometheus-stack prometheus-community/kube-prometheus-stack \
    --set='grafana.sidecar.dashboards.enabled=true' \
    --set='grafana.sidecar.dashboards.searchNamespace=true' \
    --set='prometheus.prometheusSpec.ruleSelectorNilUsesHelmValues=false' \
    --set='prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false' \
    --set='prometheus.prometheusSpec.podMonitorSelectorNilUsesHelmValues=false' \
    --set='prometheus.prometheusSpec.probeSelectorNilUsesHelmValues=false' \
    --create-namespace \
    --namespace monitoring \
    --version 43.2.0 \
    -f prom.yaml

prom.yaml:

prometheus:
  prometheusSpec:
    storageSpec:
      volumeClaimTemplate:
        spec:
          storageClassName: "{REPLCAE_ME_WITH_STORAGE_CLASS_NAME}"
          accessModes: ["ReadWriteOnce"]
          resources:
            requests:
              storage: 100Gi
grafana:
  persistence:
    enabled: true
    type: pvc
    storageClassName: "{REPLCAE_ME_WITH_STORAGE_CLASS_NAME}"
    accessModes: ["ReadWriteOnce"]
    size: 10Gi
    finalizers:
    - kubernetes.io/pvc-protection

For GKE/EKS installations:

helm upgrade --install kube-prometheus-stack prometheus-community/kube-prometheus-stack \
    --set='kubeControllerManager.enabled=false' \
    --set='kubeEtcd.enabled=false' \
    --set='kubeScheduler.enabled=false' \
    --set='kubeProxy.enabled=false' \
    --set='defaultRules.rules.etcd=false' \
    --set='defaultRules.rules.kubernetesSystem=false' \
    --set='defaultRules.rules.kubeScheduler=false' \
    --set='grafana.sidecar.dashboards.enabled=true' \
    --set='grafana.sidecar.dashboards.searchNamespace=true' \
    --set='prometheus.prometheusSpec.ruleSelectorNilUsesHelmValues=false' \
    --set='prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false' \
    --set='prometheus.prometheusSpec.podMonitorSelectorNilUsesHelmValues=false' \
    --set='prometheus.prometheusSpec.probeSelectorNilUsesHelmValues=false' \
    --create-namespace \
    --namespace monitoring \
    --version 43.2.0 \
    -f prom.yaml

prom.yaml:

prometheus:
  prometheusSpec:
    storageSpec:
      volumeClaimTemplate:
        spec:
          storageClassName: "gp2"
          accessModes: ["ReadWriteOnce"]
          resources:
            requests:
              storage: 100Gi
grafana:
  persistence:
    enabled: true
    type: pvc
    storageClassName: "gp2"
    accessModes: ["ReadWriteOnce"]
    size: 10Gi
    finalizers:
    - kubernetes.io/pvc-protection

Optional (Grafana Dashboards):

Import dashboards into Grafana manually or automatically with Helm:

helm upgrade --install grafana-stakewise-dashboards ethstaking/grafana-ethstaking-dashboards \
  --namespace monitoring

ETH1 (Execution) Nodes

ETH1 nodes are used by the validators to propose new ETH2 blocks. As such, running validator and beacon nodes also entail having a reliable connection to the ETH1 chain.

ETH1 nodes must be deployed first. Currently, GoEthereum, Erigon, Besu, and NetherMind are supported.

Add ethStaking Helm repository:

helm repo add stakewise https://charts.eth-staking.com
helm repo update

Clients supported Gnosis Chain: Nethermind

Before deployment, you need to generate a JSON Web Token (JWT). Authentication is used to secure the communication between the beacon node and execution client. You can generate a JWT using a command line tool, for example:

export JWT=`openssl rand -hex 32`

Depending on what client you would like to use, run one of the following commands:

# GoEthereum
helm upgrade --install geth ethstaking/geth \
  --set='global.replicaCount=3' \
  --set='global.network=mainnet' \
  --set='global.metrics.enabled=true' \
  --set='global.metrics.serviceMonitor.enabled=true' \
  --set='global.metrics.prometheusRule.enabled=true' \
  --set='JWTSecret=${JWT}' \
  --create-namespace \
  --namespace chain

# Erigon
helm upgrade --install erigon ethstaking/erigon \
  --set='global.replicaCount=3' \
  --set='global.network=mainnet' \
  --set='global.metrics.enabled=true' \
  --set='global.metrics.serviceMonitor.enabled=true' \
  --set='global.metrics.prometheusRule.enabled=true' \
  --set='JWTSecret=${JWT}' \
  --create-namespace \
  --namespace chain

# Besu
helm upgrade --install besu ethstaking/besu \
    --set='replicaCount=3' \
    --set='network=mainnet' \
    --set='metrics.serviceMonitor.enabled=true' \
    --set='metrics.prometheusRule.enabled=true' \
    --create-namespace \
    --namespace chain

# Nethermind
helm upgrade --install nethermind ethstaking/nethermind \
  --set='global.replicaCount=3' \
  --set='global.network=mainnet' \
  --set='global.metrics.enabled=true' \
  --set='global.metrics.serviceMonitor.enabled=true' \
  --set='global.metrics.prometheusRule.enabled=true' \
  --set='JWTSecret=${JWT}' \
    --create-namespace \
    --namespace chain

ETH2 (Consensus) beacon nodes

An ETH2 beacon node is responsible for running a full Proof-Of-Stake blockchain, known as a beacon chain, which uses distributed consensus to agree on blocks in the network. Validators connect to the beacon nodes to receive block attestation/proposal assignments.

Historically, an execution client alone has been enough to run a full Ethereum node. However, when The Merge happens, EL clients will not be able to track the Ethereum chain on its own. Instead, it will need to be coupled to another piece of software called a “consensus client”. In this configuration, the execution client will be responsible for transaction handling, transaction gossip, state management, and the Ethereum Virtual Machine (EVM). However, EL clients will no longer be responsible for block building, block gossiping, or handling consensus logic, and these will be in the remit of the consensus client.

Add ethStaking Helm repository:

helm repo add stakewise https://charts.eth-staking.com
helm repo update

When deploying ETH2 nodes, make sure that your ETH1 nodes are fully synced. It's possible to choose what ETH2 client to use. Currently, Prysm, Lighthouse, and Teku are supported. Choose one or two clients to install and deploy:

Note that Nimbus is only compatible with Lighthouse validator client

Consensus charts do not have a replicaCount parameter, instead you need to specify a list of execution endpoints, a separate statefulset will be created for each endpoint

# Prysm
helm upgrade --install prysm ethstaking/prysm \
  --set='global.network=mainnet' \
  --set='global.JWTSecret=${JWT}' \
  --set='global.executionEndpoints[0]=http://nethermind-0.chain:8545' \
  --set='global.executionEndpoints[1]=http://nethermind-1.chain:8545' \
  --set='global.executionEndpoints[2]=http://nethermind-2.chain:8545' \
  --set='global.metrics.enabled=true' \
  --set='global.metrics.serviceMonitor.enabled=true' \
  --set='global.metrics.prometheusRule.enabled=true' \
  --create-namespace \
  --namespace chain
  
# Lighthouse
helm upgrade --install lighthouse ethstaking/lighthouse \
  --set='global.network=mainnet' \
  --set='global.JWTSecret=${JWT}' \
  --set='global.executionEndpoints[0]=http://geth-0.chain:8551' \
  --set='global.executionEndpoints[1]=http://geth-1.chain:8551' \
  --set='global.executionEndpoints[2]=http://geth-2.chain:8551' \
  --set='global.metrics.enabled=true' \
  --set='global.metrics.serviceMonitor.enabled=true' \
  --set='global.metrics.prometheusRule.enabled=true' \
  --create-namespace \
  --namespace chain
  
# Teku
helm upgrade --install teku ethstaking/teku \
  --set='global.network=mainnet' \
  --set='global.JWTSecret=${JWT}' \
  --set='global.executionEndpoints[0]=http://erigon-0.chain:8551' \
  --set='global.executionEndpoints[1]=http://erigon-1.chain:8551' \
  --set='global.executionEndpoints[2]=http://erigon-2.chain:8551' \
  --set='global.metrics.enabled=true' \
  --set='global.metrics.serviceMonitor.enabled=true' \
  --set='global.metrics.prometheusRule.enabled=true' \
  --create-namespace \
  --namespace chain

The recommended setup is to deploy two replicas of the primary ETH2 client and one replica of the stand-by ETH2 client. The validators will be evenly connected to all the primary replicas and will automatically switch to another primary replica in case the connection to their current one fails.

If happens that there is an issue with the primary client, the validators can migrate to the stand-by client and won't need to wait for it to sync the chain.

Deploy PostgreSQL

Installing and configuring PostgreSQL is beyond the scope of this guide, and we hope that operators will be able to choose and implement the correct reliable solution on their own. PostgreSQL is used to store the validators' keys in encrypted form, as well as to store the slashing history of the web3signer database.

After the database is deployed, two databases and two users must be created:

web3signer - which stores web3signer's data
web3signer_keys - which stores validator keys generated via ethstaking-cli

Sync Validators Keys to the Web3Signer Database

Once you've successfully deployed the database and your proposal got approved by the DAO (the snapshot vote got executed), sync the validator keys using the ethStaking CLI

Run the following command to sync new validator keys to the DB:

./ethstaking-cli sync-db
Please choose the network name (mainnet, goerli, harbour_mainnet, harbour_goerli, gnosis) [mainnet]: goerli
Enter your operator wallet address: 0x777...
Enter the database connection string, ex. 'postgresql://username:pass@hostname/dbname': postgresql://example:example@localhost/web3signer_keys
Enter your mnemonic separated by spaces (" "): 

Syncing key pairs		  [------------------------------------]  1/1
Synced 1 key pairs, apply changes to the database? [Y/n]: Y
The database contains 1 validator keys.
Please upgrade the 'web3signer-validators' helm chart with 'validatorsCount' set to 1
Set 'DECRYPTION_KEY' env to '<decryption key>'

Deploy Web3Signer

Web3Signer is an open-source signing service developed under the Apache 2.0 license and written in Java.

Web3Signer is capable of signing on multiple platforms using private keys stored in an external vault, or encrypted on a disk.

Web3Signer Documentation.

Add ethStaking Helm repository:

helm repo add stakewise https://charts.eth-staking.com helm repo update

Deploy Web3Signer:

helm upgrade --install web3signer ethstaking/web3signer \
  --set='replicaCount=3' \
  --set='network=mainnet' \
  --set='dbUrl=jdbc:postgresql://cloudsqlproxy.default/web3signer' \
  --set='dbUsername=username' \
  --set='dbPassword=password' \
  --set='dbKeystoreUrl=postgresql://example:example@cloudsqlproxy.default/web3signer_keys' \
  --set='decryptionKey=<decryption key from the CLI' \
  --create-namespace \
  --namespace validators

Recommendations

The database storage and disk subsystem of the worker nodes must have sufficient margin of IOPS for normal operation. For example in AWS, it is recommended to use disks of 100G and larger, not for disk space but for better performance.

Validators

Validators are responsible for storing data, processing transactions, and adding new blocks to the blockchain. This will keep Ethereum secure for everyone and earn new ETH in the process.

Before deploying the validators make sure you have deployed Web3Signer and synchronized validator keys in the steps above.

Make sure you have the right number of validators running and restart them so that they will synchronize the latest changes from the Web3Signer.

Add the ethStaking helm repository:

helm repo add stakewise https://charts.eth-staking.com
helm repo update

Deploy the chart, after specifying all neded values:

helm upgrade --install validators ethstaking/validators \
  --set='network=mainnet' \
  --set='type=lighthouse' \
  --set='validatorsCount=8' \
  --set='beaconChainRpcEndpoints[0]=http://lighthouse-0.chain:5052' \
  --set='beaconChainRpcEndpoints[1]=http://lighthouse-1.chain:5052' \
  --set='beaconChainRpcEndpoints[2]=http://lighthouse-2.chain:5052' \
  --set='web3signerEndpoint=http://web3signer:6174' \
  --set='dbKeystoreUrl=postgresql://example:example@cloudsqlproxy.default/web3signer_keys' \
  --set='graffiti=ethStaking' \
  --set='metrics.enabled=true' \
  --set='metrics.serviceMonitor.enabled=true' \
  --set='metrics.prometheusRule.enabled=true' \
  --create-namespace \
  --namespace validators

Commit Operator

Once you're 100% ready for attestation/proposals assignments to the validators, commit your operator:

Go to the PoolValidators smart contract (Goerli, Perm Goerli, Gnosis Chain, Mainnet)
Click on Connect to Web3 button and connect your wallet. The address must match the one used during DAO proposal generation.
Call commitOperator function.

Congratulations on becoming ethStaking Node Operator🎉. Your validators will get assignments, and you would be able to claim your operator rewards from Farms Page.

PreviousethStaking Infra Package NextMaintenance

Last updated 1 year ago