id: getting-started
title: Getting started
Hyperledger Fabric Operator
Features
- Create certificates authorities (CA)
- Create peers
- Create ordering services
- Create resources without manual provisioning of cryptographic material
- Domain routing with SNI using Istio
- Run chaincode as external chaincode in Kubernetes
- Support Hyperledger Fabric 2.3+
- Managed genesis for Ordering services
- E2E testing including the execution of chaincodes in KIND
- Renewal of certificates
Stay Up-to-Date
hlf-operator
is currently in stable. Watch releases of this repository to be notified for future updates:
Discord
For discussions and questions, please join the Hyperledger Foundation Discord:
https://discord.com/invite/hyperledger
The channel is located under BEVEL
, named bevel-operator-fabric
.
Tutorial Videos
Step-by-step video tutorials to setup hlf-operator in Kubernetes
This workshop provides an in-depth hands on discussion and demonstration of using Bevel and the new Bevel-Operator-Fabric to deploy Hyperledger Fabric on Kubernetes.
Hyperledger Meetup
You can watch this video to see how to use it to deploy your own network:
Hyperledger Workshops
This workshop provides an in-depth, hands-on discussion and demonstration of using Bevel and the new Bevel-Operator-Fabric to deploy Hyperledger Fabric on Kubernetes.
Getting started
Tutorial
Resources:
Create Kubernetes Cluster
To start deploying our red fabric we have to have a Kubernetes cluster. For this we will use KinD.
Ensure you have these ports available before creating the cluster:
If these ports are not available this tutorial will not work.
cat << EOF > kind-config.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
image: kindest/node:v1.25.8
extraPortMappings:
- containerPort: 30949
hostPort: 80
- containerPort: 30950
hostPort: 443
EOF
kind create cluster --config=./kind-config.yaml
Install Kubernetes operator
In this step we are going to install the kubernetes operator for Fabric, this will install:
- CRD (Custom Resource Definitions) to deploy Certification Fabric Peers, Orderers and Authorities
- Deploy the program to deploy the nodes in Kubernetes
To install helm: https://helm.sh/docs/intro/install/
helm repo add kfs https://kfsoftware.github.io/hlf-helm-charts --force-update
helm install hlf-operator --version=1.9.2 -- kfs/hlf-operator
Install the Kubectl plugin
To install the kubectl plugin, you must first install Krew:
https://krew.sigs.k8s.io/docs/user-guide/setup/install/
Afterwards, the plugin can be installed with the following command:
kubectl krew install hlf
Install Istio
Install Istio binaries on the machine:
curl -L https://istio.io/downloadIstio | sh -
Install Istio on the Kubernetes cluster:
kubectl create namespace istio-system
istioctl operator init
kubectl apply -f - <<EOF
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
name: istio-gateway
namespace: istio-system
spec:
addonComponents:
grafana:
enabled: false
kiali:
enabled: false
prometheus:
enabled: false
tracing:
enabled: false
components:
ingressGateways:
- enabled: true
k8s:
hpaSpec:
minReplicas: 1
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 100m
memory: 128Mi
service:
ports:
- name: http
port: 80
targetPort: 8080
nodePort: 30949
- name: https
port: 443
targetPort: 8443
nodePort: 30950
type: NodePort
name: istio-ingressgateway
pilot:
enabled: true
k8s:
hpaSpec:
minReplicas: 1
resources:
limits:
cpu: 300m
memory: 512Mi
requests:
cpu: 100m
memory: 128Mi
meshConfig:
accessLogFile: /dev/stdout
enableTracing: false
outboundTrafficPolicy:
mode: ALLOW_ANY
profile: default
EOF
Deploy a Peer
organization
Environment Variables for AMD (Default)
export PEER_IMAGE=hyperledger/fabric-peer
export PEER_VERSION=2.5.5
export ORDERER_IMAGE=hyperledger/fabric-orderer
export ORDERER_VERSION=2.5.5
export CA_IMAGE=hyperledger/fabric-ca
export CA_VERSION=1.5.7
Environment Variables for ARM (Mac M1)
export PEER_IMAGE=hyperledger/fabric-peer
export PEER_VERSION=2.5.5
export ORDERER_IMAGE=hyperledger/fabric-orderer
export PEER_VERSION=2.5.5
export CA_IMAGE=hyperledger/fabric-ca
export CA_VERSION=1.5.7
kubectl apply -f - <<EOF
kind: ConfigMap
apiVersion: v1
metadata:
name: coredns
namespace: kube-system
data:
Corefile: |
.:53 {
errors
health {
lameduck 5s
}
rewrite name regex (.*)\.localho\.st istio-ingressgateway.istio-system.svc.cluster.local
hosts {
fallthrough
}
ready
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
ttl 30
}
prometheus :9153
forward . /etc/resolv.conf {
max_concurrent 1000
}
cache 30
loop
reload
loadbalance
}
EOF
Deploy a certificate authority
kubectl hlf ca create --image=$CA_IMAGE --version=$CA_VERSION --storage-class=standard --capacity=1Gi --name=org1-ca \
--enroll-id=enroll --enroll-pw=enrollpw --hosts=org1-ca.localho.st --istio-port=443
kubectl wait --timeout=180s --for=condition=Running fabriccas.hlf.kungfusoftware.es --all
Check that the certification authority is deployed and works:
curl -k https://org1-ca.localho.st:443/cainfo
Register a user in the certification authority of the peer organization (Org1MSP)
# register user in CA for peers
kubectl hlf ca register --name=org1-ca --user=peer --secret=peerpw --type=peer \
--enroll-id enroll --enroll-secret=enrollpw --mspid Org1MSP
Deploy a peer
kubectl hlf peer create --statedb=leveldb --image=$PEER_IMAGE --version=$PEER_VERSION --storage-class=standard --enroll-id=peer --mspid=Org1MSP \
--enroll-pw=peerpw --capacity=5Gi --name=org1-peer0 --ca-name=org1-ca.default \
--hosts=peer0-org1.localho.st --istio-port=443
kubectl hlf peer create --statedb=leveldb --image=$PEER_IMAGE --version=$PEER_VERSION --storage-class=standard --enroll-id=peer --mspid=Org1MSP \
--enroll-pw=peerpw --capacity=5Gi --name=org1-peer1 --ca-name=org1-ca.default \
--hosts=peer1-org1.localho.st --istio-port=443
kubectl wait --timeout=180s --for=condition=Running fabricpeers.hlf.kungfusoftware.es --all
Check that the peer is deployed and works:
openssl s_client -connect peer0-org1.localho.st:443
openssl s_client -connect peer1-org1.localho.st:443
Deploy an Orderer
organization
To deploy an Orderer
organization we have to:
- Create a certification authority
- Register user
orderer
with password ordererpw
- Create orderer
Create the certification authority
kubectl hlf ca create --image=$CA_IMAGE --version=$CA_VERSION --storage-class=standard --capacity=1Gi --name=ord-ca \
--enroll-id=enroll --enroll-pw=enrollpw --hosts=ord-ca.localho.st --istio-port=443
kubectl wait --timeout=180s --for=condition=Running fabriccas.hlf.kungfusoftware.es --all
Check that the certification authority is deployed and works:
curl -vik https://ord-ca.localho.st:443/cainfo
Register user orderer
kubectl hlf ca register --name=ord-ca --user=orderer --secret=ordererpw \
--type=orderer --enroll-id enroll --enroll-secret=enrollpw --mspid=OrdererMSP --ca-url="https://ord-ca.localho.st:443"
Deploy orderer
kubectl hlf ordnode create --image=$ORDERER_IMAGE --version=$ORDERER_VERSION \
--storage-class=standard --enroll-id=orderer --mspid=OrdererMSP \
--enroll-pw=ordererpw --capacity=2Gi --name=ord-node1 --ca-name=ord-ca.default \
--hosts=orderer0-ord.localho.st --admin-hosts=admin-orderer0-ord.localho.st --istio-port=443
kubectl hlf ordnode create --image=$ORDERER_IMAGE --version=$ORDERER_VERSION \
--storage-class=standard --enroll-id=orderer --mspid=OrdererMSP \
--enroll-pw=ordererpw --capacity=2Gi --name=ord-node2 --ca-name=ord-ca.default \
--hosts=orderer1-ord.localho.st --admin-hosts=admin-orderer1-ord.localho.st --istio-port=443
kubectl hlf ordnode create --image=$ORDERER_IMAGE --version=$ORDERER_VERSION \
--storage-class=standard --enroll-id=orderer --mspid=OrdererMSP \
--enroll-pw=ordererpw --capacity=2Gi --name=ord-node3 --ca-name=ord-ca.default \
--hosts=orderer2-ord.localho.st --admin-hosts=admin-orderer2-ord.localho.st --istio-port=443
kubectl wait --timeout=180s --for=condition=Running fabricorderernodes.hlf.kungfusoftware.es --all
Check that the orderer is running:
kubectl get pods
openssl s_client -connect orderer0-ord.localho.st:443
Create channel
To create the channel we need to first create the wallet secret, which will contain the identities used by the operator to manage the channel
Register and enrolling OrdererMSP identity
# register
kubectl hlf ca register --name=ord-ca --user=admin --secret=adminpw \
--type=admin --enroll-id enroll --enroll-secret=enrollpw --mspid=OrdererMSP
# enroll
kubectl hlf ca enroll --name=ord-ca --namespace=default \
--user=admin --secret=adminpw --mspid OrdererMSP \
--ca-name tlsca --output orderermsp.yaml
Register and enrolling Org1MSP Orderer identity
# register
kubectl hlf ca register --name=org1-ca --user=admin --secret=adminpw \
--type=admin --enroll-id enroll --enroll-secret=enrollpw --mspid=Org1MSP
# enroll
kubectl hlf ca enroll --name=org1-ca --namespace=default \
--user=admin --secret=adminpw --mspid Org1MSP \
--ca-name tlsca --output org1msp-tlsca.yaml
Register and enrolling Org1MSP identity
# register
kubectl hlf ca register --name=org1-ca --namespace=default --user=admin --secret=adminpw \
--type=admin --enroll-id enroll --enroll-secret=enrollpw --mspid=Org1MSP
# enroll
kubectl hlf ca enroll --name=org1-ca --namespace=default \
--user=admin --secret=adminpw --mspid Org1MSP \
--ca-name ca --output org1msp.yaml
# enroll
kubectl hlf identity create --name org1-admin --namespace default \
--ca-name org1-ca --ca-namespace default \
--ca ca --mspid Org1MSP --enroll-id admin --enroll-secret adminpw
Create the secret
kubectl create secret generic wallet --namespace=default \
--from-file=org1msp.yaml=$PWD/org1msp.yaml \
--from-file=orderermsp.yaml=$PWD/orderermsp.yaml
Create main channel
export PEER_ORG_SIGN_CERT=$(kubectl get fabriccas org1-ca -o=jsonpath='{.status.ca_cert}')
export PEER_ORG_TLS_CERT=$(kubectl get fabriccas org1-ca -o=jsonpath='{.status.tlsca_cert}')
export IDENT_8=$(printf "%8s" "")
export ORDERER_TLS_CERT=$(kubectl get fabriccas ord-ca -o=jsonpath='{.status.tlsca_cert}' | sed -e "s/^/${IDENT_8}/" )
export ORDERER0_TLS_CERT=$(kubectl get fabricorderernodes ord-node1 -o=jsonpath='{.status.tlsCert}' | sed -e "s/^/${IDENT_8}/" )
kubectl apply -f - <<EOF
apiVersion: hlf.kungfusoftware.es/v1alpha1
kind: FabricMainChannel
metadata:
name: demo2
spec:
name: demo2
adminOrdererOrganizations:
- mspID: OrdererMSP
adminPeerOrganizations:
- mspID: Org1MSP
channelConfig:
application:
acls: null
capabilities:
- V2_0
policies: null
capabilities:
- V2_0
orderer:
batchSize:
absoluteMaxBytes: 1048576
maxMessageCount: 10
preferredMaxBytes: 524288
batchTimeout: 2s
capabilities:
- V2_0
etcdRaft:
options:
electionTick: 10
heartbeatTick: 1
maxInflightBlocks: 5
snapshotIntervalSize: 16777216
tickInterval: 500ms
ordererType: etcdraft
policies: null
state: STATE_NORMAL
policies: null
externalOrdererOrganizations: []
peerOrganizations:
- mspID: Org1MSP
caName: "org1-ca"
caNamespace: "default"
identities:
OrdererMSP:
secretKey: orderermsp.yaml
secretName: wallet
secretNamespace: default
Org1MSP:
secretKey: org1msp.yaml
secretName: wallet
secretNamespace: default
externalPeerOrganizations:
- mspID: Org2MSP
tlsRootCert: |
-----BEGIN CERTIFICATE-----
MIICRjCCAeugAwIBAgIQHA5nWgCvnS9ECuauCtat6TAKBggqhkjOPQQDAjBtMQsw
CQYDVQQGEwJFUzERMA8GA1UEBxMIQWxpY2FudGUxETAPBgNVBAkTCEFsaWNhbnRl
MRkwFwYDVQQKExBLdW5nIEZ1IFNvZnR3YXJlMQ0wCwYDVQQLEwRUZWNoMQ4wDAYD
VQQDEwV0bHNjYTAeFw0yMzExMTIxNTA3MTlaFw0zMzExMTMxNTA3MTlaMG0xCzAJ
BgNVBAYTAkVTMREwDwYDVQQHEwhBbGljYW50ZTERMA8GA1UECRMIQWxpY2FudGUx
GTAXBgNVBAoTEEt1bmcgRnUgU29mdHdhcmUxDTALBgNVBAsTBFRlY2gxDjAMBgNV
BAMTBXRsc2NhMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEVB/ZqhVePy96JJH/
QhYCT6hRlH7xQVSrTwI1QjLG6GgENV2c10m2EkXH/BHpLJCjZ8ZkGiqVVx/XMxZd
E6p1B6NtMGswDgYDVR0PAQH/BAQDAgGmMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggr
BgEFBQcDATAPBgNVHRMBAf8EBTADAQH/MCkGA1UdDgQiBCBcmb8svFB8Yn96e3d4
Ft+8wZTrAYLielLn6Je/zxbjODAKBggqhkjOPQQDAgNJADBGAiEA3ad5GVhA5RSr
mjhnKlazFh53einWYfWxcYNy42v+EbcCIQCt+Fc4nzlMg0ebG3HDlpa9wjJpX9MW
caEunJxSdY4vWg==
-----END CERTIFICATE-----
signRootCert: |
-----BEGIN CERTIFICATE-----
MIICQTCCAeagAwIBAgIRALIMDeWXSDaW2cYssTwLrTAwCgYIKoZIzj0EAwIwajEL
MAkGA1UEBhMCRVMxETAPBgNVBAcTCEFsaWNhbnRlMREwDwYDVQQJEwhBbGljYW50
ZTEZMBcGA1UEChMQS3VuZyBGdSBTb2Z0d2FyZTENMAsGA1UECxMEVGVjaDELMAkG
A1UEAxMCY2EwHhcNMjMxMTEyMTUwNzE5WhcNMzMxMTEzMTUwNzE5WjBqMQswCQYD
VQQGEwJFUzERMA8GA1UEBxMIQWxpY2FudGUxETAPBgNVBAkTCEFsaWNhbnRlMRkw
FwYDVQQKExBLdW5nIEZ1IFNvZnR3YXJlMQ0wCwYDVQQLEwRUZWNoMQswCQYDVQQD
EwJjYTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABAomf+HmUFMdRS7D+IZRCEkQ
QEDyrakkXYEE/vwSlXhSW5PsVVXQdi/wmXj2YwoPqsDk2IrXc7KT2tni5wOxJ8mj
bTBrMA4GA1UdDwEB/wQEAwIBpjAdBgNVHSUEFjAUBggrBgEFBQcDAgYIKwYBBQUH
AwEwDwYDVR0TAQH/BAUwAwEB/zApBgNVHQ4EIgQgESLOdwga9AciHgTGFTR6Zxzh
Z/vpQ2gC3V0MbzFvfOYwCgYIKoZIzj0EAwIDSQAwRgIhAKLN7BlISJJzjjHOqJ25
EFfWSywv6MgddOsGgrQxsedCAiEAxhMj/EJcYvVu5rr+xiHo7TarixkdvmA5k/eW
kOWiji0=
-----END CERTIFICATE-----
ordererOrganizations:
- caName: "ord-ca"
caNamespace: "default"
externalOrderersToJoin:
- host: ord-node1
port: 7053
mspID: OrdererMSP
ordererEndpoints:
- orderer0-ord.localho.st:443
orderersToJoin: []
orderers:
- host: orderer0-ord.localho.st
port: 443
tlsCert: |-
${ORDERER0_TLS_CERT}
EOF
Join peer to the channel
export IDENT_8=$(printf "%8s" "")
export ORDERER0_TLS_CERT=$(kubectl get fabricorderernodes ord-node1 -o=jsonpath='{.status.tlsCert}' | sed -e "s/^/${IDENT_8}/" )
kubectl apply -f - <<EOF
apiVersion: hlf.kungfusoftware.es/v1alpha1
kind: FabricFollowerChannel
metadata:
name: demo2-org1msp
spec:
anchorPeers:
- host: peer0-org1.localho.st
port: 443
hlfIdentity:
secretKey: org1msp.yaml
secretName: wallet
secretNamespace: default
mspId: Org1MSP
name: demo2
externalPeersToJoin: []
orderers:
- certificate: |
${ORDERER0_TLS_CERT}
url: grpcs://ord-node1.default:7050
peersToJoin:
- name: org1-peer0
namespace: default
EOF
Install a chaincode
Prepare connection string for a peer
To prepare the connection string, we have to:
-
Get connection string without users for organization Org1MSP and OrdererMSP
-
Register a user in the certification authority for signing (register)
-
Obtain the certificates using the previously created user (enroll)
-
Attach the user to the connection string
-
Get connection string without users for organization Org1MSP and OrdererMSP
kubectl hlf inspect --output org1.yaml -o Org1MSP -o OrdererMSP
- Register a user in the certification authority for signing
kubectl hlf ca register --name=org1-ca --user=admin --secret=adminpw --type=admin \
--enroll-id enroll --enroll-secret=enrollpw --mspid Org1MSP
- Get the certificates using the user created above
kubectl hlf ca enroll --name=org1-ca --user=admin --secret=adminpw --mspid Org1MSP \
--ca-name ca --output peer-org1.yaml
- Attach the user to the connection string
kubectl hlf utils adduser --userPath=peer-org1.yaml --config=org1.yaml --username=admin --mspid=Org1MSP
# remove the code.tar.gz chaincode.tgz if they exist
rm code.tar.gz chaincode.tgz
export CHAINCODE_NAME=asset
export CHAINCODE_LABEL=asset
cat << METADATA-EOF > "metadata.json"
{
"type": "ccaas",
"label": "${CHAINCODE_LABEL}"
}
METADATA-EOF
## chaincode as a service
Prepare connection file
cat > "connection.json" <<CONN_EOF
{
"address": "${CHAINCODE_NAME}:7052",
"dial_timeout": "10s",
"tls_required": false
}
CONN_EOF
tar cfz code.tar.gz connection.json
tar cfz chaincode.tgz metadata.json code.tar.gz
export PACKAGE_ID=$(kubectl hlf chaincode calculatepackageid --path=chaincode.tgz --language=node --label=$CHAINCODE_LABEL)
echo "PACKAGE_ID=$PACKAGE_ID"
kubectl hlf chaincode install --path=./chaincode.tgz \
--config=org1.yaml --language=golang --label=$CHAINCODE_LABEL --user=admin --peer=org1-peer0.default
kubectl hlf chaincode install --path=./chaincode.tgz \
--config=org1.yaml --language=golang --label=$CHAINCODE_LABEL --user=admin --peer=org1-peer1.default
Deploy chaincode container on cluster
The following command will create or update the CRD based on the packageID, chaincode name, and docker image.
kubectl hlf externalchaincode sync --image=kfsoftware/chaincode-external:latest \
--name=$CHAINCODE_NAME \
--namespace=default \
--package-id=$PACKAGE_ID \
--tls-required=false \
--replicas=1
Check installed chaincodes
kubectl hlf chaincode queryinstalled --config=org1.yaml --user=admin --peer=org1-peer0.default
Approve chaincode
export SEQUENCE=1
export VERSION="1.0"
kubectl hlf chaincode approveformyorg --config=org1.yaml --user=admin --peer=org1-peer0.default \
--package-id=$PACKAGE_ID \
--version "$VERSION" --sequence "$SEQUENCE" --name=asset \
--policy="OR('Org1MSP.member')" --channel=testbft02
Commit chaincode
kubectl hlf chaincode commit --config=org1.yaml --user=admin --mspid=Org1MSP \
--version "$VERSION" --sequence "$SEQUENCE" --name=asset \
--policy="OR('Org1MSP.member')" --channel=testbft02
Invoke a transaction on the channel
kubectl hlf chaincode invoke --config=org1.yaml \
--user=admin --peer=org1-peer0.default \
--chaincode=asset --channel=testbft02 \
--fcn=initLedger -a '[]'
Query assets in the channel
kubectl hlf chaincode query --config=org1.yaml \
--user=admin --peer=org1-peer0.default \
--chaincode=asset --channel=testbft02 \
--fcn=GetAllAssets -a '[]'
At this point, you should have:
- Ordering service with 1 nodes and a CA
- Peer organization with a peer and a CA
- A channel demo2
- A chaincode install in peer0
- A chaincode approved and committed
If something went wrong or didn't work, please, open an issue.
Prepare connection string for a peer
To prepare the connection string, we have to create a CRD of type FabricNetworkConfig
with the following command:
kubectl apply -f - <<EOF
apiVersion: hlf.kungfusoftware.es/v1alpha1
kind: FabricNetworkConfig
metadata:
name: nc
namespace: default
spec:
channels:
- testbft02
identities:
- name: org1-admin
namespace: default
internal: false
namespaces: []
organization: ''
organizations:
- Org1MSP
- OrdererMSP
secretName: nc-networkconfig
EOF
Launch the explorer
export API_HOST=operator-api.localho.st
export HLF_SECRET_NAME="nc-networkconfig"
export HLF_MSPID="Org1MSP"
export HLF_SECRET_KEY="config.yaml" # e.g. networkConfig.yaml
export HLF_USER="org1-admin-default"
kubectl hlf operatorapi create --name=operator-api --namespace=default --version="v0.0.17-beta9" --hosts=$API_HOST --ingress-class-name=istio \
--hlf-mspid="${HLF_MSPID}" --hlf-secret="${HLF_SECRET_NAME}" --hlf-secret-key="${HLF_SECRET_KEY}" \
--hlf-user="${HLF_USER}"
Cleanup the environment
kubectl delete fabricorderernodes.hlf.kungfusoftware.es --all-namespaces --all
kubectl delete fabricpeers.hlf.kungfusoftware.es --all-namespaces --all
kubectl delete fabriccas.hlf.kungfusoftware.es --all-namespaces --all
kubectl delete fabricchaincode.hlf.kungfusoftware.es --all-namespaces --all
kubectl delete fabricmainchannels --all-namespaces --all
kubectl delete fabricfollowerchannels --all-namespaces --all
Troubleshooting
Chaincode installation/build error
Chaincode installation/build can fail due to unsupported local kubertenes version such as minikube.
$ kubectl hlf chaincode install --path=./fixtures/chaincodes/fabcar/go \
--config=org1.yaml --language=golang --label=fabcar --user=admin --peer=org1-peer0.default
Error: Transaction processing for endorser [192.168.49.2:31278]: Chaincode status Code: (500) UNKNOWN.
Description: failed to invoke backing implementation of 'InstallChaincode': could not build chaincode:
external builder failed: external builder failed to build: external builder 'my-golang-builder' failed:
exit status 1
If your purpose is to test the hlf-operator please consider to switch to kind that is tested and supported.