hdinsight

Documentation

Index

• func CreateHadoopCluster(resourceGroup, clusterName string, info StorageAccountInfo) (*hdinsight.Cluster, error)
• type StorageAccountInfo

Examples

• CreateHadoopCluster

Functions

func CreateHadoopCluster

func CreateHadoopCluster(resourceGroup, clusterName string, info StorageAccountInfo) (*hdinsight.Cluster, error)

CreateHadoopCluster creats a simple hadoop 3.6 cluster

Example

rgName := config.GenerateGroupName("HadoopClusterExample")
config.SetGroupName(rgName)

ctx, cancel := context.WithTimeout(context.Background(), 6000*time.Second)
defer cancel()
defer resources.Cleanup(ctx)

_, err := resources.CreateGroup(ctx, rgName)
if err != nil {
	util.LogAndPanic(err)
}
util.PrintAndLog("created resource group")

storageAccountName := strings.ToLower(config.AppendRandomSuffix("exampleforhadoop"))
sa, err := storage.CreateStorageAccount(context.Background(), storageAccountName, rgName)
if err != nil {
	util.LogAndPanic(err)
}
util.PrintAndLog("created storage account")

containerName := strings.ToLower(config.AppendRandomSuffix("hadoopfilesystem"))
_, err = storage.CreateContainer(context.Background(), storageAccountName, rgName, containerName)
if err != nil {
	util.LogAndPanic(err)
}
util.PrintAndLog("created container")

keys, err := storage.GetAccountKeys(context.Background(), storageAccountName, rgName)
if err != nil {
	util.LogAndPanic(err)
}
util.PrintAndLog("retrieved storage account keys")

clusterName := strings.ToLower(config.AppendRandomSuffix("exhadoop36cluster"))
_, err = CreateHadoopCluster(rgName, clusterName, StorageAccountInfo{
	Name:      fmt.Sprintf("%s.blob.core.windows.net", *sa.Name), // TODO: can we get the full URL from the service?
	Container: containerName,
	Key:       *(*keys.Keys)[0].Value,
})
if err != nil {
	util.LogAndPanic(err)
}
util.PrintAndLog("created cluster")

Output:

created resource group
created storage account
created container
retrieved storage account keys
creating hadoop cluster
waiting for hadoop cluster to finish deploying, this will take a while...
created cluster

Types

type StorageAccountInfo

type StorageAccountInfo struct {
	Name      string
	Container string
	Key       string
}

StorageAccountInfo describes the storage account used for the cluster's file system.