mesos

package
v0.0.0-...-c9014b7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 15, 2017 License: Apache-2.0, CC-BY-SA-4.0 Imports: 26 Imported by: 0

README

Using Docker Swarm and Mesos

Swarm comes with a built-in scheduler that works with the Swarm manager to schedule container resources. You can completely replace the built-in scheduler with a 3rd party scheduler. For example, you can replace it with the Mesos scheduler as described here.

When using Docker Swarm and Mesos, you use the Docker client to ask the Swarm manager to schedule containers. The Swarm manager then schedules those containers on a Mesos cluster.

The Docker Swarm on Mesos integration is experimental and not production ready, please use with caution.

Prerequisites

Each node in your Swarm must run a Mesos agent. The agent must be capable of starting tasks in a Docker Container using the --containerizers=docker option.

You need to configure two TCP ports on the agent. One port to listen for the Swarm manager, for example 2375. And a second TCP port to listen for the Mesos master, for example 3375.

Start the Docker Swarm manager

If you use a single Mesos master:

$ docker run -d -p <swarm_port>:2375 -p 3375:3375 \
    swarm manage \
        -c mesos-experimental \
        --cluster-opt mesos.address=<public_machine_ip> \
        --cluster-opt mesos.port=3375 \
        <mesos_master_ip>:<mesos_master_port>

The command above creates a Swarm manager listening at <swarm_port>. The <mesos_master_ip> value points to where Mesos master lives in the cluster. Typically, this is localhost, a hostname, or an IP address. The <public_machine_ip> value is the IP address for Mesos master to talk to Swarm manager. If Mesos master and Swarm manager are co-located on the same machine, you can use the 0.0.0.0 or localhost value.

If you use multiple Mesos masters:

$ docker run -d -p <swarm_port>:2375 -p 3375:3375 \
    swarm manage \
        -c mesos-experimental \
        --cluster-opt mesos.address=<public_machine_ip> \
        --cluster-opt mesos.port=3375 \
        zk://<mesos_masters_url>

Once the manager is running, check your configuration by running docker info as follows:

$ docker -H tcp://<swarm_ip>:<swarm_port> info
Containers: 0
Offers: 2
  Offer: 20150609-222929-1327399946-5050-14390-O6286
     └ cpus: 2
     └ mem: 1006 MiB
     └ disk: 34.37 GiB
     └ ports: 31000-32000
  Offer: 20150609-222929-1327399946-5050-14390-O6287
     └ cpus: 2
     └ mem: 1006 MiB
     └ disk: 34.37 GiB
     └ ports: 31000-32000

If you run into Abnormal executor termination error, you might want to run the Swarm container with an additional environment variable: SWARM_MESOS_USER=root.

Exporting the environment variable DOCKER_HOST=<swarm_ip>:<swarm_port> will facilitate interaction with the Docker client as it will not require -H anymore.

Limitations

Docker Swarm on Mesos provides a basic Mesos offers negotiation It is also possible to use Docker-compose to communicate to the Docker Swarm manager how to schedule Docker containers on the Mesos cluster. Docker Swarm on Mesos supports Mesos master detection with Zookeeper

Functionality

Docker Swarm on Mesos supports the following subset of Docker CLI commands:

  • attach
  • build
  • commit
  • cp
  • diff
  • events
  • export
  • history
  • images
  • info
  • inspect
  • logs
  • network
    • create
    • remove
    • ls
    • inspect
    • connect
    • disconnect
  • port
  • ps
  • run
  • save
  • search
  • stats
  • top

Unsupported docker commands

The current Swarm scheduler for Mesos implementation is not feature complete. Hence the following Docker CLI commands are unsupported and may cause unpredictable results:

  • create
  • import
  • kill
  • load
  • login
  • logout
  • pause
  • pull
  • push
  • rename
  • restart
  • rm
  • rmi
  • start
  • stop
  • tag
  • unpause
  • update
  • volume
  • wait

Known issues

  • Docker Swarm on Mesos only uses unreserved resources; and if Mesos offers reserved resources, the role info is also ignored which is rejected by Mesos master. See here for a proposal on letting Docker Swarm on Mesos use both un-reserved/reserved resources.
  • See here for a proposal on letting Docker Swarm on Mesos use revocable resources.
  • Restarted Mesos agents have flaky recovery in conjunction with Docker Swarm. When Mesos agents restart, Mesos master don’t always send offers for those agents to Docker Swarm. This issue seems to be solved with the Swarm 1.1 release.

To have a global view of the tracked issues please refer to Mesos issues in Github.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewCluster 

func NewCluster(scheduler *scheduler.Scheduler, TLSConfig *tls.Config, master string, options cluster.DriverOpts, engineOptions *cluster.EngineOpts) (cluster.Cluster, error)

NewCluster for mesos Cluster creation

Types

type Cluster 

type Cluster struct {
	sync.RWMutex

	TLSConfig *tls.Config
	// contains filtered or unexported fields
}

Cluster struct for mesos

func (*Cluster) BuildImage 

func (c *Cluster) BuildImage(buildContext io.Reader, buildImage *types.ImageBuildOptions, out io.Writer) error

BuildImage builds an image

func (*Cluster) Container 

func (c *Cluster) Container(IDOrName string) *cluster.Container

Container returns the container with IdOrName in the cluster

func (*Cluster) Containers 

func (c *Cluster) Containers() cluster.Containers

Containers returns all the containers in the cluster.

func (*Cluster) CreateContainer 

func (c *Cluster) CreateContainer(config *cluster.ContainerConfig, name string, authConfig *types.AuthConfig) (*cluster.Container, error)

CreateContainer for container creation in Mesos task

func (*Cluster) CreateNetwork 

func (c *Cluster) CreateNetwork(name string, request *types.NetworkCreate) (*types.NetworkCreateResponse, error)

CreateNetwork creates a network in the cluster

func (*Cluster) CreateVolume 

func (c *Cluster) CreateVolume(request *volumetypes.VolumesCreateBody) (*types.Volume, error)

CreateVolume creates a volume in the cluster

func (*Cluster) Handle 

func (c *Cluster) Handle(e *cluster.Event) error

Handle callbacks for the events

func (*Cluster) Image 

func (c *Cluster) Image(IDOrName string) *cluster.Image

Image returns an image with IdOrName in the cluster

func (*Cluster) Images 

func (c *Cluster) Images() cluster.Images

Images returns all the images in the cluster.

func (*Cluster) Import 

func (c *Cluster) Import(source string, ref string, tag string, imageReader io.Reader, callback func(what, status string, err error))

Import image

func (*Cluster) Info 

func (c *Cluster) Info() [][2]string

Info gives minimal information about containers and resources on the mesos cluster

func (*Cluster) LaunchTask 

func (c *Cluster) LaunchTask(t *task.Task) bool

LaunchTask selects node and calls driver to launch a task

func (*Cluster) Load 

func (c *Cluster) Load(imageReader io.Reader, callback func(where, status string, err error))

Load images

func (*Cluster) Networks 

func (c *Cluster) Networks() cluster.Networks

Networks returns all the networks in the cluster.

func (*Cluster) Pull 

func (c *Cluster) Pull(name string, authConfig *types.AuthConfig, callback func(where, status string, err error))

Pull pulls images on the cluster nodes

func (*Cluster) RANDOMENGINE 

func (c *Cluster) RANDOMENGINE() (*cluster.Engine, error)

RANDOMENGINE returns a random engine.

func (*Cluster) RefreshEngine 

func (c *Cluster) RefreshEngine(hostname string) error

RefreshEngine is

func (*Cluster) RefreshEngines 

func (c *Cluster) RefreshEngines() error

RefreshEngines is

func (*Cluster) RegisterEventHandler 

func (c *Cluster) RegisterEventHandler(h cluster.EventHandler) error

RegisterEventHandler registers an event handler.

func (*Cluster) RemoveContainer 

func (c *Cluster) RemoveContainer(container *cluster.Container, force, volumes bool) error

RemoveContainer removes containers on mesos cluster

func (*Cluster) RemoveImage 

func (c *Cluster) RemoveImage(image *cluster.Image) ([]types.ImageDelete, error)

RemoveImage removes an image from the cluster

func (*Cluster) RemoveImages 

func (c *Cluster) RemoveImages(name string, force bool) ([]types.ImageDelete, error)

RemoveImages removes images from the cluster

func (*Cluster) RemoveNetwork 

func (c *Cluster) RemoveNetwork(network *cluster.Network) error

RemoveNetwork removes network from the cluster

func (*Cluster) RemoveVolumes 

func (c *Cluster) RemoveVolumes(name string) (bool, error)

RemoveVolumes removes volumes from the cluster

func (*Cluster) RenameContainer 

func (c *Cluster) RenameContainer(container *cluster.Container, newName string) error

RenameContainer renames a container

func (*Cluster) Scale 

func (c *Cluster) Scale(scaleConfig common.ScaleAPI) []string

Scale is

func (*Cluster) StartContainer 

func (c *Cluster) StartContainer(container *cluster.Container, hostConfig *dockerclient.HostConfig) error

StartContainer starts a container

func (*Cluster) TagImage 

func (c *Cluster) TagImage(IDOrName string, ref string, force bool) error

TagImage tags an image

func (*Cluster) TotalCpus 

func (c *Cluster) TotalCpus() int64

TotalCpus returns the total memory of the cluster

func (*Cluster) TotalMemory 

func (c *Cluster) TotalMemory() int64

TotalMemory returns the total memory of the cluster

func (*Cluster) UnregisterEventHandler 

func (c *Cluster) UnregisterEventHandler(h cluster.EventHandler)

UnregisterEventHandler unregisters a previously registered event handler.

func (*Cluster) Volumes 

func (c *Cluster) Volumes() cluster.Volumes

Volumes returns all the volumes in the cluster.

type Scheduler 

type Scheduler struct {
	scheduler.Scheduler
	// contains filtered or unexported fields
}

Scheduler structure for mesos driver

func NewScheduler 

func NewScheduler(config mesosscheduler.DriverConfig, cluster *Cluster, sched *scheduler.Scheduler) (*Scheduler, error)

NewScheduler for Scheduler mesos driver creation

func (*Scheduler) Disconnected 

func (s *Scheduler) Disconnected(mesosscheduler.SchedulerDriver)

Disconnected method

func (*Scheduler) Error 

func (s *Scheduler) Error(d mesosscheduler.SchedulerDriver, msg string)

Error method

func (*Scheduler) ExecutorLost 

func (s *Scheduler) ExecutorLost(mesosscheduler.SchedulerDriver, *mesosproto.ExecutorID, *mesosproto.SlaveID, int)

ExecutorLost method

func (*Scheduler) FrameworkMessage 

func (s *Scheduler) FrameworkMessage(mesosscheduler.SchedulerDriver, *mesosproto.ExecutorID, *mesosproto.SlaveID, string)

FrameworkMessage method

func (*Scheduler) OfferRescinded 

func (s *Scheduler) OfferRescinded(_ mesosscheduler.SchedulerDriver, offerID *mesosproto.OfferID)

OfferRescinded method

func (*Scheduler) Registered 

func (s *Scheduler) Registered(driver mesosscheduler.SchedulerDriver, fwID *mesosproto.FrameworkID, masterInfo *mesosproto.MasterInfo)

Registered method for registered mesos framework

func (*Scheduler) Reregistered 

func (s *Scheduler) Reregistered(mesosscheduler.SchedulerDriver, *mesosproto.MasterInfo)

Reregistered method for registered mesos framework

func (*Scheduler) ResourceOffers 

func (s *Scheduler) ResourceOffers(_ mesosscheduler.SchedulerDriver, offers []*mesosproto.Offer)

ResourceOffers method

func (*Scheduler) SlaveLost 

func (s *Scheduler) SlaveLost(mesosscheduler.SchedulerDriver, *mesosproto.SlaveID)

SlaveLost method

func (*Scheduler) StatusUpdate 

func (s *Scheduler) StatusUpdate(_ mesosscheduler.SchedulerDriver, taskStatus *mesosproto.TaskStatus)

StatusUpdate method

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.