Implements Imperative Deployment for ScaleIO #111
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Initialization of Store using Mesos' Zookeeper Fix default value Add logic to use zookeeper for metadata storage Fix the package name for common.go Fix remaining logic for creating metadata and scaleio state nodes Remove the index property Beginnings of implementing the manual definition for provisioning scaleio Fix block device in the state definition Change some of the encapsulation in the classes. Also start to implement the mesos attributes Fix the scope of the Err Implement UpdateAddNode in ScaleIONode class Start the processing for the agent attributes for scaleio Fix types Some restructuring to folders in the scheduler component Correctly build up the ScaleIO metadata from Mesos agent attributes. Working on processing that data. Add more debugging Begin adding server code for monitoring for state changes. Add new AddResourceToScaleIO state... implemented on the executor side. Now needs to on the scheduler side. Add control flow for states server side Fix server state messages Fix compile errors Fix compile error Fix compile error Fix compile error Fix the executor compile issues Fix get self node func Code clean up Initial goscaleio introduction to project Add ScaleIO API version. Default is 2.0 Add in functions to get/set metadata for pd and sp in the kv store Add in code to scafolding to handle add/delete pd, sp, dev Fix compile issue Make change to kvstore ProtectionDomain Add the "add" property to kvstore objects for explicit add of objects. Add a Device object to metadata. Need to know if a device has been deleted or added. Adding in goscaleio APIs Add in Sds object Processing the SDS add Process SDS add More debugging Fix zk endpoint parsing Add fatal when store init fails Add in functionality to delete the metadata from kv store Fix compile error Add in debug Add recursive delete of kvstore Remove levels Fix recursive Fix crash More debugging Fix executor crashing Remove TODOs Remove TODO 2 Add debug for crash Debugging Fix the clone function Add functionality to dump the contents of the key store Fix dump Fix dump Fix dump Fix delete and dump Fix dump Fix dump Dump tree final fix Fix delete Fix the metadata saving node state Fix automatic device addition to sds Fix compile error Delete unncessary call Fix key store Add debugging Fix go tests Fix compile error Fix endpoint Make map Create maps to prevent crash Fix meatdata processing Init slices Fix store init More init slices More ScaleIO API refinement Prevent the framework from re-creating objects a second time. Fix the device name in the metadata Check to see if the scaleio object exists already Use simple sds names Debug More debugging More debugging Use Index for SDS naming Allow for the addition of new nodes Fix state issue! Optimization to not create the scaleio client until there is a change. Dont keep recreating the client Fix selection of MDM nodes on restart. Provide method for modifying keyvalues. Provide method for modifying keys for debugging Fix compile error Change variable names Fix GetConfigured Remove Index and replace with hostname Need a flag to advertise devices if not explicitly declared and even if in state 1024. Add user del and add store Function hhooks Fix the location of the advertise Fix scini dep on rexray. cmd doesnt work. Fix regex Final test Fix manual prefix Fix the declarative property types Remove mandatory requirement to contribute device to sds on every node. Fix rexray config to use Consumes, then Produces, then config default. Final commit for feature
dvonthenen
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This feature implements PR #82 Imperative approach to deploying ScaleIO. This is a significant feature and therefore required a lot of restructuring to the code base. If the declarative approach is not used, the prior method for adding ScaleIO nodes to the cluster is still supported (but has been enhanced - explained further down in this PR).
Imperative Deployment:
To declare what Block/Storage Devices (if any) contributes to what StoragePools and what StoragePools contribute to which ProtectionDomains, the following Mesos Agent nodes attributes (read: http://mesos.apache.org/documentation/latest/attributes-resources/) must be defined: scaleio-sds-domains, scaleio-sds-[domain], and scaleio-sds-[pool]. An example of this is given below:
/etc/mesos-slave/attributes/scaleio-sds-domains = mydomain,myotherdomain
/etc/mesos-slave/attributes/scaleio-sds-mydomain1 = mypool
/etc/mesos-slave/attributes/scaleio-sds-myotherdomain = saltwaterpool
/etc/mesos-slave/attributes/scaleio-sds-mypool = /dev/xvdf
/etc/mesos-slave/attributes/scaleio-sds-saltwaterpool = /dev/xvdg
To declare what Mesos Agent nodes consume storage out of which ProtectionDomain/StoragePool combination, the following Mesos Agent nodes attributes must be defined: scaleio-sdc-domains and scaleio-sdc-[domain]. (Note the sds vs sdc difference). IMPORTANT: There is a limitation currently in rexray such that you can only consume a single ProtectionDomain/StoragePool combination. What ever the first attribute pair discover by the Mesos Agent will be the pair selected. If no sdc is defined, the first attribute pair from the sds attributes will be used. Else the default in the Configuration flag will be used (default values are both default for ProtectionDomain and StoragePool). Example below:
/etc/mesos-slave/attributes/scaleio-sdc-domains = mydomain
/etc/mesos-slave/attributes/scaleio-sdc-mydomain1 = mypool
Other Enhancements
In order to implement this feature, state of the ScaleIO cluster and the nodes had to be introduced and therefore libkv is being used to store ScaleIO metadata. The configuration of this metadata store by default uses the zookeeper configuration which happens to be picked up automatically. Otherwise an external KeyValue store can be used (consul, zookeeper, etcd).
Another feature that has been enhanced is that when new nodes are brought online, storage devices will be add to the ScaleIO cluster based on the method being used.