Executor

The Executor allows for VM modification and state checking after creation through the use of Feature, Inject and Condition type Deputy packages. Redis is used to keep track of the installed files and their metadata.

Execution of these deputy packages is done in several steps. First the package assets are copied to the VM, then the package action is executed. In case of the Condition services stream endpoint, the action is executed continuously every Interval seconds until the stream is closed by Ranger.

Queueing

Due to VMware API limitations, operations on VMs are executed one at a time per VM until max_connections is reached. This means that if a VM is currently being modified, any other operations on that VM will be queued until the previous operation is finished. This includes actions such as file transfers and running commands. For this reason the Executor implements a queue system that aims to give fair access for all operations and all VMs. The queue is primarily leveraged by the Condition stream endpoint as it is the most resource intensive. For this reason it is discouraged to use quick intervals for Condition type packages as the VMware API is relatively slow and cannot keep up with the amount of requests. In case a queue is created, it is only evident in Ranger in the scoring graph where the interval for individual Conditions is longer than the one specified in the Deputy package.

Operations that run once, such as Inject and Feature type packages are prioritized over Condition type packages as the queue priority is based on the amount of times an action has been run.

Services

The Executor provides several gRPC services dependant on the type of Deputy package that is to be executed.

The Executor assumes that the VM is already created and running and VM tools are installed on the target VM. During setup, restarting the VM is only permissible if it is defined in the package under the Deputy packages restarts field. After setup, the VM may be restarted at any time and the streaming conditions should be able to recover.

Feature

Feature type packages allow for installing additional services, configurations and more on the VM immediately after creation. There are two gRpc endpoints for feature type packages:

create

The create endpoint is used to install the listed assets on the VM and run its accompanying action. Feature create requests are sent by Ranger immediately after the VM is created.

Request

message Feature {
  string name = 1;
  string virtual_machine_id = 2;
  FeatureType featureType = 3;
  Source source = 4;
  Account account = 5;
  repeated string environment = 7;
}

For reference, the FeatureType, Source and Account messages are defined as follows:

enum FeatureType {
  service = 0;
  configuration = 1;
  artifact = 2;
}

message Source {
    string name = 1;
    string version = 2;
}

message Account {
    string username = 1;
    string password = 2;
    string private_key = 3;
}

Successful Response

message ExecutorResponse {
    Identifier identifier = 1;
    string stdout = 2;
    string stderr = 3;
}

delete

Deletes the listed assets from the VM. Currently, if the feature installs additional files or services on the VM, they are not removed.

Request

message Identifier {
    string value = 1;
}

Successful Response

message Empty {}

Inject

Injects are in essence the same as Feature type packages with the exception that they are run on the VM during the runtime of the exercise and not immediately after the VM is created.

create

Request

message Inject {
  string name = 1;
  string virtual_machine_id = 2;
  Source source = 3;
  Account account = 4;
  repeated string to_entities = 5;
  repeated string environment = 7;
}

Successful Response

message ExecutorResponse {
    Identifier identifier = 1;
    string stdout = 2;
    string stderr = 3;
}

delete

Request

message Identifier {
    string value = 1;
}

Successful Response

message Empty {}

Condition

Conditions are scripts that run every Interval seconds on the VM and return a value between 0 and 1.0 to indicate its current state. They are installed on the VM during its creation and then continuously run with their results streamed back to Ranger unless closed by Ranger.

The condition streams may be closed by Ranger ungracefully if the Condition has served its purpose, for example when a conditional Event has been triggered, said Condition will be closed.

create

The create endpoint is used to install the Condition on the VM.

Request

message Condition {
  string name = 1;
  string virtual_machine_id = 2;
  Account account = 3;
  Source source = 4;
  string command = 5;
  int32 interval = 6;
  repeated string environment = 7;
}

Successful Response

message ExecutorResponse {
    Identifier identifier = 1;
    string stdout = 2;
    string stderr = 3;
}

stream

The stream gRPC endpoint is used to continuously run the Condition on the VM and stream the results back to Ranger.

Request

message Identifier {
    string value = 1;
}

Successful Response

The Response field indicates the ID of the condition run.

stream message ConditionStreamResponse {
  string response = 1;
  float command_return_value = 2;
}

delete

The delete gRPC endpoint is used to uninstall the Condition from the VM.

Request

message Identifier {
    string value = 1;
}

Successful Response

message Empty {}

Configuration

The following configuration options are available for the Executor handler:

Variable	Mandatory	Type	Rules
user	yes	string	vCenter username
password	yes	string	vCenter password
hostname	yes	string	Address of the vCenter environment
insecure	yes	bool	Enables or disables certificate verification for the vSphere client
template_folder_path	yes	string	The storage folder for templates
resource_pool_path	yes	string	The resource pool to be used
exercise_root_path	yes	string	The VM folder root path for exercises under which VMs are created
redis_address	yes	string	Redis address
redis_password	yes	string	Redis password
server_address	yes	string	The handlers own address and port

Example configuration file:

user: vsphere_username
password: vsphere_password
hostname: my-vsphere.net
insecure: true
template_folder_path: /Datacenter/vm/Templates/*
resource_pool_path: /Datacenter/host/Cluster/Resources/Handlers
exercise_root_path: /Datacenter/vm/Exercises
redis_address: redis-container:6379
redis_password: redis_password
server_address: 0.0.0.0:7777

Additional configuration options are available for the Executor handler:

Variable	Mandatory	Type	Default	Description
max_connections	no	int	50	vSphere has a limit of concurrent tasks such as file manipulation on separate VMs. If the limit is reached and additional API queries are made the vSphere API will return an error. This only really impacts the responsiveness of Condition streams, if the limit is reached then excess connections are queued by the Executor
vm_tools_timeout_sec	no	int	600	The maximum amount of time the Executor will wait for the vm_tools to come online after a VM has been created.
vm_tools_retry_sec	no	int	5	The retry frequency of the VM tools status
vm_properties_timeout_sec	no	int	360	Timeout for waiting for a VMs properties to be available
executor_run_timeout_sec	no	int	300	The total timeout for retrying to Execute Package Actions on the VM
executor_run_retry_sec	no	int	2	Duration between retries of failing Execute Package Actions