easy_vision.python.model

easy_vision.python.model.classification_model

class easy_vision.python.model.classification_model.ClassificationModel(model_config, feature_dict, label_dict=None, mode='predict', categories=None)[source]

Bases: easy_vision.python.model.cv_model.CVModel

__init__(model_config, feature_dict, label_dict=None, mode='predict', categories=None)[source]

Init classification model

Parameters:
  • model_config – a protobuf config object
  • feature_dict – a dict of tensor, used as features
  • label_dict – a dict of tensor, used as labels
  • mode – indicats to build a model used in train, evaluate, predict
  • categories – a list of dicts, each dist has two keys id and name
build_loss_graph()[source]

calculate classification loss

Parameters:
  • NxC tensor, where N is batch num, C is number of classes (self._prediction_dict['prob']) –
  • NxC tensor, one-hot encoding label (self._feature_dict['label']) –
Returns:

a dict of tensor containing different kinds of loss
Return type:

loss_dict
build_metric_graph(eval_config)[source]

add metrics ops to graph :param eval_config: protobufer object, see python/protos/eval.proto.

Returns:a dict of metric_op, each metric_op is a tuple of (update_op, value_op)
Return type:metric_dict
build_predict_graph()[source]

build classification network

Parameters:self._prediction_dict['preprocessed_images'] – a tensor of size [batch_size, h, w, c]
Returns:a tensor of size [batch_size, num_classes], the logits self._prediction_dict[‘prob’]: a tensor of size [batch-size， num_classes], the softmax of logits
Return type:self._prediction_dict[‘logits’]
classmethod create_class(name)
get_outputs()[source]

return a list of output key, which can be used to index output tensor result in prediction_dict

get_scopes_of_levels()[source]

return a list of variable scope list order by levels ( outputs -> heads -> backbone -> inputs).

easy_vision.python.model.cv_estimator

class easy_vision.python.model.cv_estimator.CVEstimator(cv_estimator_config, cv_model_cls, model_dir=None, config=None, params={})[source]

Bases: tensorflow.python.estimator.estimator.Estimator

__init__(cv_estimator_config, cv_model_cls, model_dir=None, config=None, params={})[source]

Estimator sub class for cv tasks

Parameters:
  • cv_estimator_config – a CVEstimatorConfig pb object
  • cv_model_cls – a subclass of CVModel
  • model_dir – save model dir
  • config – tf.estimator.RunConfig
easy_vision.python.model.cv_estimator.build_cv_model_fn(cv_estimator_config, cv_model_class)[source]

easy_vision.python.model.cv_head

class easy_vision.python.model.cv_head.CVHead(feature_dict, head_config, label_dict=None, mode='predict')[source]

Bases: object

__init__(feature_dict, head_config, label_dict=None, mode='predict')[source]

x.__init__(…) initializes x; see help(type(x)) for signature

build_loss_graph()[source]
build_postprocess_graph()[source]

for post process of the head it is necessary to separate predict and postprocess because postprocess is not needed during train but needed during test

build_predict_graph()[source]

easy_vision.python.model.cv_model

class easy_vision.python.model.cv_model.CVModel(model_config, feature_dict, label_dict=None, mode='predict', categories=None)[source]

Bases: object

__init__(model_config, feature_dict, label_dict=None, mode='predict', categories=None)[source]

x.__init__(…) initializes x; see help(type(x)) for signature

build_loss_graph()[source]
build_metric_graph()[source]
build_predict_graph()[source]
classmethod create_class(name)
get_outputs()[source]

return a list of output key, which can be used to index output tensor result in prediction_dict

get_restore_filter()[source]
Returns:filter, type of Filter in restore_filter.py scope_drop, type of ScopeDrop in restore_filter.py
get_scopes_of_levels()[source]

return a list of variable scope list order by levels ( outputs -> heads -> backbone -> inputs). e.g. [[“RPNHead”, “RCNNBoxPredictor”, “RCNNMaskPredictor”, “FPN”]] +

self._backbone.get_scopes_of_levels(with_logits=False)
numpy_ckpt_restore_hook(ckpt_path, ckpt_var_map_path='')[source]

Hook of restoring variables from numpy ckpt_path

Parameters:
  • ckpt_path – numpy checkpoint path to restore from
  • ckpt_var_map_path – variable map from graph variables to variables in a checkpoint each line consists of: variable name in graph variable name in ckpt

Return: NumpyCheckpointRestoreHook

restore(ckpt_path, include_global_step=False, ckpt_var_map_path='', force_restore_shape_compatible=False)[source]

Restore variables from ckpt_path

  1. list the variables in graph that need to be restored
  2. inspect checkpoint and find the variables that could restore from checkpoint substitute scope names in case necessary
  3. call tf.train.init_from_checkpoint to restore the variables
Parameters:
  • ckpt_path – checkpoint path to restore from
  • include_global_step – whether to restore global_step variable
  • ckpt_var_map_path – variable map from graph variables to variables in a checkpoint each line consists of: variable name in graph variable name in ckpt
  • force_restore_shape_compatible – if variable shape is incompatible, clip or pad variables in checkpoint, and then restore

return: IncompatibleShapeRestoreHook if force_shape_compatible else None

easy_vision.python.model.detection_helper

easy_vision.python.model.detection_helper.batch_decode_boxes(box_coder, box_encodings, anchor_boxes)[source]

Decodes box encodings with respect to the anchor boxes.

Parameters:
  • box_encodings – a 4-D tensor with shape [batch_size, num_anchors, num_classes, box_coder.code_size] representing box encodings.
  • anchor_boxes – [batch_size, num_anchors, box_coder.code_size] representing decoded bounding boxes. If using a shared box across classes the shape will instead be [total_num_proposals, 1, box_coder.code_size].
Returns:

a

[batch_size, num_anchors, num_classes, box_coder.code_size] float tensor representing bounding box predictions (for each image in batch, proposal and class). If using a shared box across classes the shape will instead be [batch_size, num_anchors, 1, box_coder.code_size].
Return type:

decoded_boxes
easy_vision.python.model.detection_helper.change_coordinate_to_original_image(detection_boxes, true_image_shape, original_image_shape)[source]

Change detection boxes coordinate to original image

Parameters:
  • detection_boxes – A float32 tensor with shape [batch_size, num_proposals, box_code_size] containing proposal boxes in absolution coordinates on preprocessed image.
  • true_image_shape – A int32 tensor with shape [batch_size, 3] containing preprocessed valid image shapes
  • original_image_shape – A int32 tensor with shape [batch_size, 3] containing original valid image shapes
easy_vision.python.model.detection_helper.compute_clip_window(image_shapes)[source]
Parameters:image_shapes – the shape of each image(padded) in a batch[[h0, w0], [h1, w1], …]

Return: clip rect for each image(y0, x0, y1, x1)

easy_vision.python.model.detection_helper.unstack_batch(tensor_dict, unpad_groundtruth_tensors=True)[source]
Parameters:
  • tensor_dict – batched label dict to be unstacked each tensor first dimension size == batch_size
  • unpad_groundtruth_tensors – if True, drop end paddings by slicing, the keeped size is stored in num_groundtruth_tensors
Returns:

a dict of unstacked list of tensors(each list is of batch size len)
Raise:
in case num_groundtruth_tensors not in tensor_dict, an ValueError
will be raised.

easy_vision.python.model.detection_model

class easy_vision.python.model.detection_model.DetectionModel(model_config, feature_dict, label_dict=None, mode='predict', categories=None)[source]

Bases: easy_vision.python.model.cv_model.CVModel

__init__(model_config, feature_dict, label_dict=None, mode='predict', categories=None)[source]

x.__init__(…) initializes x; see help(type(x)) for signature

build_metric_graph(eval_config)[source]

add metrics ops to graph :param eval_config: protobufer object, see python/protos/eval.proto.

self-used args:

self._prediction_dict[fields.DetectionResultFields.detection_boxes] self._prediction_dict[fields.DetectionResultFields.detection_scores] self._prediction_dict[fields.DetectionResultFields.detection_classes] self._prediction_dict[fields.DetectionResultFields.num_detections]

self._feature_dict[fields.InputDataFields.original_image_shape]

self._padded_label_dict[fields.InputDataFields.groundtruth_boxes] self._padded_label_dict[fields.InputDataFields.groundtruth_classes] self._padded_label_dict[fields.InputDataFields.num_groundtruth_boxes]

classmethod create_class(name)
get_outputs()[source]

return a list of output key, which can be used to index output tensor result in prediction_dict

get_scopes_of_levels()[source]

return a list of variable scope list order by levels ( outputs -> heads -> backbone -> inputs).

easy_vision.python.model.estimator_utils

class easy_vision.python.model.estimator_utils.CheckpointSaverHook(checkpoint_dir, save_secs=None, save_steps=None, saver=None, checkpoint_basename='model.ckpt', scaffold=None, listeners=None, write_graph=True)[source]

Bases: tensorflow.python.training.basic_session_run_hooks.CheckpointSaverHook

Saves checkpoints every N steps or seconds.

__init__(checkpoint_dir, save_secs=None, save_steps=None, saver=None, checkpoint_basename='model.ckpt', scaffold=None, listeners=None, write_graph=True)[source]

Initializes a CheckpointSaverHook.

Parameters:
  • checkpoint_dir – str, base directory for the checkpoint files.
  • save_secs – int, save every N secs.
  • save_steps – int, save every N steps.
  • saver – Saver object, used for saving.
  • checkpoint_basename – str, base name for the checkpoint files.
  • scaffold – Scaffold, use to get saver object.
  • listeners – List of CheckpointSaverListener subclass instances. Used for callbacks that run immediately before or after this hook saves the checkpoint.
Raises:
  • ValueError – One of save_steps or save_secs should be set.
  • ValueError – At most one of saver or scaffold should be set.
after_create_session(session, coord)[source]

Called when new TensorFlow session is created.

This is called to signal the hooks that a new session has been created. This has two essential differences with the situation in which begin is called:

  • When this is called, the graph is finalized and ops can no longer be added
    to the graph.
  • This method will also be called as a result of recovering a wrapped
    session, not only at the beginning of the overall session.
Parameters:
  • session – A TensorFlow Session that has been created.
  • coord – A Coordinator object which keeps track of all threads.
before_run(run_context)[source]

Called before each call to run().

You can return from this call a SessionRunArgs object indicating ops or tensors to add to the upcoming run() call. These ops/tensors will be run together with the ops/tensors originally passed to the original run() call. The run args you return can also contain feeds to be added to the run() call.

The run_context argument is a SessionRunContext that provides information about the upcoming run() call: the originally requested op/tensors, the TensorFlow Session.

At this point graph is finalized and you can not add ops.

Parameters:run_context – A SessionRunContext object.
Returns:None or a SessionRunArgs object.
class easy_vision.python.model.estimator_utils.ExitBarrierHook(num_worker, is_chief)[source]

Bases: tensorflow.python.training.session_run_hook.SessionRunHook

ExitBarrier to make sure that workers wait for the master, and exit at the same time. After training finish, master has to do evaluation and model export, so master exits a little late than workers.

__init__(num_worker, is_chief)[source]
Parameters:config – tf.Estimator.RunConfig
after_create_session(session, coord)[source]

Clean up the queue, as there are sometimes ps is not exit, the last run enqueued elements will remain in the queue

begin()[source]

Count the number of workers and masters, and setup barrier queue

end(session)[source]

Only when all workers and master enqueue an element, will the end method exit

class easy_vision.python.model.estimator_utils.IncompatibleShapeRestoreHook(incompatible_shape_var_map)[source]

Bases: tensorflow.python.training.session_run_hook.SessionRunHook

Restore variable with incompatible shapes

__init__(incompatible_shape_var_map)[source]
Parameters:incompatible_shape_var_map – a variables mapping with incompatible shapes, map from real variable to temp variable, real variable is the variable used in model, temp variable is the variable restored from checkpoint.
after_create_session(session, coord)[source]

Called when new TensorFlow session is created.

This is called to signal the hooks that a new session has been created. This has two essential differences with the situation in which begin is called:

  • When this is called, the graph is finalized and ops can no longer be added
    to the graph.
  • This method will also be called as a result of recovering a wrapped
    session, not only at the beginning of the overall session.
Parameters:
  • session – A TensorFlow Session that has been created.
  • coord – A Coordinator object which keeps track of all threads.
begin()[source]

Called once before using the session.

When called, the default graph is the one that will be launched in the session. The hook can modify the graph by adding new operations to it. After the begin() call the graph will be finalized and the other callbacks can not modify the graph anymore. Second call of begin() on the same graph, should not change the graph.

class easy_vision.python.model.estimator_utils.MultipleCheckpointsRestoreHook(ckpt_paths)[source]

Bases: tensorflow.python.training.session_run_hook.SessionRunHook

Restore variable from numpy checkpoint

SEP = ';'
__init__(ckpt_paths)[source]
Parameters:
  • ckpt_paths – multiple checkpoint path, seperated by ;
  • name2var_map – var name in numpy ckpt to variable map
after_create_session(session, coord)[source]

Called when new TensorFlow session is created.

This is called to signal the hooks that a new session has been created. This has two essential differences with the situation in which begin is called:

  • When this is called, the graph is finalized and ops can no longer be added
    to the graph.
  • This method will also be called as a result of recovering a wrapped
    session, not only at the beginning of the overall session.
Parameters:
  • session – A TensorFlow Session that has been created.
  • coord – A Coordinator object which keeps track of all threads.
begin()[source]

Called once before using the session.

When called, the default graph is the one that will be launched in the session. The hook can modify the graph by adding new operations to it. After the begin() call the graph will be finalized and the other callbacks can not modify the graph anymore. Second call of begin() on the same graph, should not change the graph.

class easy_vision.python.model.estimator_utils.NumpyCheckpointRestoreHook(ckpt_path, name2var_map)[source]

Bases: tensorflow.python.training.session_run_hook.SessionRunHook

Restore variable from numpy checkpoint

__init__(ckpt_path, name2var_map)[source]
Parameters:
  • ckpt_path – numpy checkpoint path to restore from
  • name2var_map – var name in numpy ckpt to variable map
after_create_session(session, coord)[source]

Called when new TensorFlow session is created.

This is called to signal the hooks that a new session has been created. This has two essential differences with the situation in which begin is called:

  • When this is called, the graph is finalized and ops can no longer be added
    to the graph.
  • This method will also be called as a result of recovering a wrapped
    session, not only at the beginning of the overall session.
Parameters:
  • session – A TensorFlow Session that has been created.
  • coord – A Coordinator object which keeps track of all threads.
begin()[source]

Called once before using the session.

When called, the default graph is the one that will be launched in the session. The hook can modify the graph by adding new operations to it. After the begin() call the graph will be finalized and the other callbacks can not modify the graph anymore. Second call of begin() on the same graph, should not change the graph.

class easy_vision.python.model.estimator_utils.ProgressHook(num_steps, filename, is_chief)[source]

Bases: tensorflow.python.training.session_run_hook.SessionRunHook

__init__(num_steps, filename, is_chief)[source]
Parameters:
  • num_steps – total train steps
  • filename – progress file name
  • is_chief – is chief worker or not
after_run(run_context, run_values)[source]

Called after each call to run().

The run_values argument contains results of requested ops/tensors by before_run().

The run_context argument is the same one send to before_run call. run_context.request_stop() can be called to stop the iteration.

If session.run() raises any exceptions then after_run() is not called.

Parameters:
  • run_context – A SessionRunContext object.
  • run_values – A SessionRunValues object.
before_run(run_context)[source]

Called before each call to run().

You can return from this call a SessionRunArgs object indicating ops or tensors to add to the upcoming run() call. These ops/tensors will be run together with the ops/tensors originally passed to the original run() call. The run args you return can also contain feeds to be added to the run() call.

The run_context argument is a SessionRunContext that provides information about the upcoming run() call: the originally requested op/tensors, the TensorFlow Session.

At this point graph is finalized and you can not add ops.

Parameters:run_context – A SessionRunContext object.
Returns:None or a SessionRunArgs object.
end(session)[source]

Called at the end of session.

The session argument can be used in case the hook wants to run final ops, such as saving a last checkpoint.

If session.run() raises exception other than OutOfRangeError or StopIteration then end() is not called. Note the difference between end() and after_run() behavior when session.run() raises OutOfRangeError or StopIteration. In that case end() is called but after_run() is not called.

Parameters:session – A TensorFlow Session that will be soon closed.

easy_vision.python.model.large_scale_classification_model

class easy_vision.python.model.large_scale_classification_model.LargeScaleClassification(model_config, feature_dict, label_dict=None, mode='predict', categories=None)[source]

Bases: easy_vision.python.model.classification_model.ClassificationModel

__init__(model_config, feature_dict, label_dict=None, mode='predict', categories=None)[source]

Init large-scale classification model

Parameters:
  • model_config – a protobuf config object
  • feature_dict – a dict of tensor, used as features
  • label_dict – a dict of tensor, used as labels
  • mode – indicats to build a model used in train, evaluate, predict
  • categories – a list of dicts, each dist has two keys id and name
build_loss_graph()[source]

calculate classification loss

Parameters:
  • NxC tensor, where N is batch num, C is number of classes (self._prediction_dict['prob']) –
  • NxC tensor, one-hot encoding label (self._feature_dict['label']) –
Returns:

a dict of tensor containing different kinds of loss
Return type:

loss_dict
build_metric_graph(eval_config)[source]

add metrics ops to graph

Parameters:eval_config – protobufer object, see python/protos/eval.proto.
Returns:a dict of metric_op, each metric_op is a tuple of (update_op, value_op)
Return type:metric_dict
build_predict_graph()[source]

build classification network

Parameters:self._prediction_dict['preprocessed_images'] – a tensor of size [batch_size, h, w, c]
Returns:a tensor of size [batch_size, num_classes], the logits self._prediction_dict[‘prob’]: a tensor of size [batch-size， num_classes], the softmax of logits
Return type:self._prediction_dict[‘logits’]
classmethod create_class(name)

easy_vision.python.model.text_model_helper

easy_vision.python.model.text_model_helper.correct_text_keypoints_order(text_keypoints, text_direction)[source]

Correct text keypoints order according to text directions, first keypoint will appear in the real top-left of text region, e.g.

0—1 3—0 | A | dir = 1 => | A | | B | | B | 3—2 2—1
Parameters:
  • text_keypoints – keypoints with shape [batch_size, num_keypoints, 2]
  • text_direction – text_direction with shape [batch_size]
Returns:

text_keypoints with shape [batch_size, num_instance, 2]
easy_vision.python.model.text_model_helper.get_text_keypoints_region_aspect_ratio(text_keypoints)[source]

Calculate the aspect ratios of keypoints region boxes, width/height

Parameters:text_keypoints – keypoints absolute coords on image with shape [batch_size, num_keypoints, 2]
Returns:aspect_ratios with shape [batch_size]
easy_vision.python.model.text_model_helper.get_text_keypoints_region_height_width(text_keypoints)[source]

Calculate the heights and widths of keypoint region boxes

Parameters:text_keypoints – keypoints absolute coords on image with shape [batch_size, num_keypoints, 2]
Returns:with shape [batch_size] widths: with shape [batch_size]
Return type:heights
easy_vision.python.model.text_model_helper.get_texts_keypoints_instance_aspect_ratio(texts_instance_keypoints)[source]

Calculate the aspect ratios for detection instance keypoints boxes

Parameters:texts_instance_keypoints – keypoints absolute coords on image with shape [batch_size, num_instance, num_keypoints, 2]
Returns:aspect_ratios with shape [batch_size, num_instance]

easy_vision.python.model.video_classification_2stream_model

class easy_vision.python.model.video_classification_2stream_model.VideoClassificationTwoStreamModel(model_config, feature_dict, label_dict=None, mode='predict', categories=None)[source]

Bases: easy_vision.python.model.classification_model.ClassificationModel

__init__(model_config, feature_dict, label_dict=None, mode='predict', categories=None)[source]

Init video classification model

Parameters:
  • model_config – a protobuf config object
  • feature_dict – a dict of tensor, used as features
  • label_dict – a dict of tensor, used as labels
  • mode – indicats to build a model used in train, evaluate, predict
  • categories – a list of dicts, each dist has two keys id and name
build_loss_graph()[source]

calculate classification loss

Parameters:
  • NxC tensor, where N is batch num, C is number of classes (self._prediction_dict['prob']) –
  • NxC tensor, one-hot encoding label (self._feature_dict['groundtruth_image_classes']) –
Returns:

a dict of tensor containing different kinds of loss
Return type:

loss_dict
build_metric_graph(eval_config)[source]

add metrics ops to graph

Parameters:eval_config – protobufer object, see python/protos/eval.proto.
Returns:a dict of metric_op, each metric_op is a tuple of (update_op, value_op)
Return type:metric_dict
build_predict_graph()[source]

build video classification network

Parameters:self._prediction_dict['preprocessed_images'] – a tensor of size [batch_size, l, h, w, c]
Returns:a tensor of size [batch_size, num_classes], the logits self._prediction_dict[‘prob’]: a tensor of size [batch-size， num_classes], the softmax of logits
Return type:self._prediction_dict[‘logits’]
classmethod create_class(name)

easy_vision.python.model.video_classification_model

class easy_vision.python.model.video_classification_model.VideoClassificationModel(model_config, feature_dict, label_dict=None, mode='predict', categories=None)[source]

Bases: easy_vision.python.model.classification_model.ClassificationModel

__init__(model_config, feature_dict, label_dict=None, mode='predict', categories=None)[source]

Init video classification model

Parameters:
  • model_config – a protobuf config object
  • feature_dict – a dict of tensor, used as features
  • label_dict – a dict of tensor, used as labels
  • mode – indicats to build a model used in train, evaluate, predict
  • categories – a list of dicts, each dist has two keys id and name
build_loss_graph()[source]

calculate classification loss

Parameters:
  • NxC tensor, where N is batch num, C is number of classes (self._prediction_dict['prob']) –
  • NxC tensor, one-hot encoding label (self._feature_dict['groundtruth_image_classes']) –
Returns:

a dict of tensor containing different kinds of loss
Return type:

loss_dict
build_metric_graph(eval_config)[source]

add metrics ops to graph

Parameters:eval_config – protobufer object, see python/protos/eval.proto.
Returns:a dict of metric_op, each metric_op is a tuple of (update_op, value_op)
Return type:metric_dict
build_predict_graph()[source]

build video classification network

Parameters:self._prediction_dict['preprocessed_images'] – a tensor of size [batch_size, l, h, w, c]
Returns:a tensor of size [batch_size, num_classes], the logits self._prediction_dict[‘prob’]: a tensor of size [batch-size， num_classes], the softmax of logits
Return type:self._prediction_dict[‘logits’]
classmethod create_class(name)

easy_vision.python.model.video_multilabel_classification_model

class easy_vision.python.model.video_multilabel_classification_model.VideoMultiLabelClassificationModel(model_config, feature_dict, label_dict=None, mode='predict', categories=None)[source]

Bases: easy_vision.python.model.video_classification_model.VideoClassificationModel

__init__(model_config, feature_dict, label_dict=None, mode='predict', categories=None)[source]

Init video classification model

Parameters:
  • model_config – a protobuf config object
  • feature_dict – a dict of tensor, used as features
  • label_dict – a dict of tensor, used as labels
  • mode – indicats to build a model used in train, evaluate, predict
  • categories – a list of dicts, each dist has two keys id and name
build_loss_graph()[source]

calculate classification loss

Parameters:
  • NxC tensor, where N is batch num, C is number of classes (self._prediction_dict['prob']) –
  • NxC tensor, one-hot encoding label (self._feature_dict['groundtruth_image_classes']) –
Returns:

a dict of tensor containing different kinds of loss
Return type:

loss_dict
build_metric_graph(eval_config)[source]

add metrics ops to graph

Parameters:eval_config – protobufer object, see python/protos/eval.proto.
Returns:a dict of metric_op, each metric_op is a tuple of (update_op, value_op)
Return type:metric_dict
build_predict_graph()[source]

build video classification network

Parameters:self._prediction_dict['preprocessed_images'] – a tensor of size [batch_size, l, h, w, c]
Returns:a tensor of size [batch_size, num_classes], the logits self._prediction_dict[‘prob’]: a tensor of size [batch-size， num_classes], the softmax of logits
Return type:self._prediction_dict[‘logits’]
classmethod create_class(name)
get_outputs()[source]

return a list of output key, which can be used to index output tensor result in prediction_dict

easy_vision.python.model.whale_model

easy_vision.python.model.whale_model.is_whale_model(model_name_or_class)[source]