Compute

This module was created in order to house the calculations that may be needed by the stereo mechanisms and those required by the neural networks.

Calculate the reprojection error

It can be useful when you need to obtain the calibration of the error of the cameras, the calibration is when you find the intrinsic and extrinsic parameters of a camera, such as: distortions, center of projections, focal lengths and so on.

py2vision.compute.error_compute.re_projection_error(objpoints, rvecs, tvecs, mtx, imgpoints, dist)

Calculates the error when calibrating a camera. when the error approaches zero, the more accurate are the parameters found. Given the intrinsic, distortion, rotation and translation matrices, we must first transform the translation matrices, we must first transform the object point to the point of the camera. the object point to image point using cv.projectPoints(). Then, we will calculate the absolute norm between what we obtained with our transformation and the algorithm. obtained with our transformation and the corner search algorithm. To find the average error, we calculate the arithmetic average of the errors calculated for all calibration images.

Parameters:

• objpoints – array of object points expressed in the world coordinate frame. A 1-channel 3xN/Nx3 or 3-channel 1xN/Nx1, where N is the number of points in the view.
• rvecs – the rotation vector (Rodrigues) which, together with tvec, performs a change of base from the world coordinate system to the camera, see calibrateCamera for more details.
• tvecs – The translation vector, see the description of the parameters above.
• mtx – The intrinsic matrix of the camera.
• imgpoints – reference points to compare with the reprojected objpoints.
• dist – Input vector of the distortion coefficients.

Returns

a float with the mean error with L2 norm.

Yolo V3 Computations

The class YoloV3Calculus is responsible for adding the prediction layer in a YOLO version 3 network, converting the matrices containing the bounding box coordinates from a format ($xmin$, $ymin$, $xmax$, $ymax$) to a format ($cx$, $cy$, $w$, $h$) where $cx$ and $cy$ are the coordinates of the center point of the bounding box and the variables $w$ and $h$ are the width and height of the box respectively, such conversion can be done in both directions.

It is also in charge of calculating the IoU value and applying the Non maximum suppression algorithm to filter the bounding boxes that do not meet a certain threshold, in order to obtain predictions with a lower amount of noise and calculate the network error.

class py2vision.compute.yolov3_calculus.YoloV3Calculus

Useful methods for calculating the IOU, decode network output when training, nms, yolov3 loss, and bounding box offsets. and bounding box offsets.

bbox_ciou(boxes1, boxes2)

Compute Complete Intersection Over Union between bounding boxes.

Parameters:

• boxes1 – an array or tensor with a shape (n, 4).
• boxes2 – an array or tensor with a shape (n, 4).

Returns:

A value between (0, 1) that correspond with CIoU.

bbox_giou(boxes1, boxes2)

Compute Generalized Intersection Over Union between bounding boxes.

Parameters:

• boxes1 – an array or tensor with a shape (n, 4).
• boxes2 – an array or tensor with a shape (n, 4).

Returns:

A value between (0, 1) that correspond with GIoU.

bbox_iou(boxes1, boxes2)

Compute Intersection Over Union between anchor boxes and bounding boxes.

Parameters:

• boxes1 – an array or tensor with a shape (n, 4).
• boxes2 – an array or tensor with a shape (n, 4).

Returns:

A value between (0, 1) that correspond with IoU.

best_bboxes_iou(boxes1, boxes2)

Compute Intersection Over Union between bounding boxes and return the best choices to apply nms algorithm.

Parameters:

• boxes1 – an array or tensor with a shape (n, 4).
• boxes2 – an array or tensor with a shape (n, 4).

Returns:

A value between (0, 1) that correspond with IoU.

centroid2minmax(boxes)

Centroid to minmax format (cx, cy, w, h) to (xmin, ymin, xmax, ymax).

Parameters:boxes – Batch of bounding boxes in centroid format. Returns:Batch of boxes in minmax format Return type:minmax

decode(conv_output, num_class, i=0, strides=[8, 16, 32], anchors=[[[10, 13], [16, 30], [33, 23]], [[30, 61], [62, 45], [59, 119]], [[116, 90], [156, 198], [373, 326]]])

A piece of code that receives convolutional layers and returns the prediction layers.

Parameters:

• conv_output – output of the Yolo model.
• num_class – an integer representing how many classes the model has.
• i – an integer that can be 0, 1 or 2 to correspond to the three scales of the grid.
• strides – a list with a length of 3 corresponding to the strides of the prediction layer.
• anchors – a 3-dimensional list of anchor sizes.

Returns:

the predicted probability category box object.

loss(pred, conv, label, bboxes, num_class, i=0, strides=[8, 16, 32], loss_thresh=0.5)

Calculate a loss vector to train a yolo network using GIoU, confidence and probability losses.

Parameters:

• pred – the prediction of the model.
• conv – the last convolutional layer of a yolo model.
• label – expected label.
• bboxes – ground truth.
• num_class – an integer with the number of classes to detect.
• i – an integer which can be 0, 1, or 2 to correspond to the three grid scales.
• strides – a list with a len of 3 that correspond with the strides between each prediction.
• loss_thresh – a number between (0, 1) which if IoU is less than it, it is considered that the prediction box contains no objects.

Returns:

A tuple with a len of 3 where the first argument is the GIoU loss, next Confidence loss and the last one the probability loss.

minmax2centroid(boxes)

Minmax to centroid format (xmin, ymin, xmax, ymax) to (cx, cy, w, h).

Parameters:boxes – Batch of bounding boxes in minmax format. Returns:A Batch of boxes in centroid format

nms(bboxes, iou_threshold, sigma=0.3, method='nms')

Compute Non maximum supression algorithm. Note: see this paper to understand soft-nms https://arxiv.org/pdf/1704.04503.pdf.

Parameters:

• bboxes – (xmin, ymin, xmax, ymax, score, class).
• iou_threshold – a parameter between (0, 1).
• sigma – a parameter between (0, 1).
• method – a string that can be ‘nms’ or ‘soft-nms’.

Returns:

Better bounding boxes.

postprocess_boxes(pred_bbox, original_image, input_size, score_threshold)

Improve predicted bounding boxes and resize them.

Parameters:

• pred_bbox – a predicted bonding box.
• original_image – an image before resizing.
• input_size – the dimension of original image after resizing like an square image.
• score_threshold – if the score of a bounding boxes is less than score_threshold, it will be discard.

Returns:

Bounding boxes that are inside the range, valids and with a score greather than score_threshold.