about the dataset #13
Comments
|
Thank you for your interest in our work. |
|
@mapooon What crop parameters were used in the dataset generation? I would like to try the model on novel data, but I assume I need to pre-process all my images with the alignment that the model expects (here, "dog" for example). I'd be happy to submit a PR with my re-ID script so others can benefit. I've been able to run most of the pre-processing steps with my own images (AnyFace keypoint generation), but it's not clear from the paper or repository how you choose the crop aside from the output being 224x224 after the keypoint alignment step. The bounding box from AnyFace is not square in most cases, and many of the photos in the dataset are more zoomed out than if the box was used directly. I've been able to roughly replicate using a crop of size: def crop(image_path, box, keypoints):
out = []
image = Image.open(image_path)
if keypoints.shape[-1] == 2:
keypoints = np.expand_dims(keypoints, 0)
for (b, k) in zip(box, keypoints):
x, y, w, h = b
kp_mean = np.mean(k, axis=0)
x_c, y_c = float(kp_mean[0]), float(kp_mean[1])
# Let crop size be 2x eye distance (or half-image-size, whichever is smaller)
eye_width = abs(k[1][0] - k[0][0])
crop_radius = int(min(image.width // 2, image.height // 2, 2 * eye_width))
left = int(x_c - crop_radius)
upper = int(y_c - crop_radius)
right = int(x_c + crop_radius)
lower = int(y_c + crop_radius)
cropped = image.crop((left, upper, right, lower))
# Shift keypoints to cropped coordinate frame
k_cropped = k - np.array([left, upper])
out.append((cropped, k_cropped))
return out
def align(img, lmk, src, size):
M, _ = cv2.estimateAffinePartial2D(src, lmk, method=cv2.LMEDS)
img_aligned = cv2.warpAffine(img, M[:2], (size, size), flags=cv2.INTER_AREA)
return img_alignedAn example of how this is used: # I have a custom class to load AnyFace as an ONNX model
anyface = KeypointExtractor(onnx_path)
keypoints, box = anyface.predict(test_image_path, visualize=False)
# Crop image + shifted keypoints
cropped, kp = crop(test_image_path, box, keypoints)[0]
# Alignment
img = np.array(cropped)
h,w=img.shape[:2]
src = kp
target = "dog.npy"
tgt = np.load(target).reshape((5,2))
img_aligned = Image.fromarray(align(img,tgt,src,224))I wasn't able to get a projective warp to work as in the AnyFace keypoints + crop as above: Approximate warp to target face keypoints: Compared to a random example in the dataset:
|
|
@jveitchmichaelis Thank you for reporting your issue. |
does the dataset only include the cropped faces? will the original image be provided? thanks for the great work!
The text was updated successfully, but these errors were encountered: