This commit is contained in:
jha11aditya 2021-03-07 12:03:19 +05:30
parent b0bb218b2a
commit 7afc2fa94b
474 changed files with 344974 additions and 1 deletions

0
1503.03832.pdf Normal file → Executable file
View File

54
adience_align-master/.gitignore vendored Normal file
View File

@ -0,0 +1,54 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
# C extensions
*.so
# Distribution / packaging
.Python
env/
bin/
build/
develop-eggs/
dist/
eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.cache
nosetests.xml
coverage.xml
# Translations
*.mo
# Mr Developer
.mr.developer.cfg
.project
.pydevproject
# Rope
.ropeproject
# Django stuff:
*.log
*.pot
# Sphinx documentation
docs/_build/

View File

@ -0,0 +1,32 @@
This implements a face alignment method, as preprocessing to tasks such as age and gender estimation,
and face recognition as described in [1] and [2].
The code calls an executable of facial landmarks detection, by X.Zhu and D. Ramanan, implementing the algorithm described in [3].
(The rest of this text is a quotation from their code: Copyright (C) 2012 Xiangxin Zhu, Deva Ramanan)
It includes pre-trained face models.
Much of the detection code is built on top of part-based model implementation of [4].
The training code implements a quadratic program (QP) solver described in [5].
In the training code, we use the positive samples from MultiPIE dataset (available at www.multipie.org) and the negative images from the INRIAPerson dataset [6] (included in the package).
Acknowledgements: We graciously thank the authors of the previous code releases and image benchmarks for making them publicly available.
References
==========
[1] E. Eidinger, R. Enbar, T. Hassner, Age and Gender Estimation of Unfiltered Faces, submitted to IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, 2014
[2] http://www.openu.ac.il/home/hassner/Adience/links.html
[3] X. Zhu, D. Ramanan. Face Detection, Pose Estimation and Landmark Localization in the Wild. CVPR 2012.
[4] P. Felzenszwalb, R. Girshick, D. McAllester. Discriminatively Trained Deformable Part Models. http://people.cs.uchicago.edu/~pff/latent.
[5] D. Ramanan. Dual Coordinate Descent Solvers for Large Structured Prediction Problems. UCI Technical Report, to appear.
[6] N. Dalal, B. Triggs. Histograms of Oriented Gradients for Human Detection. CVPR 2005.

View File

@ -0,0 +1,21 @@
Copyright (C) 2014 Adience SER Ltd. (www.adience.com)
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

@ -0,0 +1,49 @@
adience_align
========
This project provides alignment tools for faces, to be used as a preprocessing step before computer vision tasks on face images.
Homepage for the project: http://www.openu.ac.il/home/hassner/Adience/
See the test for example usage.
Specificaly, the "pipeline" test, shows how to use the full process (just remember to change the location of the model files to where you stor the *.xml and other model files)
Installation
=========
in the root of the repository:
```
python setup.py sdist
sudo pip install dist/adience-<version_number>.tar.gz
```
CopyRight
=========
(contact: Eran Eidinger (eran@adience.com), Roee Enbar (roee.e@adience.com))
See the LICENSE.txt file (basically, an MIT license).
With any publication that uses this alignment code, or it's derivative, we kindly ask that you cite the paper:
E. Eidinger, R. Enbar, and T. Hassner, Age and Gender Estimation of Unfiltered Faces, Transactions on Information Forensics and Security (IEEE-TIFS), special issue on Face Recognition in the Wild
For more details, please see:
http://www.openu.ac.il/home/hassner/Adience/publications.html
Compilation notes
========
1. The shared objects were compiled for linux 64bit on Ubuntu 13.10
2. The SO uses boost-1.53, so make sure it is installed on your system and available at /usr/local/, or use LD_LIBRARY_PATH="yourpath" to point it at the right place. Alternatively, place "libboost_system.so.1.53.0" and "libboost_filesystem.so.1.53.0". at the "adiencealign/resources/" subfolder
3. For landmarks detection, we use the file libPartsBasedDetector.so, compiled from the project https://github.com/wg-perception/PartsBasedDetector. You can either compile it yourselves, or use the version under "resources" subfolder, compiled with boost 1.53, on a linux ubuntu 14.04 machine.
We will release the source code for the shared object in the near future
Running the test
========
1. run ```./clear_test.sh``` to delete results of old tests.
2. run ```python test_pipeline.py```
3. results are in the "outputs" subfolder

View File

@ -0,0 +1,20 @@
'''
Created on May 7, 2014
@author: eran
'''
from adiencealign.common.landmarks import fidu_transform, shift_vector,\
WEIGHTS3
class AffineAligner(object):
def __init__(self, fidu_model_file, ):
self.shift = ( 0.25, 0.25 )
fidu_model = [(int(x.split(',')[1]),int(x.split(',')[2])) for x in file(fidu_model_file,'r')]
self.fidu_model = shift_vector(fidu_model, self.shift)
self.WEIGHTS3 = WEIGHTS3
def align(self, img, fidu_points):
# create bs1 image
funneled_img, R = fidu_transform(self.fidu_model, fidu_points, WEIGHTS3, img, self.shift)
return funneled_img, R

View File

@ -0,0 +1,354 @@
'''
Created on May 7, 2014
@author: eran
'''
import cv2
import pickle
import numpy as np
from shapely.geometry.polygon import Polygon
import math
from adiencealign.common.files import make_path, expand_path
from adiencealign.common.images import pad_image_for_rotation
class CascadeDetector(object):
'''
This is a haar cascade classifier capable of detecting in multiple angles
'''
def __init__(self, cascade_file = './resources/haarcascade_frontalface_default.xml',
min_size = (10, 10),
min_neighbors = 20,
scale_factor = 1.04,
angles = [0],
thr = 0.4,
cascade_type = 'haar'):
'''
cascade_type - is a string defining the type of cascade
'''
print(expand_path('.'))
self.cascade_file = cascade_file.rsplit('/',1)[1]
self._cascade_classifier = cv2.CascadeClassifier(cascade_file)
self.scale_factor = scale_factor
self.min_neighbors = min_neighbors
self.min_size = min_size
self.cascade_type = cascade_type
self.angles = angles
self.thr = thr
def __str__(self):
return ''.join([str(x) for x in ['cascade_file:',self.cascade_file,
',scale_factor:',self.scale_factor,
',min_neighbors:',self.min_neighbors,
',min_neighbors:',self.min_neighbors,
',cascade_type:',self.cascade_type
]])
def save_configuration(self, target_file):
file_path = target_file.rsplit('/',1)[0]
make_path(file_path)
config = {'min_size':self.min_size, 'min_neighbours':self.min_neighbors, 'scale_factor':self.scale_factor, 'cascade_file':self.cascade_file}
pickle.dump(obj=config, file = open(target_file,'w'), protocol = 2)
@staticmethod
def load_configuration(target_file):
return pickle.load(open(target_file,'r'))
def detectMultiScaleWithScores(self, img, scaleFactor = None, minNeighbors = None, minSize = None, flags = 4):
scaleFactor = self.scale_factor if not scaleFactor else scaleFactor
minNeighbors = self.min_neighbors if not minNeighbors else minNeighbors
minSize = self.min_size if not minSize else minSize
return self._cascade_classifier.detectMultiScale(img,
scaleFactor = scaleFactor,
minNeighbors = minNeighbors,
minSize = minSize,
flags = flags)
def detectWithAngles(self, img, angels = None, resolve = True, thr = None ):
'''
angles - a list of angles to test. If None, default to the value created at the constructor (which defaults to [0])
resolve - a boolean flag, whether or not to cluster the boxes, and resolve cluster by highest score.
thr - the maximum area covered with objects, before we break from the angles loop
returns - a list of CascadeResult() objects
'''
if thr == None:
thr = self.thr
original_size = img.shape[0] * img.shape[0]
if angels == None:
angels = self.angles
results = []
total_area = 0
for angle in angels:
# the diagonal of the image is the diameter of the rotated image, so the big_image needs to bound this circle
# by being that big
big_image, x_shift, y_shift, diag, rot_center = pad_image_for_rotation(img)
# find the rotation and the inverse rotation matrix, to allow translations between old and new coordinates and vice versa
rot_mat = cv2.getRotationMatrix2D(rot_center, angle, scale = 1.0)
inv_rot_mat = cv2.invertAffineTransform(rot_mat)
# rotate the image by the desired angle
rot_image = cv2.warpAffine(big_image, rot_mat, (big_image.shape[1],big_image.shape[0]), flags=cv2.INTER_CUBIC)
faces = self.detectMultiScaleWithScores(rot_image, scaleFactor = 1.03, minNeighbors = 20, minSize = (15,15), flags = 4)
for face in faces:
xp = face[0]
dx = face[2]
yp = face[1]
dy = face[3]
score = 1
dots = np.matrix([[xp,xp+dx,xp+dx,xp], [yp,yp,yp+dy,yp+dy], [1, 1, 1, 1]])
# these are the original coordinates in the "big_image"
# print dots
originals_in_big = inv_rot_mat * dots
# print originals_in_big
shifter = np.matrix([[x_shift]*4, [y_shift]*4])
# print shifter
# these are the original coordinate in the original image
originals = originals_in_big - shifter
# print originals
points = np.array(originals.transpose())
x = points[0,0]
y = points[0,1]
box_with_score = ([x,y,dx,dy], score)
cascade_result = CascadeResult.from_polygon_points(points, score, self.cascade_type)
# print cascade_result
results.append(cascade_result)
#################
# test and see, if we found enough objects, break out and don't waste our time
total_area += cascade_result.area
if resolve:
return resolve_angles(results, width = img.shape[1], height = img.shape[0])
else:
return results
class BoxInImage(object):
def __init__(self, originals, dx, dy, score = None, angle = 0):
self.originals = originals
self.dx = dx
self.dy = dy
self.score = score
self.angle = angle
def __str__(self):
return ",".join([str(x) for x in [self.originals, self.dx, self.dy, self.score, self.angle]])
def resolve_angles(list_of_results, width, height, thr = 0.3):
'''
we want to cluster the boxes into clusters, and then choose the best box in each cluster by score
* thr - decides what the maximum distance is for a box to join a cluster, in the sense of how much of it's area is covered by the best box in the cluster
note, that two squares, centered, with 45 degrees rotation, will overlap on 77% of their area (thr == 0.22)
'''
clusters = []
for box in list_of_results:
# total_polygon = Polygon([(0,0), (width,0), (width,height), (0,height)])
# if box.polygon.intersection(total_polygon).area < box.area:
# # this means the box is outside the image somehow
# continue
area = box.area
closest_cluster = None
dist_to_closest_cluster = 1.0
for n,cluster in enumerate(clusters):
dist = 1.0
for cluster_box in cluster:
local_dist = 1.0 - box.overlap(cluster_box)/area
dist = min(dist, local_dist)
if dist < dist_to_closest_cluster:
dist_to_closest_cluster = dist
closest_cluster = n
if closest_cluster == None or dist_to_closest_cluster > thr:
# no good cluster was found, open a new cluster
clusters.append([box])
else:
clusters[n].append(box)
centroids = []
for cluster in clusters:
centroids.append(sorted(cluster,key=lambda x: x.score)[-1])
return centroids
def resolve_boxes(dict_of_list_of_cascade_results, min_overlap = 0.7):
'''
Say you tried two different cascades to detect faces.
enter a dictionary (the key is a string describing a cascade type) of detected objects
This function returns a unified results list, where it resolves overlapping boxes, and chooses one of them.
The bigger boxes are selected instead of smaller ones, whether they contain them, or enough of them, determined by min_overlap
'''
final_faces = []
for cascade_str, faces in dict_of_list_of_cascade_results.items():
# go through each cascade type
for face in faces:
if type(face) == CascadeResult:
new_res = face
else:
new_res = CascadeResult(face,cascade_type = cascade_str)
to_add = True
for old_index,old_res in enumerate(final_faces):
ratio = new_res.area / old_res.area
if ratio >1.0:
# new_box is bigger
if new_res.overlap(old_res)/old_res.area > min_overlap:
# the new box contains the old one, we want to replace it:
final_faces[old_index] = new_res
to_add = False
break
if ratio <=1.0:
# the new_box is smaller
if new_res.overlap(old_res)/new_res.area > min_overlap:
# the old box contains the new one, we therefore dont need to add the new box:
to_add = False
break
if to_add:
# if there was no hit, this is a new face, we can add it
final_faces.append(new_res)
return final_faces
def most_centered_box( cascade_results, xxx_todo_changeme ):
( rows, cols ) = xxx_todo_changeme
best_err = 1e10
for i, cascade in enumerate( cascade_results ):
err = ( cascade.x + cascade.dx / 2 - cols / 2 ) ** 2 + ( cascade.y + cascade.dy / 2 - rows / 2 ) ** 2
if err < best_err:
index = i
return cascade_results[ index ]
class CascadeResult(object):
def __init__(self, box_with_score, cascade_type = None, angle = 0):
self.x = box_with_score[0][0]
self.y = box_with_score[0][1]
self.dx = box_with_score[0][2]
self.dy = box_with_score[0][3]
self.score = box_with_score[1]
self.cascade_type = cascade_type
self.angle = angle
@staticmethod
def from_polygon_points(points, score, cascade_type = None):
'''
an alternative generator, allows giving the polygon points instead of [x,y,dx,dy]
'''
x = points[0,0]
y = points[0,1]
top = points[1,] - points[0,]
left = points[3,] - points[0,]
dx = math.sqrt(sum([i*i for i in top]))
dy = math.sqrt(sum([i*i for i in left]))
angle = math.atan(float(top[1])/top[0]) * 180 / math.pi if top[0] != 0 else (970 if top[1] >0 else -90)
return CascadeResult(([x,y,dx,dy],score), cascade_type, angle)
def __str__(self):
return ''.join([str(x) for x in ['center:',self.center,
',\nx:',self.x,
',\ny:',self.y,
',\ndx:',self.dx,
',\ndy:',self.dy,
',\nscore:',self.score,
',\nangle:',self.angle,
',\ncascade_type:',self.cascade_type,
',\npoints_int:\n',self.points_int
]])
@property
def points(self):
x = self.x
y = self.y
dx = self.dx
dy = self.dy
a = self.angle/180.0*math.pi
dots = np.matrix([[x,y,1],[x+dx,y,1],[x+dx,y+dy,1],[x,y+dy,1]])
dots = dots.transpose()
rot_mat = cv2.getRotationMatrix2D((dots[0,0],dots[1,0]), -self.angle, scale = 1.0)
points = rot_mat * dots
points = points.transpose()
return points
@property
def center(self):
return tuple(int(x) for x in (self.points.sum(0)/4.0).tolist()[0])
@property
def points_int(self):
return self.points.astype(int)
@property
def score_with_type(self):
if self.cascade_type:
return self.cascade_type + ' ' + str(self.score)
else:
return str(self.score)
@property
def filename_encode(self):
return '_'.join([str(x) for x in ['loct'] + self.cvformat_result[0] + ['ang', int(self.angle),self.cascade_type, self.score]])
@property
def cvformat_coords(self):
if self.angle == 0:
return [int(x) for x in [self.x, self.y, self.dx, self.dy]]
else:
raise Exception('cannot return [x,y,dx,dy] for a box with angle, use cvformat_result() instead')
@property
def cvformat_result(self):
return ([int(x) for x in [self.x, self.y, self.dx, self.dy]], self.score, self.angle)
# @property
# def rot_matrix(self):
# return array([[cos(math.radians(self.angle)), -sin(math.radians(self.angle))],
# [sin(math.radians(self.angle)), cos(math.radians(self.angle))]])
@property
def top_left(self):
return tuple(self.points[0,].tolist()[0])
@property
def top_right(self):
return tuple(self.points[1,].tolist()[0])
@property
def bottom_right(self):
return tuple(self.points[2,].tolist()[0])
@property
def bottom_left(self):
return tuple(self.points[3,].tolist()[0])
@property
def polygon(self):
return Polygon([self.top_left, self.top_right, self.bottom_right, self.bottom_left])
def overlap(self, otherRect):
return float(self.polygon.intersection(otherRect.polygon).area)
@property
def area(self):
return float(self.polygon.area)
def __gt__(self,b):
return self.area>b.area
def __ge__(self,b):
return self.area>=b.area
def __lt__(self,b):
return self.area<b.area
def __le__(self,b):
return self.area<=b.area

View File

@ -0,0 +1,353 @@
'''
Created on May 7, 2014
@author: eran
'''
import cv2
import pickle
import numpy as np
from shapely.geometry.polygon import Polygon
import math
from adiencealign.common.files import make_path, expand_path
from adiencealign.common.images import pad_image_for_rotation
class CascadeDetector(object):
'''
This is a haar cascade classifier capable of detecting in multiple angles
'''
def __init__(self, cascade_file = './resources/haarcascade_frontalface_default.xml',
min_size = (10, 10),
min_neighbors = 20,
scale_factor = 1.04,
angles = [0],
thr = 0.4,
cascade_type = 'haar'):
'''
cascade_type - is a string defining the type of cascade
'''
print expand_path('.')
self.cascade_file = cascade_file.rsplit('/',1)[1]
self._cascade_classifier = cv2.CascadeClassifier(cascade_file)
self.scale_factor = scale_factor
self.min_neighbors = min_neighbors
self.min_size = min_size
self.cascade_type = cascade_type
self.angles = angles
self.thr = thr
def __str__(self):
return ''.join([str(x) for x in ['cascade_file:',self.cascade_file,
',scale_factor:',self.scale_factor,
',min_neighbors:',self.min_neighbors,
',min_neighbors:',self.min_neighbors,
',cascade_type:',self.cascade_type
]])
def save_configuration(self, target_file):
file_path = target_file.rsplit('/',1)[0]
make_path(file_path)
config = {'min_size':self.min_size, 'min_neighbours':self.min_neighbors, 'scale_factor':self.scale_factor, 'cascade_file':self.cascade_file}
pickle.dump(obj=config, file = open(target_file,'w'), protocol = 2)
@staticmethod
def load_configuration(target_file):
return pickle.load(open(target_file,'r'))
def detectMultiScaleWithScores(self, img, scaleFactor = None, minNeighbors = None, minSize = None, flags = 4):
scaleFactor = self.scale_factor if not scaleFactor else scaleFactor
minNeighbors = self.min_neighbors if not minNeighbors else minNeighbors
minSize = self.min_size if not minSize else minSize
return self._cascade_classifier.detectMultiScale(img,
scaleFactor = scaleFactor,
minNeighbors = minNeighbors,
minSize = minSize,
flags = flags)
def detectWithAngles(self, img, angels = None, resolve = True, thr = None ):
'''
angles - a list of angles to test. If None, default to the value created at the constructor (which defaults to [0])
resolve - a boolean flag, whether or not to cluster the boxes, and resolve cluster by highest score.
thr - the maximum area covered with objects, before we break from the angles loop
returns - a list of CascadeResult() objects
'''
if thr == None:
thr = self.thr
original_size = img.shape[0] * img.shape[0]
if angels == None:
angels = self.angles
results = []
total_area = 0
for angle in angels:
# the diagonal of the image is the diameter of the rotated image, so the big_image needs to bound this circle
# by being that big
big_image, x_shift, y_shift, diag, rot_center = pad_image_for_rotation(img)
# find the rotation and the inverse rotation matrix, to allow translations between old and new coordinates and vice versa
rot_mat = cv2.getRotationMatrix2D(rot_center, angle, scale = 1.0)
inv_rot_mat = cv2.invertAffineTransform(rot_mat)
# rotate the image by the desired angle
rot_image = cv2.warpAffine(big_image, rot_mat, (big_image.shape[1],big_image.shape[0]), flags=cv2.INTER_CUBIC)
faces = self.detectMultiScaleWithScores(rot_image, scaleFactor = 1.03, minNeighbors = 20, minSize = (15,15), flags = 4)
for face in faces:
xp = face[0]
dx = face[2]
yp = face[1]
dy = face[3]
score = 1
dots = np.matrix([[xp,xp+dx,xp+dx,xp], [yp,yp,yp+dy,yp+dy], [1, 1, 1, 1]])
# these are the original coordinates in the "big_image"
# print dots
originals_in_big = inv_rot_mat * dots
# print originals_in_big
shifter = np.matrix([[x_shift]*4, [y_shift]*4])
# print shifter
# these are the original coordinate in the original image
originals = originals_in_big - shifter
# print originals
points = np.array(originals.transpose())
x = points[0,0]
y = points[0,1]
box_with_score = ([x,y,dx,dy], score)
cascade_result = CascadeResult.from_polygon_points(points, score, self.cascade_type)
# print cascade_result
results.append(cascade_result)
#################
# test and see, if we found enough objects, break out and don't waste our time
total_area += cascade_result.area
if resolve:
return resolve_angles(results, width = img.shape[1], height = img.shape[0])
else:
return results
class BoxInImage(object):
def __init__(self, originals, dx, dy, score = None, angle = 0):
self.originals = originals
self.dx = dx
self.dy = dy
self.score = score
self.angle = angle
def __str__(self):
return ",".join([str(x) for x in [self.originals, self.dx, self.dy, self.score, self.angle]])
def resolve_angles(list_of_results, width, height, thr = 0.3):
'''
we want to cluster the boxes into clusters, and then choose the best box in each cluster by score
* thr - decides what the maximum distance is for a box to join a cluster, in the sense of how much of it's area is covered by the best box in the cluster
note, that two squares, centered, with 45 degrees rotation, will overlap on 77% of their area (thr == 0.22)
'''
clusters = []
for box in list_of_results:
# total_polygon = Polygon([(0,0), (width,0), (width,height), (0,height)])
# if box.polygon.intersection(total_polygon).area < box.area:
# # this means the box is outside the image somehow
# continue
area = box.area
closest_cluster = None
dist_to_closest_cluster = 1.0
for n,cluster in enumerate(clusters):
dist = 1.0
for cluster_box in cluster:
local_dist = 1.0 - box.overlap(cluster_box)/area
dist = min(dist, local_dist)
if dist < dist_to_closest_cluster:
dist_to_closest_cluster = dist
closest_cluster = n
if closest_cluster == None or dist_to_closest_cluster > thr:
# no good cluster was found, open a new cluster
clusters.append([box])
else:
clusters[n].append(box)
centroids = []
for cluster in clusters:
centroids.append(sorted(cluster,key=lambda x: x.score)[-1])
return centroids
def resolve_boxes(dict_of_list_of_cascade_results, min_overlap = 0.7):
'''
Say you tried two different cascades to detect faces.
enter a dictionary (the key is a string describing a cascade type) of detected objects
This function returns a unified results list, where it resolves overlapping boxes, and chooses one of them.
The bigger boxes are selected instead of smaller ones, whether they contain them, or enough of them, determined by min_overlap
'''
final_faces = []
for cascade_str, faces in dict_of_list_of_cascade_results.iteritems():
# go through each cascade type
for face in faces:
if type(face) == CascadeResult:
new_res = face
else:
new_res = CascadeResult(face,cascade_type = cascade_str)
to_add = True
for old_index,old_res in enumerate(final_faces):
ratio = new_res.area / old_res.area
if ratio >1.0:
# new_box is bigger
if new_res.overlap(old_res)/old_res.area > min_overlap:
# the new box contains the old one, we want to replace it:
final_faces[old_index] = new_res
to_add = False
break
if ratio <=1.0:
# the new_box is smaller
if new_res.overlap(old_res)/new_res.area > min_overlap:
# the old box contains the new one, we therefore dont need to add the new box:
to_add = False
break
if to_add:
# if there was no hit, this is a new face, we can add it
final_faces.append(new_res)
return final_faces
def most_centered_box( cascade_results, ( rows, cols ) ):
best_err = 1e10
for i, cascade in enumerate( cascade_results ):
err = ( cascade.x + cascade.dx / 2 - cols / 2 ) ** 2 + ( cascade.y + cascade.dy / 2 - rows / 2 ) ** 2
if err < best_err:
index = i
return cascade_results[ index ]
class CascadeResult(object):
def __init__(self, box_with_score, cascade_type = None, angle = 0):
self.x = box_with_score[0][0]
self.y = box_with_score[0][1]
self.dx = box_with_score[0][2]
self.dy = box_with_score[0][3]
self.score = box_with_score[1]
self.cascade_type = cascade_type
self.angle = angle
@staticmethod
def from_polygon_points(points, score, cascade_type = None):
'''
an alternative generator, allows giving the polygon points instead of [x,y,dx,dy]
'''
x = points[0,0]
y = points[0,1]
top = points[1,] - points[0,]
left = points[3,] - points[0,]
dx = math.sqrt(sum([i*i for i in top]))
dy = math.sqrt(sum([i*i for i in left]))
angle = math.atan(float(top[1])/top[0]) * 180 / math.pi if top[0] != 0 else (970 if top[1] >0 else -90)
return CascadeResult(([x,y,dx,dy],score), cascade_type, angle)
def __str__(self):
return ''.join([str(x) for x in ['center:',self.center,
',\nx:',self.x,
',\ny:',self.y,
',\ndx:',self.dx,
',\ndy:',self.dy,
',\nscore:',self.score,
',\nangle:',self.angle,
',\ncascade_type:',self.cascade_type,
',\npoints_int:\n',self.points_int
]])
@property
def points(self):
x = self.x
y = self.y
dx = self.dx
dy = self.dy
a = self.angle/180.0*math.pi
dots = np.matrix([[x,y,1],[x+dx,y,1],[x+dx,y+dy,1],[x,y+dy,1]])
dots = dots.transpose()
rot_mat = cv2.getRotationMatrix2D((dots[0,0],dots[1,0]), -self.angle, scale = 1.0)
points = rot_mat * dots
points = points.transpose()
return points
@property
def center(self):
return tuple(int(x) for x in (self.points.sum(0)/4.0).tolist()[0])
@property
def points_int(self):
return self.points.astype(int)
@property
def score_with_type(self):
if self.cascade_type:
return self.cascade_type + ' ' + str(self.score)
else:
return str(self.score)
@property
def filename_encode(self):
return '_'.join([str(x) for x in ['loct'] + self.cvformat_result[0] + ['ang', int(self.angle),self.cascade_type, self.score]])
@property
def cvformat_coords(self):
if self.angle == 0:
return [int(x) for x in [self.x, self.y, self.dx, self.dy]]
else:
raise Exception('cannot return [x,y,dx,dy] for a box with angle, use cvformat_result() instead')
@property
def cvformat_result(self):
return ([int(x) for x in [self.x, self.y, self.dx, self.dy]], self.score, self.angle)
# @property
# def rot_matrix(self):
# return array([[cos(math.radians(self.angle)), -sin(math.radians(self.angle))],
# [sin(math.radians(self.angle)), cos(math.radians(self.angle))]])
@property
def top_left(self):
return tuple(self.points[0,].tolist()[0])
@property
def top_right(self):
return tuple(self.points[1,].tolist()[0])
@property
def bottom_right(self):
return tuple(self.points[2,].tolist()[0])
@property
def bottom_left(self):
return tuple(self.points[3,].tolist()[0])
@property
def polygon(self):
return Polygon([self.top_left, self.top_right, self.bottom_right, self.bottom_left])
def overlap(self, otherRect):
return float(self.polygon.intersection(otherRect.polygon).area)
@property
def area(self):
return float(self.polygon.area)
def __gt__(self,b):
return self.area>b.area
def __ge__(self,b):
return self.area>=b.area
def __lt__(self,b):
return self.area<b.area
def __le__(self,b):
return self.area<=b.area

View File

@ -0,0 +1,153 @@
'''
Created on May 7, 2014
@author: eran
'''
from adiencealign.common.images import extract_box
import glob
import os
import time
from adiencealign.cascade_detection.cascade_detector import CascadeDetector,\
resolve_boxes, CascadeResult
import cv2
import csv
'''
Created on Dec 18, 2013
@author: eran
'''
'''
Created on Nov 26, 2013
@author: eran
'''
class CascadeFaceFinder(object):
def __init__(self,
min_size = 32,
drawn_target_res = 360*360,
hangles = [0, -22, 22],
langles = [0,-45,-22,22,45],
haar_file = 'haarcascade_frontalface_default.xml',
lbp_file = 'lbpcascade_frontalface.xml'):
'''
finder = CascadeFaceFinder(min_size = 32, drawn_target_res = 360*360, hangles = [0], langles = [0,-45,-22,22,45], parts_threshold = 0)
finder.get_faces_in_folder(input_folder, output_dir, drawn_folder, is_small_drawn)
or
finder.get_faces_in_photo(full_file, output_dir, drawn_folder, is_small_drawn)
'''
self.min_size = (min_size,min_size)
self.drawn_target_res = drawn_target_res
self._hangles = hangles
self._langles = langles
self.recalc_detectors(haar_file, lbp_file)
# self.funnel = FaceFunnel()
@property
def hangles(self):
return self._hangles
@hangles.setter
def hangles(self,hangles):
self._hangles = hangles
self.recalc_detectors()
@property
def langles(self):
return self._langles
@langles.setter
def langles(self,langles):
self._langles = langles
self.recalc_detectors()
def recalc_detectors(self, haar_file, lbp_file):
self.haar_dtct = CascadeDetector(cascade_file = haar_file,
min_size = self.min_size,
min_neighbors = 20,
scale_factor = 1.03,
cascade_type = 'haar',
thr = 0.4,
angles = self.hangles)
self.lbp_dtct = CascadeDetector(cascade_file = lbp_file,
min_size = self.min_size,
min_neighbors = 15,
scale_factor = 1.04,
cascade_type = 'lbp',
thr = 0.4,
angles = self.langles)
def get_faces_list_in_photo(self, img):
if self.hangles:
haar_faces = self.haar_dtct.detectWithAngles(img, resolve = True)
else:
haar_faces = []
lbp_faces = self.lbp_dtct.detectWithAngles(img, resolve = True)
faces = resolve_boxes({'haar':haar_faces, 'lbp':lbp_faces}, min_overlap = 0.6)
return faces
def create_faces_file(self, fname, is_overwrite = False, target_file = None):
'''
Runs facial detection on fname (say a.jpg, or a.png), and creates a results file (a.faces.txt)
target_file - override, and specify a specific target file
is_overwrite - allow overwriting an existing results file
'''
faces = self.get_faces_list_in_photo(cv2.imread(fname))
results_file = fname.rsplit('.',1)[0] + '.faces.txt' if target_file is None else target_file
if os.path.exists(results_file) and not is_overwrite:
print("Warning, faces result file", results_file, "exists")
else:
with open(results_file,'w') as csvfile:
csv_writer = csv.writer(csvfile, delimiter=',')
header = ['x', 'y','dx','dy', 'score', 'angle', 'type']
csv_writer.writerow(header)
for face in faces:
csv_writer.writerow([str(i) for i in [int(face.x), int(face.y), int(face.dx), int(face.dy), face.score, face.angle, face.cascade_type]])
return results_file
def get_sub_images_from_file(self,original_image_file, faces_file):
'''
extracts all the face sub-images from an image file, based on the results in a faces file
returns - the list of face images (numpy arrays)
'''
img = cv2.imread(original_image_file)
faces_reader = csv.reader(open(faces_file))
next(faces_reader) # discard the headings
padded_face_images = []
for line in faces_reader:
x, y, dx, dy, score, angle, cascade_type = line
[x,y,dx,dy,score, angle] = [int(float(i)) for i in [x,y,dx,dy,score, angle]]
face = CascadeResult(([x,y,dx,dy], score), cascade_type, angle)
padded_face, bounding_box_in_padded_face, _, _ = extract_box(img, face, padding_factor = 0.25)
padded_face_images.append(padded_face)
return padded_face_images
def create_sub_images_from_file(self, original_image_file, faces_file, target_folder = None, img_type = 'png'):
'''
reads a faces file, created by "self.create_faces_file" and extracts padded faces from the original image
The faces will be created in the same folder as the faces file, unless specified otherwise by "target_folder"
returns - the list of face files (strings)
'''
target_folder = os.path.split(faces_file)[0] if target_folder is None else target_folder
padded_face_images = self.get_sub_images_from_file(original_image_file, faces_file)
base_image_name = os.path.split(faces_file)[1].split('.')[0]
face_files = []
for n_face, face_img in enumerate(padded_face_images):
face_file = os.path.join(target_folder, base_image_name + '_face_%d.%s' %(n_face, img_type))
cv2.imwrite( face_file , face_img )
face_files.append(face_file)
return face_files

View File

@ -0,0 +1,153 @@
'''
Created on May 7, 2014
@author: eran
'''
from adiencealign.common.images import extract_box
import glob
import os
import time
from adiencealign.cascade_detection.cascade_detector import CascadeDetector,\
resolve_boxes, CascadeResult
import cv2
import csv
'''
Created on Dec 18, 2013
@author: eran
'''
'''
Created on Nov 26, 2013
@author: eran
'''
class CascadeFaceFinder(object):
def __init__(self,
min_size = 32,
drawn_target_res = 360*360,
hangles = [0, -22, 22],
langles = [0,-45,-22,22,45],
haar_file = 'haarcascade_frontalface_default.xml',
lbp_file = 'lbpcascade_frontalface.xml'):
'''
finder = CascadeFaceFinder(min_size = 32, drawn_target_res = 360*360, hangles = [0], langles = [0,-45,-22,22,45], parts_threshold = 0)
finder.get_faces_in_folder(input_folder, output_dir, drawn_folder, is_small_drawn)
or
finder.get_faces_in_photo(full_file, output_dir, drawn_folder, is_small_drawn)
'''
self.min_size = (min_size,min_size)
self.drawn_target_res = drawn_target_res
self._hangles = hangles
self._langles = langles
self.recalc_detectors(haar_file, lbp_file)
# self.funnel = FaceFunnel()
@property
def hangles(self):
return self._hangles
@hangles.setter
def hangles(self,hangles):
self._hangles = hangles
self.recalc_detectors()
@property
def langles(self):
return self._langles
@langles.setter
def langles(self,langles):
self._langles = langles
self.recalc_detectors()
def recalc_detectors(self, haar_file, lbp_file):
self.haar_dtct = CascadeDetector(cascade_file = haar_file,
min_size = self.min_size,
min_neighbors = 20,
scale_factor = 1.03,
cascade_type = 'haar',
thr = 0.4,
angles = self.hangles)
self.lbp_dtct = CascadeDetector(cascade_file = lbp_file,
min_size = self.min_size,
min_neighbors = 15,
scale_factor = 1.04,
cascade_type = 'lbp',
thr = 0.4,
angles = self.langles)
def get_faces_list_in_photo(self, img):
if self.hangles:
haar_faces = self.haar_dtct.detectWithAngles(img, resolve = True)
else:
haar_faces = []
lbp_faces = self.lbp_dtct.detectWithAngles(img, resolve = True)
faces = resolve_boxes({'haar':haar_faces, 'lbp':lbp_faces}, min_overlap = 0.6)
return faces
def create_faces_file(self, fname, is_overwrite = False, target_file = None):
'''
Runs facial detection on fname (say a.jpg, or a.png), and creates a results file (a.faces.txt)
target_file - override, and specify a specific target file
is_overwrite - allow overwriting an existing results file
'''
faces = self.get_faces_list_in_photo(cv2.imread(fname))
results_file = fname.rsplit('.',1)[0] + '.faces.txt' if target_file is None else target_file
if os.path.exists(results_file) and not is_overwrite:
print "Warning, faces result file", results_file, "exists"
else:
with open(results_file,'w') as csvfile:
csv_writer = csv.writer(csvfile, delimiter=',')
header = ['x', 'y','dx','dy', 'score', 'angle', 'type']
csv_writer.writerow(header)
for face in faces:
csv_writer.writerow([str(i) for i in [int(face.x), int(face.y), int(face.dx), int(face.dy), face.score, face.angle, face.cascade_type]])
return results_file
def get_sub_images_from_file(self,original_image_file, faces_file):
'''
extracts all the face sub-images from an image file, based on the results in a faces file
returns - the list of face images (numpy arrays)
'''
img = cv2.imread(original_image_file)
faces_reader = csv.reader(open(faces_file))
faces_reader.next() # discard the headings
padded_face_images = []
for line in faces_reader:
x, y, dx, dy, score, angle, cascade_type = line
[x,y,dx,dy,score, angle] = [int(float(i)) for i in [x,y,dx,dy,score, angle]]
face = CascadeResult(([x,y,dx,dy], score), cascade_type, angle)
padded_face, bounding_box_in_padded_face, _, _ = extract_box(img, face, padding_factor = 0.25)
padded_face_images.append(padded_face)
return padded_face_images
def create_sub_images_from_file(self, original_image_file, faces_file, target_folder = None, img_type = 'png'):
'''
reads a faces file, created by "self.create_faces_file" and extracts padded faces from the original image
The faces will be created in the same folder as the faces file, unless specified otherwise by "target_folder"
returns - the list of face files (strings)
'''
target_folder = os.path.split(faces_file)[0] if target_folder is None else target_folder
padded_face_images = self.get_sub_images_from_file(original_image_file, faces_file)
base_image_name = os.path.split(faces_file)[1].split('.')[0]
face_files = []
for n_face, face_img in enumerate(padded_face_images):
face_file = os.path.join(target_folder, base_image_name + '_face_%d.%s' %(n_face, img_type))
cv2.imwrite( face_file , face_img )
face_files.append(face_file)
return face_files

View File

@ -0,0 +1,33 @@
'''
Created on May 7, 2014
@author: eran
'''
import cv2
from adiencealign.cascade_detection.cascade_detector import CascadeResult
import numpy as np
def draw_rect(img, r, angle = 0, color=(255,255,255), thickness = 4, alpha = 0.5):
'''
accepts:
1. a (x,y,dx,dy) list
4. a [(x,y,dx,dy),score] list, as returned by cv2.CascadeClassifier.detectMultiScaleWithScores()
5. a CascadeResult object
'''
if type(r) == CascadeResult:
color = tuple(list(color) + [alpha])
cv2.polylines(img, pts = [r.points_int], isClosed = True, color = color, thickness = thickness)
return
elif len(r)==4 or len(r)==2: # [x,y,dx,dy]
if len(r)==2:
if len(r[0]) == 4:
r = r[0]
else:
raise Exception("bad input to draw_rect...")
pt1 = int(round(r[0])), int(round(r[1]))
pt2 = int(round(r[0]+r[2])), int(round(r[1]+r[3]))
color = tuple(list(color) + [alpha])
cv2.rectangle(img, pt1, pt2, color, thickness = thickness)
else:
raise Exception("bad input to draw_rect...")
return

View File

@ -0,0 +1,22 @@
'''
Created on May 7, 2014
@author: eran
'''
import os
import shutil
from os.path import expanduser
def make_path(path, delete_content_if_exists = False):
if not os.path.exists(path):
os.makedirs(path)
elif delete_content_if_exists:
shutil.rmtree(path)
def expand_path(file_or_path):
if file_or_path.startswith('~/'):
home = expanduser("~")
file_at = os.path.join(home,file_or_path[2:])
return file_at
else:
return file_or_path

View File

@ -0,0 +1,87 @@
'''
Created on May 7, 2014
@author: eran
'''
import math
import numpy as np
import cv2
def pad_image_for_rotation(img):
# we pad the image so, when we rotate it, it would never be clipped
rot_y,rot_x = img.shape[:2]
rot_x = int(rot_x / 2.0)
rot_y = int(rot_y / 2.0)
diag = int(math.sqrt(sum([math.pow(x,2) for x in img.shape])))
diag = int(math.ceil(diag / 2.0) * 2.0) # make sure it is even
if len(img.shape) == 3:
big_image = np.zeros((diag, diag, 3), dtype=np.uint8)
else:
big_image = np.zeros((diag, diag), dtype=np.uint8)
x_shift = int(diag/2-rot_x)
y_shift = int(diag/2-rot_y)
# the shift of the old image within big_image
if len(img.shape) == 3:
big_image[y_shift :y_shift+img.shape[0], x_shift:x_shift+img.shape[1], :] = img
else:
big_image[y_shift :y_shift+img.shape[0], x_shift:x_shift+img.shape[1]] = img
# the rotation center is no the radius (half the old image diagonal)
rot_center = diag/2, diag/2
return big_image, x_shift, y_shift, diag, rot_center
def extract_rect(img, rect, factor = 0.2):
(x,y,dx,dy) = rect
new_x = max(0, int(x-dx*factor))
new_y = max(0, int(y-dy*factor))
new_dx = min(int(dx+2*factor*dx), img.shape[1] - new_x)
new_dy = min(int(dy+2*factor*dy), img.shape[0] - new_y)
Dx = x - new_x
Dy = y - new_y
#return [new_x, new_y, new_dx, new_dy]
return img[new_y:new_y+new_dy,new_x:new_x+new_dx,:], Dx, Dy
def extract_box(img, box, padding_factor = 0.2):
'''
we can search for whatever we want in the rotated bordered image,
Any point found can be translated back to the original image by:
1. adding the origins of the bordered area,
2. rotating the point using the inverse rotation matrix
'''
if box.angle != 0:
b_w = max(img.shape)*2
b_h = b_w
dx_center = b_w / 2 - box.center[0]
dy_center = b_h / 2 - box.center[1]
new_img = np.zeros((b_w, b_h, 3), dtype = img.dtype)
new_img[dy_center:(dy_center + img.shape[0]), dx_center:(dx_center + img.shape[1]), :] = img
box_in_big_image = box.points + np.c_[np.ones((4,1)) * dx_center, np.ones((4,1)) * dy_center]
rot_mat = cv2.getRotationMatrix2D((b_w/2, b_h/2), box.angle, scale = 1.0)
inv_rot_mat = cv2.invertAffineTransform(rot_mat)
rot_image = cv2.warpAffine(new_img, rot_mat, (new_img.shape[1],new_img.shape[0]), flags=cv2.INTER_CUBIC)
box_UL_in_rotated = (rot_mat * np.matrix([box_in_big_image[0,0], box_in_big_image[0,1], 1]).transpose()).transpose().tolist()[0]
box_coords_in_rotated = np.matrix(np.c_[box_in_big_image, np.ones((4,1))]) * rot_mat.T
box_coords_in_rotated = box_coords_in_rotated[0,:].tolist()[0] + [box.dx, box.dy]
else:
rot_mat = cv2.getRotationMatrix2D(box.center, box.angle, scale = 1.0)
inv_rot_mat = cv2.invertAffineTransform(rot_mat)
# for efficiency
rot_image = img.copy()
box_UL_in_rotated = (rot_mat * np.matrix([box.points[0,0], box.points[0,1], 1]).transpose()).transpose().tolist()[0]
box_coords_in_rotated = box_UL_in_rotated + [box.dx, box.dy]
img_with_border, Dx, Dy = extract_rect(rot_image, box_coords_in_rotated, padding_factor)
box_coords_in_bordered = [Dx, Dy] + [box.dx, box.dy]
border_UL_in_rotated = [box_UL_in_rotated[0]-Dx, box_UL_in_rotated[1]-Dy]
return img_with_border, box_coords_in_bordered, border_UL_in_rotated, inv_rot_mat

View File

@ -0,0 +1,113 @@
'''
Created on May 7, 2014
@author: eran
'''
import csv
import cv2
from numpy import linalg
import numpy as np
WEIGHTS3 = [11.1088851746,15.8721645013,12.3189439894,15.9467104922,13.9265119716,17.2447706133,11.4118267639,17.0728365324,12.7831886739,17.1908773151,9.6639887492,13.8443342456,8.76890470223,11.4441704453,7.52083144762,10.3245662427,6.35563072919,7.55739887985,6.42340544936,7.48786881875,10.8720924456,8.1349958353,12.3664410374,9.58137800608,6.29390307208,9.47697088783,8.49859202931,9.43946799727,7.92920023102,10.6126442536,10.2953809171,11.299323189,11.1181958685,12.9374719654,12.3764338392,14.7823225327,13.086272904,16.0571795811,15.079169884,17.5936174739,8.39112414861,7.68532826996,8.89386612449,8.70173923211,10.0826620269,8.70286074207,8.13121344224,9.80805203263,7.76044090777,9.2502084627,7.61334683331,10.4813589698,8.64831020289,11.0452512508,9.19528177019,13.0171747152,10.1204323102,14.0189765809,11.0232436734,14.7355286373,12.4881579947,15.4279914333,11.5785971474,16.7942051778