{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h1>Imports </h1>"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import os as os # for iteration throug directories\n",
"import pandas as pd # for Series and DataFrames\n",
"import cv2 # for OpenCV \n",
"import numpy as np # for arrays\n",
"import time # for time calculations\n",
"from feature_extraction_try import imgCrawl, getClassLabels #for fetching images\n",
"from skimage.feature import hog #calculates HOGs \n",
"from sklearn.cluster import MiniBatchKMeans #for clustering"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h1>HOG computation :</h1>\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this file, we aim to compute a HOG descriptor thas has the same dimension for each image. So we used a bag of word approach : \n",
"<ul>\n",
" <li>Resizing images to a 0mod5 height and width</li>\n",
" <li>Sequencing each image in 5*5 cells</li>\n",
" <li>Computing local histograms on each cell</li>\n",
" <li>Clustering the local histograms</li>\n",
" <li>Binning the local histograms to create our feature</li>\n",
"</ul>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h2>Resizing images : </h2>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We decided to resize images by duplicating the missing pixels on each side (width and height) because it seem to be the less disturbing way for gradient computation. \n",
"Another possible way is to crop each image, but it may delete useful information. \n",
"Returns the resized image as a np.array "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def reSize(image, CELL_DIMENSION):\n",
" height, width, channels = image.shape\n",
" \n",
" if height%CELL_DIMENSION==0 and width%CELL_DIMENSION==0:\n",
" resizedImage = image\n",
" \n",
" elif width%CELL_DIMENSION==0:\n",
" missingPixels = CELL_DIMENSION-height%CELL_DIMENSION\n",
" resizedImage = cv2.copyMakeBorder(image,0,missingPixels,0,\\\n",
" 0,cv2.BORDER_REPLICATE)\n",
" \n",
" elif height%CELL_DIMENSION==0:\n",
" missingPixels = CELL_DIMENSION-width%CELL_DIMENSION\n",
" resizedImage = cv2.copyMakeBorder(image,0,0,0,missingPixels,\\\n",
" cv2.BORDER_REPLICATE)\n",
" \n",
" else:\n",
" missingWidthPixels = CELL_DIMENSION-width%CELL_DIMENSION\n",
" missingHeightPixels = CELL_DIMENSION-height%CELL_DIMENSION\n",
" resizedImage = cv2.copyMakeBorder(image,0,missingHeightPixels,0,\\\n",
" missingWidthPixels,cv2.BORDER_REPLICATE)\n",
" return resizedImage"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h2>Sequencing images : </h2>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, we just tranfrom our corpus in $(5*5)$ cells to be able to execute the bag of words approach. \n",
"\n",
"Returns a (nbImages$*$imageHeight mod 5$*$imageWidth mod 5$*$nbChannels) np.array"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def imageSequencing(npImages, CELL_DIMENSION):\n",
" cells=[]\n",
" \n",
" for k in range(len(npImages)):\n",
" image = cv2.imread(npImages[k][1])\n",
" resizedImage = reSize(image, CELL_DIMENSION)\n",
" height, width, channels = resizedImage.shape\n",
" cells.append(\\\n",
" np.array([\\\n",
" resizedImage[\\\n",
" j*CELL_DIMENSION:j*CELL_DIMENSION+CELL_DIMENSION,\\\n",
" i*CELL_DIMENSION:i*CELL_DIMENSION+CELL_DIMENSION] \\\n",
" for i in range(width/CELL_DIMENSION) \\\n",
" for j in range(height/CELL_DIMENSION)\\\n",
" ])\\\n",
" )\n",
" return np.array(cells) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h2>Compute HOG descriptor on each cell : </h2>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here we convert our images in grayscale toi be able to apply our localHistogram function (doc : http://scikit-image.org/docs/dev/auto_examples/plot_hog.html)\n",
"\n",
"Then we compute local HOGs on our cells with NB_ORIENTATIONS orientations"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def computeLocalHistograms(cells, NB_ORIENTATIONS, CELL_DIMENSION):\n",
" localHistograms = np.array([\\\n",
" np.array([\\\n",
" hog(cv2.cvtColor( cell, \\\n",
" cv2.COLOR_BGR2GRAY), \\\n",
" orientations=NB_ORIENTATIONS, \\\n",
" pixels_per_cell=(CELL_DIMENSION,\\\n",
" CELL_DIMENSION),\\\n",
" cells_per_block=(1,1)) \\\n",
" for cell in image]) \\\n",
" for image in cells])\n",
" return localHistograms"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h2>Clustering local HOGs : </h2>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here we search for NB_CLUSTERS clusters in all our localHistograms to be able to bin later our localHistograms.\n",
"\n",
"doc for MiniBatchKMeans : http://scikit-learn.org/stable/modules/generated/sklearn.cluster.MiniBatchKMeans.html"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def clusterGradients(localHistograms, NB_CLUSTERS, MAXITER):\n",
" sizes = np.array([len(localHistogram) for localHistogram in localHistograms])\n",
" nbImages = len(localHistograms)\n",
" flattenedHogs = np.array([cell for image in localHistograms for cell in image])\n",
" miniBatchKMeans = MiniBatchKMeans(n_clusters=NB_CLUSTERS, max_iter=MAXITER, \\\n",
" compute_labels=True)\n",
" localHistogramLabels = miniBatchKMeans.fit_predict(flattenedHogs)\n",
" return localHistogramLabels, sizes"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h2>Binning local HOGs : </h2>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then we just need to bin our local HOGs to avec a constant-sized feature based on HOG to describe our images"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def makeHistograms(labels, NB_CLUSTERS, sizes):\n",
" indiceInLabels = 0\n",
" hogs = []\n",
" for image in sizes:\n",
" histogram = np.zeros(NB_CLUSTERS)\n",
" for i in range(image):\n",
" histogram[labels[indiceInLabels+i]] += 1\n",
" hogs.append(histogram)\n",
" indiceInLabels+=i \n",
" return np.array(hogs)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h2>Global function</h2>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def extractHOGFeature(npImages, CELL_DIMENSION, NB_ORIENTATIONS, \\\n",
" NB_CLUSTERS, MAXITER):\n",
" cells = imageSequencing(npImages, CELL_DIMENSION)\n",
" localHistograms = computeLocalHistograms(cells)\n",
" localHistogramLabels, sizes = clusterGradients(localHistograms, \\\n",
" NB_CLUSTERS, MAXITER)\n",
" hogs = makeHistograms(localHistogramLabels, NB_CLUSTERS, sizes)\n",
" return hogs"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h1>Test zone</h1>"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"if __name__ == '__main__':\n",
"\n",
"\n",
" start = time.time()\n",
" path ='/home/doob/Dropbox/Marseille/OMIS-Projet/03-jeux-de-donnees/101_ObjectCategories'\n",
" testNpImages = [ [1,'testImage.jpg'] ]\n",
" CELL_DIMENSION = 5\n",
" NB_ORIENTATIONS = 8\n",
" NB_CLUSTERS = 12\n",
" MAXITER = 100\n",
"\n",
" print \"Fetching Images in \" + path\n",
" # get dictionary to link classLabels Text to Integers\n",
" sClassLabels = getClassLabels(path)\n",
" # Get all path from all images inclusive classLabel as Integer\n",
" dfImages = imgCrawl(path, sClassLabels)\n",
" npImages = dfImages.values\n",
" extractedTime = time.time()\n",
" print \"Extracted images in \" + str(extractedTime-start) +'sec'\n",
" print \"Sequencing Images ...\"\n",
" blocks = imageSequencing(testNpImages, 5)\n",
" sequencedTime = time.time()\n",
" print \"Sequenced images in \" + str(sequencedTime-extractedTime) +'sec'\n",
" print \"Computing gradient on each block ...\"\n",
" gradients = computeLocalHistograms(blocks, NB_ORIENTATIONS, CELL_DIMENSION)\n",
" hogedTime = time.time()\n",
" print \"Computed gradients in \" + str(hogedTime - sequencedTime) + 'sec'\n",
" print \"Clustering gradients ...\"\n",
" gradientLabels, sizes = clusterGradients(gradients, NB_CLUSTERS, MAXITER)\n",
" clusteredItme = time.time()\n",
" print \"Clustered gradients in \" + str(hogedTime - sequencedTime) + 'sec'\n",
" print \"Computing histograms ...\"\n",
" histograms = makeHistograms(gradientLabels, NB_CLUSTERS, sizes)\n",
" end = time.time()\n",
" print \"Computed histograms in \" + str(int(end - hogedTime)) + 'sec'\n",
" print \"Histogram shape : \" +str(histograms.shape)\n",
" print \"Total time : \" + str(end-start) + 'sec'\n",
" #hogs = extractHOGFeature(testNpImages, CELL_DIMENSION, \\\n",
" # NB_ORIENTATIONS, NB_CLUSTERS, MAXITER)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.11"
}
},
"nbformat": 4,
"nbformat_minor": 0
}