Computer Vision with Python and openCV

Of late I have been obsessed with computer vision. This is in part due to my ambition of creating my own butler and the 3d scanner project. What this led to was a long and extensive study of the mathematics involved behind computer vision.

After some days of searching I discovered the git repository of OpenCV. A wonderful library full of interesting mathematical features and so on. Since there was no simple pip install as is the case with most non-trivial installations, I spent quiet some time building and installing this piece of code.

Once installed I was at a complete loss of knowledge because every possible documentation was for C/C++. I could not find any(partly because I was not using google. I use duckduckgo.) After a while I did find some documentation and it was quiet fun.

About half an hour of understanding the math and finally moving on to the code I began with getting the webcam feed to show up.

import cv,cv2

def get_live_feed():

    window=cv.NamedWindow('live',0)
    #calibrate the camera
    #required to adjust for lighting
    for i in range(10):
        img=cv.QueryFrame(cam)
    #capture and show the feed
    while True:
        img=cv.QueryFrame(cam)
        if img!=0:
            cv.ShowImage('live',img)
        c=cv.WaitKey(10)
        if c==27:break
    cv.DestroyWindow('live')

if __name__=='__main__':
    get_live_feed()

With this I had a live feed working.Now came the part where I had to detect my face in the frames obtained. Hence with a few documentation snippets and code from here and there I had the following.

import cv2
import sys

casc = sys.argv[1]
faceCascade = cv2.CascadeClassifier(casc)

video_capture = cv2.VideoCapture(0)

while True:
    ret, frame = video_capture.read()
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    faces = faceCascade.detectMultiScale(
        gray,
        scaleFactor=1.1,
        minNeighbors=5,
        minSize=(30, 30),
        flags=cv2.cv.CV_HAAR_SCALE_IMAGE
    )

    # Draw a rectangle around the faces
    for (x, y, w, h) in faces:
        cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
    # Display the resulting frame
    cv2.imshow('Video', frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# When everything is done, release the capture
video_capture.release()
cv2.destroyAllWindows()

That led to the following video being created. The vision is still far off from what the Butler must see , I will probably teach it to recognize other objects like keys etc. Also after face detection comes the task of face recognition. Expect a post soon on such a topic.The thing is a little off but works fine generally speaking.

O lovely thing

O lovely thing of god's creation,
how long I have fed my eyes,
yet with every passing sun I,
can not quench this thirst of lies.

Your mirth gives birth,
to a smile which so,
makes my mind forget all dearth,
of love, I have no more.

Leave my side I beg of you,
for you were never here,
yet O godly thing on earth,
the yearning is mine to bear.

I have loved you, or have i not?
from a universe away,
so why do I still feel so near,
while I keep you at bay.

O godly thing, are you not?
a beauty so angelic it shines,
I am none so special
not angel nor demon but
measly human in your eyes.

Courage I have none,
not even the strength,
all i have is the pine.
Is that why you have been aloof,
for I never made you mine?

3D scanning with blender and python

For my physics project this year I arrived on the conclusion that I needed to make a 3d scanner. It was a gutsy move since I knew that if I became committed to this I could not buy this anywhere in the market and would have to build it myself. This burnt all the white flags I may have had and made sure I made my own project. Let me tell you it was horrible. What I had in mind was something awesome, what I obtained was something organic. Graciously Anurag helped me with this herculean task.

First was the problem of the laser. It was too damn expensive. I bought a laser diode from a friend and quickly burnt. What I did not realize was that intensity of the laser depended on the current provided and not on the voltage.

Then came the problem of making a line laser out of a point laser. The first method I stumbled upon was to use a rotating mirror placed in front of the point which would cause a circle of laser light. This failed miserably as the required RPM was not met. Then I came upon the idea of using a cylindrical lens. A glass stirrer cut to size( not cut in prototype 1) was nimbly attached to the front of the laser pointer and lo behold we had a laser line.


Now to tackle the problem of holding my mobile phone upright. This was indeed a messy one. After some time I gave up and replaced it with a friend's DSLR. Problem solved. The camera now sits perfectly on it's own body.

The rotating mechanism was simplicity itself and was simply too easy to build. A little DIY(or jugaad for that matter) and we had a rotating pedestal.

What came next was the mathematics. After digging around a lot I still could not understand exactly how this thing was supposed to work. Then came a moment of truth and everything was a walk in the park. Using blender I managed to extract frames from the video and ended up with about a thousand frames to work with.

Next came the cleaning of the frames. A simple blur, increased contrast and grey scale conversion gave me a very good image of the laser. Then we selected the brightest point in every row of the image and marked it as the laser line.

Next came the reconstruction. With a simple python script I managed to get the cylindrical coordinates of every point in the picture. With the knowledge that the object was rotated 360 degrees and with the assumption that the rate of rotation did not change much I  recreated the scene.

A collection of points was created and saved as a scene. This was then put into blender to create a 3d representation of the object which was very very wrong. What had happened was that the glow of the laser had reflected off the laser and created data points where there should have been none. This created a scan which had a lot of errors.


I later realized that I was calculating angles in degrees and python implicitly(any good program) uses radians. Recalculating the slices led to a new plot which fared a lot better than the previous ones. A lot of the reconstruction was noise but I could make out the nose, and ears of the Buddha statue. It was a magical moment.

Finally Anurag scanned another object, an emergency flashlight. The results were good and funny at the same time. The flashlight had a small volume and so the point cloud was very dense. During the scan the strap attached to the flashlight was also scanned. It was pleasing to note that the scan reconstructed the strap too. Due to the dense point cloud it was not easy to make out the rest of the geometry of the flashlight.

To see the  geometry we moved the point of view to inside the flashlight and could see the objects clearly.

The next problem to be tackled was the problem of mesh regeneration from the point cloud. The problem was that our cloud had non uniform density. This led to some algorithms being discarded. Ball Pivoting Algorithm and Poisson Surface Reconstruction are what got my eye. Will be writing about them soon. All the source code is available on my Github Page.