Monthly Archives: February 2015

CW 7 & 8 (9th February – 22nd February) Caffe installation

The main focus for CW 7 was installation of ‘caffe’, which had been chosen as a solution to extract the input image features. The main reason for choosing caffe was because it has been used in one of the projects by Insight, mentioned in a previous post. This gives me opportunity to seek advice and help from people who have gained practical knowledge and experience with caffe along with the online resources available. Another important reason for choosing this framework is because it provides pre-trained models that can be modified to suit specific needs and it also provides a model that I will need to extract image features from the query image.

The installation of caffe is very difficult as it requires many pre-requisites such as CUDA driver and libraries, BLAS, Boost, OpenCV, IO Libraries, Python, Python libraries and Python caffe module and some others. Some versions of these do not work well with others or they need to be configured in different ways. I have been trying to install caffe on Ubuntu in CW7 and I have not managed to do it. I asked Dian for help in CW 8, and after many failed installations and corrected errors, caffe still fails some of the tests after compiling it. In a worst case scenario I would get access to caffe on one of the lab machines in Insight.

While Dian was still trying to install caffe on his own machine (CW 8), with final success at the end of the week (and promised to give it another try on my laptop) I have concentrated on learning Python basics. Caffe is written in C++ and provides two interfaces: Matlab and Python, to access the libraries and we have selected Python as this would be easier for me to learn from basics because of similarity to Java, than Matlab, but also because ‘Annoy’ that I am planning to use for indexing of the dataset, can also be accessed with Python interface.

The past two weeks brought many technical challenges (and there is many more to come) and installation of caffe, which is much more difficult than I would ever expect, have delayed my schedule which I will try to get back on during the next two weeks.


CW 6 (2nd February – 8th February) LIRe or AlexNet? Caffe and Annoy?

This week, after few improvements, I have submitted the Initial Delivery Documentation.

I have also completed the first task from the project timeline, ‘Development of Uploading Interface’, which will be used by the user to upload a query image for which tags will be suggested. This has been done in PHP. I have faced few technical problems with Ubuntu, that I partly fixed myself, and received help from Dian with the rest.

Dian (who I will be mentioning a lot 🙂 ), from Insight Research Centre, is working with the same Flickr dataset, for the EPA project, mentioned in the previous post.

We (as I have no experience in image processing I rely on Dian’s suggestions and advice) have started to look into the image feature extraction. This is one of the biggest tasks and challenges of this project. As the LIRE features are now available we have considered the extraction of LIRE from query image and base the search for similar images in the dataset on some of the low level features provided by LIRE (http://www.lire-project.net/ , http://www.semanticmetadata.net/lire/).

After reviewing the paper on ‘Instance Search and Semantic Indexing’ project from TRECVid, available at http://doras.dcu.ie/20287/ we are considering to use AlexNet features (not yet available for the dataset) and extract these with Caffe, a deep learning framework(http://caffe.berkeleyvision.org/), as it seems to be a better approach in terms of processing time and capabilities it provides than LIRE. I have read some documents to familiarize myself with different aspects such as framework, features, installation etc.

During the meeting with Dian on Wednesday, we have also discussed a possible approach for indexing, which is another big task in this project. Spotify Annoy may be a possible solution. This will be further examined after image feature extraction task is completed.