Semaine 1 (Week 1) lundi

~ lundi ~

Finished reading Ting-Fan's paper [1].
Noted down some plans for our project on the way. Still developing plans.

• First, need to clean data, so that the problem can become simpler. Data should have no clutter and have distinguishable plate color from food, ideally all white.
• In order to clean data, first need to run some samples so we can know how clean the data needs to be. So need to run the segmentation software.
• Lighting conditions: Ting-Fan had problems with illumination because the heat lamp and fluorescent light actually emitted different colors, that can be hard to adjust to, but for simple darker and lighter brightnesses, it might be doable. I wonder if there is a way to adjust the brightness of all input images to a threshold, so that they all have the same level of brightness, and thus similar color distributions on the histogram? That's something to Google and see
• Color spaces: Ting-Fan didn't use HSV at all. I wonder why? If brightness can be tuened, HSV may be useful, so the hue can be used, and the value can all be tuned to a similar level, maybe?
• Textons: Need to take a look at the textons paper from Ting-Fan's references and see what exactly it is and how it works. Seems like it works like a combination of color and texture, and it works better than color and gradient combined.

Got JSEG to compile and run in Linux [2].

• Download the source of jseg and jpeg-b6 from the URL. Unzip anywhere.
• To set up jpeg-b6 library, use dos2unix command to convert the jseg-b6/configure file to UNIX format, run ./configure to generate the jconfig.h file.
• Change the make file in JSEG directory to point to the jpeg-b6 file path for -L and -I CFLAGS. Then run make in JSEG directory. It should run and create "segdist" executable.
• Run segdist executable on images.
• First image tried with JSEG (original image came from the web):

• Image is ran with default parameters, so very over-segmented. Will need to tune later and find out if this software is suitable.

/* Never mind this doesn't make sense. I DO need to segment out some parts in order to figure out the ingredient!

In using JSEG, the difference between my goal and Ting-Fan's goal is that while he had multiple dishes on a plate and thus needed to segment each dish out from a single plate, I only want to segment out the entire food area from the plate. It could be difficult to segment out a continuous area for dishes with discrete objects, such as sushi. For others, such as spaghetti, it should be easier.

*/

As for the rationale of "segment then classify" or "classify then segment," my choice is clearly the former, because the nature of the project is to figure out the different ingredients on a plate first, and then use the combination of ingredients to classify the plate to a cuisine.

Thus I will be using JSEG to segment out the different ingredients, try to identify each one, and then from the ingredients, classify to a cuisine. So it's a two-step process:

1. Identify ingredients using some kind of features - color, texture, shape, etc.
2. Classify cuisine using a second set of features (, using the identified ingredients as input?).

Final outputs would be country / region names where the cuisine is from, e.g. Japanese, Italian, Mexican, Indian, etc. For each image, the output should be a set of probabilities (higher than a set threshold) of the cuisine, for example, an output could look like:

67.5 Japanese

23.8 Chinese

(7.2 Mexican, percentage lower than threshold of 15% and ommited from output)

Next step TODO:

• Tune JSEG and see how well it works to segment out the entire food area from the plate, instead of over-segmenting
• Further develop project plans and slim down things, using notes from reading Ting-Fan's paper as a reference
• Brain wrinkling: brightness tuning, textons
• Could look up if useful: LoG, DoG, whatever else in paper

[1] Wu, Ting-Fan. Cafeteria Vision: Identification and Amount Measurement of Foods in a Plate. http://cseweb.ucsd.edu/classes/fa06/cse252c/projects/twu.pdf

[2] Y. Deng and b. s. Manjunath. Unsupervised segmentation of color-texture regions in images and
video. IEEE Trans. Pattern Anal. Mach. Intell., 23(8):800–810, 2001.
 http://vision.ece.ucsb.edu/segmentation/jseg/software/