Progress update

It's been a while since I posted an update on my project, so here it is!

So I've finally got the short-throw projector which allows me to project onto the screen at a decent brightness without using mirrors and such! It's amazing how big the image is given such a short throw, in fact the image is too big now and I've had to make the image smaller to get it to fit on the screen. The image is also much brighter than my previous arrangement and I had to turn down the brightness and contrast as it was too bright as default. It looks like the width of shelves I want to have should be all OK with the new projector too.

Short throw projector in the bookcase, just fits!

I've also got hold of a wide-angle webcam from Creative which has an 80 degree viewing angle apparently. This is needed so I can view the whole screen from the webcam inside the bookcase. When placed at the back of the bookcase the webcam can indeed see the entire screen. I had planned to attach the webcam to the underside of the bookcase shelf but I also realised a problem, in that the webcam needs to be at the front of the shelf otherwise the shelf will obscure the view of the screen. Although eventually the shelves will be much narrorwer this still means the webcam won't quite see all of the screen. However, I think it's the best I'm going to get given the bookcase dimensions. The webcam also has correction for the wide angle lens built into the drivers which is quite handy as I was worried this would cause problems!

Upon testing out the webcam with my OpenCV program I found that it wasn't as "good" as the other cheap old webcam was. I assume this is because firstly the webcam can see a lot more and secondly it's a lot more sensitive. Adjusting the settings of the drivers and my program gave some improvements but it was still not as good as previously. After much time fiddling I decided that the card/tape I was using to detect the position of the finger was too dull. I got a few packs of large brightly coloured sticky dots and tried with these instead. Using these the performance seemed much better. There were also now issues due to light reflecting back off the screen from the projected image which hadn't been a problem previously as the projector wasn't as bright. Adjusting the brightness and contrast of the projector helped a bit.

I'm still not very happy with the OpenCV detection as it doesn't seem to be as robust as before and I don't think detecting angle of point is going to be an option, which will be a pain. I'm going to play around with different lighting. I've also implemented some stuff to load and save settings from the OpenCV program which should make adjustments easier as I can build on what I did last time rather than starting from scratch each time.

The webcam duct taped to the back of the bookcase

Since my last post I've also spent a fair amount of time working on my Java code. I've added functionality to look up book information from the Amazon API based on the ISBN number read from an XML file. The Amazon API is massive and it took some time to figure out which bits I needed to use, but it all works fine now. The only thing I need to work on is extracting keywords and reviews but currently it gets author, title, publisher, date and an image all from the Internet. I've also made the Java code more general and tidied it up a bit, although I think that's going to be an ongoing process!

The image the new projector shows; much brighter and no problems with keystone or focus, notice the new GUI and information from Amazon

I also met with a guy from the Bristol Museum a couple of weeks ago, they were very interested in what I am doing and he said they'd be interested in seeing it in action and/or running a demo at their demo case in the museum.

Finally I've also been thinking about the Computer Sciency bits of my project. I think the main questions I need to answer are;

• Where is it best to display the information on screen given that people want to see the physical objects and the digital information. Also, what information should be displayed?
• How do you figure out what the person is pointing at given that they could be pointing at the screen or an object in the bookcase. In both cases the actual position of their finger tip and the angle it is pointing as viewed by the webcam will be dependant not only on what they are pointing at but also on the distance from the screen, as the webcam views the world in persepective. This is exagguarated with a wide angle lens.
  
• Once the problem of what the user is pointing at is solved then I still need to figure out if they are pointing at the screen or a book as there are two layers of information. Moving between these layers could be done in a number of ways, for example, explicitly by having a way to switch between the layers (e.g. a close button) or by using some sort of heuristic.
• How should the user get feedback about what they are pointing at? A cursor on the screen seems a simple solution, but is this too much like a normal computer?
• How can a user select objects? The options here are dwell time, clicking by moving the finger somehow or using a different input method for clicking (voice for example)

I'm sure there will be more questions as time goes on, hopefully next time I'll have some answers too!