Ben Organ

2008-04-04T16:20:00.003+01:00

Screenshots

As promised here are some screenshots;

And some of what it looks like on the actual screen;

2008-04-02T13:31:00.004+01:00

Progress update

This week I've mostly been working on the business plan part of my project, which is going quite well. I've also found an interesting article on BBC News about humans and computers becoming closer together. It says; "The keyboard, mouse and monitor will increasingly be replaced by more intuitive forms of interaction and display, including tablet computers, speech recognition systems and fingertip-operated surfaces." This is exactly what my project is setting out to achieve, blurring the boundaries between physical and digital.

I've also been working on getting Wiimote input working, along with some general refactoring of the code to make it a bit nicer. I'm using the WiiremoteJ library along with Bluecove and my Macbook (as it has bluetooth built in). Bluetooth is a massive pain and took me ages to get it working nicely.

However, the Wiimote input now works; it picks up the IR light and pressing the button "clicks" on things. I'm not quite sure yet how it will work with finding the angle of point since both the IR light position and the accelerometer are affected by the pointing direction of the controller. I'm going to get some IR LEDs hooked up later this week so hopefully will be able to get a better understanding of how it will work then.

Next tasks are to get the shelves cut and get the internal lighting sorted in the bookcase as this will really allow me to see how everything works together.

I've also knocked up a quick webpage at http://duality.benorgan.co.uk

Some more screenshots should be posted later this week as I've made a few more tweaks to the GUI.

2008-03-24T17:54:00.007Z

Touchpacity is no more...

After an hour of searching through thesauruses, Wikipedia and using the Dotamator I've decided on a new name for my project.

Duality

du·al·i·ty (dū-ăl'ĭ-tē, dyū-) pronunciation
n.
The quality or character of being twofold; dichotomy.

2008-03-23T16:46:00.003Z

GUIness

I've been playing around with my Java GUI today and have made a few aesthetic tweaks. Hope you like it!

2008-03-21T17:36:00.007Z

Javadoc

Javadoc documentation is now available at http://benorgan.co.uk/duality/javadoc/. This will be kept up to date as the project progresses.

2008-03-19T12:52:00.005Z

Progress update

It's been a while since I posted an update on my project, so here it is!

So I've finally got the short-throw projector which allows me to project onto the screen at a decent brightness without using mirrors and such! It's amazing how big the image is given such a short throw, in fact the image is too big now and I've had to make the image smaller to get it to fit on the screen. The image is also much brighter than my previous arrangement and I had to turn down the brightness and contrast as it was too bright as default. It looks like the width of shelves I want to have should be all OK with the new projector too.

Short throw projector in the bookcase, just fits!

I've also got hold of a wide-angle webcam from Creative which has an 80 degree viewing angle apparently. This is needed so I can view the whole screen from the webcam inside the bookcase. When placed at the back of the bookcase the webcam can indeed see the entire screen. I had planned to attach the webcam to the underside of the bookcase shelf but I also realised a problem, in that the webcam needs to be at the front of the shelf otherwise the shelf will obscure the view of the screen. Although eventually the shelves will be much narrorwer this still means the webcam won't quite see all of the screen. However, I think it's the best I'm going to get given the bookcase dimensions. The webcam also has correction for the wide angle lens built into the drivers which is quite handy as I was worried this would cause problems!

Upon testing out the webcam with my OpenCV program I found that it wasn't as "good" as the other cheap old webcam was. I assume this is because firstly the webcam can see a lot more and secondly it's a lot more sensitive. Adjusting the settings of the drivers and my program gave some improvements but it was still not as good as previously. After much time fiddling I decided that the card/tape I was using to detect the position of the finger was too dull. I got a few packs of large brightly coloured sticky dots and tried with these instead. Using these the performance seemed much better. There were also now issues due to light reflecting back off the screen from the projected image which hadn't been a problem previously as the projector wasn't as bright. Adjusting the brightness and contrast of the projector helped a bit.

I'm still not very happy with the OpenCV detection as it doesn't seem to be as robust as before and I don't think detecting angle of point is going to be an option, which will be a pain. I'm going to play around with different lighting. I've also implemented some stuff to load and save settings from the OpenCV program which should make adjustments easier as I can build on what I did last time rather than starting from scratch each time.

The webcam duct taped to the back of the bookcase

Since my last post I've also spent a fair amount of time working on my Java code. I've added functionality to look up book information from the Amazon API based on the ISBN number read from an XML file. The Amazon API is massive and it took some time to figure out which bits I needed to use, but it all works fine now. The only thing I need to work on is extracting keywords and reviews but currently it gets author, title, publisher, date and an image all from the Internet. I've also made the Java code more general and tidied it up a bit, although I think that's going to be an ongoing process!

The image the new projector shows; much brighter and no problems with keystone or focus, notice the new GUI and information from Amazon

I also met with a guy from the Bristol Museum a couple of weeks ago, they were very interested in what I am doing and he said they'd be interested in seeing it in action and/or running a demo at their demo case in the museum.

Finally I've also been thinking about the Computer Sciency bits of my project. I think the main questions I need to answer are;

Where is it best to display the information on screen given that people want to see the physical objects and the digital information. Also, what information should be displayed?
How do you figure out what the person is pointing at given that they could be pointing at the screen or an object in the bookcase. In both cases the actual position of their finger tip and the angle it is pointing as viewed by the webcam will be dependant not only on what they are pointing at but also on the distance from the screen, as the webcam views the world in persepective. This is exagguarated with a wide angle lens.
Once the problem of what the user is pointing at is solved then I still need to figure out if they are pointing at the screen or a book as there are two layers of information. Moving between these layers could be done in a number of ways, for example, explicitly by having a way to switch between the layers (e.g. a close button) or by using some sort of heuristic.
How should the user get feedback about what they are pointing at? A cursor on the screen seems a simple solution, but is this too much like a normal computer?
How can a user select objects? The options here are dwell time, clicking by moving the finger somehow or using a different input method for clicking (voice for example)

I'm sure there will be more questions as time goes on, hopefully next time I'll have some answers too!

2008-03-19T12:46:00.003Z

New Enterprise Competition

As part of the Business Plan component of my project we were asked to submit an entry to the University of Bristol New Enterprise Competition. Being a bit of a business geek I've entered every year I've been at Bristol, but didn't win once.

So I was very surprised when they announced my entry, Touchpacity, as one of the seven finalists this year! Not only do I win £200 as a cash prize but I also get help and advice from some really experienced people, get to go to the Enterprise Dinner in June and have the chance of winning some of the £35,000 prize fund.

I've got to develop a full business plan and presentation for before the dinner in June, but I need to do that anyway for my course so it's all good.

2008-03-07T00:41:00.005Z

Java GUI

Also this week I've been spending a lot of time developing my GUI application in Java. Lots of time spent figuring out how they work again and trying to remember how Listeners work.

Anyway, I now have an application which will read information from an XML file (or other sources once I write the code) and take input from the OpenCV application I knocked up a few weeks ago (or other sources once I write the code). It then uses these to create an on screen representation of the books which can be selected using your finger pointing at a webcam. As there is no notion of "clicking" I am currently using dwell time to select things. I've also made an on-screen keyboard where you can "type" in letters and search for strings within the books and it then highlights all the relevant books on-screen.

A screenshot of the amazing GUIness

Picture of me using the input to select books. Notice my (green clad) finger at the top left in front of the webcam. The image is then processed using OpenCV to detect the position of my fingertip (on the right of the screen) and used to control the cursor on my application (left of the screen).

Next week I'm planning on getting my application working on the screen and spending a lot more time on the input (specifically figuring out if my plan is actually going to work!).

I'm still lacking a name (and not sure about Touchpacity). New ideas include Tiffany and Midas. I will buy anyone who comes up with a name I end up using a pint.

2008-03-07T00:12:00.008Z

More projection...

I spent a bit more time this week experimenting with the projection onto the screen. I managed to find a mirror (which was actually part of a table mat set from a very cheap shop in Wales for £5) which I could use to reflect the projection off. This not only gives me control over where the image goes but also increases the throw of the projector, making the image on the Holoscreen larger.

Hi-tech mirror setup for reflecting the projected image

Image after being reflected off the mirror, with some keystoning this will be about the right size. However, it's only really visible with the lights off due to the low power of the projector

As you can see the mirror makes the image much bigger although it's not very bright. I'm still waiting for another projector (either a standard one or a fancy short-throw one) so in the mean time I bought in my personal projector to see how much difference a bright projector made (although the bulb is past-due to be replaced). I forgot to bring the DVI adapter I needed so unfortunately the image is just of the projector logo, but you get the idea.

As you can see this projector is much brighter, even with the lights on. Certainly bright enough for what I need.

And a close up shot, ignore the lines on the screen; they are due to the DLP projector and you can't see them in real-life

The next thing I wanted to find out was how deep the shelves could be before they restricted the image. Obviously this depends on a number of factors; mostly the angle of projection (determined by the mirror), the depth of the shelves and the height of the lowest shelf. I stuck a bit of card hanging off a shelf at the top of the bookcase at the depth where I want the shelves to be (I'm planning on cutting them). This would then provide an object to obstruct the projection so I could see how big of a problem it would be. If you look closely at the image (apart from my reflection) you can see the cardboard inside dangling down, and then you can see a black area where the card is causing a shadow right at the top of the projected image. As you can see this shouldn't present too much of a problem as the shelves can be fairly low down before interfering with the image.

Pic showing how the shelves might cause problems for the projected image (ignore the reflection of me taking a photo!)

2008-02-25T16:41:00.003Z

Webcams

After much frustration with the PC I'm using for my project I managed to get a webcam hooked up to it. Unfortunately it seems that the webcam when placed at the back of the bookcase can only see about 1/2 of the screen width which is bad news. I'm going to try with a wide angle lens in front of the webcam (although I'll need to correct the distortion) or I might look into getting a wide-angle webcam (such as the Trust WB-6200p).

I still haven't managed to get my OpenCV application running on the computer as its configuration is a bit messed up, hopefully I'll be able to reinstall it soon which should fix things.

I also played around with the webcam whilst projecting onto the screen and discovered another, more fundamental problem. Most of the image picked up is of what's being displayed on the screen rather than through the screen which could present a real problem for my tracking. Using a webcam which has infra-red LEDs helped a little bit (and might be better if reflective tape was put on the finger) but it's not really an ideal solution.

The alternatives are to do the tracking from webcams which are outside of the bookcase (although I think I'll need 2 webcams in this case) or switch to an alternative tracking technology such as the Wiimote idea. Ideas on a postcard please...!

2008-02-25T16:12:00.013Z

Projection

Samsung SP-P300 projector

I spent some time today playing around with projection onto the screen. At the moment I'm using a Samsung SP-P300 projector (shown above) which is amazingly small but the projection isn't very bright (only 24 lumens) and the throw distance (distance required between projector and screen) to get a good sized image is fairly large.

Ideally I want to place the projector inside the bookcase as this will prevent occlusion problems when people use the screen and also the HoloScreen apparently works better when the image is projected from the rear. However, due to the throw distance of the projector putting it inside the bookcase doesn't give an image which fills the entire screen.

With the lights on the image isn't particularly bright and doesn't fill the entire screen

With the lights out the image is much brighter

For now I decided to project from the front of the screen using a throw of around a metre. This fills the screen although the image isn't very bright and also projects onto the back of the bookcase. I also used NVKeystone to adjust the keystone effect caused by the projector being at a 30 degree angle to the screen. Having this feature is really handy, particularly for projectors like the SP-P300 which don't let you do proper keystone adjustment. Pictures comparing front projection with and without keystoning are shown below;

Front projection without (left) and with (right) keystoning

I'm still thinking rear-projection is the way forward so I either need to find a standard projector with a shorter throw, or use something like the NEC WT-610 which only needs a 6.4cm throw distance for a 40" screen (I've spied one of these being used in the Department but they are really expensive), or the alternative is to use some kind of mirror arrangement at the back of the bookcase to get the required image size.

2008-02-22T19:18:00.009Z

Holoscreen is attached!

Yes I know it's Friday night and I should be at the pub but my colleagues are growing anxious at my lack of blogging so I thought I better post something to pacify them.

This week I finally got the DNP Holoscreen attached to the bookcase. It's a temporary bodge at the moment just so I can figure out if / how everything is going to work. I'm hoping to neaten things up and adjust the positioning of the screen so it fits onto the front of the bookcase a bit better next week, but this will do for now. Below are a few pics;

DNP logo and felt covered hinge which attaches the glass to the bookcase

Dodgy cardboard washer and bolt arrangement (this will be sorted out soon!)

View of the bookcase with the glass screen closed (notice the overlap on the right, that will be fixed soon too!)

View of the bookcase with the door open. Only one shelf at the moment. Eventually the shelves will be a bit shallower.

Also this week I've been playing round with OpenCV and managed to detect coloured bits of card which I stuck to my finger (yes I looked crazy when testing it, waving my cardboard covered finger around in mid-air). Using this I can find out the position of my finger in front of a webcam and also the angle (by doing some maths on the two known points). The angle part isn't working very well yet (need to add some Kalman filtering or similar) but the position seems to work pretty well. Managed to knock up a simple Java app which let me move the pointer around the screen using my finger and select boxes by hovering over them for a second. I'm still not sure whether to use this or the Wiimote (or something else), but hopefully testing next week should sort that out.

Picture of my hand with the bits of card on. My OpenCV program looks for the green and blue coloured parts of the image and uses backprojection, thresholding and blob detection to work out the positions of the finger tip and joint. This information is then sent over the network to my Java app.

Finally I've also been finalising the business plan competition entry this week, which is due in on Tuesday.

I'm still looking for a better name than Touchpacity, suggestions would be most welcome.

Anyway, time to stop working and head to the pub!

2008-02-18T19:16:00.007Z

Project progress

So far I've read a shed load of papers and done a lot of thinking and planning.

I've also been playing around with some OpenCV stuff to detect where fingers are pointing but at the moment I'm thinking this might be too complicated and unreliable to use. I also hooked up my mate's Wii controller to my PC by Bluetooth and played around with WiinRemote to control the cursor on the screen. Although this would require users to hold a device it's a lot more accurate and reliable and means I don't have to spend weeks working on a computer vision problem (which isn't really under the remit of my project).

I've also made some progress with the hardware; the bookcase is built and has hinges and I've been playing around with NVKeystone to give the keystone correction I need for the projection onto the screen.

Today I fired up Netbeans and wrote my first few lines of code - it was a bit disconcerting starting from a completely blank screen realising that eventually this will be my project! So far I've got a Java GUI full screen with a custom cursor. Not very exciting, but it's a start!

2008-02-18T19:12:00.003Z

4th Year Project underway

After a long absence from any blog posts (my fault entirely) I have decided to ressurect my blog to chronicle my 4th year project for my Computer Science degree at the University of Bristol.

The basic idea is a using a DNP HoloScreen to provide a touch interface that combines the real and digital worlds. The example I am using is a bookcase which has (physical) books in it and a glass HoloScreen on the front. Obviously you can see "through" the screen, and information is projected "on" to it. In this way users could point at books and at information on the screen and get the best of physical and digital interfaces.

I'm currently trying to think of a good name for it. The best I have so far are; Touchpacity, LucentTouch and SurfaceTouch. Any good suggestions are welcome!

2007-07-13T20:33:00.000+01:00

Get a life?

I've been working at IBM for three weeks now as part of Extreme Blue where we're working on a project in Second Life, the virtual world which has received much media attention of late. I can't go into the details because it's all top secret suffice to say it involves real and virtual world interaction.

Many people are sceptical of virtual worlds such as Second Life, dismissing them as stupid, for "geeks" and of no real value. Get A First Life is a mock of Second Life and, while I was initially sceptical and there may be some elements of truth in what people say, I don't think we should dismiss virtual worlds (or multiverses) altogether.

The rise of virtual worlds and MMORPGs mimics the rise of the early Internet. It is dominated by a number of proprietary browsers, it is slow and error prone, it's been taken up by the "geeks", and it's been criticised by many. Second Life can be seen like the AOL of the early Internet. Even if the technology at the moment doesn't have a really useful purpose and even if people do stuff just because "it's cool" that doesn't mean to say we should dismiss it. The same was true of the early Internet, people did stuff because they could. The really useful applications, like online shopping, email and video conferencing came along later once people had figured out how to use technology. It also has echoes of the telephone network. Alexander Graham Bell constructed phone lines all around the States without knowing what they would be used for. It was only later that others developed phone calls, faxes and eventually the Internet. And where would we be today without all of this?

I guess the message I'm trying to get across is to not be dismissive of virtual worlds, because if things end up as they are looking likely to then this is just the start, and the 3D Internet will become a big part of everyone's lives.

So what is coming next and where will it end up? Who knows. What's fairly certain is that standards will need to be adopted and, as ever, computers will need to get a lot faster. Whatever happens, it's an exciting time and potentially the start of something massive.

2007-06-19T17:38:00.000+01:00

Cyclists

Now this isn't a rant at all cyclists, most of them are excellent and completely sensible. However, what annoys me is cyclists who can't decide if they are a vehicle or a pedestrian. The ones who will ride straight through a red light, ride down the pavement instead of pushing and the ones who ride the wrong way down roads.

The Highway Code clearly states that if you are riding a bike then you are a vehicle and need to abide by the law as such. That means stopping at red lights like cars and not cycling on the pavement. If you are pushing your bike then you are like a pedestrian, and should abide by the law as such. The cyclists who get this confused also seem to be the ones who don't wear helmets and don't have lights at night, and consequently probably the ones which end up in hospital with a broken leg.

Imagine my joy when walking down a footpath today which had been narrowed (due to roadworks) and was barely wide enough for two people and what should come towards me but a cyclist wobbling around. I carried on walking straight, making sure I took up as much of the pavement as possible. As he neared and realised I wasn't going to move he looked up, still wobbling, and said "Excuse me". I glared at him and replied "You aren't supposed to cycle on the pavement" before brushing past him. Hopefully he fell off.

2007-06-19T17:31:00.000+01:00

Cars = fun

I had forgotten just how much fun you can have with cars until I went to the University of Bristol Motor Club weekend.

We went road rallying on Saturday night which involved decoding clues and finding a route around the middle of nowhere and then driving there to arrive at the allocated time. We started in a dark car park in the rain somewhere near Marlborough around 10pm and 7 hours later finished with the sun coming up. Whilst the time allocation gives you an average speed of 30mph, which doesn't seem very fun, by the time you've worked out where to go, got lost and run over an animal you end up having to average 60 down lanes not much wider than the car. We only managed to get properly lost once and ended up coming 3rd out of 5, although two people didn't make it to the end.

The next day after some snatched sleep in a tent we did an autotest. This was supposed to be driving round a slightly damp, muddy field attempting to go between cones and do J-turns and handbrake turns, but actually it ended up with people going very sideways and wheelspinning all over the place. Great fun.

The only problem is that I now have a worrying urge to buy a car...

2007-06-14T01:38:00.000+01:00

Outlook 2007 and Google Calendar

I've wanted to be able to sync up Google Calendar to my Exchange account on Microsoft Outlook for a long time now, to let others check out my calendar and to make it more accessible to myself when I'm not at my desk.

I tried installing Remote Calendars with Outlook 2003 but the toolbar or options dialogue never appeared, so I put it down to the fact I was using an Exchange server and left it. Having recently upgraded to the fancy new Outlook 2007 I decided to give it another try. The dialogue box appeared and let me put all the details in, however, nothing seemed to happen when I'm sure it was supposed to, so I gave up.

After some more Googling I found another couple of programs, SyncMyCal and gSyncit both of which give you the option of a cut down free version, or a paid version with useful features like autosync.

After spending 10 minutes trying to get SyncMyCal working, I gave up - it wasn't working. Thinking maybe this wasn't such a great idea after all I finally installed gSyncit as a last attempt. Amazingly it worked superbly, just as it says on the tin, and only took me a minute or so to set up.

The annoying nag screen and lack of autosync in the free version were frustrating, and although I wouldn't normally buy software like this (especially when Remote Calendars was free, although it didn't work!), I splashed out on the $9.99 for it. It's been well worth it, my only quibble is that when an autosync happens a window pops up and steals focus, but apart from that it is a really useful and well written little app.

2007-06-07T14:40:00.000+01:00

Double standards

Now don't get me wrong, I'm all for equal rights and less prejudice in this world, however, there seems to be a growing amount of double standards with regards to equal rights.

This article (http://news.bbc.co.uk/1/hi/world/asia-pacific/6697469.stm) on BBC News recently explained how an Australian gay pub has banned heterosexual people. Fair enough, straight people were intimidating the customers. However, what were to happen if you were to ban homosexuals from a "straight" pub. There would be outrage, people would moan, there would be a petition against you and you'd probably get taken to court.

Equal rights are all well and good, but only if they really are equal.

2007-06-07T13:38:00.000+01:00

Is Fairtrade all that fair?

I've been thinking about Fairtrade quite a bit recently and I'm not sure it is all that fair really. I've tried to buy Fairtrade tea and fruit where possible but haven't really gone out of my way. Anyway, after some thought and reading around I've decided to no longer buy Fairtrade, here are my reasons.

Whilst Fairtrade pays people involved in the supply chain above average wages this isn't necessarily a good thing in the long term. There are still plenty of other people being paid the standard rate, and paying people differentially for the same work is never a good long term solution. Imagine if suddenly some farmers in this country got paid £1m a year and the consequences of that. Also, increased wages drives up costs overall in the area, meaning food, clothing, etc. will rise in price, and the people still being paid a standard wage will suffer. What is needed is for a uniform increase in wages and more of an effort to improve living conditions.
Artificially fixing prices leads to lack of economic signals to an economy. Usually if the price of something drops then this is a signal to move on to producing something else, without these price signals the whole economy gets confused and as a consequence actually ends up being worse off. Artificially increasing the price people get paid encourages people to start making tea, or coffee, which ends up being over supplied. The non-Fairtrade farmers then end up being even worse off as their produce isn't in demand and their wages aren't fixed.
Fairtrade goods are sold at a higher price (in general) than non-Fairtrade items. Although it seems like this gets passed on down the supply chain in reality only around 10% of the extra you pay goes to the people who "need" it. The extra commission is swallowed up by the supermarkets, importers and transport companies.
The supermarkets clearly use people who buy Fairtrade and organic produce as an identifier for people who are likely to make emotive purchases without considering the price, or price insensitive customers.
90% of Fairtrade's profit goes straight back into promoting their own brand. You have to question just how ethical that really is.

Fairtrade is just a brand, in the same way Nike and Coca Cola are. It's simply a way to generate profit from middle classes by making them feel good about themselves. Far greater good would be achieved in other ways, but they aren't as easy to profit from or don't make people feel good about themselves like boasting about having Fairtrade tea in their cupboard does.

Artificially inflating the price people get paid is not good for an economy. Whilst I don't dispute that help given to Third World countries and people who are "poor" compared to us is a great thing, I believe it needs to be done very carefully to ensure we aren't actually causing more harm than good.

The article on the Fair trade debate at Wikipedia makes some good points.

2007-06-07T13:35:00.000+01:00

Hello everybody

Well hello, this is my very first ever blog post. I've decided to start a blog to let people know what I'm up to, my thoughts and as a place for me to write things I don't get a chance to say anywhere else. Hopefully it will also motivate me to give my personal site an overhaul during the summer.