CS6825: Computer Vision

Main
Goals
Data Input
Research
Getting Started
Proposal

Implementation
Requirements AND Data Output

Deliverables
Other

Project 3: Mobile Imaging Research

You are going to propose a Mobile Imaging Application that in some way will assist a blind person. It is up to you to think of this application (i.e. an app that tells you what is the salt and what is the pepper shaker).

Think out of the box --but, make it doable (this is hard).....even if you solve the problem paritally I may be happy ---talk to me.

  • salt and peper shaker recognition or how about in coffee shop --cream versus milk, sugar versus substitute?
  • country road walking --- follow the white line on side of road
  • city road walking ---stay on sidewalk
  • Help! Get me out of Here App ---- and entrance or door finding in a room app --- tell user straight ahead to left or right of scene.
  • label reading in a grocery store -- even the start of this ---recognizing OCR characters and then (someone else could use text to speech to read them out).
  • what about a low vision person ---what could you do to help them out ?
  • Traffic Light Crossing App --- here a low vision or blind person comes to intersection without the crossing audio beeps/sound to say it is safe to cross. User can point up camera to intersection and using video stream your app will find the traffic light and tell user to hold phone still and then wait for NEXT green light and tell user to walk. (next green light so not telling user to walk at end of a green light)

      traffic light

  • ????


Application Specifications

Every student will most likely propose a different mobile imaging application. You will submit a proposal that will be approved or the instuctor will make suitable suggestions for improvement (simplication or possibly more capabilities will be required). Because the purpose of the application can widly varry I am only making some general requirements as listed below.

  • You must as part of the program take pictures (one or more) using the phone's camera either automatically or with the assistance of the user.
  • You need to have some kind of visual response (even if temporarily) showing the photo(s) taken.
  • You need to have some kind of visual reponse showing results of your application ( examples not for you to necessarily replicate- web search listing response for product that has the bar code took picture of, could be a movie/slideshow showing pieced together images taken, could be some image processing results, could be some kind of upload feature, could be some kind of security pass, etc).
  • (part of proposal and final project turn in ) You will need to define buisness model in terms of audience (demographics-age, income, location\,tc) and projected cost of application
  • (part of proposal and final project turn in) You will need to define use case diagram and description of your system. This needs to include prototypes of interface as would be seen and experienced by user along with hypothetical results.



    Research:

    1) Ideas - come up with one or at most two ideas that you might like to implement (e.g. iris detector on a mobile device).

    2) 5 papers - for the idea(s) you came up with in #1, find a minimum of 5 papers you are going to post to the Blackboard Discussion board for Project 3 Research Postings.

    3) pick 5 papers posted by someone else and review them and give your opinion of what is going on.

     

    Proposal:

    before beginning this option/project you must submit and proposal which must include the following sections

    • Concept Summary - a few paragraphs on purpose of Android Imaging Application

    • Audience - demographics of intended audience.

    • Application Cost and Projected success - tell me what you will charge for this application (99 cents or 4.99 or what?) and why it will be successful for your audience---why are they going to buy it.

    • Interface Mockups - drawings (hand drawing is okay if readable) of interfaces seen by user as they use the application --- you should have more than one as the application must do something

    • Use Case - Diagram and Description of use of application

    • References - any (ideally online or electronic you can attach) references you used.

    • Image Processing Routines - brief description of what kinds of image processing /algorithms you will do on the one or more images your application will take (possibly with assistance of user). Note that you can not simply display the image taken or say even one that has a single image processing routine done on it (unless you can convince me of some powerful business use for it that does not currently exist)---this is way too trivial of an application. You need to think about the business part and try to come up with an idea that you think will sell.

    Implementation

    Make sure you choose a later (4.0 or on) Android SDK so that you can capter the latest new features. You can also choose to test on later SDK like 3.0 so you get "older" devices too. Note later versions of Android SDK and emulator do not let you capture your laptops camera picture for testing in emulation mode!!!!


 

 

A Few Trending Apps for the Blind / Low Vision

ps designed specifically to help the blind and visually impaired.

1. LookTel Money Reader

itunes imageApple iTunes

The LookTel Money Reader ($1.99) recognizes US currency in standard denominations ($1, $2, $5, $10, $20, $50, and $100 bills) enabling blind and visually impaired people to quickly identify and count bills. Point the iPhone camera at any US bill and LookTel's object recognition technology via VoiceOver tells users the denomination in real time. Best to organize bills before hitting the nightclub; the app doesn't work as well in low light.



2. SayText

itunes imageApple iTunes Store

SayText (free), developed by Norfello Oy, scans text within any image, such as a medical form or restaurant menu, and reads it aloud. Center the document under the iPhone camera and double-tap the "Take Picture" button. Then raise it slowly: a beep indicates that the entire document is in the phone's frame. The app's Optical Character Recognition utility then scans the text. Tap the screen for status updates. Once scanned, swipe right on the screen to hear the document.

3. Color Identifier

itunes imageApple iTunes

GreenGar Studios' Color Identifier ($1.99) uses the iPhone camera to identify and speak color names aloud. Shades identified are specific to the point of annoyance (Paris Daisy, Moon Mist) for some users. The company makes a free app called Color ID Free that sticks to basic colors. Blind people will never wear mix-matched socks or the wrong shirt again. An interesting offshoot is using the app to distinguish shades of sky, enabling one to experience sunsets or gauge possible weather changes.

4. TalkingTag LV

itunes imageApple iTunesTalkingTagâ„¢ LV ($9.99) from TalkingTag enables blind people to label everyday items with special coded stickers. Users scan each sticker with the iPhone camera and record and replay via VoiceOver up to a 1-minute audio message identifying what's being labeled. The app is ideal for organizing a DVD collection, locating boxes during a move, or picking the right jelly jar from the refrigerator. Stickers can be erased and recorded over.

 

7. Navigon MobileNavigator North America

imageAppl iTunes

NAVIGON's MobileNavigator North America ($44.99) transforms the iPhone into a fully functional mobile navigation system that uses the latest NAVTEQ map material. The app offers text-to-speech voice guidance, enhanced pedestrian navigation, a turn-by-turn RouteList, location sharing via email, and a Take Me Home function. It also provides direct access and navigation to iPhone address book contacts. Navigation is automatically resumed after an incoming phone call.

8. Big Clock

imageThe Coding Monkeys

The Coding Monkeys' Big Clock HD app ($0.99) is a must for visually impaired travelers. Just double tap to rotate iPad orientation to landscape view and set it atop a hotel room TV or table. You'll be able to read it with a glance while lying in bed. The clock displays time and date in the region format and language the device is set to. The app prevents devices from auto-locking when displaying the time.

 

 

© Lynne Grewe