CS6825

CS6825: Computer Vision

Project 1 Research Tutorial

(points: 260) class reviewed

proposal due:start of class Oct 12 (note: topics/featured articles are first-come first-serve) if time check off in class 20th(40 points)

tutorial due: start of class Nov. 2 (200 points)

review of peer's tutorials due: 11pm, Nov 10 (25 points)

**** You will be using the cs Server

host = csweb01.csueastbay.edu
login/password = your netid login and password
access = via a ssh tool or sftp tool

you will need to create a public_html directory and all webages and content must be inside this directory (or sub-directories under it).

if you have a file called references.html inside the public_html directory you access it by http://csweb01.csueastbay.edu/~YOURNETID/references.html

Your are going to create an interactive Web-based tutorial that you will post for review by your classmates. This project has 3 phases:

Proposal (must be approved by instructor),
Project delivered
Peer Reviews

Proposal Submission Guidelines

Select a topic from the list presented below (these topics will help with our Project 3). Topics are given out on a first come first serve basis. The list of topics is repeated on a discussion board topic "Project 1 Proposals". You MUST look there and not select some topic that has already been claimed via another student's posting. The proposal must contain the following information (please number them as seen here) AND IMPORTANTLY the subject title of your posting should be the title of your topic e.g. "Computer Vision for Automatic Driving".

Title of topic (e.g. "Mobile: Previous work on Sign reading")
Author (your name)
1 paragraph detailing what you are going to talk about (this requires that you have done some research /reading on the subject).
table of contents - break down the subject into sub-topics (each could be a major link on your website). For example, you might have for "Mobile: Previous work on Sign reading"
1. Introduction and Problem Definition
  - what is the problem of mobile sign reading- a person using a mobile camera app is walking around (inside/outside) and the application using still images or video detects signs (what is a sign) and reads the writing (or sybmols ---think bathroom) and conveys this information to the user.
2. Sensor Possibilities
  - describe different sensors that people have used to solve this problem in the past --- i.e. still images from mobile devices or video?
3. Finding and Reading Signs
  -here describe (maybe choose 1 or a few) different techniques people have used to find find signs in an image and then how they might be read
  -give details on image processing algorithms used to accomplish this
4. Success and Failures
  -describe (the 1 or few you are comparing) system(s) success and where / why it (they) failed
5. The Challenges
  -I (meaning you) will give readers some sense of what the greatest challenges are (i.e. speed of user movement in enviornment and quickness of Computer Vision app response, sensor problems with missallignment, noise from enviornment, etc).
  -I (meaning you) will also give the results of an interview with a professor/student doing research in this field (contact one of the authors via email)
6. The Future and Conclusions
  -I (meaning you again) will let readers know about the research labs (CMU, etc) doing research in this area and what they are currently working on (you should send and email and contact professor in charge of lab to ask about it).
At least 2 sources you have read (preferrably online) on the subject and used to create the paragraph.

On-line Tutorial Requirements:

(40 points) Proposal: Adequately given proposal posted on time on discussion board.
(10 points) Format& Ease of Use: You must implement the tutorial as a web tutorial. That means HTML/CSS with supporting media.
(10 points) Mixed-Media: You must have a minimum of 5 pictures or diagrams illustrating the concept you are discussing. As appropriate include video or audio also. If you borrow images, you must reference in the legend where you retrieved them from. You must add voice to each of your individual web pages (not necessary for pages that only show blow-ups of images).
(10 points) Organization: The tutorial should be organized. It SHOULD not look like a long paper simply poped-up on the web. You should have a nice intro page with an easy navigation scheme present throughout the site's pages.
(20 points) Originality: Your writing should be original. You can NOT paraphrase large blocks of text. If you need to paraphrase smaller items, you need to refer to the article in which the text comes from and surround the text with quotes. You should keep to a bare minimum the number of times you paraphrase others words.
(20 points) Length: Someone should be able to complete the tutorial within 30 minutes, and it should minimally take 15 minutes unassisted.
(10 points) Grammar: good grammar
(60 points) Level of Detail: This is a graduate level course and as such, I expect that you will delve into your topic at a sufficient level of detail for the audience to be able to understand how things are implemented or created and with possible reference to other articles (that you quote) be able to recreate or build the system(s) described. Includes Detail -overall (summary of concepts, results given--how well works) , Detail- tech algorithms/system details, Detail -applications discussed. Detail-limitations of system discussed. For example
- If you are discussing a topic that involves the development of software or algorithms, you should present a minimum of one algorithm or piece of code on the topic. You should discuss how the algorithm works and if there are alternatives compare them to others, summarizing with what you think (and others may think) are superior algorithms (and why). Also, give the user an idea when you might apply one technique over another.
- If you are discussing a topic that is hardware-based, you should discuss the various hardware options, the components of the hardware system(s), how the hardware works, the conditions under which the hardware system(s) should be used, the output format of the hardware. Discuss the limitations of the hardware and assumptions made about the working environment. Contrast and compare different hardware system(s) that may accomplish the same or similar (or different if appropriate) goals.
(10 points) Interactivity: You are to incorporate some kind of interactive activity - either a test/ quiz (you don't have to grade it) or some kind of special video of a demonstration (you create) or a slide show (you create) ,etc.
(10 points) Annotated Bibliography: You are required to have links in your on-line (or powerpoint) tutorial to an annotated bibliography. You must have a minimum of 5 different references (please note different pages on the same web-site do NOT count). Throughout your on-line tutorial, each time you wish to quote a reference, it should be represented by a number that is hyper-text linked to the annotated bibliography. In this bibliography page, you will have listed in order of appearance (hence the numbers) your references each followed by an annotation of the contents of the reference. Specifically, you must include not only the title, author, date, source, but also, a sysnopsis, and a reliability rating. For example, for the second reference you would have:
- [2] "Java is Great", Lynne Grewe, PC Weekly, May, 1996., p.g. 21-33.

synopsis: this is a journal article that discusses the benefits Java brings to the programming world and contrasts it with other high-level languages.

reliablity: L. Grewe is an independent researcher and professor. PC Weekly is an popular computer industry magazine that publishes technical articles, mainly to a non-research but, technical audience.

(40 points) Presentation: You need to create a small power point to help you create a YouTube video that is a presentation using BOTH this powerpoint AND your Website Tutorial --that you go over and your voice should be hear clearly as well as any
text in the YouTube video. See Syllbus for tips on how to create and upload from a smart phone. YOU MUST MAKE your YouTube video unlisted and turn in this URL as one of your 2 deliverables --student must be able to access it. YOu can not have
the video be private. Unlisted is a mode where anyone who knows the URL can access it but, does not have to be invited to it...however, it can't be searched in youtube.
- NOTE: this is part of item 13 in google doc

Grading Criteria:

Your grade will be termined in part through peer review. Here is the formula: average of points determined by your peers..

REVIEW - online with shared google doc MUST be invited to share...should do this first day of class
in person. IR is your responsibility to email me before AT LEAST 1 WEEK BEFORE THE due date with your email address to get the invitation.

How to use Google Document FOR EACH STUDENT YOU WANT TO REVIEW

Go to Google Doc, say your name is Lynne Grewe and you are going to review the student "Patel" (last name) work.

Go to the "Patel" tab, and find your row with your last name on it...in this example we go to the Grewe row. NOTE: every person has a different colored row --this is to help avoid you editing the wrong row. So, in this example,

Student Abe, will review Patel by going to Patel tab and ONLY editing the pink/purple cells

Student Grewe, will review Patel by going to Patel tab and ONLY editing the grey cells

Student Grant, will review Patel by going to Patel tab and ONLY editing the greeen cells

Student Zan, will review Patel by going to Patel tab and ONLY editing the yellow cells

************** IF YOU DO NOT UNDERSTAND THIS WE NEED TO MEET ***************

Fill in your review ONLY in your row for Patel's work (this is where you, Grewe in our example, are reviewing Patel's work)

NEVER fill out anything in the RED boxes or in other peoples rows.

How to Turn in/ Deliverables (turn in times are start of class on the due date)

1)PROPOSALS: submit your proposals on the Blackboard discussion board called "BB->Discussion Board-> Project 1 - Proposals". Note that the title of your posting must be the topic of your tutorial i.e. "Mobile: Previous work on Sign reading"

this IS FIRST COME FIRST SERVE....meaning if a prior (by time/date) posting has taken the topic you may NOT re-use that topic.

This means you should post your topic quickly and then go back and update your posting to include the full proposal. You don't want to have spent a lot of time researching a topic to find someone else has already taken it.

2)PROJECTS turn in the URL to your web tutorial AND the YouTube video of you presenting the tutorial. NOTE: many students choose to create an adidtional shorter powerpoint to help with the YouTube presentation (it can be impossible to see webpages in small text) and use the power point AND the web based tutorial both to present in the YouTube video.
UPLOAD URL to YouTube video AND URL to Web tutorial in a SINGLE posting to BB->Discussion Board->Project 1-Final Tutorial

3)PEER REVIEWS: REVIEW OTHERS by looking at their YouTube Presentation & Web Tutorial botth posted on each student's blackboard discussion board "Project 1 - Final Tutorial". Make your reviews for each person on their tab in the google document AND save your reviews locally in case we have a problem.

You will have approximately a week to go through ALL your fellow classmates tutorials and give a review. This review will be available for ALL to see and we will use the online shared google doc you were invited to share. If you did not get an invitation it is YOUR RESPONSIBILITY to email the instructor to ask for an invitation.
Note: if there are 15 students in class you will do 14 evaluation postings (1 for every student except yourself).
IMPORTANT: you must save a local copy on your machine of all the reviews you make of others --why? Sometimes people make mistakes and edit the wrong field in the document and we have to fix this.
IMPORTANT: if you do not evaluate adequately you will loose points on this.

Possible Topics:

IMPORTANT: You can NOT choose a topic that is taken already by someone else --who ever post's first on discussion board their proposal gets that topic.

IMPORTANT: choose a topic ideally related to what app you might develop for our last Project 3.

IMPORTANT: I DO NOT WANT ANYONE FOR PROJECT 3 and 1 to do simple OCR ---that is done and apps out there ---you can apply it to very specific tasks like sign reading etc.

Mobile: Previous work on Cross Walk Guidance/ Navigation
Mobile: Previous work on Door Finding
Mobile: Previous work on use of 2D vision for obstacle detection on mobile phones (here you monitor the environment and tell user audibly when an obstacle is in their path)
Mobile: Previous work on use of 3D vision for obstacle detection on mobile phones (will need to purchase special dual lens recording for 3D data generation and detect obstacles in person way when say walking and warn them audibly)
**** this would require purchasing hardware*****
Mobile: Previous work on use of 3D vision - NON obstacle detection applications (will need to purchase special dual lens recording for 3D data generation and DO something useful for a low vision person with it)
**** this would require purchasing hardware*****
caution: you would need to think of how 3D would benefit a low vision person ---for Project 3 you couldn't just have a 3D display of the 3D input data
Mobile: Previous work on Label Reading -- cans, packages, price tags
Mobile: Previous work on Money Reading - read all US demoninations
Mobile: Previos work indoor navigation -- using vision
Mobile: Previos work on Fashion Matching (help blind match clothes)
Mobile: Previous work on people detection (tell blind person number of people in scene)
Mobile: Previous work on scene understanding - not people detection(by this I mean more specifically describe contents of scene ---this is hard anything can be in a scene you could focus on
indoor furniture like chairs, tables, couches)
Mobile: Previous work on interpertreting facial expressions/emotion (can assist blind in "reading" emotions of others)
Mobile: Previoius work on smile detection --very similar to pevious facial expressions/emotion but, focus only on smile
Mobile: Previous work on augmented reality without headset (visual enhancements with speech controls)
Mobile: Previouis work on use of headsets for augmented reality (visual enhancement etc but, with headset so 3D)**** this would require purchasing headset*****
caution: you would need to think of how 3D would benefit a low vision person ---for Project 3 you couldn't just have a 3D display of data---there has to be a purpuse
Mobile: Previous work on Electronic Sign Reading (like reading airport signs for take off /landing)
Mobile: how can use computer vision to recognize food on plate (for assisting blind, see Purdue U. research on plate monitoring for diet tracking as a hint)
Mobile: Previous work on detecting vehicles in scene
Mobile: Using Mobile vision to assit blind with small vehicle (bike, scooter) navigation (look to some of the mobile vehicle vision based tracking)
Mobile: Previous work on Sign reading (bathrooms, room signs)
Mobile: Previous work on detection Stop and Yield Signs (for when walking)
Computer Vision For Blind: Review of work, see workshop on computer vision applications for the visually impaired
caution -- this is a review and as such may help you in choosing a topic for Project 3 but, may not give enough review of the topic you eventually choose.
Mobile: Picking out ripe fruit or rejected bad fruit using 2D color vision. (idea here would be for project 3 to select say a minimum of 4 fruit that you would assist user with this process).
Mobile: Combining Vision and Haptic (touch) -
caution: would require some significant arduino type work --hardware kit ---
like here suit that buzes in direction on shirt where computer vision senses some dramatic change
like replacing peripheral vision
Mobile: Face Recognition - blind can know who is approaching and phone can tell them who it is
Mobile: Recognizing Agressive actions - (like security applications, hard problem on phone)
Mobile: Assistance with Makeup Application for low vision
caution - is this even a topic???? how could you direct the person to apply makeup even?
Mobile: Magnification App with ZOOM Lens (this would require you purchase a lens and caution ---you would need to control not just display video do something interesting with it) --- t
his is tricky and not so much a research topic but, going towards an app idea ---and maybe too trivial you could not just display the magnified image --but, how could it be
controlled/ used for a low vision person.

=======THE FOLLOWING ARE GENERIC AND NOT APPLICATION ORIENTED=========
Mobile: Previous work on creating apps for 3D mobile Google platform (not really blind focused but, could be???) **** this would require purchasing hardware*****
Mobile: Computer Vision Frameworks specifically for mobile platform (i.e. https://developer.qualcomm.com/mobile-development/add-advanced-features/computer-vision-fastcv, OpenCV on android/ios --how does it perform)
Mobile: Project Tango and how to use and applications (not really focused on blind and needs special hardware but, if someone wants to explore)
Mobile AND WEB Crowd Sourcing: letting say blind take pictures and other people tell what is in them and annotating and delivering in some kind of augemented speech reality --- don't know if this exits.

IMPORTANT: If a topic you is presented in our class materials your treatement of it must go significiantly beyond what we will be discussing