Reading

• MV- Chapter 1

Class Lecture, Materials

• Introduction lecture
• Names
• Applications
• Illusions
• Server Account
• Unix Tips
  
• Peer Evaluations Using a Google Shared Document: Flash Movie (see details when used in your project descriptions)

1

2
Our Visual System

Reading

• CV-chapter 1, cameras (local copy),
• (recommended/ not required CVA-p.g. 12-15)
  

Class Lecture, Materials

• Human Abilities lecture
• The Human Visual System
• Illumination. CVA- 2.6

3
Creating Images

Reading

• MV- chapter 25
  

Class Lecture, Materials

• Image formation lecture 1
• Image formation lecture 2
• Sampling and Quantization
• Bits and Bytes
• Lenses

1

4
Image/Video Formats

Reading

• OPTIONAL Video: Chapter 2 of "Practical Image Processing in C" by C. Lindley
• OPTIONAL Image Formats: p.g. 185-187,214-240 of "Practical Image Processing in C" by C. Lindley

Class Lecture, Materials

• image formats
  • image header example
• video standards
• display types

1

2

5
Simple Image Operations

Reading 

• MV- Chapter 2

Class Lecture, Materials

• Point-based Processing Lecture
• Point Operators
• Histogram
• Binary Images

2

6
Software

Class Lecture, Materials

• Arrays and Indexing (or see specific data structures/classes used in any package your plan to employ)
• java

2

7
Probability

Reading

• CV-chapter 7, probability (local copy)

Class Lecture, Materials

• probability basics
• distribution basics
• more basics
• more yet: MSE and crosscorrelation

2

8
Histograms, Equalizaton

Reading

• MV- Chapter 4 (thresholding)

Class Lecture, Materials

• Lecture on Histograms and Equalization
• Histograms
• Histogram Equalization
  

2

Lecture: Module A4 (see below)
ASSIGN: Exercise A2

READ: IP -chapter 1 and chapter 2 (only creating our application)

Recommended : on Safari index on library.csueastbay.edu read chapter 2 of "OpenCV Android Programming By Example", by A. Mohammad - covers histograms and histogram equalization

3

9
Color

Reading

• CV-chapter 4, color (local copy)
• (recommended/not required CVA-p.g.139-148)
• OPTIONAL: Opencv reading from safari book(Android Application Programming with OpenCV 3) read chapter 3, section "mixing color channels"

Class Lecture, Materials

• Color lecture
• basics
• tutorial

10
Geometric Relationships:
Scaling/Sizing, Rotation, Mirror Images

Reading

• CV-section 6.1, analytical (local copy)
  

Class Lecture, Materials

• simple geometric operations
• basics
• math-tranformations: linear algebra lecture, part1, part2

Week3

11
Area Processing: edges and filters

Reading

• CV-chapter 8, linearfilters (local copy)
• CV-chapter 9, edgedetection (local copy)
• MV- Chapter 3,5
• OPTIONAL CVA-section 9.7 (edge)

Class Lecture, Materials

• Neighborhoods
• Area Filters lecture
• Area Processing

4

12
Image Noise and intro to the frequency domain

Reading

• CV-chapter 8, chapter 10, nonlinearfilts (local copy),
• MV- see part of Chapter 2
• (recommend/ not required Fourier Transform: CVA:section 8.6-8.7 )

Class Lecture, Materials

• frequency:
  • frequency lecture
  • intro
  • fourier
  • sample code
• Noise lecture
• kinds of noise and reductions
  • exercise (not turned in)

4

13
Some fun- Morphing

** you will NOT be assessed on this subject

Class Lecture, Materials

• Gryphon SW:
  • morphing
  • warping
  • local copy of code (windows version)

4

14
Binary Image processing

Reading

• MV: chapter 7,9
• (recommended/not required Binary Image Processing: Morphology, Thinning, Component Features: p.g. 25-38, 40,47, 55-70 of "Machine Vision" by Jain, Kasturi, Schunck. )

Class Lecture, Materials

• binary images and morphology lecture
• binary image: geometric feature lecture
• binary image processing, blob detection, labeling and more
  • geometric features

4

Discuss Project 3

• Discuss Project 3

5

15
Features and Texture

Reading

• Texture: CV-chapter 11, texture (local copy), MV-chapter 8
• Features and hough: MV- Chapter 6, 9,10,11, 12, 13, CV- chapter 17(local copy) ,
• SIFT - local features (scale and rotation invariant) READ IP- Chapter 3
  • ALTERNATIVE that comes part of standard OpenCV is ORB and matching with ORB
• paper comparing different Local features for recognition in different viewing conditions(image transformations) (PAMI 2005)
  • OpenCV and SIFT explained. (in python but, same idea)

Class Lecture, Materials

• texture lecture
• feature detection
• features part 2 (attributes of features --see Davies book, 6.10 -shape description by moments, )
• hough transform
  • exercise (not turned in)

5

More On Project 3

Reading

• Exploring Kinect and Android on Own

Class Lecture, Materials

• Discuss Ideas, Finalize Poject Choice

5

Reading: IP - Chapter 2 (read all of the rest) including Hough&Contours,etc., Chapter 3: SIFT & more

Recommended : on Safari index on library.csueastbay.edu read chapter 3 of "OpenCV Android Programming By Example", by A. Mohammad - covers edges, shapes with Hough transform, AND chapter - covers corner detection and matching features

Week6

16
Segmentation and Fitting

Reading

• CV-chapter 16, clustering (local copy)
• Fitting: CV-chapter 17, fitting (local copy) (chapter 18 for those who are interested...not covered)
• (recommended / not required p.g. 113-125, "A Guided Tour of Computer Vision" by Nalwa. )
• IP: p.g. 42-44 (previously assigned - contours). See also OpenCV and contours

Class Lecture, Materials

• segmentation lecture
• segmentation

6,7

17
Recognition

Reading

• Model-Based: CV-chapter 20, csppose (local copy)
• Statistical Classifiers: CV-21.0 to 21.3, classifiers (local copy), MV-chapter 14, CVA-section 15.1-15.3 (not good),
• Neural Networks: CV-21.4, classifiers (local copy)
• Tensorflow (using deep neural network):tensorflow.org, tutorial, tutorial 2, on mobile, setup on android, how to create your own android tensorflow model app
• Google Cloud Vision API & Google Cloud Machine Learning (related to tensorflow)
• Applications and more: MV- Chapter 20-23
• Generalized Hough Transform and template matching: MV- Chapter 13 (previously assigned)
• With OPENCV: IP-Chapter 3,4

Class Lecture, Materials

• Intro
• Recognition Algorithms - a sampling
  • Statistical Classification
  • Hashing
  • Neural Networks lecture
  • Eigenspaces
  • Optional: Hidden Markov Models
  • Bag of Words (based on Text retrieval ideas)
    • paper on topic
  • Nearest Neighbor
  • BoostingColor
• Applications lecture
• Deep Learning and CNN:
  • lecture
  • deep learning Video
  • CNN
  
  
• Google Cloud Vision API - coding: VIDEO part 1, VIDEO part 2
  
  
  
• Tensorflow resources
  
  • VIDEO -explaining the Android Tensorflow example found here
    
    
    
• GETTING started on Project 3 with Google Cloud Vision OR Tensorflow
  
  
• intro, more (web)
  • graph matching
  • interpretation tree search
    • demo
  • scene labeling
  • neural networks
  • class on CNN (convolutional NN)

Exercise: A4 (see below)

6

More On Project 3

LAB Work and At Home Work

6

Vision as Backend Process

Cloud and Vision

• OpenCV deployed with Docker onto Google Cloud (IaaS)
• For PaaS that don't include OpenCV support directly
  • OpenCV Java bindings packaged with native libraries, delivered as a turn-key Maven dependency -- good if instead of building a project for each platform you provide the OpenCV binaries for inclusion as a dependency in Maven projects.
• OpenCV on AWS Lambda with Python deploy script (Paas)
• OpenCV on AWS Lambda with NodeJS (PaaS)
• Google Vision Library in cloud (specific tasks like ocr, object recognition, face recognition, label detection, not as image processing rich as OpenCV)
• Google Cloud Machine Learning

6

Reading: IP - Chapter 3 Matching Featuers, Chapter 4 - Cascade filters(HOG), Chapter Chapter 7 :OCR with KNN, OCR with Support Vector Machines (SVM)
NOTE: there is an OCRTesseract class that does OCR relatively well that I recommend using if OCR is not a main topic you want to alter do research on. This chapter's reading is more about how to do KNN and SVM

Recommended : on Safari index on library.csueastbay.edu read chapter 6 of "OpenCV Android Programming By Example", by A. Mohammad - covers cascade classifier (using Haar-like featuers and adaptive boosting)

Week7

18
Data Structures in Vision

Reading - none

Class Lecture, Materials

• structures

7

19
Motion

Reading

• Motion: MV- Chapter 19, (recommended/not required - p.g. 243-275 of "A Guided Tour of Computer Vision" by Nalwa.) ,
• Tracking- an application of motion detection: CV-19.6 only, tracking (local copy),
• Applications: CVA-p.g. 347-349 (applications)

Class Lecture, Materials

• Motion Part 1 lecture
• Part 2 lecture
• general
  • differential
  • advanced: CVA-section 10.2-10.3 optical flow

7

7

More On Project 3

LAB Work and At Home Work

Week8

20
Image Databases

Reading

• Digital Libraries: CV-Chapter 25 , diglib (local copy )

Class Lecture, Materials

• about

8

21
3D Imaging

Reading

• Stereo vision: CV--chapter 12, (local copy), chapter 13 stereo (local copy),
• 3D Imaging: CV- chapter 24 range (local copy), MV- Chapter 15, (recommended /not required CVA- Chapter 7)
• advanced (not tested): Depth-from-Focus: CVA-section 11.3, Depth from motion with OpenCV (non android version)
• other: (recommended/not required Chapter 13 p.g. 299-301, 316-322 of "Robot Vision" by Horn ;  CVA-section 11.1-11.2Chapter 11 of "Machine Vision" by Jain,et. al.)

Class Lecture, Materials

• sensing
• stereo vision
• shape from shading
• reconstruction

8

More On Project 3

LAB Work and At Home Work

Week9

22
Wavelets

Reading

Class Lecture, Materials

• basics

9

23
Compression

Reading

Class Lecture, Materials

• introduction lecture
• basics

9

24 Other topics - Remote Sensing, Robot Vision, Navigation, Surveillance, etc.

Reading

• MV- previously assigned chapters 20-23

Class Lecture, Materials

• about and index

Android *****NOTE: Many of the links will take you to CS4521 website*****

• Beginning Android and How to create an Android Project
  • LECTURE: Intro
    • asPowerPoint alone,    as MP4 Video OR as YouTube
      
      
  • LECTURE: START Programming: (HOW to Create a Project in Android Studio)
    • as Powerpoint, , as MP4 Video OR as YouTube

NEW***Demo - overview of using Android Studio: MP4 Video OR as YouTube

Emulator and AVD Manager and Running on a Real Device

• Android Emulator, AVD Manager and how to creat an AVD
  • AVD Sample Configs (give me your favorite to add)
  • can take a long time to launch emulator ---snapshots?
• How to Debug/Run on a Physical (real) Device
  • Install APK on Physical Device video demo (from previous student --mpg --download and play on machine)

SDK Manager

• Android SDK manager
  
  

Loading Existing APK, Pushing/Pulling Files

• Loading Existing Android Project (what you need and how to do it)
• Loading Existing APK (application)
• Pulling/Pusing File from/to Device in Emulator

• Exercise A1:(30 points) Install on your laptop the AndroidStudio IDE and Android SDKs, Download and run the Hello World code. Turn in to Blackboard->Exercises->Exercise A1 screen shots of running on BOTH emulator and a real Android Physical device DUE -before start of class Sept 28
  

Useful Apps / Tools (including important SCREEN RECORDER)

:

Android

• LECTURE: more ideas and review of Android Activity
  • as Powerpoint alone, as MP4, as YouTube

A.3 Android and event handling and intents

Android

• LECTURE: Android Interface
• as Power point, as MP4 , as YouTube

YOUR APPS MUST use OpenCV in them

Android & making A Computer Vision App --the options

*** OPTIONS With and Without OpenCV ( --you must use OpenCV option)

Android & OpenCV apps <<< this is the option we are using

VIDEOS:

1) Discussion of OpenCV w/ Android setup and code MP4 YouTube
2) Demo OpenCVAndroid proj. Create from Scratch:MP4 YouTube
3) Demo - overview of using Android Studio: MP4 YouTube

• Writing your Own pixel manipulation algorithms and Speed issues
  

• The OpenCV Manager

• Create Project OpenCV Android & Code Explained

**** please use AndroidStudio and NOT Eclipse as your IDE ******

SAMPLE ZIP FILE OF Android Studio Beginning OpenCV Application (uses Android SDK 23 and OpenCV 3.1.0)

Example Extended (adding a spinner and image processing) EXPLAINED

• OpenCV Tips
  
  
• Special Note on running on Device
  
  
• Incorporating NonFree Module (opencv3* has non-open/free modules , some free for research)
• word document showing steps for OpenCV3.*
• NOTE: ALTERNATIVE for SIFT/SURF (not free) that comes part of standard OpenCV is ORB and matching with ORB
  
  
  
• Setup for using Tesseract (OCR) with OpenCV
  
  
• Samples Downloaded with OpenCV: http://opencv.org/platforms/android/opencv4android-samples.html

• optional: consider running Sample – image-manipulations – this example demonstrates how OpenCV can be used as an image processing and manipulation library. It does operations like our Project 2.

• OpenCV API for version 3.1 (if you are using a different version search for api online)
  
  

3rd party OpenCV videos: (some desktop not android examples)

• object tracking
• hand recognition
• face recognition
• face detection
• image processing and another image processing
• SIFT operator
• OCR
• Sodoku Solver
• Object Recognition
• with a "robot"
  
• for the blind: color detector reading signs money reading
  crowd sourcing + "be my eyes" everything
• many more ---search "OpenCV Android XXX"
  

Android & Camera without OpenCV --- not using this

• LECTURE: Android using the Camera lecture WITHOUT OpenCV, as MP4, as YouTube
  
  
• Camera Issues and Fake Image
• Camera api 2
• READ latest on developer
  • Basics Overview
  • Building Your Own Camera App
  • Newer android.hardware.camera2 (only added API 21) and tutorial on camera2 api
• Web pages:
  • Example Using Camera (android 4.0 or greater)
  • Example Using Camera -built into App
  • Example Using Camera 2 api (api 21 or greater)

Exercises

• Exercise A2(30 points): Also follow these instructions to create a project with OpenCV on Android.. Turn in to Blackboard->Exercises->Exercise A2 screen shots of running on BOTH emulator and a real Android Physical device DUE -start of class Oct. 5
  
  
• Exercise A3(30 points): Modify your A3 using extended sample code here AND by following instructions in Chapter 2 &3 of IP book and create app to do some image processing as shown in the chapters using OpenCV classes and methods and demo. Note instead of doing it on an image from a file do it inside the onCameraFrame method for every frame taken by the camera.. Turn in to Blackboard->Exercises->Exercise A3 screen shots of running on BOTH emulator and a real Android Physical device DUE - start of class Oct. 12 You must alter spinner in extended sample to contain choices of the following that like the extended sample code is done on LIVE VIDEO (no need to load from files as shown in book):
• Threshold (already done),
• Mean Blur (chapter 1),
• Gaussian Blur (chapter 1)
• Dialation (chapter 1)
• Erosion (chapter 1)
• Adaptive Thresholding(chapter 1)
• Difference of Gaussian(chapter 2)
• Canny Edge (chapter 2)
• Sobel Edge (chapter 2)
• Corner Detection (chapter 2)
• Hough Line Transform (chapter 2) and OPTIONAL Safari book (OpenCV Android Programming By Example) read Chapter 3 (section detecting shapes)
• Exercise A4 (60 points): Choose one of the following depending on which you choose for your Project 3 - either Google Vision API or Tensorflow. You work on this withyour project 3 groupDue Nov. 14. Get the code to work and demo it it in a YouTube Video and upload this URL to BB->Exerxices->A4
  • GOOGLE VISION OPTION -In small groups, search and find off google an android application that uses Google TensorFlow (you will need to download code for this --see lecture materials on TensorFlow) and uses a PRE-EXISTING trainned CNN. Get the code to work and demo it either in person/office hours or if instructor chooses in a YouTube Video. Note -I have given you this code (see recognition module on the outline)
  • TENSORFLOW OPTION - In small groups, follow instructions here on some NEW CODE ***** I created for you for an Android OpenCV Based application --see video inside to see it working
    • after the TensorFLow Detectors class code that reads
      detector.recognizeImage() immediate write some code that takes the Image Mat and processes it to do edge detection
    • then you should continue on with drawing of the recognition results but, now on top of the edge image.

Demo - Git and Android Studio

• create new github repository and commit from Android Studio
• create new bitbucket repository and commit from Android Studio (if versioning controll not yet on in your project do VCS->Enable Versioning Control prior to what is shown in video)

ASSESSMENTS:

• Assessment and Project Presentations.

• See class Announcement about any materials allowed.

WeekO1

Speech

Class Lecture, Materials

• speech interfaces

Fuzzy Image Processing

Reading

• CVA-Chapter 16

O3

23 Camera Calibration

Reading

• CV-section 6.3-6.4, analytical (local copy) ,
• CVA- Chapter 6

HTML basics

Class Lecture, Materials

• URL and Hostnames
• REVIEW: About WWW browsing, HelperApplications
• FreeSpeech, powerpoint
• HTML intro
  • Design Issues
  • Broswer Safe Color
• Link to CS2020 and CS3520

Kinect

• Kinect for Windows Sensor Overview
• Microsoft Kinect Getting Started (for version 1.8) ( this will reference how to dowload SDK, runtime tools and setup your sensor)
• Visual Studio and Microsoft Visual Studio Online for Code Versioning, Collaboration and Cloud Hosting
• Kinect Developer Site (and SW)
• Developer site (note we will use kinect for windows and 1.8 version of sdk and runtime)
• Tutorials
• Code Samples
• Version 1.8 general info
• Exercises:
  • Ex K1: Download Kinect Studio and Play --- recorder showing color, depth
  • Ex K2: Download Code currently the C# sample called "Skeleton Basics -WPF C#" , see http://www.microsoft.com/en-us/kinectforwindows/develop/sample-code.aspx or if link has changed search ON MICROSOFT page for it. Turn in screen shots of running
    
    • this will perform skeleton tracking and display it. Alter the code to write the Skeleton Tracking ID (you will need to read about Microsoft.Kinect.Skeleton class ---see http://msdn.microsoft.com/en-us/library/dn791989.aspx)