Computational Perception Extended

Intro 01

* * *

1. Intro

About this course

Introduction to machine learning

The goal of the module is to explore different applications of ML/AI/DL with a focus on design and art.

We will first learn how neural networks work with simple code examples. From there we will experiment with different techniques of Deep Learning for Computer Vision (image classification, objects detection, pose estimation...) and Generative Deep Learning (DeepDream, style transfer, Pix2Pix / CycleGAN...).

About Me

UX, UI, CODE

Designer and coder. Background in code, worked mostly in design...

Principles

Peer learning

"Peer learning is an educational practice in which students interact with other students to attain educational goals."

Evaluation Criteria

Grading Collaboration, Research, Exploration
Deliverable Presentation + project + documentation
Date Week 51 / 52

1.0 Students intro

1.1 AI/ML/DL

Artificial intelligence (AI) is an area of computer science that emphasizes the creation of intelligent machines that work and react like humans.

Machine Learning at its most basic is the practice of using algorithms to parse data, learn from it, and then make a determination or prediction about something in the world.

--

So rather than hand-coding software routines with a specific set of instructions to accomplish a particular task, the machine is “trained” using large amounts of data and algorithms that give it the ability to learn how to perform the task.

1.2 History

Analytical Engine

“ The Analytical Engine has no pretensions whatever to originate anything. It can do whatever we know how to order it to perform… Its province is to assist us in making available what we are already acquainted with ”

Ada Lovelace (1843)

Brief history of ML

View reference

1.3 The steps of ML

1.4 Different kinds of ML

Supervised Learning

Learning to map input data to known targets (also called annotations),
given a set of examples (often annotated by humans)

Unsupervised Learning

Finding interesting transformations of the input data without the help of any targets

Reinforcement Learning

In reinforcement learning, an "agent" receives information about its environment and learns to pick actions that will maximize some reward.

1.5 What is a neural network?

1.6 What is Deep Learning

Deep learning is a specific subfield of machine learning, a new take on learning representations from data which puts an emphasis on learning successive "layers" of increasingly meaningful representations.

The 'Deep' in Deep Learning

Understanding how deep learning works in three figures

1.7 Learning steps

Step 01: Forward Propagation

View reference

Step 02: Gradient Descent

View reference

Step 03: Back Propagation

View reference

Application of DL

Self driving vehicles
Face tracking / recognition
Medical image interpretation
Speech processing

1.8 Why Now?

2. Maths

Essence of linear algebra

View reference

3. Machine Learning

3.1 Tasks

Image classification

Image regression

Text classification

Sound classification (text to speech)

Object Detection

Pose Detection

Image Segmentation

3.2 Models Architectures

CNN - Convolutional Neural Networks

RNN - Reccurent Neural Networks

LSTM - Long Short Term Memory

GAN - Generative Adversorial Networks

4. Examples

4.1 Personal

Blog

4.2 Others

X Degrees of Separation

The hidden paths through culture

url: https://artsexperiments.withgoogle.com/xdegrees/
built by: Mario Klingemann & Simon Doury
blog post: https://experiments.withgoogle.com/x-degrees-of-separation

Move Mirror

Explore pictures in a fun new way, just by moving around.

url: Move Mirror
built by: Google Creative Lab

Teachable Machine

Teach a machine using your camera, live in the browser - no coding required.

url: Teachable Machine
built by: Google Creative Lab

Neural Doodle

Use a deep neural network to borrow the skills of real artists and turn your two-bit doodles into masterpieces!

url: https://github.com/alexjc/neural-doodle
built by: https://github.com/alexjc
based on: Semantic Style Transfer
blog post: https://nucl.ai/blog/neural-doodles/

Text 2 Image

Text 2 Image translation

url: t2i.cvalenzuelab.com
based on: AttnGan
built by: @c_valenzuelab

Fast photo style

Given a content photo and a style photo, the code can transfer the style of the style photo to the content photo.

url: https://github.com/nvidia/fastphotostyle
tutorial: https://github.com/NVIDIA/FastPhotoStyle/blob/master/TUTORIAL.md
built by: https://github.com/NVIDIA

Everybody Dance Now

Simple method for "do as I do" motion transfer: given a source video of a person dancing the model can transfer that performance to a novel (amateur) target after only a few minutes of the target subject performing standard moves

vid2vid

photorealistic video-to-video translation. It can be used for turning semantic label maps into photo-realistic videos, synthesizing people talking from edge maps, or generating human motions from poses.

url: https://github.com/NVIDIA/vid2vid
project: https://tcwang0509.github.io/vid2vid/
built by: https://github.com/NVIDIA

Voyage in Sentence Space

Imagine a sentence. “I went looking for adventure.” Imagine another one. “I never returned.” Now imagine a sentence gradient between them

url: https://www.robinsloan.com/notes/voyages-in-sentence-space/
built by: Robin Sloan

Writing with the machine

...If I had to offer an extravagant analogy (and I do) I’d say it’s like writing with a deranged but very well-read parrot on your shoulder. Anytime you feel brave enough to ask for a suggestion, you press tab, and...

url: https://www.robinsloan.com/notes/writing-with-the-machine/
built by: Robin Sloan

I should say clearly: I am absolutely 100% not talking about an editor that “writes for you,” whatever that means. The world doesn’t need any more dead-eyed robo-text.

The animating ideas here are augmentation; partnership; call and response.

The goal is not to make writing “easier”; it’s to make it harder.

The goal is not to make the resulting text “better”; it’s to make it different — weirder, with effects maybe not available by other means.

Robin Sloan