Computational Perception Extended - Fall 2024

★★★ This repository is published here ★★★

Results

Coming Soon!

Intro

In this module we will explore different applications of ML/AI/DL with a particular focus on design and art. We will first learn how neural networks work with simple code examples, then we will experiment with different techniques of Deep Learning:

DL applied to Computer Vision (image classification, objects detection, pose estimation…)
Generative Deep Learning (LLM, SD, etc…)

Once we get a good grasp of the different techniques, we will experiment further by building our own ‘AI’ project.

Schedule

Week 01 (KW09) |

Students intro
Intro: General
ML / AI / DL: History & Example
Inner working of a Neural Network
Intro to tools: Colab / Jupyter

Week 02 (KW10) |

Communication
ImageGen - Part 01 - Getting started

Week 03 (KW11) |

Step back
ImageGen - Part 02 - What the #@ˆ% are we doing?

Week 04 (KW12) |

Projects - Small talk
ImageGen - Part 03 - Techniques & tools, diving in…

Week 05 (KW13) |

LLM
Platforms, API, libraries

Week 06 (KW14) |

Projects - Sharing

Week 07 (KW15) |

Audio Generation
ML vs HL
Principles

Week 08 (KW16) |

ML from scratch
Model conversion
Other frameworks (JS)
Hardware / local setup (good practice)
Conda, docker, etc…

Week 09 (KW17) |

Open discussions:
- Prompting
- Deploying / Using LLM

Week 10 (KW18) |

Open discussions
Mechanism of learning (slides from KI_Philo class)
Help on projects

Week 11 (KW19)

Week 12 (KW20)

Week 13 (KW21)

Week 14 (KW22)

Presentations, etc…

Evaluation

Evaluation criteria:

Attendance, participation and engagement in others’ projects (1/3)
Research, Documentation (1/3)
Final Project (1/3)

Deliverables:

Presentation / Source code / Documentation

Academic integrity

(Copied from Golan Levin’s 2020 CMU class)

Use of free and open source code

Credit is perhaps the most important form of currency in the economies of commons-based peer production and open-source media arts. You are expected to cite the source of any code you use. Please note the following expectations and guidelines:

Use Libraries. In your Projects, the use of general, reusable libraries is strongly encouraged. The people who developed and contributed these components to the community worked hard, often for no pay; acknowledge them by citing their name and linking to their repository.

Be Careful. It sometimes happens that an artist places the entire source code for their sketch or artwork online, as a resource from which others can learn. The assignments professors give in new-media arts courses are often similar; you may discover the work of a student in some other class or school, who has posted code for a project which responds to a similar assignment. You should probably avoid this code. At the very least, you should be very, very careful about approaching such code for possible re-use. If it is necessary to do so, it is best to extract components that solve a specific technical problem, rather than those parts which operate to create a unique experience. Your challenge, if and/or when you work with others’ code, is to make it your own. It should be clear that forking an artwork from someone’s page on GitHub, Glitch, OpenProcessing, etc., and simply changing the colors would be disgracefully lazy. Doing so without proper citation would be plagiarism.

Informal colaborations

Our course places a very high value on civic responsibility that includes, but is not limited to, helping others learn. In this course, we strongly encourage you to give help (or ask others for help) in using various toolkits, algorithms, libraries, or other facilities. Please note the following expectations:

In this class, it’s OK to give and receive help. In fact, it’s better than OK! But students who receive help from someone else are obliged to acknowledge that person in their project report, clarifying the nature of the help that was received.
We are all teachers. Students with advanced skills are expected to help others, yet refrain from doing another’s work for them. You can usually tell when you’re about to cross the line: Ask yourself whether you are teaching someone to fish, or merely giving them the fish.
When in doubt: give credit to the people who have helped you.

Formal colaborations

The assignments in this course are primarily intended to be executed by individuals. That said, I am in favor of students collaborating if such collaborations arise organically and can be conducted safely. Please note the following expectations:

Use proper social distancing. In light of the ongoing the COVID-19 pandemic, please respect University and other health guidelines regarding personal distance. Do not share computer keyboards, sit at least 6 feet apart, etcetera.
Notify the Professor. It’s helpful for me to know who is working with whom. Students who wish to collaborate should jointly inform the professor as early as possible.
Only pairs. Unless permission is explicitly granted by the Professor, collaborations in this course are restricted to pairs of students.
Describe who did what. Written reports for collaborative projects should describe how your effort was distributed.
Only known collaborators. Your project collaborator, if you have one, must be in this class. You may not collaborate with people from outside the course (e.g. your housemate).
Avoid co-dependency. You may not collaborate with the same person (i.e. submit an assignment jointly) on more than two projects.

COMPP24