Prompt Engineering (PROMPT H26)
Practical prompting for images, video, music, sound, and text, with an emphasis on creative experimentation, documented workflows, and critical reflection.
Overview
These pages cover GM’s three sessions. We move from visual generation to multimodal creation, then evaluate whether changes to prompts and workflows improve the results.
Each session lasts 2 h 20 min (140 minutes), including a 10-minute break. All sessions can run online or in person, with the same activities and learning goals. The delivery format will be confirmed before each session.
Sessions
| Session | Topic | Main activity | Materials |
|---|---|---|---|
| 1 | Image & Video Prompting and Workflow Design | Artistic Telephone, storyboard, and visual refinement | Session materials |
| 2 | Prompting for Audio, Music & Multimodal Workflows | PromptVision: visual-to-audio country competition | Session materials |
| 3 | Generative AI Evaluation Clinic | Human and automated evaluation of generated outputs | H26 session materials coming next |
Session 1: Image & Video Prompting and Workflow Design
- Prompt structure and techniques for image and video generation
- Storyboarding, scene descriptions, and visual consistency
- Designing and iterating multi-step image/video workflows
Start with Replicate’s web UI, then use a prepared Jupyter Notebook to record prompts, generate images, and compare iterations. Artistic Telephone makes interpretation drift visible; a storyboard and refinement task introduce deliberate consistency. We will also explore how to turn a still image into a short video. You can try this as an optional extension.
Open Session 1, then start with the Replicate basics notebook. For a structured version of the same API exercise, use Replicate scene prompts. The Artistic Telephone notebook adds iteration, comparison and selected-image passing. At the end of the game, we will compare the image chains and reveal their prompts. See the game guide and Jupyter setup instructions.
Session 2: Prompting for Audio, Music & Multimodal Workflows
- Prompting for music and sound: genre, mood, tempo, voice, and other characteristics
- Combining text, image, audio, and video in multimodal workflows
- Choosing and iterating with the right media and tools for a project
In PromptVision, teams of 2–3 translate a source image into a sonic identity for an imaginary digital country, generate a new cover image and enter a Eurovision-style listening show. We use browser tools, a minimal automatic local presentation and ScoreWiz voting; no student notebook is required. See the activity guide.
Session 3: Generative AI Evaluation Clinic
- Evaluation approaches across text, image, audio, and video generation
- Hands-on evaluation of text outputs, with image evaluation as a secondary focus
- Human feedback and simple automated evaluation to compare and improve prompts/workflows
We will compare Arena rankings with our own ratings, try a prepared Ragas text-evaluation exercise, and explore an ImageReward example. We will also discuss what can be evaluated in audio and video. Session materials will be available here.
Competencies
Professional
- Design effective prompts for diverse AI tools and contexts.
- Adapt prompts to different models and use cases.
- Understand the limits and appropriate use of generative systems.
- Recognize privacy, security, and intellectual-property risks when using AI.
Methodological
- Test, compare, and refine prompts systematically.
- Document the inputs, outputs, and decisions in a creative workflow.
- Choose evaluation criteria appropriate to the medium and task.
Personal
- Engage critically and creatively with AI tools.
- Communicate transparently about AI-assisted work.
- Reflect on ethical and societal implications.
Working tools
- Replicate: use the web UI and prepared API calls with your individual account.
- Jupyter Notebook: keep prompt versions, generated media, observations, and evaluation results together. Session 1 includes the basics notebook, a scene-prompt version and Artistic Telephone; you edit prompts and run supplied cells. Session 2 uses browser audio tools and a small local instructor presentation generator; no new student notebook is required.
- Model selection: use the model list provided for each activity.
- SWITCHdrive: share media within groups and submit work through the class folders. Folder links will be supplied in class.
Keep API tokens private and out of notebooks, screenshots, and submissions.
Ethics & Reflection
Topics across the sessions include copyright and dataset provenance; bias and representation; disclosure of AI-assisted work; and authorship and responsibility when combining generated media.