Overview

Your mission (with a partner) is to build on the Web Audio Visualizer ICE and create a unique interactive audio visualization experience that utilizes the Web Audio and Canvas APIs.

This could be a great portfolio piece for you - so give it your best effort!

The assignment is worth 10% of your final grade, is graded out of 100 points, and no late submissions will be accepted, so post what you have before the due date to receive partial credit. An A grade will be awarded only for meeting the requirements below, and going sufficiently "above and beyond" the what we did in the Audio Visualizer ICE.

Requirements

First, get a partner, and post both of your names to the discussion thread in mycourses. If you like, give your team a name.

1. Usability and overall UX (25%)

• The UI should be intuitive - a typical user will be able to easily use the app without any verbal instructions. Label your widgets clearly, and provide tool tips and text instructions as necessary.
• The user should feel they have sufficient and satisfying control of the app.
• The experience of using the app should be pleasing to the user. Visual and audio effects should not be there just to "meet the requirements", but should make esthetic sense.
• You will be required to implement many user controls of this app - the user will likely be able to manipulate the controls in such a way that the visual experience is not pleasing. So be sure that the default settings of the controls results in an app in a visually pleasing state.

2. Interaction Design (15%)

• The user will be able to choose between at least 2 songs or sound clips. If you have the server space for more sound files, feel free to give the user more choices.
• The user will be able to manipulate at least 1 <select> (besides the sound track chooser), 2 sliders and 3 checkboxes that change aspects of the audio and visual presentation of the app. You may end up needing even more widgets than the minimum requirements - add as many as you need. Radio buttons may be used to replace the select if you wish.
• Giving the user the ability to manipulate the app via mouse interaction (probably by clicking and dragging on the canvas) is encouraged but not required.

3. Canvas API (25%)

• Bitmap info fetched with ctx.getImageData() will be manipulated and displayed to the user in some way using ctx.putImageData(). Ideally the user will be able to control or toggle the effects on and off.
• Audio data (frequency or waveform) must be sampled and used to draw graphics on the canvas:
  • Lines
  • Bezier and Cubic Bezier curves (see SG-2)
  • Circles/Ovals
  • A gradient will be used in at least one fill() or stroke() operation.
  • Rectangles (optional)
• Use context state variables where appropriate such as .strokeStyle, .fillStyle, .lineWidth, .lineCap, .lineJoin, .globalAlpha etc ...
• Use the canvas transforms where appropriate - ctx.translate(), ctx.rotate() and ctx.scale()
• Use "push" and "pop" the canvas drawing state where appropriate. Recall that if your functions manipulate any drawing state variables, it's a good idea to ctx.save() the drawing state at the top of the function, and ctx.restore() the state at the bottom of the function.

4. Web Audio API (15%)

• Give the user the ability to view both the frequency AND the waveform data (not necessarily simultaneously).
• Add at least one audio node (besides the analyzer), and give the user the ability to manipulate it.

5. Media and Presentation and CSS/HTML (10%)

• The sound clips should be in mp3 format and at least 30 seconds in length.
• Embed a web font for use on the HTML portion of the project, and use it in the heading or banner of the page.
• Give the rest of the page an attractive visual design, with good choices made for colors, fonts, and the layout and spacing of elements.
• Using HTML5 semantic tags, on a valid page would be nice.

6. Code (10%)

• No jQuery for anything without prof's prior permission. If you feel you need it for something special, just ask first.
• No other JS libraries may be used without prior approval. I won't automatically say no, but you need to ask first.
• "use strict";
• Everything is in an IIFE - no globals allowed.
• D.R.Y. - Don't Repeat Yourself. Repeated blocks of nearly identical code should be factored out and placed in a separate function.
• Naming conventions: variable and function names must begin with a lowercase letter.
• Every JS statement ends with a semi-colon - don't rely on the JS interpreter to insert them for you.
• Well-commented code. Each and every function gets a comment indicating what it does. Every block of drawing code that produces an effect on the canvas should have additional comments explaining what's going on.
• Delete or comment out your console.log() calls and debugger; statements.
• You will only receive credit for code that you write beyond what you've inherited from our ICEs.
• Any borrowed libraries or code fragments must be credited (author and URL) in both your code comments, and in the final documentation of the project.

7. Penalties (There are no set values for penalties. The more penalties you make, the more points you will lose.)

• There are point deductions for:
  • Runtime errors
  • Poorly written or improper code - single character variables (besides loop iterators and such), unclear variable names, not indenting code, unused variables, using deprecated elements, etc
  • Violating the Academic Honesty Policy (see syllabus) will result in an F in the course and an email to the IGM department chair.

Hints

• The value of the title attribute of an HTML element will be displayed as a tooltip when the mouse hovers over that element.
• Don't know how to embed a web font? View the HTML source of this page, and then head over to Google Fonts for hundreds of font choices.

Ideas

• Your drawing doesn't have to be on a black background like the Audio Visualizer ICE is.
• Certain drawing could go beyond the raw data for the various bins (frequency ranges), it could instead be aggregate data such as average loudness of all frequencies, or changes in the average of certain frequencies, or tied to beat detection.
• Moving sprites (like in SG-2) could change their direction/speed/behavior based on user control or the characteristics of the audio data.
• You can use ctx.scale() to squash or stretch shapes - to create ovals for example.
• The Audio Visualizer ICE redraws the entire canvas every turn. Consider NOT doing so, and allowing drawing to accumulate on the canvas - a moving sprite drawing a trail for example - and then the entire layer either slowly fading or being wiped and started over.
• The above could be implemented on a second canvas like we did in the Paint Part II ICE. This second canvas could also be toggled on and off by the user.
• Not everything has to be drawn at 60 frames/second. Create a global frameCounter variable and do things once a second i.e. if(frameCounter % 60 == 0){...} or at any other interval.
• Not all effects need be user controllable. Consider having certain effects toggle on and off or change every 5 to 10 seconds or so.
• Consider using motion detection or face tracking data to influence your visualization.

Resources

• WebAudio spec
• Simple Audio Viz Examples - using the DOM
• joshondesign.com - Web Audio Viz Example - this example does not wipe the canvas every frame like we do in the Viz ICEs - it gives a different kind of effect.
• www.html5rocks.com - Getting Started with Web Audio API
• A WebAudio Spectrum Analyzer

Deliverables

• Prototype to show in class - see dropbox for due date.
• Final version, posted to both dropbox and web. The location of the live version will be put in the comments field of the dropbox. See dropbox for due date. Documentation (see below) will also be posted to the dropbox.
• ** Both partners must produce code that appears in the final version of the project. Failing to do so will result in a grade penalty for the non-contributing partner. **

Documentation (-10% off of final grade if not completely done)

Both team members will individually submit docs. It's a good idea to document things as you are working on the project. Consider setting up a google doc right away so that you can posts links and other information there as you are working.

• Elucidate how you met the 6 categories of requirements, and be specific about any extras you did - i.e. where you went "above and beyond".
• Discuss what went right and what went wrong. List what features you would have liked to add to the project if you'd had more time/energy.
• Declare any non-course resources (libraries, sounds, images, tutorials, sample code) you utilized, in both your code comments, and in the final documentation of the project. Note: You don't have to document that you used the code I gave you in class.
• Precisely detail the contributions of each partner to the overall project.
• Grade yourself, your partner, and the overall project and justify it. Use a grade of 0-100%.