Multimedia combines text, images, audio, video and animation in one interactive system. Video calls, streaming platforms, online classes, games and social media are all multimedia applications, and they account for most Internet traffic. The subject covers how each medium is represented digitally, how it is compressed, how it is delivered over networks and how applications are built. This article revisits the five units of the JNTUK course.
Unit 1: Introduction and media fundamentals
What multimedia is
| Medium | Nature |
|---|---|
| Text | Characters and fonts |
| Graphics and images | Still pictures, bitmap (pixels) or vector (shapes) |
| Audio | Sound |
| Video | A sequence of images over time with sound |
| Animation | Generated moving images |
Media are discrete (text, images: no time dimension) or continuous / time-based (audio, video: they must be played at a fixed rate). Multimedia is interactive (users choose their path), uses hypermedia links and has demanding needs for storage, bandwidth and synchronisation.
Digital representation
| Medium | How it is digitised | Size example |
|---|---|---|
| Image | Pixels × colour depth. A 1920 × 1080 image at 24 bits per pixel | about 6 MB uncompressed |
| Audio | Sampling (Nyquist: at least twice the highest frequency), quantisation and encoding. CD audio: 44.1 kHz, 16 bits, stereo | about 1.4 Mbit/s, 10 MB per minute |
| Video | Frames per second × frame size. 1080p at 30 fps, 24 bits | about 1.5 Gbit/s uncompressed |
The huge sizes explain why compression is essential.
Colour and text
Colour models: RGB (screens), CMYK (print), HSV/HSL (intuitive), and YCbCr (luminance and colour difference, used in JPEG and video because the eye is less sensitive to colour detail). Text: character sets from ASCII to Unicode (UTF-8), which covers Telugu, Hindi and all scripts, and the fonts and markup (HTML, XML) used to display it.
Today: the uncompressed numbers above are why a video platform spends so much on encoding, storage and bandwidth. Unicode support is the reason a modern app must handle many languages and scripts correctly.
Unit 2: Compression
Compression removes redundancy to save space and bandwidth. Lossless methods allow exact reconstruction; lossy methods discard details that people barely notice, for much higher compression.
Lossless techniques
| Technique | Idea |
|---|---|
| Run-length encoding (RLE) | Replace repeated values with a count: AAAAAB becomes 5A1B |
| Huffman coding | Shorter codes for frequent symbols |
| Arithmetic coding | Encodes a whole message as one number; closer to the entropy limit |
| Dictionary methods (LZ77, LZ78, LZW) | Replace repeated phrases with references |
| Formats | ZIP, gzip, PNG, GIF |
Lossy image compression: JPEG
| Step | Purpose |
|---|---|
| Colour conversion and subsampling | RGB to YCbCr; store less colour detail |
| 8 × 8 blocks and DCT | The discrete cosine transform converts each block into frequency components |
| Quantisation | Divide by a table and round; drops high-frequency detail (the lossy step; quality setting) |
| Entropy coding | Zig-zag scan, run-length and Huffman coding |
At high compression JPEG shows blocky artefacts. Newer formats: WebP, AVIF, HEIC, and JPEG 2000 (wavelet-based). Vector formats (SVG) store shapes, not pixels.
Audio compression
Based on psychoacoustics: the ear misses very high or low frequencies and sounds that are masked by louder ones nearby. MP3 (MPEG-1 Layer 3), AAC and Opus (used for calls and streaming) use this. FLAC is lossless. Speech codecs (G.711, AMR, Opus) exploit the structure of voice, and VoIP relies on them.
Video compression
| Idea | Meaning |
|---|---|
| Spatial redundancy | Compress each frame as an image (DCT) |
| Temporal redundancy | Neighbouring frames are similar: store the differences |
| Motion estimation and compensation | Find where blocks moved between frames and send only the motion vector and the small residual |
| Frame types | I-frames (full images), P-frames (predicted from past frames), B-frames (predicted from past and future) in a group of pictures (GOP) |
Standards: MPEG-1, MPEG-2 (DVDs and TV), H.264/AVC (the most widespread), H.265/HEVC, VP9, AV1 (royalty-free, more efficient) and H.266/VVC. Each generation roughly halves the bitrate for the same quality at the cost of more computation. A codec is the encoder-decoder pair; a container (MP4, MKV, WebM) packages video, audio and subtitles.
Today:
ffmpegis the standard tool for encoding and converting media. Video platforms encode each upload into several bitrate ladders (240p to 4K), often on GPU or dedicated hardware, so transcoding pipelines are a heavy compute and cost item in the cloud.
Unit 3: Audio, video and animation technology
Audio
| Topic | Key points |
|---|---|
| Sound properties | Frequency (pitch), amplitude (loudness), waveform |
| Formats | WAV and AIFF (uncompressed), MP3, AAC, OGG, FLAC |
| MIDI | Instructions for notes (instrument, pitch, duration), not recorded sound; tiny files |
| Processing | Mixing, filtering, noise reduction, equalisation, speech recognition and synthesis |
| Stereo and surround | Multiple channels for position |
Video
| Topic | Key points |
|---|---|
| Analog TV standards | NTSC, PAL (India uses PAL), SECAM |
| Digital video | Resolution (720p, 1080p, 4K, 8K), frame rate (24, 30, 60), aspect ratio (16:9), interlaced vs progressive scan |
| Capture and editing | Cameras, non-linear editing (Premiere, DaVinci Resolve), effects, colour grading |
| HDR and high frame rate | Better contrast and smooth motion |
Animation
| Type | Description |
|---|---|
| Cel / frame-by-frame | Each frame drawn |
| Keyframe animation | Define key poses and let the computer interpolate (tween) the frames between |
| 2D and 3D | 3D needs modelling, rigging, texturing, lighting and rendering |
| Motion capture | Record real movement to drive a model |
| Procedural and particle effects | Fire, smoke, water by rules |
| Web animation | CSS, SVG, canvas, Lottie |
Animation is typically shown at 24 to 30 frames per second, and rendering a film frame can take hours on a render farm.
Today: MIDI-style “instructions instead of data” lives on in game engines and vector graphics. Render farms and GPU-based rendering run on cloud instances that are spun up for a job and shut down after, a good fit for batch scheduling and spot instances.
Unit 4: Multimedia systems, networking and streaming
Requirements
Multimedia needs high bandwidth, low delay and jitter for real-time media and synchronisation between streams (lip-sync). Quality of Service (QoS) captures these needs in numbers: bandwidth, latency, jitter and loss.
Storage and databases
Large objects are stored on disks, SSDs and object storage (Amazon S3); content is indexed with metadata and searched using content-based retrieval (images by colour or texture, audio by fingerprint). Standards: MPEG-7 (description), MPEG-21. Digital rights management (DRM) and watermarking protect content.
Delivery over networks
| Approach | Behaviour |
|---|---|
| Download | Receive the whole file, then play |
| Progressive download | Play while downloading |
| Streaming | Play as data arrives, without storing the whole file |
| Live streaming and conferencing | Real-time, very low delay |
| Protocol | Role |
|---|---|
| RTP / RTCP | Real-time Transport Protocol: carries media with timestamps and sequence numbers; RTCP reports quality |
| RTSP | Control of streams (play, pause) |
| SIP, H.323 | Call setup for voice and video |
| WebRTC | Real-time browser-to-browser audio, video and data using UDP, with encryption |
| HLS and MPEG-DASH | Adaptive bitrate streaming over HTTP: the video is cut into short segments in several qualities, and the player switches quality according to the network |
| Multicast | One stream to many receivers |
Handling jitter uses a playout buffer; handling loss uses forward error correction or retransmission (rarely, for live media, because late packets are useless).
Content delivery networks
A CDN (CloudFront, Akamai, Cloudflare) caches content at edge servers near users, which cuts latency, load on the origin and cost. Peer-to-peer delivery is a further option.
Today: YouTube, Netflix and JioHotstar use adaptive streaming (HLS or DASH) over HTTPS from CDNs, with several encodings per title. Video calls (Meet, Teams, Zoom) use WebRTC or RTP with echo cancellation and jitter buffers. For a DevOps team, cache hit ratio, origin shielding, segment length, egress cost and playback start time are the key operational numbers.
Unit 5: Authoring, application development and trends
Authoring tools and process
Multimedia authoring combines the media into an interactive product. Tools in the syllabus (Macromedia Director, Flash, Authorware) have given way to Adobe Premiere, After Effects, Photoshop, Figma, Unity, Unreal and web technologies. Authoring metaphors: timeline-based, card or page-based, icon-based flow and scripting.
The development process for a multimedia application:
- Concept and planning: audience, goals, budget.
- Design: storyboards, navigation structure, user interface.
- Asset production: create or collect text, images, audio, video.
- Authoring and integration of the assets.
- Testing: functionality, usability, performance on target devices.
- Delivery and maintenance: web, app store, DVD or kiosk.
Teams include a producer, designers, programmers, video and audio specialists, and writers.
Application development
| Platform | Technology |
|---|---|
| Web | HTML5 <video> and <audio>, Canvas, WebGL, Web Audio API, WebRTC, Media Source Extensions |
| Mobile | Android and iOS media frameworks, ExoPlayer, AVFoundation |
| Desktop and games | Unity, Unreal Engine |
| Back end | FFmpeg pipelines, queue-based transcoding, object storage, CDN, DRM licensing |
Features to design for: accessibility (captions, audio description, keyboard control), localisation, responsive layouts and offline playback. Copyright and licensing (Creative Commons, royalty-free assets) and DRM (Widevine, FairPlay) are legal and business concerns.
Trends
AR/VR and the metaverse, 360° and volumetric video, AI-generated media (images, voice, video) and deepfakes with authenticity and watermarking concerns, AI-based upscaling and noise suppression, cloud gaming, low-latency live streaming and spatial audio.
Today: a video platform’s architecture is an upload service, an object store, a queue of transcoding jobs on autoscaled workers, a packager producing HLS and DASH, a CDN, a licence server for DRM and an analytics pipeline. Building and operating that pipeline is a typical cloud and DevOps workload.
One-minute recap
| Unit | Key idea | You meet it in |
|---|---|---|
| 1. Fundamentals | Discrete vs continuous media, sampling, colour models, huge raw sizes | Unicode handling, storage and bandwidth planning |
| 2. Compression | Lossless vs lossy; JPEG DCT; I, P, B frames; H.264, AV1 | ffmpeg, transcoding pipelines, WebP and AVIF |
| 3. Audio, video, animation | MIDI, frame rates, keyframes, rendering | Render farms, GPU instances, game engines |
| 4. Systems and streaming | QoS, RTP, WebRTC, HLS and DASH, CDNs | Video platforms, video calls, CDN cache metrics |
| 5. Authoring and development | Development process, HTML5 media APIs, DRM, trends | Media back ends, accessibility, AI-generated media |
Next in Engineering Revisited: Software Project Management.
Written by Claude, Anthropic’s AI assistant, as part of Engineering Revisited, a quick refresher of the B.Tech subjects one at a time.