Scene 1 (0s)
Multimedia Application Design Presented by : Dr. Oussama DERNI [email protected] 03 & 10 & 17 November 2024 ةــيبعشلا ةــيطارقميدلا ةــيرئازجلا ةــيروهمجلا Algerian Democratic and Popular Republic يملعلا ثحبلاو يلاعلا ميلعتلا ةرازو Ministry of Higher Education and Scientific Research Course CHAPTER 04 University of Relizane Faculty of Science & Technology Computer Science Department.
Scene 2 (13s)
PLAN • Introduction • A Brief History of SMIL • SMIL presentation Basics • Timing and Synchronization • Timing Model Basics • Advanced Timing and Synchronization Attributes • SMIL animation • SMIL conditional elements SMIL (Synchronized Multimedia 01 Integration Language).
Scene 3 (25s)
SMIL (Synchronized Multimedia Integration Language).
Scene 4 (31s)
I n t r o d u c t i o n • SMIL (Synchronized Multimedia Integration Language) is an XML-based language designed to synchronize multimedia presentations. • It enables users to create multimedia documents that include text, audio, video, and animations, which can be synchronized to play in a specific sequence or in parallel. • It is used for educational materials, multimedia presentations, and certain types of digital signage. • It is also used in some legacy systems and applications. 4.
Scene 5 (52s)
A B r i e f H i s t o r y o f S M I L • In 1996, W3C started an activity to determine how audio and video could best be supported in the context of the a-temporal HTML text and image content that was then dominant on the World Wide Web. • This activity, lead by Philipp Hoschka, resulted in the definition of the Synchronized Multimedia (SYMM) working group. • After about nine months of work, the group published the first version of SMIL in June 1998. 5.
Scene 6 (1m 14s)
A B r i e f H i s t o r y o f S M I L SMIL 1.0 introduced the basic syntax and elements required for creating synchronized multimedia presentations. ❑ Basic Structure: Introduced the core elements such as <smil>, <head>, and <body>. ❑ Media Integration: Allowed for simple integration of audio, video, images, and text. ❑ Sequential and Parallel Layouts: Supported <seq> (sequential) and <par> (parallel) elements to control the order of media playback. 6 SMIL 1.0.
Scene 7 (1m 36s)
A B r i e f H i s t o r y o f S M I L SMIL 2.0 (2001) brought significant enhancements and new features, making it more flexible for complex multimedia presentations. ❑ Enhanced Timing and Synchronization: Improved control over media synchronization with more precise timing attributes. ❑ New Elements: Introduced new elements such as <switch> for conditional media playback and <layout> for positioning media. ❑ Styling and Animation: Incorporated basic styling and animation capabilities. ❑ Extensible Markup Language (XML): More extensive use of XML for better integration and extensibility. 7 SMIL 2.0.
Scene 8 (2m 0s)
A B r i e f H i s t o r y o f S M I L SMIL 2.1 (2005) introduced several improvements over SMIL 2.0, including better support for multimedia content and enhancements to timing and layout. ❑ Refined Timing and Synchronization: Improved attributes for managing timing and synchronization of multimedia content. ❑ Enhanced Layout and Regions: Better support for defining regions and layouts for multimedia elements. ❑ Integration with Other Standards: Improved integration with other web standards and technologies. 8 SMIL 2.1.
Scene 9 (2m 21s)
A B r i e f H i s t o r y o f S M I L SMIL 3.0 (2008) introduced a number of new features aimed at enhancing multimedia presentations and making SMIL more robust for modern applications. ❑ Improved Synchronization: Advanced features for synchronizing and sequencing multimedia elements. ❑ New Elements and Attributes: Added new elements such as <animate> for animations and improved support for media control. ❑ Compatibility: Enhanced compatibility with modern web standards and multimedia formats. 9 SMIL 3.0.
Scene 10 (2m 42s)
S M I L p r e s e n t a t i o n B a s i c s • A SMIL presentation is an XML-formatted specification • Contains references to media content objects, a temporal scheduling, and synchronization model. • Synchronization model determines when these objects are presented (and how persistent they are) • Layout model can help a playback agent determine where the objects should be placed relative to one another. • The specification also allows transitions, metadata, temporal hyperlinks, and a host of other secondary presentation features to be defined. 10.
Scene 11 (3m 4s)
S M I L p r e s e n t a t i o n B a s i c s 11 <!DOCTYPE SMIL PUBLIC "-//W3C//DTD SMIL 3.0 Language//EN" "http://www.w3.org/2008/SMIL30/SMIL30Language.dtd"> <smil xmlns="http://www.w3.org/ns/SMIL" version="3.0" baseProfile="Language"> <head> <meta name="title" content="SlideShow"/> <meta name="generator" content="GRINS for SMIL 3.0"/> <meta name="author" content="Dick Bulterman"/> <layout> <root-layout xml:id="Winter" backgroundColor="#ffffcc" width="240" height="269"/> <region xml:id="Title" left="0" width="240" top="0" height="29"/> <region xml:id="Image" left="0" width="240" top="29" height="180" z-index="2"/> <region xml:id="Text" left="0" width="240" top="209" height="42" fit="meet"/> <region xml:id="Buttons" left="0" width="240" top="251" height="15"/> <region xml:id="Sound"/> </layout> </head>.
Scene 12 (3m 36s)
SMIL presentation <img region:" Title" src="title.gif"/> <video region="lmage" src="vO.mp4"/> <video region="lmage" src="v1.mp4"/> <video region="lmage" src="v2.mp4"/> <video region="lmage" src="v3.mp4"/> </seq> </par> </smil> Basics 12.
Scene 13 (3m 50s)
T i m i n g a n d S y n c h r o n i z a t i o n • SMIL timing defines when elements in a presentation get scheduled and, once scheduled, how long they will be active. • The SMIL timing facilities are the core contribution of the SMIL standard: using the elements and attributes defined in the SMIL Recommendation, time can be integrated into any XML language. • In a SMIL-based document, every media object and nearly every structural element has a specific timing scope. • While media timing plays an important role in determining the overall duration of a presentation, the structure of the document is also used to simplify and optimize the rendering of media presentations. 13.
Scene 14 (4m 18s)
T i m i n g a n d S y n c h r o n i z a t i o n • SMIL timing defines a collection of elements that determine the relative start and end times of document objects and a collection of attributes that control the duration, persistence, repetition, and accuracy of timing relations in a document. • SMIL can be used directly as the host language for a document (as is done in the various SMIL profiles) • But it can also serve as the basis for integrating time-based coordination of otherwise static elements (as is done in SVG animation and in XHTML + SMIL). 14.
Scene 15 (4m 42s)
T i m i n g M o d e l B a s i c s • The timing approach used by SMIL to specify the (relative) begin times of media objects and their durations is based on a structured timing model. • This means that the nested presentation structure in a SMIL document—and not only hard-coded clock values—is used to define the high-level activation and synchronization of objects. • For many simple SMIL documents, this timing is implied: The SMIL agent can figure them out at presentation time. • If more precise control over a presentation is required (such as inserting delays or specifying interactive behavior), SMIL also provides more complex timing mechanisms. 15.
Scene 16 (5m 9s)
T i m i n g M o d e l B a s i c s • The presentation contains a single background image on top of which a sequence of slides is placed, each with an accompanying image containing a text label and an audio file containing spoken commentary. • It contains a single background music object that is played throughout the presentation. • The timing in this presentation is dominated by two object sets: a background music object, which determines the duration of the total presentation, and various spoken commentary objects, which determine the duration of each of the image slides. • This means that an outer time base for the entire presentation is defined and a set of inner time bases for each slide in the presentation. 16 A Simple Slideshow Presentation.
Scene 17 (5m 39s)
17 T i m i n g M o d e l B a s i c s A Simple Slideshow Presentation.
Scene 18 (5m 46s)
T i m i n g M o d e l B a s i c s • A basic property of multimedia presentations is that they require some degree of temporal coordination among the objects being presented. • The more complex a presentation—in terms of either number of simultaneous objects or number of synchronization control points—the greater the amount of control information required. • SMIL uses timing elements and timing attributes to provide the activation and synchronization control information in a presentation. • In general, timing attributes are used to control the timing behavior of media object, and timing elements are used to control the behavior of the presentation as a whole. 18 Media Object and Presentation Timing Definitions.
Scene 19 (6m 13s)
T i m i n g M o d e l B a s i c s Media Timing: • Multimedia presentations typically contain two types of media objects: discrete media and continuous media. • Discrete media objects, such as the text labels, the background image, and each of the slide images of the example, have no implicit duration. ❑ If referenced in a SMIL file without any additional timing attributes, their duration will be 0 s—which is not very long. • Continuous media, such as the background music object and each of the spoken commentaries of the example, have implicit durations defined within their media encodings. 19 Media Object and Presentation Timing Definitions.
Scene 20 (6m 40s)
T i m i n g M o d e l B a s i c s Media Timing: ❑ If referenced in a SMIL file without any additional timing attributes, they will be rendered for the full duration defined by the object. • In the example, each slide image and the associated text labels should be displayed for duration that is defined by the accompanying spoken commentaries. • (Each slide/text/audio group will have different durations, since not all spoken commentary is equally long.) SMIL provides a range of attributes that allow the duration of objects to be explicitly defined and refined. • This allows the durations of the images and text to match that of the spoken audio. 20 Media Object and Presentation Timing Definitions.
Scene 21 (7m 9s)
T i m i n g M o d e l B a s i c s Presentation Timing: • A SMIL file contains references to one or more of media objects and a set of timing primitives that determine when these objects get started relative to one another. • The total timing of each of the media objects, plus any additional timing control defined in the SMIL file, determines the duration of the composite presentation. • Sometimes, this composite duration can be calculated in advance, but often no. • The basic timing of the slideshow presentation described in the example is deterministic: That is, we can determine the full timing in advance of the presentation’s execution by evaluating the timing of each of the continuous media objects. 21 Media Object and Presentation Timing Definitions.
Scene 22 (7m 40s)
T i m i n g M o d e l B a s i c s Presentation Timing: • However, that the presentation is only deterministic if several potential presentation- time delays are ignored—these include any streaming delays associated with bringing the media object from a server to the presentation device, or any delays at the client associated with decoding and rendering individual media objects. • For local presentations, such as CD-ROM multimedia, it is safe to assume that all the delays in the system can be predicted in advance and factored into the presentation timing. 22 Media Object and Presentation Timing Definitions.
Scene 23 (8m 5s)
T i m i n g M o d e l B a s i c s Presentation Timing: • For Web-based multimedia, where the delays when obtaining media may be considerable (and unpredictable) & where there may be wide variability in the performance of end-user devices, assuming that presentations are fully deterministic is a dangerous strategy. • A presentation with uncertain timing characteristics is non-deterministic. • In addition to the network streaming delays discussed above, non-deterministic timing can also be the result of content substitution within the presentation or as a result of using interactive, event-based presentation timing. • SMIL has elements and attributes to handle these cases as well. 23 Media Object and Presentation Timing Definitions.
Scene 24 (8m 33s)
T i m i n g M o d e l B a s i c s • A timeline is a simple graph showing time on one axis and one or more media objects on the other axis. • An example timeline, showing the elements and objects of the previous example, is shown in next slide. • This timeline shows the media sorted by layout: The left axis shows the various classes of media objects used, and the bottom axis shows the cumulative duration. • It is also possible to define separate lines for each media object, but this is usually less space efficient. • A timeline exposes the exact temporal relationships among media items. 24 SMIL and Timelines.
Scene 25 (9m 0s)
T i m i n g M o d e l B a s i c s 25 SMIL and Timelines.
Scene 26 (9m 7s)
T i m i n g M o d e l B a s i c s • These can translated to a text format by assigning explicit begin times for each media object and defining durations to discrete objects. • One such encoding is the time-list structure of the following code fragment. • The background audio and image objects (lines 1 and 2) are followed by the set of slide images (lines 3–9), the spoken commentary (lines 10–16), and the image- encoded text labels (lines 17–23). • The begin times are determined by the duration of the spoken commentary objects. • While this example could be used as the basis for SMIL timing, this would be unwise, since it is insensitive to delays and requires that all timing relationships be pre- calculated. 26 SMIL and Timelines.
Scene 27 (9m 38s)
T i m i n g M o d e l B a s i c s 1. <audio xml:id="M0" begin="0s" .../> 2. <img xml:id="B0" begin="0s" dur="43s".../> 3. <img xml:id="I0" begin="0s" dur="3s" .../> 4. <img xml:id="I1" begin="3s" dur="4s" .../> 5. <img xml:id="I2" begin="7s" dur="9s" .../> 6. <img xml:id="I3" begin="16s" dur="9s" .../> 7. <img xml:id="I4" begin="25s" dur="4s" .../> 8. <img xml:id="I5" begin="29s" dur="5s" .../> 9. <img xml:id="I6" begin="34s" dur="9s" .../> 10. <audio xml:id="C0" begin="0s".../> 11. <audio xml:id="C1" begin="3s" .../> 12. <audio xml:id="C2" begin="7s".../> 27 SMIL and Timelines 13. <audio xml:id="C3" begin="16s" .../> 14. <audio xml:id="C4" begin="25s" .../> 15. <audio xml:id="C5" begin="29s" .../> 16. <audio xml:id="C6" begin="34s" .../> 17. <img xml:id="L0" begin="0s" dur="3s" .../> 18. <img xml:id="L1" begin="3s" dur="4s" .../> 19. <img xml:id="L2" begin="7s" dur="9s" .../> 20. <img xml:id="L3" begin="16s" dur="9s" .../> 21. <img xml:id="L4" begin="25s" dur="4s" .../> 22. <img xml:id="L5" begin="29s" dur="5s" .../> 23. <img xml:id="L6" begin="34s" dur="9s" .../>.
Scene 28 (10m 32s)
T i m i n g M o d e l B a s i c s • The main advantage of the timeline model is that it is an easy-to-understand representation of continuous media objects under deterministic timing conditions. • While deterministic timing is good for modeling video tape, it does not scale well to most Web environments: If one of the image objects arrives later than planned, or if the presentation agent is slow in rendering the audio, the timeline does not really help in maintaining order among objects. • Things become even more troublesome if we don’t know the implicit duration of the audio items when constructing the timeline or if the duration of the object changes over the lifetime of the presentation. 28 SMIL and Structure-Based Timing.
Scene 29 (11m 2s)
T i m i n g M o d e l B a s i c s • (Since the SMIL file does not contain the media—it contains a pointer to the media— the timing and the update histories of the media object are decoupled from its use.) • If we add content substitution or interactive timing to the presentation (such as having the follow-on slide begin on a mouse click rather than at a fixed time), the timeline representation loses almost all of its utility. • In order to provide a more realistic framework for Web documents, SMIL is based on a structured timing framework in which the structured relationships among objects can be used to define most timing. 29 SMIL and Structure-Based Timing.
Scene 30 (11m 30s)
T i m i n g M o d e l B a s i c s • SMIL encodes its timing relationships by defining a logical timing hierarchy rather than an exact timeline. • The hierarchy for discussed example is shown in next slide. • Here, we see a set of yellow logical parallel nodes (P0–P7) and one blue logical sequential node (S0). • The parallel components say activate the sub-components together as a unit, and the sequential component says activate the sub-components sequentially. • The SMIL textual encoding of the presentation hierarchy is as follows: 30 SMIL and Structure-Based Timing.
Scene 31 (11m 53s)
T i m i n g M o d e l B a s i c s 31 SMIL and Structure-Based Timing.
Scene 32 (12m 1s)
T i m i n g M o d e l B a s i c s 32 SMIL and Structure-Based Timing <par xml:id="P0"> <audio xml:id="M0" .../> <image xml:id="B0" .../> <seq xml:id="S0"> <par xml:id="P1"> <img xml:id="I0" .../> <audio xml:id="C0" .../> <img xml:id="L0" .../> </par> <par xml:id="P2"> <img xml:id="I1" .../> <audio xml:id="C1" .../> <img xml:id="L1" .../> </par> ……………. ………… …….. <par xml:id="P7"> <img xml:id="I6" .../> <audio xml:id="C6" .../> <img xml:id="L6" .../> </par> </seq> </par>.
Scene 33 (12m 27s)
T i m i n g M o d e l B a s i c s 33 SMIL and Structure-Based Timing • While a timeline can state that objects 𝐶2, 𝐼2, and 𝐿2 all start at 7 s into the presentation and that they each have duration of 9 s, the SMIL hierarchy can state what is really going on logically: ❑ That objects 𝐶𝑖, 𝐼𝑖, and 𝐿𝑖 are to be treated as a logical group that get scheduled together (that is, they begin and end together); ❑ That the duration of 𝐼𝑖, and 𝐿𝑖 depends on the duration of 𝐶𝑖, ❑ That all three objects are to begin after object 𝐶𝑖−1—and, by extension, 𝐼𝑖−1 and 𝐿𝑖−1—end..
Scene 34 (12m 55s)
T i m i n g M o d e l B a s i c s 34 SMIL and Structure-Based Timing • None of relationships depends on the exact duration of any of the objects—you can construct a SMIL file before you know anything about the actual media being used. • A single timeline for one instance of a SMIL specification (one run-time uses of the presentation) can be constructed by combining the structured composition of objects with a model of the execution environment that contains information on the performance of the network connection, the preferences of the user, etc. • A timeline model based on the explicit media timings alone is not rich enough to model the various structured paths throughout a SMIL presentation..
Scene 35 (13m 22s)
T i m i n g M o d e l B a s i c s 35 Durations, Time, and Timebases • One of the most powerful features of SMIL is a flexible time model in which various aspects of an element’s behavior can be determined by the context in which it is being presented. • In order to use this model, it is important to understand a number of temporal distinctions and constraints applied by the SMIL model. • These include defining the active period of an element and defining the way that delays and relative starting/ending times can be expressed in a document..
Scene 36 (13m 47s)
T i m i n g M o d e l B a s i c s 36 Durations, Time, and Timebases Defining the Active Period of an Element: • Most media formats require that the duration of all of the component media objects be explicitly defined. • SMIL has several attributes that support direct duration definition, but it also provides attributes that allow you to specify or limit several layers of logical object durations..
Scene 37 (14m 5s)
T i m i n g M o d e l B a s i c s 37 Durations, Time, and Timebases Defining the Active Period of an Element: • Intrinsic duration: ❑ The duration of a media object as encoded in the (external to SMIL) media file. ❑ Most discrete media items such as images/plain text have an intrinsic duration of 0s ❑ Some quasi-discrete media such as animated images may have a longer intrinsic duration. ❑ Continuous media objects have duration that is equal to the temporal length of the object. ❑ Many media formats define the intrinsic duration explicitly in the media encoding..
Scene 38 (14m 30s)
T i m i n g M o d e l B a s i c s 38 Durations, Time, and Timebases Defining the Active Period of an Element: • Implicit duration: ❑ the duration that SMIL uses as the basis for scheduling an object. ❑ It is usually equal to the intrinsic duration, if available. ❑ Discrete objects are modeled as having an implicit duration of 0 s. ❑ If the intrinsic duration for continuous objects is not available (often MP3), the SMIL agent typically will have to scan the entire object to determine its duration. ❑ The implicit duration forms the starting point for the calculation of other logical timing durations have been defined within SMIL..
Scene 39 (14m 57s)
T i m i n g M o d e l B a s i c s 39 Durations, Time, and Timebases Defining the Active Period of an Element: • Simple duration: ❑ It is possible to modify an object’s implicit duration with an explicit duration via SMIL’s dur attribute. ❑ The result of applying an explicit duration (if any) to an object yields its simple duration. (by default the implicit duration & simple durations are the same.) ❑ Simple duration of an object may be longer or shorter than the implicit duration. ❑ Simple durations can also be defined to have special values that logically limit or stretch the duration of objects; these are the media and indefinite values.
Scene 40 (15m 24s)
T i m i n g M o d e l B a s i c s 40 Durations, Time, and Timebases Defining the Active Period of an Element: • Active duration: ❑ SMIL defines a number of attributes that allow an element to be repeated. ❑ These attributes modify the element’s simple duration, and the resulting repeated duration is called the object’s active duration. ❑ If an element’s simple duration is shorter than its implicit duration, only the first part of the element will be repeated. If the element’s simple duration is longer than its implicit duration, the entire element plus the temporal difference between the implicit and simple durations will be repeated..
Scene 41 (15m 51s)
T i m i n g M o d e l B a s i c s 41 Durations, Time, and Timebases Defining the Active Period of an Element: • Active duration: ❑ (During this “extra” time, either nothing will be rendered or the final frame/sample of the media object will be rendered: The behavior depends on the media type.) ❑ The end attribute can be used to define when the active duration ends. ❑ If an element does not repeat and is not shortened by end, its simple and active durations are the same..
Scene 42 (16m 14s)
T i m i n g M o d e l B a s i c s 42 Durations, Time, and Timebases Defining the Active Period of an Element: • Rendered duration: ❑ The active duration of an element ends after its dur/end and repeat attributes have been applied. ❑ This does not mean that an object disappears at the end of its active duration. ❑ SMIL provides an attribute to control the persistence of an object after its active duration has ended: the fill attribute. If fill is set to “freeze” the element will remain rendered until the end of its parent time container..
Scene 43 (16m 39s)
T i m i n g M o d e l B a s i c s 43 Durations, Time, and Timebases Defining the Active Period of an Element: • Rendered duration: ❑ If fill is set to “remove,” the object is removed from the screen as soon as its active duration ends. ❑ For discrete media with a fill = “freeze” attribute, the object will simply be rendered as if its active duration was extended; ❑ for visual continuous media, the last frame or sample of the object will be rendered..
Scene 44 (17m 1s)
T i m i n g M o d e l B a s i c s 44 Durations, Time, and Timebases Clock Values: • Many timing attributes are based on clock values. • These values can take several different forms, and they may serve as all or part of an attribute’s time value. • All clock values represent a relative time and have meaning only within the context of a time container..
Scene 45 (17m 19s)
T i m i n g M o d e l B a s i c s 45 Durations, Time, and Timebases Clock Values: Clock values may be given in four general forms: • Full clock values: ❑ These are times represented as a colon-separated list of hours, minutes, seconds, and fractions of a second. ❑ (If days, months, or years need to be specified, then absolute wall clock timing may be used instead.) ❑ This is a relative time and has meaning only within the context of a time container’s syncbase..
Scene 46 (17m 42s)
T i m i n g M o d e l B a s i c s 46 Durations, Time, and Timebases Clock Values: Clock values may be given in four general forms: • Partial clock values: ❑ These are times represented as a shorthand notation for full clock values, containing minutes, seconds, and (optionally) fractions of a second. • Timecount values: ❑ Are numbers with an optional type string and an optional fractional component. ❑ An integer clock value with no type string (such as “10”) implies a timecount in seconds; it is equivalent to “10 s.” Allowed type strings are “h,” “min,” “s,” and “ms.”.
Scene 47 (18m 8s)
T i m i n g M o d e l B a s i c s 47 Durations, Time, and Timebases Clock Values: Clock values may be given in four general forms: • Wallclock values: ❑ These are absolute times represented in three parts: a date field, a time field, and an (optional) timezone field. ❑ These times are absolute..
Scene 48 (18m 24s)
T i m i n g M o d e l B a s i c s 48 Durations, Time, and Timebases Syncbases: • Except in the special case of wallclock timing, every clock value in SMIL is relative to some other part of the document. • The child elements of a <par> container are started relative to the start time of that parent, while the child elements of a <seq> container are started relative to the end of their predecessor (except for the first child, which starts at the beginning of its parent). • Every element has a specific temporal reference point: the syncbase. • Most elements never have to specify their syncbase reference explicitly since the common SMIL time containers do this by default..