1st Edition

Handbook on Networked Multipoint Multimedia Conferencing and Multistream Immersive Telepresence using SIP Scalable Distributed Applications and Media Control over Internet

By Radhika Ranjan Roy Copyright 2021
    1066 Pages 120 B/W Illustrations
    by CRC Press

    1066 Pages 120 B/W Illustrations
    by CRC Press

    Handbook on Networked Multipoint Multimedia Conferencing and Multistream Immsersive Telepresence using SIP: Scalable Distributed Applications and Media Control over Internet is the first book to put together all IETF request for comments (RFCs), and the internet drafts standards related to the multipoint conferencing and immersive telepresence.

    This book includes mandatory and optional texts of all standards in a chronological and systematic way almost with one-to-one integrity from the beginning to end, allowing the reader to understand all aspects of the highly complex real-time applications.

    It is a book that network designers, software developers, product manufacturers, implementers, interoperability testers, professionals, professors, and researchers will find to be immensely useful. Practitioners and engineers in all spectrums who are concentrating on building the real-time, scalable, interoperable multipoint applications, can use this book to make informed choices based on technical standards in the market place, on all proprietary non-scalable and non-interposable products. This book will provide focus and foundation for these decision makers.

    Contents

    Preface...........................................................................................................................................xxxi

    Author...........................................................................................................................................xxxv

    Chapter 1 Scalable Networked Multipoint Multimedia Conferencing and Telepresence..............1

    1.1 Networked Multipoint Multimedia Conferencing..............................................1

    1.2 Basic Conferencing Protocol Architecture.........................................................4

    1.3 Third-Party Multiparty Conferencing.............................................................. 12

    1.4 Multistream Immersive Telepresence...............................................................23

    1.5 Scalable Centralized Multipoint Multimedia Conferencing Systems

    and Protocols Framework.................................................................................24

    1.6 Terminology.....................................................................................................28

    1.7 Summary..........................................................................................................28

    1.8 Problems........................................................................................................... 41

    References................................................................................................................... 42

    Chapter 2 Centralized Conferencing System Architecture......................................................... 43

    2.1 Introduction...................................................................................................... 43

    2.2 Convention........................................................................................................ 43

    2.3 Terminology..................................................................................................... 43

    2.4 Overview..........................................................................................................44

    2.5 Centralized Conferencing Data........................................................................ 45

    2.6 Centralized Conferencing Constructs and Identifiers...................................... 47

    2.7 Conferencing System Realization....................................................................50

    2.8 Conferencing Mechanisms............................................................................... 55

    2.9 Conference Scenario Realizations....................................................................56

    2.10 Relationships between SIP and Centralized Conferencing System

    Frameworks......................................................................................................69

    2.11 Security Considerations....................................................................................69

    2.12 Summary.......................................................................................................... 71

    2.13 Problems........................................................................................................... 72

    Chapter 3 Media Server Control Architecture............................................................................. 73

    3.1 Introduction...................................................................................................... 73

    3.2 Terminology..................................................................................................... 75

    3.3 Architecture Overview..................................................................................... 75

    3.4 NSIP Usage.......................................................................................................77

    3.5 Media Control for IVR Services...................................................................... 78

    3.6 Media Control for Conferencing Services........................................................80

    3.7 Security Considerations....................................................................................85

    3.8 Summary..........................................................................................................86

    3.9 Problems...........................................................................................................87

    Chapter 4 Conferencing Information Data Model for Centralized Conferencing.......................89

    4.1 Introduction......................................................................................................89

    4.2 Terminology.....................................................................................................89

    4.3 Overview..........................................................................................................89

    Comparison.......................................................................... 91

    4.4 Data Model Definition......................................................................................92

    4.5 RELAX NG Schema...................................................................................... 103

    4.6 XML Schema Extensibility............................................................................ 103

    4.7 XML Example................................................................................................ 103

    4.8 Non-Normative RELAX NG Schema in XML Syntax (Appendix A of

    RFC 6501)....................................................................................................... 103

    4.9 Relax Schema in XML Syntax (Appendix B of RFC 6501).......................... 103

    4.10 Security Considerations.................................................................................. 104

    4.11 Summary........................................................................................................ 108

    4.12 Problems......................................................................................................... 109

    References................................................................................................................. 111

    Chapter 5 Centralized Conferencing Manipulation Protocol.................................................... 129

    5.1 Introduction.................................................................................................... 129

    5.2 Conventions and Terminology........................................................................ 129

    5.3 XCON Conference Control System Architecture.......................................... 129

    5.4 Protocol Overview.......................................................................................... 131

    5.5 CCMP Messages............................................................................................ 134

    5.6 Example of CCMP in Action......................................................................... 153

    5.7 Locating a Conference Server........................................................................ 162

    5.8 Managing Notifications.................................................................................. 163

    5.9 HTTP Transport............................................................................................. 165

    5.10 Security Considerations.................................................................................. 166

    5.11 XML Schema................................................................................................. 168

    5.12 IANA Considerations..................................................................................... 169

    5.13 Summary........................................................................................................ 176

    5.14 Evaluation of Other Protocol Models and Transports Considered for

    CCMP (Appendix A of RFC 6503)................................................................ 176

    5.15 Problems......................................................................................................... 178

    References................................................................................................................. 179

    Chapter 6 Binary Floor Control Protocol.................................................................................. 181

    6.1 Introduction.................................................................................................... 181

    6.2 Terminology................................................................................................... 181

    6.3 Scope.............................................................................................................. 181

    6.4 Overview of Operation................................................................................... 183

    6.5 Packet Format................................................................................................. 186

    6.6 Transport........................................................................................................200

    6.7 Lower-Layer Security..................................................................................... 201

    6.8 Protocol Transactions..................................................................................... 201

    6.9 Authentication and Authorization..................................................................202

    6.10 Floor Participant Operations..........................................................................202

    6.11 Chair Operations............................................................................................205

    6.12.1 Requesting Information about Floors...............................................206

    6.13 Floor Control Server Operations....................................................................208

    6.14 Security Considerations.................................................................................. 214

    6.15 BFCP Connection Establishment................................................................... 215

    6.16 IANA Considerations..................................................................................... 218

    6.17 Summary........................................................................................................ 221

    6.18 Problems.........................................................................................................222

    Chapter 7 XCON Notification Service.......................................................................................225

    7.1 Introduction....................................................................................................225

    7.2 Terminology...................................................................................................225

    7.3 Notification Formats.......................................................................................225

    7.4 Full Notifications............................................................................................225

    7.5 Partial Notifications........................................................................................227

    7.6 IANA Considerations.....................................................................................229

    7.7 Security Considerations..................................................................................229

    7.7.1 Connection Security..........................................................................230

    7.7.2 Authorization Considerations ...........................................................230

    7.8 Summary........................................................................................................230

    7.9 Problems......................................................................................................... 231

    Reference................................................................................................................... 231

    Chapter 8 Media Channel Control Framework.......................................................................... 233

    8.1 Introduction.................................................................................................... 233

    8.2 CFW Capabilities........................................................................................... 233

    8.3 Overview........................................................................................................ 235

    8.4 Control Channel Setup................................................................................... 238

    8.4.1 Control Client SIP UAC Behavior..................................................... 238

    8.4.2 Control Server SIP UAS Behavior.................................................... 239

    8.5 Establishing Media Streams – Control Client SIP UAC Behavior.................240

    8.6 Control Framework Interactions..................................................................... 241

    8.7 Response Code Descriptions.......................................................................... 247

    8.8 Control Packages............................................................................................ 247

    8.9 Formal Syntax................................................................................................249

    8.10 Examples........................................................................................................ 251

    8.11 Extensibility....................................................................................................254

    8.12 Security Considerations..................................................................................254

    8.12.1 Session Establishment.......................................................................254

    8.13 IANA Considerations.....................................................................................256

    8.14 Common Package Components (Appendix A of RFC 6230).........................256

    8.15 Summary........................................................................................................ 257

    8.16 Problems......................................................................................................... 257

    Chapter 9 Mixer Package for Media Control Channel Framework........................................... 259

    9.1 Introduction.................................................................................................... 259

    9.2 Conventions and Terminology........................................................................260

    9.3 Control Package Definition............................................................................260

    9.4 Element Definitions........................................................................................ 262

    9.5 Formal Syntax................................................................................................290

    9.6 Examples........................................................................................................ 291

    9.6.1 AS-MS Framework Interaction Examples........................................ 291

    9.6.1.1 Creating a Conference Mixer and Joining a

    Participant..........................................................................309

    9.6.1.2 Receiving Active Talker Notifications...............................309

    9.6.1.3 Conference Termination....................................................309

    9.6.2 Mixing Examples.............................................................................. 310

    9.6.2.1 Audio Conferencing........................................................... 310

    9.6.2.2 Bridging Connections........................................................ 312

    xvi Contents

    9.6.2.3 Video Conferencing........................................................... 312

    9.7 Security Considerations.................................................................................. 313

    9.8 IANA Considerations..................................................................................... 315

    9.9 Summary........................................................................................................ 315

    9.10 Problems......................................................................................................... 315

    References................................................................................................................. 316

    Chapter 10 Media Session Recording.......................................................................................... 317

    10.1 Use Cases and Requirements for SIP-Based Media Recording................... 317

    10.2 Media Recording Architecture....................................................................324

    10.3 Media Recording Metadata.......................................................................... 331

    XML Body.................................................................344

    10.3.9 XML Schema Definition for Recording Metadata....................... 345

    10.3.10 Security Considerations................................................................348

    10.3.11 IANA Considerations...................................................................349

    10.4 Media Session Recoding Protocol...............................................................349

    10.5 Summary...................................................................................................... 371

    10.6 Problems....................................................................................................... 372

    References................................................................................................................. 373

    Chapter 11 Media Resource Brokering....................................................................................... 375

    11.1 Introduction.................................................................................................. 375

    11.2 Conventions and Terminology..................................................................... 375

    11.3 Problem Discussion...................................................................................... 377

    11.4 Deployment Scenario Options..................................................................... 378

    11.5 MRB Interface Definitions.......................................................................... 381

    11.6 MRB Acting as a B2BUA............................................................................405

    11.7 Multimodal MRB Implementations.............................................................405

    11.8 Relative Merits of Query Mode, IAMM, and IUMM.................................406

    11.9 Examples......................................................................................................407

    11.10 Media Service Resource Publisher Interface XML Schema....................... 425

    11.11 Media Service Resource Consumer Interface XML Schema......................444

    11.12 Security Considerations............................................................................... 461

    11.13 IANA Considerations...................................................................................463

    11.14 Summary......................................................................................................463

    11.15 Problems.......................................................................................................464

    References.................................................................................................................465

    Chapter 12 Media Control Channel Framework for Interactive Voice Response ....................... 467

    12.1 Introduction.................................................................................................. 467

    12.2 Conventions and Terminology.....................................................................469

    12.3 Control Package Definition..........................................................................469

    12.4 Element Definitions...................................................................................... 471

    12.5 Formal Syntax.............................................................................................. 511

    12.6 Examples...................................................................................................... 534

    12.8 IANA Considerations...................................................................................544

    12.9 Using VoiceXML as a Dialog Language.....................................................544

    12.10 Summary...................................................................................................... 549

    12.11 Problems....................................................................................................... 550

    Chapter 13 SIP Interface to VoiceXML Media Services............................................................ 551

    13.1 Introduction.................................................................................................. 551

    13.2 VoiceXML Session Establishment and Termination................................... 553

    13.3 Media Support..............................................................................................560

    13.4 Returning Data to the Application Server................................................... 563

    13.5 Outbound Calling.........................................................................................565

    13.6 Call Transfer................................................................................................. 565

    13.7 Security Considerations............................................................................... 569

    13.8 IANA Considerations...................................................................................569

    13.9 Summary......................................................................................................569

    13.10 Problems....................................................................................................... 570

    References................................................................................................................. 571

    Chapter 14 Media Resource Control Protocol Version 2............................................................. 573

    14.1 Introduction.................................................................................................. 573

    14.2 Conventions.................................................................................................. 574

    14.3 Architecture................................................................................................. 575

    14.4 MRCPv2 Basics........................................................................................... 576

    14.5 MRCPv2 Specification................................................................................. 583

    14.5.5 Events........................................................................................... 587

    14.6 MRCPv2 Generic Methods, Headers, and Result Structure........................ 588

    14.7 Resource Discovery..................................................................................... 597

    14.8 Speech Synthesizer Resource...................................................................... 597

    14.10 Recorder Resource....................................................................................... 656

    14.11 Speaker Verification and Identification........................................................663

    14.12 Security Considerations...............................................................................684

    14.13 IANA Considerations...................................................................................686

    14.14 Examples......................................................................................................686

    14.15 ABNF Normative Definition........................................................................697

    14.16 XML Schemas.............................................................................................706

    14.17 Summary...................................................................................................... 711

    14.18 Problems....................................................................................................... 713

    References................................................................................................................. 714

    Chapter 15 Media Control Channel Framework (CFW) Call Flow Examples........................... 715

    15.1 Introduction.................................................................................................. 715

    15.2 Conventions.................................................................................................. 715

    15.3 Terminology................................................................................................. 716

    15.4 A Practical Approach................................................................................... 716

    15.4.1 State Diagrams............................................................................. 716

    15.5 Control Channel Establishment................................................................... 718

    15.6 Use-Case Scenarios and Examples.............................................................. 726

    15.7 Media Resource Brokering..........................................................................792

    15.8 Security Considerations...............................................................................820

    15.9 Summary......................................................................................................827

    15.10 Problems.......................................................................................................830

    Reference................................................................................................................... 831

    Chapter 16 Multistream Immersive Telepresence Conferencing Systems.................................. 833

    16.1 Requirements for Telepresence Multistreams.............................................. 833

    16.2 Use Cases for Telepresence Multistreams.................................................... 837

    16.3 Framework for Telepresence Multistreams.................................................. 845

    16.4 XML Schema for the CLUE Data Model....................................................885

    16.5 CLUE Protocol............................................................................................. 933

    16.6 CLUE Protocol Data Channel.....................................................................966

    16.7 Mapping RTP Streams to CLUE Media Captures.......................................980

    16.8 Session Signaling for Controlling Multiple Streams for Telepresence

    (CLUE)......................................................................................................... 983

    16.9 Summary......................................................................................................997

    16.10 Problems..................................................................................................... 1001

    References...............................................................................................................1002

    Appendix....................................................................................................................................... 1005

    Index............................................................................................................................................. 1013

    Biography

    Radhika Ranjan Roy has been an electronics engineer, United States Army Research, United States

    Command, Control, Communications, Computers, Combat Systems, Intelligence, Surveillance,

    and Reconnaissance (C5ISR)-S&TCD Laboratories (previously known as CERDEC), Aberdeen

    Proving Ground (APG), Maryland, United States since 2009. Dr. Roy is leading research and development

    efforts in the development of scalable large-scale SIP-based VoIP/Multimedia networks and

    services, cybersecurity, artificial intelligence and machine/deep learning (AI/ML/DL) architecture,

    mobile ad hoc networks (MANETs), Peer-to-Peer (P2P) networks, cybersecurity detecting application

    software and network vulnerability, jamming detection, supporting an array of the Army/

    Department of Defense’s Nationwide and Worldwide Warfighter Networking Architectures and

    participating in technical standards development in Multimedia/Real-Time Services Collaboration,

    IPv6, Radio Communications, Enterprise Services Management, and Information Transfer of

    Department of Defense (DoD) Technical Working Groups (TWGs). He received his PhD in Electrical

    Engineering with Major in Computer Communications from the City University of New York

    (CUNY), NY, United States in 1984 and his MS in Electrical Engineering from the Northeastern

    University, Boston, MA, United States in 1978. He received his BS in Electrical Engineering from

    the Bangladesh University of Engineering & Technology (BUET), Dhaka, Bangladesh in 1967. He

    was born in the renowned country town of Derai, Bangladesh.

    Prior to joining CERDEC, Dr. Roy worked as the lead system engineer at CACI, Eatontown,

    NJ from 2007 to 2009 and developed Army Technical Resource Model (TRM), Army Enterprise

    Architecture (AEA), DoD Architecture Framework (DoDAF), and Army LandWarNet (LWN)

    Capability Sets, and technical standards for Joint Tactical Radio System (JTRS), Mobile IPv6,

    MANET, and SIP, supporting Army Chief Information Officer (CIO)/G-6. Dr. Roy worked as senior

    system engineer, SAIC, Abingdon, MD from 2004 to 2007, supporting Modeling, Simulations,

    Architectures, and System Engineering of many Army projects: WIN-T, FCS, and JNN.

    During his career, Dr. Roy worked in AT&T/Bell Laboratories, Middletown, NJ as senior consultant

    from 1990 to 2004 and led a team of engineers in designing AT&T’s Worldwide SIP-based

    VoIP/Multimedia Communications Network Architecture, consisting of wired and wireless, from

    the preparation of Request for Information (RFI) to the evaluation of vendor RFI responses and

    interactions with all selected major vendors related to their products. He participated in and contributed

    to the development of VoIP/H.323/SIP multimedia standards in ITU-T, IETF, ATM, and

    Frame Relay standard organizations.

    Dr. Roy worked as senior principal engineer in CSC, Falls Church, VA from 1984 to 1990 and

    worked in the design and performance analysis of the US Treasury nationwide X.25 packet-switching

    network. In addition, he designed many network architectures of many proposed U.S. Government

    and Commercial Worldwide and Nationwide Networks: Department of State Telecommunications

    Network (DOSTN), U.S. Secret Service Satellite Network, Veteran Communications Network, and

    Ford Company’s Dealership Network. Prior to CSC, he worked from 1967 to 1977 as deputy director,

    Design, in PDP, Dhaka, Bangladesh.

    Dr. Roy’s research interests include the areas of artificial intelligence, machine/deep learning,

    blockchain cloud and fog computing, mobile ad hoc networks, multimedia communications, peer-topeer

    networking, and quality-of-service. He has published over 60 technical papers and is holding or

    pending over 35 patents. He also participates in many IETF working groups. Dr. Roy authored three

    books Handbook on SDP for Multimedia Session Negotiations: SIP and WebRTC Telephony (CRC

    Press/Taylor & Francis, 2018), Handbook on Session Initiation Protocol: Networked Multimedia

    Communications for IP Telephony (CRC Press/Taylor & Francis, 2016), and Handbook of Mobile

    Ad Hoc Networks for Mobility Models, Springer, 2010. He lives in the historical district of Howell

    Township, New Jersy, with his wife Jharna.