sdp

0.1.1-SNAPSHOT


A parser and emitter for the Session Description Protocol (SDP) as described by RFC 4566.

dependencies

org.clojure/clojure
1.7.0
org.clojure/core.match
0.3.0-alpha4
org.clojure/tools.logging
0.3.1
dire
0.5.3



(this space intentionally left almost blank)
 

The multimedia.streaming.sdp namespace provides the published API for parsing and emitting SDP strings.

(ns multimedia.streaming.sdp
  (:require [multimedia.streaming.sdp.parser :as parser]
            [clojure.string :as string]))

What is SDP?

SDP is intended for describing multimedia sessions for the purposes of session announcement, session invitation and other forms of multimedia session initiation.

An SDP session description is entirely textual using the ISO 10646 character set in UTF-8 encoding. SDP field names and attribute names use only the US-ASCII subset of UTF-8, but textual fields and attribute values may use the full ISO 10646 character set. Field and attribute values that use the full UTF-8 character set are never directly compared, hence there is no requirement for UTF-8 normalisation.

An Example

An SDP session description consists of a number of lines of text of the form

type=value

where type must be exactly one case-significant character and value is structured text whose format depends upon type. In general, value is either a number of fields delimited by a single space character or a free format string, and is case-significant unless a specific field defines otherwise. Whitespace must not be used on either side of the = sign.

Session descriptions are divided into sections and consist of a session-level section followed by zero or more media-level sections. The session-level part starts with a 'v=' line and continues to the first media-level section. Each media-level section starts with an 'm=' line and continues to the next media-level section or to the end of the whole session description. In general, session-level values are the default for all media unless overridden by an equivalent media-level value.

(def example-sdp-string
  "v=0
   o=jdoe 2890844526 2890842807 IN IP4 10.47.16.5
   s=SDP Seminar
   i=A Seminar on the session description protocol
   u=http://www.example.com/seminars/sdp.pdf
   e=j.doe@example.com (Jane Doe)
   p=+44 (0)1445 637948
   c=IN IP4 224.2.17.12/127
   b=CT:128
   t=2873397496 2873404696
   r=7d 1h 0 25h
   z=2882844526 -1h 2898848070 0
   k=clear:gf638ebi3rh3i3o3e35767
   a=recvonly
   m=audio 49170 RTP/AVP 0
   i=Media title
   c=IN IP4 224.2.17.14/127
   c=IN IP4 224.2.17.18/225
   b=AT:14
   a=recvonly
   a=ctlmethod:serverpush
   m=video 51372 RTP/AVP 992882844526
   a=rtpmap:99 h263-1998/90000")

Mime-Type

The mime-type string to be used for SDP data is 'application/sdp'.

(def mime-type
  "application/sdp")

Parsing

The parse function parses the provided sdp-string and returns its data structure representation.

(parse example-sdp-string)

The following flags can optionally be supplied to change the parsing behaviour.

:relaxed attempts to continue parsing an invalid SDP description, skipping erronious lines and providing default values where required.

(parse example-sdp-string :relaxed)

See Custom Representations for information on how to change the data representation of the parsed fields.

(defn parse
  [sdp-string & flags]
  (let [{:keys [relaxed]} (set flags)
        prep-lines (comp (remove string/blank?)
                         (map parser/mapify-lines)
                         parser/add-line-numbers
                         parser/add-sections
                         (parser/check-line-order relaxed))]
    (->> sdp-string
         string/split-lines
         (transduce prep-lines (completing (parser/parse-lines relaxed)) {}))))

SDP Description Structure

The parsed SDP description is a map containing at least the :version, :origin and :name keys, and optionally any of the additional keys shown here.

Note that keys are shown here in parsing order for clarity, however the parsed map is unordered.

Top level keys represent session level fields. The individual media level fields are found under the :media-descriptions key.

Vectors indicate that any number of entries is possible for that field.

(def example-sdp-description
  {:version 0
   :origin {:username "jdoe"
            :session-id "2890844526"
            :session-version "2890842807"
            :network-type "IN"
            :address-type "IP4"
            :address "10.47.16.5"}
   :name "SDP Seminar"
   :information "A Seminar on the session description protocol"
   :uri "http://www.example.com/seminars/sdp.pdf"
   :email ["j.doe@example.com (Jane Doe)"]
   :phone ["+44 (0)1445 637948"]
   :connection {:network-type "IN"
                :address-type "IP4"
                :address "224.2.17.12/127"}
   :bandwidth [{:bandwidth-type "CT"
                :bandwidth 128}]
   :timing [{:start-time 2873397496N
             :end-time 2873404696N
             :repeat [{:repeat-interval "7d"
                       :active-duration "1h"
                       :offsets-from-start "0"}]}]
   :timezone [{:adjustment-time 2882844526N
               :offset "-1h"}
              {:adjustment-time 2898848070N
               :offset "0"}]
   :encryption-key {:method "clear"
                    :payload "gf638ebi3rh3i3o3e35767"}
   :attributes [{:attribute "recvonly"}]
   :media-descriptions [{:media-type "audio"
                         :port 49170
                         :protocol "RTP/AVP"
                         :format "0"
                         :information "Media title"
                         :connection [{:network-type "IN"
                                       :address-type "IP4"
                                       :address "224.2.17.14"
                                       :ttl 127}]
                         :bandwidth [{:bandwidth-type "AT"
                                      :bandwidth 14}]
                         :attributes [{:attribute "recvonly"}
                                      {:attribute "ctlmethod"
                                       :value "serverpush"}]}
                        {:media-type "video"
                         :port 51372
                         :protocol "RTP/AVP"
                         :format "992882844526"
                         :attributes [{:attribute "rtpmap"
                                       :value "99 h263-1998/90000"}]}]})

Emitting

The emit function serialises the provided sdp-description into its textual form and retuns the string representation.

(emit example-sdp-description)

This function is not currently implemented.

(defn emit
  [sdp-description]
  :not-implemented)

Custom Representations

Each field in the SDP structure is parsed as a specific type. For each type there exists a parsing function that is called to produce the field value from the textual representation. It is sometimes useful to provide a custom parsing function that will return types suitable to the application at hand. For example, an application built upon the Java 8 platform may wish to parse instants as java.time.Instant, whereas an application using an older platform may wish to use the JodaTime library constructs or a basic java.util.DateTime.

The custom-parser-for! function overrides the default parsing functions with those specified in parser-map.

(use-custom-parsers!
  {:port identity
   :instant (fn [raw] ...)})

The supplied parsing functions must accept the raw value as a string and return the parsed representation.

Parsers can be specified for any of the fields listed in the parser-fns map detailed in the multimedia.streaming.sdp.parser namespace.

Calling this function will affect all future invocations of parse on all threads.

(defn use-custom-parsers!
  [parser-map]
  (alter-var-root #'parse
    (fn [f]
      (fn [sdp-string & flags]
        (binding [parser/parse-fns (merge parser/parse-fns parser-map)]
          (apply f example-sdp-string flags))))))
 

The multimedia.streaming.sdp.parser namespace is an internal namespace providing the implementation of the parser. It is not part of the published interface and may change without warning between versions.

(ns multimedia.streaming.sdp.parser
  (:require [clojure.string :as string]
            [clojure.core.match :refer [match]]
            [clojure.tools.logging :as log]
            [dire.core :refer [with-handler!]]))

Parser Implementation

The parser implementation is intended to be as declarative as possible. It consists of three main parts:

  • The input preparation pipeline,
  • the parser configuration, and
  • the execution core.

When data is fed to the parser it is first processed by the input preparation pipeline. This performs some simple transformations on the input to make the parsers job easier. The prepared input is then fed to the parser which parses the various fields, using a set of overridable parsing functions and inserts them into the parsed structure according to the rules in the parser configuration.

Input Preparation

The input preperation pipeline is implemented as the following set of composed transducers.

mapify-lines transforms a textual SDP line into a map of its component parts.

(defn mapify-lines
  [line]
  (let [[k v] (string/split line #"=" 2)]
    {:type (keyword (string/trim k)) :value (string/triml v)}))

add-line-numbers is a stateful transducer that adds a line number to each line. This allows the parser to produce much better error messages.

(defn add-line-numbers
  [xf]
  (let [line-number (volatile! 0)]
    (fn ([] (xf))
        ([result] (xf result))
        ([result input]
         (xf result (assoc input :line-number (vswap! line-number inc)))))))

add-sections is a stateful transducer that adds a section identifier to each line, indicating to the parser whether it is currently parsing a session-level line, or a media-level line.

(defn add-sections
  [xf]
  (let [section (volatile! :session)]
    (fn ([] (xf))
        ([result] (xf result))
        ([result {line-type :type :as input}]
         (when (= :m line-type)
           (vreset! section :media))
         (xf result (assoc input :section @section))))))

SDP has a strict line order. The line-order structure specifies, for each line type, which line types may follow.

(def line-order
  {:session
   {:v #{:o}
    :o #{:s}
    :s #{:i :u :e :p :c :b :t :z :k :a :m}
    :i #{:u :e :p :c :b :t :z :k :a :m}
    :u #{:e :p :c :b :t :z :k :a :m}
    :e #{:e :p :c :b :t :z :k :a :m}
    :p #{:p :c :b :t :z :k :a :m}
    :c #{:b :t :z :k :a :m}
    :b #{:t :z :k :a :m}
    :t #{:t :r :z :k :a :m}
    :r #{:t :z :k :a :m}
    :z #{:k :a :m}
    :k #{:a :m}
    :a #{:a :m}}
   :media
   {:m #{:m :i :c :b :k :a}
    :i #{:m :c :b :k :a}
    :c #{:c :m :b :k :a}
    :b #{:m :k :a}
    :k #{:m :a}
    :a #{:m :a}}})

check-line-order returns a transducer that verifies that the SDP lines are in the expected order as defined in the line-order structure. If the lines are not in the expected order an exception is thrown. The following flags are supported.

:relaxed causes bad ordering to be treated as a non-fatal error that simply generates a warning.

(defn check-line-order
  [& flags]
  (fn [xf]
    (let [{:keys [relaxed]} (set flags)
          allowed-lines (volatile! #{:v})]
      (fn ([] (xf))
        ([result] (xf result))
        ([result {:keys [type line-number section] :as line}]
         (let [allowed @allowed-lines
               possibilities (get-in line-order [section type])]
           (when possibilities
             (vreset! allowed-lines possibilities))
           (cond
            (allowed type) (xf result line)
            relaxed (do (log/warn "Skipping illegal line type '" type
                                  "' on line " line-number ", expected one of "
                                  allowed) result)
            :else   (throw
                     (ex-info (str "Illegal line type '" type "' on line "
                                   line-number ", expected one of " allowed)
                              {:error-type :illegal-line-type
                               :expected allowed
                               :received type
                               :line-number line-number})))))))))

The input data leaves the input preparation stage as a sequence of maps that contain all the information needed for the parsing stage. Additionally, if the parser is in its default strict mode, the lines are guaranteed to be in the correct order.

Parser Configuration

The behaviour of the parser is dictated by the parser configuration. This configuration describes the structure of the SDP protocol, how each field should be parsed and how it should be inserted into the parsed SDP description.

Insert Functions

The following functions provide the options for inserting a parsed field into the SDP description.

vectorize is an insert function that causes the parsed field to be inserted into the sdp description as a vector under key. Multiple fields of the same type, within the same section, are appended to the vector.

(defn vectorize
  [sdp key field]
  (if (vector? (key field))
    (update-in sdp [key] into (key field))
    (update-in sdp [key] (fnil conj []) (key field))))

in-last returns an insert function that causes the parsed field to be inserted as the value of key under the last entry in parent, where parent is a key in the final sdp structure, whose value is a vector. For example

(in-last :media-descriptions)

would add field as the value of key to the most recent entry under :media-descriptions.

(defn in-last
  [parent]
  (fn [sdp key field]
    (let [index (dec (count (parent sdp)))]
      (assoc-in sdp [parent index key] (key field)))))

vectorize-in-last returns an insert function that causes the parsed field to be inserted as a vector, which is the value of key, in the last entry in parent, where parent is a key in the final sdp structure. Multiple fields of the same type, within the same section, are appended to the vector. For example

(vectorize-in-last :media-descriptions)

would add field to the vector, which is the value of key in the most recent entry under :media-descriptions.

(defn vectorize-in-last
  [parent]
  (fn [sdp key field]
    (let [index (dec (count (parent sdp)))]
      (if (vector? (key field))
        (update-in sdp [parent index key] into (key field))
        (update-in sdp [parent index key] (fnil conj []) (key field))))))

Parse Rules

The parse-rules structure defines the rules for parsing the fields of the SDP protocol. It also specifies how the parsed fields should be combined into the final structure.

At the top level, the parse-rules structure maps SDP line-types to the appropriate parsing rule. Each parsing rule consists of the following fields.

:name is required and specifies the key, in the final SDP structure, under which the parsed field should be inserted.

:parse-as is required and specifies how the field should be parsed. Its value must be either

  • a keyword matching one of the parsing functions in the parse-fns structure, or
  • a parse-spec as defined below.

:on-fail is optional and specifies how to recover from an error when in relaxed parsing mode. Its value must be either

  • a keyword matching one of the error handling functions in the error-fns structure, or
  • a two element vector, where the first element is a keyword matching one of the error handling functions in error-fns, and the second element is the parameter to that function.

If the key is not present, errors for that field are always treated as fatal.

:insert is optional and specifies how the parsed field is inserted into the final SDP structure. Its value must be either

  • an insert function to be used for all parsed values, or
  • a map whose keys are section identifiers and whose values are the insert functions to be used when parsing values from the corresponding section.

If the key is not present, the field is inserted at the top level of the SDP structure. In the event of multiple values for the same field, only the last value will be present in the SDP structure.

Parse Spec
A parse-spec describes how to parse a compound field. It is a map comprising the following keys.

:separator is required and specifies how the compound field should be split into its atomic parts. Its value must be a regular expression suitable for consumption by clojure.string/split.

:fields is required and specifies the list of fields that make up the compound field. Its value must be a vector of field-specs as defined below.

:repeats? is optional and specifies whether the list of field-specs should be repeated until all components of the compound field have been consumed. If the key is present, its value is truthy, and the line contains more components than exist fields in the field-spec, the parser will continue, assuming that the remaining components are another set of the same fields. This is repeated until all input is consumed. The resulting parsed values will be inserted into the final SDP structure as a vector of compound fields, one for each cycle of the field-spec list.

Field Spec
A field-spec describes how to parse an atomic field from a compound field. It is a map comprising the following keys.

:name is required and specifies the key, in the parsed compound field, under which the sub-field should be inserted.

:parse-as is required and specifes the parse function to be used when parsing the field. Its value must address one of the parse functions in the parse-fns structure.

:expect is optional and specifies the set of possible values that are recognised as valid by the specification. If specified and the parsed value is not one of the listed values, a debug message is emitted warning about the use of a non-standard value. This can greatly aid debugging in multimedia applications.

(def parse-rules
  {:v {:name :version
       :parse-as :integer
       :on-fail [:default-to 0]}
   :o {:name :origin
       :parse-as {:separator #"\s+"
                  :fields [{:name :username
                            :parse-as :string}
                           {:name :session-id
                            :parse-as :numeric-string}
                           {:name :session-version
                            :parse-as :string}
                           {:name :network-type
                            :parse-as :string
                            :expect #{"IN"}}
                           {:name :address-type
                            :parse-as :string
                            :expect #{"IP4" "IP6"}}
                           {:name :address
                            :parse-as :host}]}}
   :s {:name :name
       :parse-as :string
       :on-fail [:default-to " "]}
   :i {:name :information
       :parse-as :string
       :insert {:media (in-last :media-descriptions)}}
   :u {:name :uri
       :parse-as :string}
   :e {:name :email
       :parse-as :email
       :insert vectorize}
   :p {:name :phone
       :parse-as :phone
       :insert vectorize}
   :c {:name :connection
       :parse-as {:separator #"\s+"
                  :fields [{:name :network-type
                            :parse-as :string
                            :expect #{"IN"}}
                           {:name :address-type
                            :parse-as :string
                            :expect #{"IP4" "IP6"}}
                           {:name :address
                            :parse-as :address}]}
       :insert {:media (vectorize-in-last :media-descriptions)}}
   :b {:name :bandwidth
       :parse-as {:separator #":"
                  :fields [{:name :bandwidth-type
                            :parse-as :string
                            :expect #{"CT" "AS"}}
                           {:name :bandwidth
                            :parse-as :integer}]}
       :insert {:session vectorize
                :media (vectorize-in-last :media-descriptions)}}
   :t {:name :timing
       :parse-as {:separator #"\s+"
                  :fields [{:name :start-time
                            :parse-as :instant}
                           {:name :end-time
                            :parse-as :instant}]}
       :insert vectorize}
   :r {:name :repeat
       :parse-as {:separator #"\s+"
                  :fields [{:name :repeat-interval
                            :parse-as :duration}
                           {:name :active-duration
                            :parse-as :duration}
                           {:name :offsets-from-start
                            :parse-as :duration}]}
       :insert (vectorize-in-last :timing)}
   :z {:name :timezone
       :parse-as {:separator #"\s+"
                  :fields [{:name :adjustment-time
                            :parse-as :instant}
                           {:name :offset
                            :parse-as :duration}]
                  :repeats? true}}
   :k {:name :encryption-key
       :parse-as {:separator #":"
                  :fields [{:name :method
                            :parse-as :string
                            :expect #{"clear" "base64" "uri" "prompt"}}
                           {:name :payload
                            :parse-as :string}]}
       :insert {:media (in-last :media-descriptions)}}
   :a {:name :attributes
       :parse-as {:separator #":"
                  :fields [{:name :attribute
                            :parse-as :string}
                           {:name :value
                            :parse-as :string}]}
       :insert {:session vectorize
                :media (vectorize-in-last :media-descriptions)}}
   :m {:name :media-descriptions
       :parse-as {:separator #"\s+"
                  :fields [{:name :media-type
                            :parse-as :string
                            :expect #{"audio" "video" "text"
                                      "application" "message"}}
                           {:name :port
                            :parse-as :port}
                           {:name :protocol
                            :parse-as :string
                            :expect #{"udp" "RTP/AVP" "RTP/SAVP"}}
                           {:name :format
                            :parse-as :string}]}
       :insert vectorize}})

Parse Functions

The following functions comprise the default parse functions used to parse atomic fields.

numeric-string is a parse function that returns the unmodified string value of the field, but throws an exception if the value is not numeric.

(defn numeric-string
  [field]
  (try (bigint field) field
    (catch NumberFormatException e
      (throw (Exception. "String is not numeric")))))

integer-in-range returns a parse function that parses an integer from the field and returns it, if and only if the value is in the range of min and max inclusive. If integer parsing fails or if the value is outside of the specified range, an exception is thrown.

(defn integer-in-range
  [min max]
  (fn [field]
    (let [value (Integer. field)]
      (if (<= min value max)
        value
        (throw (Exception. (str "Value '" field "' is outside the legal range "
                                "(min: " min ", max: " max ")")))))))

ip4-address is a regular expression that will match an IPv4 address.

(def ip4-address
  #"^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$")

multicast? is a utility function that, given an IP address string, will return true if address is a multicast address.

(defn multicast?
  [address]
  (-> address
      java.net.InetAddress/getByName
      .isMulticastAddress))

address is a parse function that will parse an IP version 4 address with an optional TTL value and an optional address count, an IP version 6 address with, an optional address count, or a fully qualified domain name from the field. The string must be in the format

<address>[/ttl][/address-count]

and is parsed into the structure

{:address string
 :ttl integer
 :address-count integer}

with the presence of the :ttl and :address-count keys being conditional upon their presence in the input string.

(defn address
  [field]
  (let [[address ttl address-count] (string/split field #"/" 3)
        address-type (cond
                       (re-matches ip4-address address) :IPv4
                       (re-matches #".*:.*" address) :IPv6
                       :else :FQDN)
        result {:address address}]
    (match [address-type ttl address-count]
      [:IPv4 nil nil]   (if (multicast? address)
                          (throw (Exception. (str "Multicast IPv4 address "
                                                  "requires a TTL value")))
                         result)
      [:IPv4 ttl nil]   (assoc result :ttl ((integer-in-range 0 255) ttl))
      [:IPv4 ttl count] (assoc result :ttl ((integer-in-range 0 255) ttl)
                                      :address-count (Integer. count))
      [:IPv6 nil nil]   result
      [:IPv6 count nil] (assoc result :address-count (Integer. count))
      [:IPv6 _ _]       (throw (Exception. "TTL value not allowed for IPv6"))
      [:FQDN nil nil]   result
      [:FQDN _ _]       (throw (Exception. "TTL value not allowed for FQDN")))))

The parse-fns structure specifies which parse function should be used to parse each field type. The functions specified here can be overriden by the user with the custom-parser-for! API function.

(def ^:dynamic parse-fns
  {:string identity
   :integer #(Integer. %)
   :numeric-string numeric-string
   :instant bigint
   :duration identity
   :host identity
   :address address
   :email identity
   :phone identity
   :port (integer-in-range 0 65535)})

Error Functions

The error-fns structure specifies how errors should be handled for each of the :on-fail conditions in the parse-rules structure.

(def error-fns
  {:default-to (fn [[rule value default line-num e]]
                 (log/info "Bad value '" value "' for " (name (:name rule))
                           " on line " line-num
                           ", substituting default value '" default "'")
                 {(:name rule) default})
   :error (fn [[rule value line-num e]]
            (throw (ex-info (str "Bad value '" value "' for "
                                 (name (:name rule)) " on line " line-num)
                    {:error-type :parse-failed
                     :field (:name rule)
                     :value value
                     :parser (:parse-as rule)} e)))})

Execution Core

The execution core has no intrinsic understanding of the SDP format. It is simply a generic parsing engine that is given a set of parse rules as described in the parse-rules structure and a pre-processed sequence of input lines in the following format.

{:type :v
 :value "0"
 :line-number 1
 :section :session}

The lines are parsed in order and the resulting values inserted into the SDP structure.

The following functions comprise the execution core.

flatten-fields flattens any values in map, that are themselves maps, into map. If map and one of its values contain the same key, the child value takes presidence.

(defn flatten-fields
  [map]
  (reduce (fn [r [k v]] (if (map? v) (merge r v) (assoc r k v))) {} map))

parse-simple-field parses an atomic field from value using the specified rule, allowing any exceptions to propogate.

(defn parse-simple-field
  [{:keys [parse-as name expect] :as rule} value line-num relaxed]
  (let [parsed ((parse-as parse-fns) value)]
    (when-not (or (nil? expect) (expect parsed))
        (log/debug "Non-standard value '" parsed "' for '" name "' on line "
                   line-num ", expected one of " expect))
    {name parsed}))

This with-handler! call wraps the parse-simple-field function in an exception handler that deligates any errors to the appropriate error function in the error-fns structure.

(with-handler! #'parse-simple-field
  Exception
  (fn [e & [{error-rule :on-fail :as rule} value line-num relaxed]]
    (cond
     (nil? relaxed)       ((:error error-fns) [rule value line-num e])
     (vector? error-rule) (let [[func param] error-rule]
                            ((func error-fns) [rule value param line-num e]))
     :else                ((:error error-fns) [rule value line-num e]))))

parse-compound-field parses a compound field from value using the rule (pased as the first parameter) and returns the parsed value.

(defn parse-compound-field
  [{{:keys [repeats? separator] field-rules :fields} :parse-as name :name}
   value line-num relaxed]
  (let [fields (string/split value separator)
        [field-rules parse-fn] (if repeats?
                                 [(cycle field-rules)
                                  (comp (map parse-simple-field)
                                        (partition-all (count field-rules))
                                        (map (partial apply merge)))]
                                 [field-rules (map parse-simple-field)])
        parsed (sequence parse-fn field-rules fields
                         (repeat line-num) (repeat relaxed))]
    (if repeats?
      {name (into [] parsed)}
      {name (transduce (map flatten-fields) merge parsed)})))

parse-lines returns a reducing function that, given a map representation of an SDP line, will parse the line value and return its data structure equiverlent. Any errors in parsing will result in an ExceptionInfo being thrown. The following flags are supported.

:relaxed causes minor errors to be automatically corrected where possible, corrections are logged at the info log level. Major errors will still throw an ExceptionInfo.

(defn parse-lines
  [& flags]
  (let [{:keys [relaxed]} (set flags)]
    (fn [result {:keys [type value line-number section] :as line}]
      (let [{:keys [insert parse-as name] :as rule} (type parse-rules)
            parsed (if (map? parse-as)
                     (parse-compound-field rule value line-number relaxed)
                     (parse-simple-field rule value line-number relaxed))
            insert (if (map? insert) (section insert) insert)
            insert (if (fn? insert)
                        insert
                       (fn [final key parsed] (conj final parsed)))]
        (insert result name parsed)))))