# Implementing XML languages with lxml

## Dr. Stefan Behnel

http://codespeak.net/lxml/

lxml-dev@codespeak.net

# What is an »XML language«?

- a language in XML notation
- aka »XML dialect«
  - except that it's not a dialect
- Examples:
  - XML Schema
  - Atom/RSS
  - (X)HTML
  - Open Document Format
  - SOAP
  - ... add your own one here

# Popular mistakes to avoid (1)

"That's easy, I can use regular expressions!"

No, you can't.

# Popular mistakes to avoid (2)

"This is tree data, I'll take the DOM!"

# Popular mistakes to avoid (2)

"This is tree data, I'll take the DOM!"

- DOM is ubiquitous, but it's as complicated as Java
- uglify your application with tons of DOM code to
  - walk over non-element nodes to find the data you need
  - convert text content to other data types
  - modify the XML tree in memory

=> write verbose, redundant, hard-to-maintain code

# Popular mistakes to avoid (3)

"SAX is so fast and consumes no memory!"

# Popular mistakes to avoid (3)

"SAX is so fast and consumes no memory!"

- but writing SAX code is not fast!
- write error-prone, state-keeping SAX code to
  - figure out where you are
  - find the sections you need
  - convert text content to other data types
  - copy the XML data into custom data classes
  - ... and don't forget the way back into XML!

=> write confusing state-machine code

=> debugging into existence

# Working with XML

> Getting XML work done

> (instead of getting time wasted)

# How can you work with XML?

- Preparation:
  - Implement usable data classes as an abstraction layer
  - Implement a mapping from XML to the data classes
  - Implement a mapping from the data classes to XML
- Workflow:
  - parse XML data
  - map XML data to data classes
  - work with data classes
  - map data classes to XML
  - serialise XML

- Approach:
  - get rid of XML and do everything in your own code

# What if you could simplify this?

- Preparation:
  - Extend usable XML API classes into an abstraction layer
- Workflow:
  - parse XML data into XML API classes
  - work with XML API classes
  - serialise XML

- Approach:
  - cover only the quirks of XML and make it work for you

# What if you could simplify this ...

- ... without sacrificing usability or flexibility?
- ... using a high-speed, full-featured, pythonic XML toolkit?
- ... with the power of XPath, XSLT and XML validation?

... then »lxml« is your friend!

# Overview

- What is lxml?
  - what & who
- How do you use it?
  - Lesson 0: quick API overview
    - ElementTree concepts and lxml features
  - Lesson 1: parse XML
    - how to get XML data into memory
  - Lesson 2: generate XML
    - how to write an XML generator for a language
  - Lesson 3: working with XML trees made easy
    - how to write an XML API for a language

# What is lxml?

- a fast, full-featured toolkit for XML and HTML handling
  - http://codespeak.net/lxml/
  - lxml-dev@codespeak.net
- based on and inspired by
  - the C libraries libxml2 and libxslt (by Daniel Veillard)
  - the ElementTree API (by Fredrik Lundh)
  - the Cython compiler (by Robert Bradshaw, Greg Ewing & me)
  - the Python language (by Guido & [paste Misc/ACKS here])
  - user feedback, ideas and patches (by you!)
    - keep doing that, we love you all!
- maintained (and major parts) written by myself
  - initial design and implementation by Martijn Faassen
  - extensive HTML API and tools by Ian Bicking

# What do you get for your money?

- many tools in one:
  - Generic, ElementTree compatible XML API: lxml.etree
    - but faster for many tasks and much more feature-rich
  - Special tool set for HTML handling: lxml.html
  - Special API for pythonic data binding: lxml.objectify
  - General purpose path languages: XPath and CSS selectors
  - Validation: DTD, XML Schema, RelaxNG, Schematron
  - XSLT, XInclude, C14N, ...
  - Fast tree iteration, event-driven parsing, ...
- it's free, but it's worth every €-Cent!
  - what users say:
    - »no qualification, I would recommend lxml for just about any HTML task«
    - »THE tool [...] for newbies and experienced developers«
    - »you can do pretty much anything with an intuitive API«
    - »lxml takes all the pain out of XML«

# Lesson 0: a quick overview

> why »lxml takes all the pain out of XML«

> (a quick overview of lxml features and ElementTree concepts)

# Namespaces in ElementTree

- uses Clark notation:

  - wrap namespace URI in `{...}`
  - append the tag name

  System Message: ERROR/3 (`lxml-ep2008.txt`, line 368)

  Unknown directive type "sourcecode".

      .. sourcecode:: pycon

        >>> tag = "{http://www.w3.org/the/namespace}tagname"
        >>> element = etree.Element(tag)

- no prefixes!

- a single, self-containing tag identifier

# Text content in ElementTree

- uses `.text` and `.tail` attributes:

  System Message: ERROR/3 (`lxml-ep2008.txt`, line 383)

  Unknown directive type "sourcecode".

      .. sourcecode:: pycon

        >>> div = html.fragment_fromstring(
        ...     "<div><p>a paragraph<br>split in two</p> parts</div>")
        >>> p = div[0]
        >>> br = p[0]

        >>> p.text
        'a paragraph'
        >>> br.text
        >>> br.tail
        'split in two'
        >>> p.tail
        ' parts'

- no text nodes!

  - simplifies tree traversal a lot
  - simplifies many XML algorithms

# Attributes in ElementTree

- uses `.get()` and `.set()` methods:

  System Message: ERROR/3 (`lxml-ep2008.txt`, line 410)

  Unknown directive type "sourcecode".

      .. sourcecode:: pycon

        >>> root = etree.fromstring(
        ...     '<root a="the value" b="of an" c="attribute"/>')

        >>> root.get('a')
        'the value'

        >>> root.set('a', "THE value")
        >>> root.get('a')
        'THE value'

- or the `.attrib` dictionary property:

  System Message: ERROR/3 (`lxml-ep2008.txt`, line 424)

  Unknown directive type "sourcecode".

      .. sourcecode:: pycon

        >>> d = root.attrib

        >>> list(sorted(d.keys()))
        ['a', 'b', 'c']
        >>> list(sorted(d.values()))
        ['THE value', 'attribute', 'of an']

# Tree iteration in lxml.etree (1)

System Message: ERROR/3 (`lxml-ep2008.txt`, line 440)

Unknown directive type "sourcecode".

    .. sourcecode:: pycon

        >>> root = etree.fromstring(
        ...   "<root> <a><b/><b/></a> <c><d/><e><f/></e><g/></c> </root>")

        >>> print([child.tag for child in root])   # children
        ['a', 'c']

        >>> print([el.tag for el in root.iter()])  # self and descendants
        ['root', 'a', 'b', 'b', 'c', 'd', 'e', 'f', 'g']

        >>> print([el.tag for el in root.iterdescendants()])
        ['a', 'b', 'b', 'c', 'd', 'e', 'f', 'g']

        >>> def iter_breadth_first(root):
        ...     bfs_queue = collections.deque([root])
        ...     while bfs_queue:
        ...         el = bfs_queue.popleft()  # pop next element
        ...         bfs_queue.extend(el)      # append its children
        ...         yield el

        >>> print([el.tag for el in iter_breadth_first(root)])
        ['root', 'a', 'c', 'b', 'b', 'd', 'e', 'g', 'f']

# Tree iteration in lxml.etree (2)

System Message: ERROR/3 (`lxml-ep2008.txt`, line 469)

Unknown directive type "sourcecode".

    .. sourcecode:: pycon

        >>> root = etree.fromstring(
        ...   "<root> <a><b/><b/></a> <c><d/><e><f/></e><g/></c> </root>")

        >>> tree_walker = etree.iterwalk(root, events=('start', 'end'))

        >>> for (event, element) in tree_walker:
        ...     print("%s (%s)" % (element.tag, event))
        root (start)
        a (start)
        b (start)
        b (end)
        b (start)
        b (end)
        a (end)
        c (start)
        d (start)
        d (end)
        e (start)
        f (start)
        f (end)
        e (end)
        g (start)
        g (end)
        c (end)
        root (end)

# Path languages in lxml

System Message: ERROR/3 (`lxml-ep2008.txt`, line 501)

Unknown directive type "sourcecode".

    .. sourcecode:: xml

        <root>
          <speech class='dialog'><p>So be it!</p></speech>
          <p>stuff</p>
        </root>

- search it with XPath

  System Message: ERROR/3 (`lxml-ep2008.txt`, line 510)

  Unknown directive type "sourcecode".

      .. sourcecode:: pycon

        >>> find_paragraphs = etree.XPath("//p")
        >>> paragraphs = find_paragraphs(xml_tree)

        >>> print([ p.text for p in paragraphs ])
        ['So be it!', 'stuff']

- search it with CSS selectors

  System Message: ERROR/3 (`lxml-ep2008.txt`, line 520)

  Unknown directive type "sourcecode".

      .. sourcecode:: pycon

        >>> find_dialogs = cssselect.CSSSelector("speech.dialog p")
        >>> paragraphs = find_dialogs(xml_tree)

        >>> print([ p.text for p in paragraphs ])
        ['So be it!']

# Summary of lesson 0

- lxml comes with various tools
  - that aim to hide the quirks of XML
  - that simplify finding and handling data
  - that make XML a pythonic tool by itself

# Lesson 1: parsing XML/HTML

> The input side

> (a quick overview)

# Parsing XML and HTML from ...

- strings: `fromstring(xml_data)`
  - byte strings, but also unicode strings
- filenames: `parse(filename)`
- HTTP/FTP URLs: `parse(url)`
- file objects: `parse(f)`
  - `f = open(filename, 'rb')` !
- file-like objects: `parse(f)`
  - only need a `f.read(size)` method
- data chunks: `parser.feed(xml_chunk)`
  - `result = parser.close()`

(parsing from strings and filenames/URLs frees the GIL)

# Example: parsing from a string

- using the `fromstring()` function:

  System Message: ERROR/3 (`lxml-ep2008.txt`, line 582)

  Unknown directive type "sourcecode".

      .. sourcecode:: pycon

        >>> root_element = etree.fromstring(some_xml_data)

- using the `fromstring()` function with a specific parser:

  System Message: ERROR/3 (`lxml-ep2008.txt`, line 588)

  Unknown directive type "sourcecode".

      .. sourcecode:: pycon

        >>> parser = etree.HTMLParser(remove_comments=True)
        >>> root_element = etree.fromstring(some_html_data, parser)

- or the `XML()` and `HTML()` aliases for literals in code:

  System Message: ERROR/3 (`lxml-ep2008.txt`, line 595)

  Unknown directive type "sourcecode".

      .. sourcecode:: pycon

        >>> root_element = etree.XML("<root><child/></root>")
        >>> root_element = etree.HTML("<p>some<br>paragraph</p>")

# Parsing XML into ...

- a tree in memory
  - `parse()` and `fromstring()` functions
- a tree in memory, but step-by-step with a generator
  - `iterparse()` generates `(start/end, element)` events
  - tree can be cleaned up to save space
- SAX-like callbacks without building a tree
  - `parse()` and `fromstring()` functions
  - pass a `target` object into the parser

# Summary of lesson 1

- parsing XML/HTML in lxml is mostly straight forward
  - simple functions that do the job
- advanced use cases are pretty simple
  - event-driven parsing using `iterparse()`
  - special parser configuration with keyword arguments
    - configuration is generally local to a parser
- BTW: parsing is very fast, as is serialising
  - don't hesitate to do parse-serialise-parse cycles

# Lesson 2: generating XML

> The output side

> (and how to make it safe and simple)

# The example language: Atom

The Atom XML format

- Namespace: http://www.w3.org/2005/Atom
- W3C recommendation derived from RSS and friends
- Atom feeds describe news entries and annotated links
  - a `feed` contains one or more `entry` elements
  - an `entry` contains `author`, `link`, `summary` and/or `content`

# Example: generate XML (1)

The ElementMaker (or E-factory)

System Message: ERROR/3 (`lxml-ep2008.txt`, line 670)

Unknown directive type "sourcecode".

    .. sourcecode:: pycon

      >>> from lxml.builder import ElementMaker
      >>> A = ElementMaker(namespace="http://www.w3.org/2005/Atom",
      ...                  nsmap={None : "http://www.w3.org/2005/Atom"})

System Message: ERROR/3 (`lxml-ep2008.txt`, line 678)

Unknown directive type "sourcecode".

    .. sourcecode:: pycon

     >>> atom = A.feed(
     ...   A.author( A.name("Stefan Behnel") ),
     ...   A.entry(
     ...     A.title("News from lxml"),
     ...     A.link(href="http://codespeak.net/lxml/"),
     ...     A.summary("See what's <b>fun</b> about lxml...",
     ...               type="html"),
     ...   )
     ... )

System Message: ERROR/3 (`lxml-ep2008.txt`, line 690)

Unknown directive type "sourcecode".

    .. sourcecode:: pycon

     >>> from lxml.etree import tostring
     >>> print( tostring(atom, pretty_print=True) )

# Example: generate XML (2)

System Message: ERROR/3 (`lxml-ep2008.txt`, line 699)

Unknown directive type "sourcecode".

    .. sourcecode:: pycon

      >>> atom = A.feed(
      ...   A.author( A.name("Stefan Behnel") ),
      ...   A.entry(
      ...     A.title("News from lxml"),
      ...     A.link(href="http://codespeak.net/lxml/"),
      ...     A.summary("See what's <b>fun</b> about lxml...",
      ...               type="html"),
      ...   )
      ... )

System Message: ERROR/3 (`lxml-ep2008.txt`, line 711)

Unknown directive type "sourcecode".

    .. sourcecode:: xml

      <feed xmlns="http://www.w3.org/2005/Atom">
        <author>
          <name>Stefan Behnel</name>
        </author>
        <entry>
          <title>News from lxml</title>
          <link href="http://codespeak.net/lxml/"/>
          <summary type="html">See what's &lt;b&gt;fun&lt;/b&gt;
                               about lxml...</summary>
        </entry>
      </feed>

# Be careful what you type!

System Message: ERROR/3 (`lxml-ep2008.txt`, line 729)

Unknown directive type "sourcecode".

    .. sourcecode:: pycon

      >>> atom = A.feed(
      ...   A.author( A.name("Stefan Behnel") ),
      ...   A.entry(
      ...     A.titel("News from lxml"),
      ...     A.link(href="http://codespeak.net/lxml/"),
      ...     A.summary("See what's <b>fun</b> about lxml...",
      ...               type="html"),
      ...   )
      ... )

System Message: ERROR/3 (`lxml-ep2008.txt`, line 741)

Unknown directive type "sourcecode".

    .. sourcecode:: xml

      <feed xmlns="http://www.w3.org/2005/Atom">
        <author>
          <name>Stefan Behnel</name>
        </author>
        <entry>
          <titel>News from lxml</titel>
          <link href="http://codespeak.net/lxml/"/>
          <summary type="html">See what's &lt;b&gt;fun&lt;/b&gt;
                               about lxml...</summary>
        </entry>
      </feed>

# Want more 'type safety'?

Write an XML generator module instead:

System Message: ERROR/3 (`lxml-ep2008.txt`, line 761)

Unknown directive type "sourcecode".

    .. sourcecode:: python

      # atomgen.py

      from lxml import etree
      from lxml.builder import ElementMaker

      ATOM_NAMESPACE = "http://www.w3.org/2005/Atom"

      A = ElementMaker(namespace=ATOM_NAMESPACE,
                       nsmap={None : ATOM_NAMESPACE})

      feed = A.feed
      entry = A.entry
      title = A.title
      # ... and so on and so forth ...

      # plus a little validation function: isvalid()
      isvalid = etree.RelaxNG(file="atom.rng")

# The Atom generator module

System Message: ERROR/3 (`lxml-ep2008.txt`, line 790)

Unknown directive type "sourcecode".

    .. sourcecode:: pycon

      >>> import atomgen as A

      >>> atom = A.feed(
      ...   A.author( A.name("Stefan Behnel") ),
      ...   A.entry(
      ...     A.link(href="http://codespeak.net/lxml/"),
      ...     A.title("News from lxml"),
      ...     A.summary("See what's <b>fun</b> about lxml...",
      ...               type="html"),
      ...   )
      ... )

      >>> A.isvalid(atom) # ok, forgot the ID's => invalid XML ...
      False

      >>> title = A.titel("News from lxml")
      Traceback (most recent call last):
        ...
      AttributeError: 'module' object has no attribute 'titel'

# Mixing languages (1)

Atom can embed serialised HTML

System Message: ERROR/3 (`lxml-ep2008.txt`, line 818)

Unknown directive type "sourcecode".

    .. sourcecode:: pycon

        >>> import lxml.html.builder as h

        >>> html_fragment = h.DIV(
        ...   "this is some\n",
        ...   h.A("HTML", href="http://w3.org/MarkUp/"),
        ...   "\ncontent")

System Message: ERROR/3 (`lxml-ep2008.txt`, line 829)

Unknown directive type "sourcecode".

    .. sourcecode:: pycon

      >>> serialised_html = etree.tostring(html_fragment, method="html")

      >>> summary = A.summary(serialised_html, type="html")

System Message: ERROR/3 (`lxml-ep2008.txt`, line 835)

Unknown directive type "sourcecode".

    .. sourcecode:: pycon

      >>> print(etree.tostring(summary))
      <summary xmlns="http://www.w3.org/2005/Atom" type="html">
         &lt;div&gt;this is some
         &lt;a href="http://w3.org/MarkUp/"&gt;HTML&lt;/a&gt;
         content&lt;/div&gt;
      </summary>

# Mixing languages (2)

Atom can also embed non-escaped XHTML

System Message: ERROR/3 (`lxml-ep2008.txt`, line 850)

Unknown directive type "sourcecode".

    .. sourcecode:: pycon

        >>> from copy import deepcopy
        >>> xhtml_fragment = deepcopy(html_fragment)

        >>> from lxml.html import html_to_xhtml
        >>> html_to_xhtml(xhtml_fragment)

        >>> summary = A.summary(xhtml_fragment, type="xhtml")

System Message: ERROR/3 (`lxml-ep2008.txt`, line 862)

Unknown directive type "sourcecode".

    .. sourcecode:: pycon

      >>> print(etree.tostring(summary, pretty_print=True))
      <summary xmlns="http://www.w3.org/2005/Atom" type="xhtml">
        <html:div xmlns:html="http://www.w3.org/1999/xhtml">this is some
        <html:a href="http://w3.org/MarkUp/">HTML</html:a>
        content</html:div>
      </summary>

# Summary of lesson 2

- generating XML is easy
  - use the ElementMaker
- wrap it in a module that provides
  - the target namespace
  - an ElementMaker name for each language element
  - a validator
  - maybe additional helper functions
- mixing languages is easy
  - define a generator module for each

... this is all you need for the output side of XML languages

# Lesson 3: Designing XML APIs

> The Element API

> (and how to make it the way you want)

# Trees in C and in Python

- Trees have two representations:
  - a plain, complete, low-level C tree provided by libxml2
  - a set of Python Element proxies, each representing one element
- Proxies are created on-the-fly:
  - lxml creates an Element object for a C node on request
  - proxies are garbage collected when going out of scope
  - XML trees are garbage collected when deleting the last proxy

# Mapping Python classes to nodes

- Proxies can be assigned to XML nodes by user code
  - lxml tells you about a node, you return a class

# Example: a simple Element class (1)

- define a subclass of ElementBase

  System Message: ERROR/3 (`lxml-ep2008.txt`, line 939)

  Unknown directive type "sourcecode".

      .. sourcecode:: pycon

        >>> class HonkElement(etree.ElementBase):
        ...    @property
        ...    def honking(self):
        ...       return self.get('honking') == 'true'

- let it replace the default Element class

  System Message: ERROR/3 (`lxml-ep2008.txt`, line 948)

  Unknown directive type "sourcecode".

      .. sourcecode:: pycon

        >>> lookup = etree.ElementDefaultClassLookup(
        ...                             element=HonkElement)

        >>> parser = etree.XMLParser()
        >>> parser.set_element_class_lookup(lookup)

# Example: a simple Element class (2)

- use the new Element class

  System Message: ERROR/3 (`lxml-ep2008.txt`, line 962)

  Unknown directive type "sourcecode".

      .. sourcecode:: pycon

        >>> root = etree.XML('<root><honk honking="true"/></root>',
        ...                  parser)

        >>> root.honking
        False
        >>> root[0].honking
        True

# Mapping Python classes to nodes

- The Element class lookup
  - lxml tells you about a node, you return a class
  - no restrictions on lookup algorithm
  - each parser can use a different class lookup scheme
  - lookup schemes can be chained through fallbacks
- Classes can be selected based on
  - the node type (element, comment or processing instruction)
    - `ElementDefaultClassLookup()`
  - the namespaced node name
    - `CustomElementClassLookup()` + a fallback
    - `ElementNamespaceClassLookup()` + a fallback
  - the value of an attribute (e.g. `id` or `class`)
    - `AttributeBasedElementClassLookup()` + a fallback
  - read-only inspection of the tree
    - `PythonElementClassLookup()` + a fallback

# Designing an Atom API

- a feed is a container for entries

  System Message: ERROR/3 (`lxml-ep2008.txt`, line 1012)

  Unknown directive type "sourcecode".

      .. sourcecode:: python

        # atom.py

        ATOM_NAMESPACE = "http://www.w3.org/2005/Atom"
        _ATOM_NS = "{%s}" % ATOM_NAMESPACE

        class FeedElement(etree.ElementBase):
            @property
            def entries(self):
               return self.findall(_ATOM_NS + "entry")

- it also has a couple of meta-data children, e.g. `title`

  System Message: ERROR/3 (`lxml-ep2008.txt`, line 1026)

  Unknown directive type "sourcecode".

      .. sourcecode:: python

        class FeedElement(etree.ElementBase):
            # ...
            @property
            def title(self):
                "return the title or None"
                return self.find("title")

# Consider lxml.objectify

- ready-to-use, generic Python object API for XML

System Message: ERROR/3 (`lxml-ep2008.txt`, line 1041)

Unknown directive type "sourcecode".

    .. sourcecode:: python

        >>> from lxml import objectify

        >>> feed = objectify.parse("atom-example.xml")
        >>> print(feed.title)
        Example Feed

        >>> print([entry.title for entry in feed.entry])
        ['Atom-Powered Robots Run Amok']

        >>> print(feed.entry[0].title)
        Atom-Powered Robots Run Amok

# Still room for more convenience

System Message: ERROR/3 (`lxml-ep2008.txt`, line 1059)

Unknown directive type "sourcecode".

    .. sourcecode:: python

        from itertools import chain

        class FeedElement(objectify.ObjectifiedElement):

            def addIDs(self):
                "initialise the IDs of feed and entries"

                for element in chain([self], self.entry):
                    if element.find(_ATOM_NS + "id") is None:
                        id = etree.SubElement(self, _ATOM_NS + "id")
                        id.text = make_guid()

# Incremental API design

- choose an XML API to start with
  - lxml.etree is general purpose
  - lxml.objectify is nice for document-style XML
- fix Elements that really need some API sugar
  - dict-mappings to children with specific content/attributes
  - properties for specially typed attributes or child values
  - simplified access to varying content types of an element
  - shortcuts for unnecessarily deep subtrees
- ignore what works well enough with the Element API
  - lists of homogeneous children -> Element iteration
  - string attributes -> .get()/.set()
- let the API grow at your fingertips
  - play with it and test use cases
  - avoid "I want because I can" feature explosion!

# Setting up the Element mapping

Atom has a namespace => leave the mapping to lxml

System Message: ERROR/3 (`lxml-ep2008.txt`, line 1111)

Unknown directive type "sourcecode".

    .. sourcecode:: python

        # ...
        _atom_lookup = etree.ElementNamespaceClassLookup(
                          objectify.ObjectifyElementClassLookup())

        # map the classes to tag names
        ns = _atom_lookup.get_namespace(ATOM_NAMESPACE)
        ns["feed"]  = FeedElement
        ns["entry"] = EntryElement
        # ... and so on
        # or use ns.update(vars()) with appropriate class names

        # create a parser that does some whitespace cleanup
        atom_parser = etree.XMLParser(remove_blank_text=True)

        # make it use our Atom classes
        atom_parser.set_element_class_lookup(_atom_lookup)

        # and help users in using our parser setup
        def parse(input):
            return etree.parse(input, atom_parser)

# Using your new Atom API

System Message: ERROR/3 (`lxml-ep2008.txt`, line 1138)

Unknown directive type "sourcecode".

    .. sourcecode:: pycon

        >>> import atom
        >>> feed = atom.parse("ep2008/atom-example.xml").getroot()

        >>> print(len(feed.entry))
        1
        >>> print([entry.title for entry in feed.entry])
        ['Atom-Powered Robots Run Amok']

        >>> link_tag = "{%s}link" % atom.ATOM_NAMESPACE
        >>> print([link.get("href") for link in feed.iter(link_tag)])
        ['http://example.org/', 'http://example.org/2003/12/13/atom03']

# Summary of lesson 3

To implement an XML API ...

1. start off with lxml's Element API
   - or take a look at the object API of lxml.objectify
2. specialise it into a set of custom Element classes
3. map them to XML tags using one of the lookup schemes
4. improve the API incrementally while using it
   - discover inconveniences and beautify them
   - avoid putting work into things that work

# Conclusion

lxml ...

- provides a convenient set of tools for XML and HTML
  - parsing
  - generating
  - working with in-memory trees
- follows Python idioms wherever possible
  - highly extensible through wrapping and subclassing
  - callable objects for XPath, CSS selectors, XSLT, schemas
  - iteration for tree traversal (even while parsing)
  - list-/dict-like APIs, properties, keyword arguments, ...
- makes extension and specialisation easy
  - write a special XML generator module in trivial code
  - write your own XML API incrementally on-the-fly