Importer/Exporter Overview

(C) 2000 Dom Lachowicz

cinamod@hotmail.com

$DATE: January 3, 2001

1. Introduction

This document gives a general overview of AbiWord´s import/export facilities. It is meant to be both a high-level overview of the code as well as a in-depth analysis of several key points and features.

2. AbiWord´s Native File Format

AbiWord´s native file format is XML-based. This means that AbiWord´s file format shares much in common with other markup formats such as (X)HTML. XML DTDs for the AbiWord format can be found at http://www.abisource.com/~sam/awml/

When viewing the AbiWord DTD, you will see various entities and attributes. These can be broken into two basic categories:

Container Containers

Data Containers

A container container (as the name implies) is merely a container which contains 1+ other containers. An example of this is the fact that any AbiWord document may contain 1 or more sections.

An example of a data container is the "c" entity tag. The "<c>" tag contains a run of textual characters meant to be displayed by AbiWord.

When examining these AbiWord DTDs, you will notice that many of the elements have a shared attribute - namely "props." It is inside of these props attributes that the actual formatting parameters are contained. For example, the following line, when interpreted by AbiWord

<c props="font-family:Courier New; font-size:14pt; font-style:italic; font-weight:bold">Bold Italic Courier New Size 14</c>

produces the following output:

Bold Italic Courier New Size 14

For an extensive list of what values can be contained inside of any element´s "props" tag, please consult either the AbiWord DTD or the MSWord importer.

3. Why an input/ouput system?

AbiWord is an excellent word processor, and its file format is an excellent representation of AbiWord´s capabilities and the expected capabilities of most modern word processors. But AbiWord does not exist in a vacuum. There are other viable WP offerings on the market. A goal of AbiWord is to inter-operate with each of these other WPs as flawlessly and seamlessly as possible. This is best achieved by being able to both read and write documents in other WPs native formats.

Some applications treat their native format importers/exporters as somewhat "sacred." By this I mean that they do not recognize their native importer/exporter as "just another" importer or exporter (take gnumeric, for example). AbiWord does not do this - our ABW importer/exporter is treated as just another importer/exporter class, with the notable exception that it maps extremely well onto AbiWord´s internal structures with a minimum of fuss.

4. The import (input) mechanism

When creating an importer class, you must first override/extend the base IE_Imp class or a class that extends the IE_Imp baseclass. After overriding the IE_Imp class, you must define several other methods:

RecognizeSuffix(const char * szSuffix)

StaticConstructor(PD_Document * pDocument, IE_Imp ** ppie)

SupportsFileType(IEFileType ft)

GetDlgLabels(const char ** pszDesc, const char ** pszSuffixList, IEFileType * ft)

RecognizeContents(const char * szBuf, UT_uint32 iNumbytes)

The code required to implement these methods is extremely trivial, usually requiring (at most) a few lines of code. For example, the RecognizeSuffix method requires that you perform a case-insensitive comparison of the passed string and your filetype. For MSWord Documents, the code is: `return (UT_stricmp (szSuffix, ".doc"));´ In most cases, you can cut-and-paste code from existing importers to implement these methods.

After that, all that is required is that you implement the `importFile(const char * szFilename)´ method. The importFile method can utilize any number of other functions or methods to import the requested file. It is assumed by this point that the passed szFilename probably represents your filetype. If it does not, you can always return UT_IE_BOGUSDOCUMENT, meaning that for some reason or another, your importer could not import the given document.

All importers contain an abstraction class of AbiWord´s document - the PD_Document class-member m_pDocument. m_pDocument has several methods of great interest to us:

appendFmt ()

appendStrux ()

appendFmt appends the passed formatting properties to the current run of text. These properties are expected to be identical to those contained/allowed in any AbiWord document. For example:

XML_Char * buf [3];

buf[0] = "props";

buf[1] = "font-family:Courier New; font-size:14pt; font-style:italic; font-weight:bold";

buf[2] = NULL;

m_pDocument->appendFmt (buf);

The appendStrux() method is used to append a section or paragraph to a given document. The first parameter is one of the enum { PTX_Section, PTX_Block }; The second parameter is identical in nature to the XML_Char ** buf used in appendFmt ();

4.1 IE_Imp_XML: an exception

A special case of an importer is an XML importer - that is, any class meant to import XML content. Dom Lachowicz created an abstract IE_Imp_XML baseclass which takes care of many of the necessary details needed to import an XML document.

The baseclass utilizes either the LibXML2 or the Expat XML library´s SAX parser to import documents. In addition to the 5 methods listed above, the XML importer class expects two other methods to exist:

_startElement (const XML_Char *name, const XML_Char **atts)

_endElement (const XML_Char *name, const XML_Char **atts)

Additionally, the XML importer provides one function for writing data into AbiWord:

_charData(const XML_Char *s, int len)

The _charData takes as parameters a string (XML_Char *s) and that string´s length (len). It also tries to take into account if white space is significant or not (ABW is, HTML is not). From the source of _charData:

// parse UTF-8 text and convert to Unicode.

// also take care of some white-space issues:

// [] convert CRLF to SP.

// [] convert CR to SP.

// [] convert LF to SP.

_startElement and _endElement are called (respectively) whenever a starting or ending tag is encountered in your XML document. Implementing these methods is extremely easy and makes writing an XML-based importer fairly simple. @name represents the tag´s name (e.g.: "table") while @atts represents any attributes that accompany that tag (e.g. "border=0 cellspacing=1 cellpadding=1"). Mapping these @name tags to integer tokens is simplified (and sped-up) by using the mapNameToToken function. See and XML-based importer for an example of usage.

5. The export (output) mechanism

Writing an exporter is usually slightly more difficult that writing an importer, but many aspects of writing an exporter are similar to those learned by writing an importer.

Firstly, your importer must implement several (4) methods:

RecognizeSuffix(const char * szSuffix)

StaticConstructor(PD_Document * pDocument, IE_Exp ** ppie)

SupportsFileType(IEFileType ft)

GetDlgLabels(const char ** pszDesc, const char ** pszSuffixList, IEFileType * ft)

These should look familiar enough to not explain them.

[TODO]