Slow SAX parsing of large CDATA? 2003-02-11 - By Daniel Rabe
I'm using SAX (Xerces 2.3.0 on Windows XP) to parse an XML file that can
contain large CDATA sections (where large is somewhere between 1 and 5 Mb).
The data is Base64-encoded. The code works properly, but when the CDATA is
over 1Mb or so, it's very slow. It seems like a 1Mb CDATA section can be
processed in several seconds, but once it gets up to about 3 or 4 Mb,
processing time goes up to about 10 minutes. It seems like Xerces is
building up a huge buffer of all the data before calling my characters
callback. (I'd prefer to get many characters callbacks so I can stream the
data to a file, rather than accumulating all the data in memory.) This is
the stack crawl I get while it's processing. Garbage collection is also very
active during this process. It doesn't seem to matter whether my max heap is
set to 128Mb or 256Mb... behavior is the same.
at org.apache.xerces.util.XMLStringBuffer.append(Unknown Source) at
org.apache.xerces.impl.XMLEntityScanner.scanData(Unknown Source) at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanCDATASection(Unkno
wn Source) at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatc
her.dispatch(Unknown Source) at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
Source) at org.apache.xerces.parsers.DTDConfiguration.parse(Unknown Source)
at org.apache.xerces.parsers.DTDConfiguration.parse(Unknown Source) at
org.apache.xerces.parsers.XMLParser.parse(Unknown Source) at
org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
Since my data is base64-encoded, I don't really need the CDATA... I can just
treat it like element data. If I do this, I get a characters callback for
each line of the encoded data, and it's wonderfully fast. Unfortunately, the
XML files that I need to process are provided by another vendor and contain
the CDATA.
Has anybody else run into this? Any workarounds, or any way to give xerces a
clue that I want more frequent characters callbacks?
Thanks,
Daniel Rabe
drabe@(protected)
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<META NAME="Generator" CONTENT="MS Exchange Server version 5.5.2653.12">
<TITLE>Slow SAX parsing of large CDATA?</TITLE>
</HEAD>
<BODY>
<P><FONT SIZE=2>I'm using SAX (Xerces 2.3.0 on Windows XP) to parse an XML file
that can contain large CDATA sections (where large is somewhere between 1 and 5
Mb). The data is Base64-encoded. The code works properly, but when the CDATA is
over 1Mb or so, it's very slow. It seems like a 1Mb CDATA section can be
processed in several seconds, but once it gets up to about 3 or 4 Mb,
processing time goes up to about 10 minutes. It seems like Xerces is building
up a huge buffer of all the data before calling my characters callback. (I'd
prefer to get many characters callbacks so I can stream the data to a file,
rather than accumulating all the data in memory.) This is the stack crawl I get
while it's processing. Garbage collection is also very active during this
process. It doesn't seem to matter whether my max heap is set to 128Mb or 256Mb
... behavior is the same.</FONT></P>
<P><FONT SIZE=2>at org.apache.xerces.util.XMLStringBuffer.append(Unknown Source
) at org.apache.xerces.impl.XMLEntityScanner.scanData(Unknown Source) at org
.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanCDATASection(Unknown
Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl
$FragmentContentDispatcher.dispatch(Unknown Source) at org.apache.xerces.impl
.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source) at org.apache
.xerces.parsers.DTDConfiguration.parse(Unknown Source) at org.apache.xerces
.parsers.DTDConfiguration.parse(Unknown Source) at org.apache.xerces.parsers
.XMLParser.parse(Unknown Source) at org.apache.xerces.parsers.AbstractSAXParser
.parse(Unknown Source)</FONT></P>
<P><FONT SIZE=2>Since my data is base64-encoded, I don't really need the CDATA.
.. I can just treat it like element data. If I do this, I get a characters
callback for each line of the encoded data, and it's wonderfully fast.
Unfortunately, the XML files that I need to process are provided by another
vendor and contain the CDATA.</FONT></P>
<P><FONT SIZE=2>Has anybody else run into this? Any workarounds, or any way to
give xerces a clue that I want more frequent characters callbacks?</FONT></P>
<P><FONT SIZE=2>Thanks,</FONT>
<BR><FONT SIZE=2>Daniel Rabe</FONT>
<BR><FONT SIZE=2>drabe@(protected)</FONT>
</P>
</BODY>
</HTML>
|
|
