使用valgrind查看xerces-c++内存使用情况

本文结构

上一篇说到xerces会耗费大量内存，今天我们用工具valgrind统计下各个函数申请堆内存的比例。我会先写一小段demo，它会创建很多节点； 然后使用valgrind massif查看heap profiler.

1) demo

本demo会创建大约5000个Product节点，每个Product有70个子节点Item(你可以试着把字符串改成全都不一样的，结果会不一样), 如下

 <Company>
          <Product1>
            <Item>MEMORY TEST 1Item>
            <Item>MEMORY TEST 1Item>
            <Item>MEMORY TEST 1Item>
            <Item>MEMORY TEST 1Item>
            <Item>MEMORY TEST 1Item>
            <Item>MEMORY TEST 1Item>
            <Item>MEMORY TEST 1Item>
          Product1>

          <Product2>
            <Item>MEMORY TEST 2Item>
            <Item>MEMORY TEST 2Item>
            <Item>MEMORY TEST 2Item>
            <Item>MEMORY TEST 2Item>
            <Item>MEMORY TEST 2Item>
            <Item>MEMORY TEST 2Item>
            <Item>MEMORY TEST 2Item>
          Product2>
        Company>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

主要代码如下，外围代码请参考xerces下的samples/createDOMDocument.cpp

DOMDocument* doc = impl->createDocument(
				   0,                    // root element namespace URI.
				   X("Company"),         // root element name
				   0);                   // document type object (DTD).

DOMElement* rootElem = doc->getDocumentElement();

for(int i=1; i<=5000; i++){
	DOMElement*  prodElem = doc->createElement(X((std::string("Product")+std::to_string(i)).c_str()));
	for(int j=1; j<70; j++){
		DOMElement*  itemElem = doc->createElement(X("Item"));
		DOMText*    itemDataVal = doc->createTextNode(X((std::string("MEMORY TEST ")+std::to_string(i)).c_str()));
		itemElem->appendChild(itemDataVal);
		prodElem->appendChild(itemElem);
   }
   rootElem->appendChild(prodElem);
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

2) heap profiler结果

$ valgrind --tool=massif ./a.out
$ ms_print massif.out.*

99.97% (111,454,411B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->99.90% (111,378,444B) 0x4FF6168: xercesc_3_1::MemoryManagerImpl::allocate(unsigned long) (MemoryManagerImpl.cpp:40)
| ->99.68% (111,132,672B) 0x4F9EB13: xercesc_3_1::DOMDocumentImpl::allocate(unsigned long) (DOMDocumentImpl.cpp:884)
| | ->48.92% (54,542,336B) 0x4FA2A36: xercesc_3_1::DOMDocumentImpl::allocate(unsigned long, xercesc_3_1::DOMMemoryManager::NodeObjectType) (DOMDocumentImpl.cpp:1502)
| | | ->33.39% (37,224,448B) 0x4FA0F1A: operator new (DOMDocumentImpl.hpp:454)
| | | | ->33.39% (37,224,448B) 0x4FA0F1A: xercesc_3_1::DOMDocumentImpl::createElement(unsigned short const*) (DOMDocumentImpl.cpp:342)
| | | |   ->32.92% (36,700,160B) 0x4024D3: main (xercesMemoryTest.cpp:175)
| | | |   |
| | | |   ->00.47% (524,288B) in 1+ places, all below ms_print's threshold (01.00%)
| | | |
| | | ->15.52% (17,301,504B) 0x4F9EFE9: operator new (DOMDocumentImpl.hpp:454)
| | | | ->15.52% (17,301,504B) 0x4F9EFE9: xercesc_3_1::DOMDocumentImpl::createTextNode(unsigned short const*) (DOMDocumentImpl.cpp:407)
| | | |   ->15.52% (17,301,504B) 0x402594: main (xercesMemoryTest.cpp:176)
| | | |
| | | ->00.01% (16,384B) in 1+ places, all below ms_print's threshold (01.00%)
| | |
| | ->17.63% (19,660,800B) 0x4FAA424: operator new (DOMDocumentImpl.hpp:469)
| | | ->17.63% (19,660,800B) 0x4FAA424: xercesc_3_1::DOMElementImpl::DOMElementImpl(xercesc_3_1::DOMDocument*, unsigned short const*) (DOMElementImpl.cpp:53)
| | |   ->17.63% (19,660,800B) 0x4FA0F2E: xercesc_3_1::DOMDocumentImpl::createElement(unsigned short const*) (DOMDocumentImpl.cpp:342)
| | |     ->17.63% (19,660,800B) 0x4024D3: main (xercesMemoryTest.cpp:175)
| | |
| | ->17.08% (19,038,208B) 0x4F9B7FE: operator new (DOMDocumentImpl.hpp:469)
| | | ->17.08% (19,038,208B) 0x4F9B7FE: xercesc_3_1::DOMCharacterDataImpl::DOMCharacterDataImpl(xercesc_3_1::DOMDocument*, unsigned short const*) (DOMCharacterDataImpl.cpp:40)
| | |   ->17.08% (19,038,208B) 0x4FBF1E2: xercesc_3_1::DOMTextImpl::DOMTextImpl(xercesc_3_1::DOMDocument*, unsigned short const*) (DOMTextImpl.cpp:47)
| | |     ->17.08% (19,038,208B) 0x4F9EFFB: xercesc_3_1::DOMDocumentImpl::createTextNode(unsigned short const*) (DOMDocumentImpl.cpp:407)
| | |       ->17.08% (19,038,208B) 0x402594: main (xercesMemoryTest.cpp:176)
| | |
| | ->15.58% (17,367,040B) 0x4FBE238: xercesc_3_1::DOMBuffer::DOMBuffer(xercesc_3_1::DOMDocumentImpl*, unsigned long) (DOMStringPool.cpp:38)
| | | ->15.58% (17,367,040B) 0x4F9B812: xercesc_3_1::DOMCharacterDataImpl::DOMCharacterDataImpl(xercesc_3_1::DOMDocument*, unsigned short const*) (DOMCharacterDataImpl.cpp:40)
| | |   ->15.58% (17,367,040B) 0x4FBF1E2: xercesc_3_1::DOMTextImpl::DOMTextImpl(xercesc_3_1::DOMDocument*, unsigned short const*) (DOMTextImpl.cpp:47)
| | |     ->15.58% (17,367,040B) 0x4F9EFFB: xercesc_3_1::DOMDocumentImpl::createTextNode(unsigned short const*) (DOMDocumentImpl.cpp:407)
| | |       ->15.58% (17,367,040B) 0x402594: main (xercesMemoryTest.cpp:176)
| | |
| | ->00.47% (524,288B) in 1+ places, all below ms_print's threshold (01.00%)
| |
| ->00.22% (245,772B) in 1+ places, all below ms_print's threshold (01.00%)
|
->00.07% (75,967B) in 1+ places, all below ms_print's threshold (01.00%)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

一共大约耗费了111M的堆内存（而如果转成字符串也就10M左右），主要耗费内存的地方：

1. 33.39% (37,224,448B) 创建DOMElementImpl（350000个Node）。
2. 15.52% (17,301,504B) 创建DOMTextImpl （350000个）
3. 17.63% (19,660,800B) 创建DOMAttrMapImpl
4. 17.08% (19,038,208B) 创建DOMBuffer
5. 15.58% (17,367,040B) 创建DOMBuffer下的capacity