How to avoid high CPU usage with pysnmp

Question

I am using pysnmp and have encountered high CPU usage. I know netsnmp is written in C and pysnmp in Python, so I would expect the CPU usage times to be about 20-100% higher because of that. Instead I am seeing 20 times higher CPU usage times.

Am I using pysnmp correctly or could I do something to make it use less resources?

Test case 1 - PySNMP:

from pysnmp.entity.rfc3413.oneliner import cmdgen
import config
import yappi

yappi.start()
cmdGen = cmdgen.CommandGenerator()
errorIndication, errorStatus, errorIndex, varBindTable = cmdGen.nextCmd(
    cmdgen.CommunityData(config.COMMUNITY),
    cmdgen.UdpTransportTarget((config.HOST, config.PORT)),
    config.OID,
    lexicographicMode=False,
    ignoreNonIncreasingOid=True,
    lookupValue=False, lookupNames=False
)
for varBindTableRow in varBindTable:
    for name, val in varBindTableRow:
        print('%s' % (val,))
yappi.get_func_stats().print_all()

Test case 2 - NetSNMP:

import argparse
import netsnmp
import config
import yappi

yappi.start()
oid = netsnmp.VarList(netsnmp.Varbind('.'+config.OID))
res = netsnmp.snmpwalk(oid, Version = 2, DestHost=config.HOST, Community=config.COMMUNITY)
print(res)
yappi.get_func_stats().print_all()

If someone wants to test for himself, both test cases need a small file with settings, config.py:

HOST = '192.168.1.111'
COMMUNITY = 'public'
PORT = 161
OID = '1.3.6.1.2.1.2.2.1.8'

I have compared the returned values and they are the same - so both examples function correctly. The difference is in timings:

PySNMP:

Clock type: cpu
Ordered by: totaltime, desc

name                                    #n         tsub      ttot      tavg
..dgen.py:408 CommandGenerator.nextCmd  1          0.000108  1.890072  1.890072
..:31 AsynsockDispatcher.runDispatcher  1          0.005068  1.718650  1.718650
..r/lib/python2.7/asyncore.py:125 poll  144        0.010087  1.707852  0.011860
/usr/lib/python2.7/asyncore.py:81 read  72         0.001191  1.665637  0.023134
..UdpSocketTransport.handle_read_event  72         0.001301  1.664446  0.023117
..py:75 UdpSocketTransport.handle_read  72         0.001888  1.663145  0.023099
..base.py:32 AsynsockDispatcher._cbFun  72         0.001766  1.658938  0.023041
..:55 SnmpEngine.__receiveMessageCbFun  72         0.002194  1.656747  0.023010
..4 MsgAndPduDispatcher.receiveMessage  72         0.008587  1.654553  0.022980
..eProcessingModel.prepareDataElements  72         0.014170  0.831581  0.011550
../ber/decoder.py:585 Decoder.__call__  1224/216   0.111002  0.801783  0.000655
...py:312 SequenceDecoder.valueDecoder  288/144    0.034554  0.757069  0.002629
..tCommandGenerator.processResponsePdu  72         0.008425  0.730610  0.010147
..NextCommandGenerator._handleResponse  72         0.008692  0.712964  0.009902
...

NetSNMP:

Clock type: cpu
Ordered by: totaltime, desc

name                                    #n         tsub      ttot      tavg
..kages/netsnmp/client.py:227 snmpwalk  1          0.000076  0.103274  0.103274
..s/netsnmp/client.py:173 Session.walk  1          0.000024  0.077640  0.077640
..etsnmp/client.py:48 Varbind.__init__  72         0.008860  0.035225  0.000489
..tsnmp/client.py:111 Session.__init__  1          0.000055  0.025551  0.025551
...

So, netsnmp uses 0.103 s of CPU time and pysnmp uses 1.890 s of CPU time for the same operation. I find the results surprising... I have also tested the asynchronous mode, but the results were even a bit worse.

Am I doing something wrong (with pysnmp)?

UPDATE:

As per Ilya's suggestion, I have tryed using BULK instead of WALK. BULK is indeed much faster overall, but PySNMP still uses cca. 20x CPU time in comparison to netsnmp:

..dgen.py:496 CommandGenerator.bulkCmd  1          0.000105  0.726187  0.726187

Netsnmp:

..es/netsnmp/client.py:216 snmpgetbulk  1          0.000109  0.044421  0.044421

So the question still stands - can I make pySNMP less CPU intensive? Am I using it incorrectly?

score 2 · Accepted Answer · answered Feb 28 '14 at 12:48

2

Try using GETBULK instead of GETNEXT. With your code and Max-Repetitions=25 setting it gives 5x times performance improvement on my synthetic test.

answered Feb 28 '14 at 12:48

Ilya Etingof

5,440
1
17
21

Ilya, thanks for the suggestion. I have been avoiding BULK because I am not 100% sure I always get all of the data... Will look into it and try to use it. However, comparing the PySNMP BULK and netsnmp BULK the problem is still the same (I have updated the question with new stats). – johndodo Mar 03 '14 at 09:21
PySNMP is indeed much slower than virtually any C-based solution. Most resources go into BER data processing so it's clearly a hotspot. In theory it could be optimized by using a special-purpose (SNMP packet specific) BER codec what would streamline octetstream processing, however such codec would become non-reusable in other (non-SNMP) applications. – Ilya Etingof Mar 03 '14 at 11:46
If you would be interested in optimizing round-trip delays, async design might help. But since CPU seems like a bottleneck here, I can't think of anything but the reduction of the number of SNMP packets to be processed. So GETBULK might help, but Net-SNMP will always win in terms of performance. – Ilya Etingof Mar 03 '14 at 11:52

How to avoid high CPU usage with pysnmp

1 Answers1