Include only the necessary functions from libpcap

Question

I have a C++ JNI Library (that I did not write myself) that uses the functionality of libpcap. There are only a few functions that are actually being used from libpcap but when the library is compiled, every function gets included. Perhaps it's a flag? I've searched and can't seem to find an answer to my question.

The older developers were using BJAM to do the compile, and I've recently been tasked to convert it to GNU Make.

Here are all the pcap symbols in my Make compile:

00000000000197f0 T _pcap_activate
0000000000019d00 T _pcap_breakloop
0000000000019070 T _pcap_can_set_rfmon
0000000000019bb0 T _pcap_close
000000000001b560 T _pcap_compile
000000000001c180 T _pcap_compile_nopcap
00000000000192c0 T _pcap_create
0000000000019d10 T _pcap_datalink
0000000000019d30 T _pcap_datalink_ext
0000000000019ff0 T _pcap_datalink_name_to_val
000000000001a080 T _pcap_datalink_val_to_description
0000000000019f70 T _pcap_datalink_val_to_name
00000000000191c0 T _pcap_dispatch
000000000003b660 T _pcap_dump
000000000003bde0 T _pcap_dump_close
000000000003bda0 T _pcap_dump_file
000000000003bdc0 T _pcap_dump_flush
000000000003b8d0 T _pcap_dump_fopen
000000000003bdb0 T _pcap_dump_ftell
000000000003b6d0 T _pcap_dump_open
000000000003b940 T _pcap_dump_open_append
000000000003a650 T _pcap_ether_aton
000000000003a780 T _pcap_ether_hostton
000000000001a240 T _pcap_file
000000000001a250 T _pcap_fileno
0000000000019220 T _pcap_findalldevs
000000000003acd0 T _pcap_fopen_offline
000000000003a910 T _pcap_fopen_offline_with_tstamp_precision
0000000000019e20 T _pcap_free_datalinks
0000000000019140 T _pcap_free_tstamp_types
000000000001af60 T _pcap_freealldevs
000000000001c1f0 T _pcap_freecode
000000000001a260 T _pcap_get_selectable_fd
00000000000197e0 T _pcap_get_tstamp_precision
000000000001a2a0 T _pcap_geterr
000000000001a2b0 T _pcap_getnonblock
000000000001a760 T _pcap_inject
000000000001a1e0 T _pcap_is_swapped
000000000001a7a0 T _pcap_lib_version
0000000000019d50 T _pcap_list_datalinks
0000000000019080 T _pcap_list_tstamp_types
000000000001b010 T _pcap_lookupdev
000000000001b0a0 T _pcap_lookupnet
0000000000019c80 T _pcap_loop
000000000001a200 T _pcap_major_version
000000000001a220 T _pcap_minor_version
000000000003a150 T _pcap_nametoaddr
000000000003a1a0 T _pcap_nametoaddrinfo
000000000003a4c0 T _pcap_nametoeproto
000000000003a520 T _pcap_nametollc
000000000003a1f0 T _pcap_nametonetaddr
000000000003a210 T _pcap_nametoport
000000000003a2c0 T _pcap_nametoportrange
000000000003a4a0 T _pcap_nametoproto
0000000000019180 T _pcap_next
00000000000191d0 T _pcap_next_ex
000000000001a770 T _pcap_offline_filter
000000000001a6c0 T _pcap_open_dead
000000000001a5f0 T _pcap_open_dead_with_tstamp_precision
00000000000199a0 T _pcap_open_live
000000000003ab20 T _pcap_open_offline
000000000003a840 T _pcap_open_offline_with_tstamp_precision
000000000001a270 T _pcap_perror
000000000001a740 T _pcap_sendpacket
0000000000019720 T _pcap_set_buffer_size
0000000000019e30 T _pcap_set_datalink
00000000000196e0 T _pcap_set_immediate_mode
00000000000195a0 T _pcap_set_promisc
00000000000195e0 T _pcap_set_rfmon
0000000000019560 T _pcap_set_snaplen
0000000000019620 T _pcap_set_timeout
0000000000019760 T _pcap_set_tstamp_precision
0000000000019660 T _pcap_set_tstamp_type
000000000001a490 T _pcap_setdirection
000000000001a480 T _pcap_setfilter
000000000001a370 T _pcap_setnonblock
000000000001a1c0 T _pcap_snapshot
000000000001a4d0 T _pcap_stats
0000000000019940 T _pcap_statustostr
0000000000019130 T _pcap_strerror
000000000001a0c0 T _pcap_tstamp_type_name_to_val
000000000001a180 T _pcap_tstamp_type_val_to_description
000000000001a140 T _pcap_tstamp_type_val_to_name

And these are the only functions that should be included (BJAM compile):

             U _pcap_close
             U _pcap_findalldevs
             U _pcap_freealldevs
             U _pcap_geterr
             U _pcap_next
             U _pcap_open_live
             U _pcap_sendpacket
             U _pcap_stats

Edit

I just noticed now that these symbols are undefined... Now I am very confused. Are they using a shared version of libpcap?

Edit 2 Output from file on make library:

custompcap-1.2-r8581.jnilib: Mach-O universal binary with 2 architectures
custompcap-1.2-r8581.jnilib (for architecture x86_64):  Mach-O 64-bit dynamically linked shared library x86_64
custompcap-1.2-r8581.jnilib (for architecture i386):    Mach-O dynamically linked shared library i386

Output from BJAM library:

target/x86_64-apple-darwin14.1.0/lib/custompcap.jnilib: Mach-O 64-bit dynamically linked shared library x86_64

Edit 3

Here is the linker

c++ -stdlib=libstdc++ -lstdc++ -Wl,-single_module -install_name custompcap.jnilib -dynamiclib  -o target/x86_64-apple-darwin14.1.0/lib/custompcap.jnilib target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/customcap/JNICapture.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/customcap/JNINetworkInterface.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/customcap/JNIcustomcap.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/util/HexDump.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/util/Timer.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/util/DataStream.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/frag/IPDefrag.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/frag/IPFrag.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/pcap/PcapInput.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/pcap/PcapOutput.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/customcap/IPPacket.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/customcap/PacketBundle.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/customcap/PacketDispatch.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/customcap/Headers.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/customcap/PacketRule.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/customcap/CIDR.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/customcap/BoyerMoore.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/customcap/Piranha.o target/x86_64-apple-darwin14.1.0/obj/src/main/cpp/customcap/Packetcustomcap.o  -L/Users/vdawn/src/3rdparty/pcap/libpcap/1.8.0-PRE-GIT/ -lpcap -lpcre -headerpad_max_install_names -Wl,-dead_strip -no_dead_strip_inits_and_terms -arch x86_64

Thanks

What file are you running `nm` on to produce the first listing, what file are you running `nm` on to produce the second listing, and what is printed if you run `file` on both of those files? (I'm assuming you're doing this on some flavor of UN\*X.) — , Feb 09 '16 at 23:06
@GuyHarris Thanks for the response. I am using `nm` without any parameters on both BJAM and Make compiled libraries. This is being done on OSX. I am updating OP to show output of file. — user0000001, Feb 10 '16 at 17:17
And the `c++` command is what produces the Make version? Or is it what produces the BJAM version? If it's what produces the BJAM version, what command produces the Make version? — , Feb 10 '16 at 18:53
@GuyHarris Unfortunately the prior developers did not provide any documentation on the environment setup. There is missing configuration for the BJAM compile, enough to not allow me to repeat the build on my OSX computer. So that is why I am having to convert this into a Makefile. The new library works as intended and I haven't had any issues with it thus far, but the size of the library is about double. And I can only assume at this point, that it's a result of libpcap and all of it's subroutines being linked. — user0000001, Feb 10 '16 at 20:12
@GuyHarris I'd consider it done but when this hits the dev channel and QA see's that there is a significant size difference in the two libraries, they will ask why. I will have to show them and then eventually it will be kicked back to me to try and provide a solution. So, if it was as simple as hand picking which functions get included, that's the route I would go. But I don't know of any such feature/flag in c++. — user0000001, Feb 10 '16 at 20:16

score 1 · Accepted Answer · answered Feb 10 '16 at 20:50

Without an indication of what command built the BJAM version, there are strict limits on what I can suggest if you want to make the Makefile build the same way BJAM did, given that I don't know what BJAM did.

However, if we look at the command used for Make, it's not linking with the system version of libpcap, it's linking with a snapshot of tcpdump.org's master branch:

c++ ... -L/Users/vdawn/src/3rdparty/pcap/libpcap/1.8.0-PRE-GIT/ -lpcap ...

If what's being linked with is a static version of libpcap, then it will include every object file in the static library that contains a symbol to which the JNI library refers.

This means that if you use, for example, pcap_open_live(), it will include the object file that defines it. That's pcap.o, and pcap.o defines the following symbols:

                 U ___bzero
                 U ___error
                 U ___snprintf_chk
                 U ___stderrp
                 U ___strlcpy_chk
                 U _atexit
                 U _bpf_filter
                 U _calloc
000000000000c160 S _capture_source_types
0000000000008ed0 s _charmap
                 U _close
000000000000c18a b _did_atexit
0000000000009000 s _dlt_choices
                 U _fcntl
                 U _fprintf
                 U _free
                 U _malloc
                 U _memcpy
00000000000007e0 T _pcap_activate
00000000000017e0 T _pcap_add_to_pcaps_to_close
0000000000000440 t _pcap_alloc_pcap_t
0000000000000cd0 T _pcap_breakloop
0000000000000000 T _pcap_can_set_rfmon
00000000000004e0 t _pcap_cant_set_rfmon
0000000000000530 T _pcap_check_activated
00000000000019d0 t _pcap_cleanup_dead
0000000000001850 T _pcap_cleanup_live_common
0000000000000b80 T _pcap_close
0000000000001790 t _pcap_close_all
0000000000000250 T _pcap_create
00000000000002d0 T _pcap_create_common
                 U _pcap_create_interface
0000000000000ce0 T _pcap_datalink
0000000000000d00 T _pcap_datalink_ext
0000000000001000 T _pcap_datalink_name_to_val
0000000000001070 T _pcap_datalink_val_to_description
0000000000000f60 T _pcap_datalink_val_to_name
                 U _pcap_debug
0000000000000150 T _pcap_dispatch
0000000000001730 T _pcap_do_addexit
0000000000001490 T _pcap_file
00000000000014a0 T _pcap_fileno
00000000000001b0 T _pcap_findalldevs
                 U _pcap_findalldevs_interfaces
0000000000000df0 T _pcap_free_datalinks
00000000000000d0 T _pcap_free_tstamp_types
                 U _pcap_freealldevs
                 U _pcap_freecode
00000000000014b0 T _pcap_get_selectable_fd
00000000000007d0 T _pcap_get_tstamp_precision
00000000000014f0 T _pcap_geterr
0000000000001500 T _pcap_getnonblock
0000000000001550 T _pcap_getnonblock_fd
0000000000001a80 T _pcap_inject
0000000000001430 T _pcap_is_swapped
0000000000001ac0 T _pcap_lib_version
0000000000000d20 T _pcap_list_datalinks
0000000000000010 T _pcap_list_tstamp_types
0000000000000c50 T _pcap_loop
0000000000001450 T _pcap_major_version
0000000000001470 T _pcap_minor_version
0000000000000110 T _pcap_next
0000000000000160 T _pcap_next_ex
0000000000001af0 t _pcap_not_initialized
0000000000001a90 T _pcap_offline_filter
                 U _pcap_offline_read
00000000000000e0 T _pcap_oneshot
00000000000019e0 T _pcap_open_dead
0000000000001910 T _pcap_open_dead_with_tstamp_precision
0000000000000990 T _pcap_open_live
0000000000000bc0 T _pcap_open_offline_common
                 U _pcap_optimizer_debug
00000000000014c0 T _pcap_perror
                 U _pcap_platform_finddevs
0000000000001800 T _pcap_remove_from_pcaps_to_close
0000000000001a60 T _pcap_sendpacket
0000000000000700 T _pcap_set_buffer_size
0000000000000e00 T _pcap_set_datalink
00000000000006c0 T _pcap_set_immediate_mode
0000000000001ae0 T _pcap_set_optimizer_debug
0000000000001ad0 T _pcap_set_parser_debug
0000000000000570 T _pcap_set_promisc
00000000000005b0 T _pcap_set_rfmon
00000000000004f0 T _pcap_set_snaplen
00000000000005f0 T _pcap_set_timeout
0000000000000740 T _pcap_set_tstamp_precision
0000000000000630 T _pcap_set_tstamp_type
00000000000016e0 T _pcap_setdirection
00000000000016d0 T _pcap_setfilter
00000000000015c0 T _pcap_setnonblock
0000000000001610 T _pcap_setnonblock_fd
0000000000001410 T _pcap_snapshot
0000000000001720 T _pcap_stats
00000000000019a0 t _pcap_stats_dead
0000000000000930 T _pcap_statustostr
000000000000c170 b _pcap_statustostr.ebuf
0000000000000fc0 T _pcap_strcasecmp
00000000000000c0 T _pcap_strerror
00000000000010c0 T _pcap_tstamp_type_name_to_val
00000000000013c0 T _pcap_tstamp_type_val_to_description
0000000000001370 T _pcap_tstamp_type_val_to_name
0000000000008fd0 s _pcap_version_string
000000000000c190 b _pcaps_to_close
                 U _strdup
                 U _strerror
                 U _strncpy
0000000000009ba0 s _tstamp_type_choices
0000000000009c30 s l_switch.table

so you're going to get all the "T" symbols in that list in your library, and there is nothing you can do about it other than significantly hack the libpcap source to put different symbols into different files.

Now, if you build against a shared version of libpcap, you won't get any of those symbols - you'll get a dynamically-linked JNI library that will, when loaded, attach to the shared version of libpcap and get the symbols from there.

That may be what the old BJAM build did. If you don't need any of the APIs added to libpcap since the 1.5.3 version (Apple's currently stuck at 1.5.3; I'm hoping that, once I add more pcapng support, and support for PKTAP, to the standard version, I can get them to pick up a newer version), you may be better off linking with the system version of libpcap rather than with a snapshot from tcpdump.org (note that anything with "PRE-GIT" in the name is Subject To Change Without Notice when a release comes out - it'll still be compatible with 1.7.x, but it's not guaranteed to be compatible with earlier 1.8.0-PRE-GIT versions).

(You're also getting a "fat" version of the JNI library in your Make build, with both IA-32 and x86-64 code, which also makes it bigger than the BJAM version.)

Oh man, I meant to switch those around. The BJAM is building for both archs, while Make is targetting x86_64. And I probably should have dug deeper into OSX libpcap, I was not aware that it shipped with it. That's great information. Thank you so much for the help and the time. I will post my results with the information you provided. — user0000001, Feb 10 '16 at 20:57
So what is printed if you run `otool -L` on the BJAM-built JNI library and on the Make-build JNI library? — , Feb 10 '16 at 21:14
The BJAM, static libpcap and the newly compiled one with the changes you mentioned above were all the same for `otool`. Without `otool`, I'd still be pulling my teeth with `libc++`. Anyway, I've made the corrections in my `Makefile` and now I am seeing the same symbols in contrast to the BJAM library. Thank you again. Cheers! — user0000001, Feb 11 '16 at 19:23
So all three of them either were, or weren't, dynamically linked with libpcap? ("Were", as in "`otool -L` lists /usr/lib/libpcap.A.dylib (compatibility version 1.0.0, current version 1.0.0)", or "weren't", as in "`otool -L` doesn't mention libpcap".) — , Feb 11 '16 at 23:56
Each of them had the libpcap dep in `otool -L`. Including the compile prior to the changes you've mentioned in this post. — user0000001, Feb 12 '16 at 18:40
I.e., one of them 1) defined all those symbols in from libpcap (I.e., "T") *AND* had libpcap.dylib as a dependency listed by `otool -L`? — , Feb 12 '16 at 21:18
Yes, the one that I compiled had both T symbols of libpcap (_pcap_*) and the libpcap dep listed by `otool -L`. — user0000001, Feb 12 '16 at 21:28
Well, that's weird; I guess it was somehow linked with *both* a static libpcap (so that it pulled into the JNI library a whole bunch of .o files from the static libpcap, defining all the symbols) *and* a dynamic libpcap (so it contained a Mach-O item saying "I depend on the dynamic libpcap") even though it didn't actually need anything from the dynamic libpcap (as it got everything it needed from the static one). — , Feb 12 '16 at 23:30

Include only the necessary functions from libpcap

1 Answers1