Debugging & troubleshooting tutorial: core dump debug

As you know, there are LIMIT settings in Linux OS which may have something to do with some frequent-happen-issue. Such as stack size, open files, core file generation, etc.

Here are some tips about ulimit, core, and some debugging tricks relevant.

1. ulimit comamnd

——————————

To view your OS limitations, you can use command:

ulimit -a

It will show you the current OS limitations for your environment like this:

[weiguohua@localhost ~]$ ulimit -a

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
pending signals                 (-i) 1024
max locked memory       (kbytes, -l) 32
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 16384
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

2. Enable core file
——————————

If you wish to generate a core file when your program crashed, you can use a command like this:

ulimit -c 123456789

The number following ulimit -c is the max core file size, you can set a number as you wish.

And there are rules for core file name, they are saved in:

1) /proc/sys/kernel/core_pattern

Used to control the core file name format
Default value: core
You can change it to “/home/weiguohua/corefile/core-%e-%p-%t” which means all the core file will be saved to the folder “/home/weiguohua/corefile”, and the core file name will be “core-[program name]-[pid]-[crash-time]”.

echo /home/weiguohua/corefile/core-%e-%p-%t > /proc/sys/kernel/core_pattern

 2) /proc/sys/kernel/core_uses_pid

0: Default setting, means the core file will not suffixed with a PID.
1: Add a PID suffix to the core file, such as “core.9721”

You can change it by command:

echo 1 > /proc/sys/kernel/core_uses_pid

You can man core to reference the naming rule of core dump files.

3. Debug a core file by GDB
——————————

Debugging a core file in some way is nothing different with debugging the program in a normal way.
Assume your program name is segment.out, and the generated core file name is core.9721, you can start your debug by command:

gdb ./segment.out core.9721

Then your can do anything you want for the core file, such as bt, p, etc.

P.S. If a core dump happened, and you can not locate the function by gdb backtrace, you can:
1) Use ldd to check out the so files(dynamic link libraries) this program depends on.
2) Use nm to list the symbols of the so files, and search the addresses in the core file.
3) Locate the functions by comparing the addresses in core file and the symbol map listed by nm command.

4. Memory maps for a running program
——————————

You can view the address map & range for a running program by simply cat the maps file in the /proc/[pid] folder.
It will print out the detail memory map of all the symbols the program loaded, including the so files it depended on.

[weiguohua@localhost 9721]# pwd
/proc/9721
[weiguohua@localhost 9721]# cat maps

00110000-00113000 r-xp 00000000 fd:00 677857     /lib/libdl-2.12.so
00113000-00114000 r–p 00002000 fd:00 677857     /lib/libdl-2.12.so
00114000-00115000 rw-p 00003000 fd:00 677857     /lib/libdl-2.12.so
00115000-00127000 r-xp 00000000 fd:00 677856     /lib/libz.so.1.2.3
00127000-00128000 rw-p 00011000 fd:00 677856     /lib/libz.so.1.2.3
00128000-00256000 r-xp 00000000 fd:00 1967906    /usr/lib/mysql/libmysqlclient.so.16.0.0
00256000-0029d000 rw-p 0012e000 fd:00 1967906    /usr/lib/mysql/libmysqlclient.so.16.0.0
0029d000-002c5000 r-xp 00000000 fd:00 677847     /lib/libm-2.12.so
002c5000-002c6000 r–p 00027000 fd:00 677847     /lib/libm-2.12.so
002c6000-002c7000 rw-p 00028000 fd:00 677847     /lib/libm-2.12.so
002c7000-00439000 r-xp 00000000 fd:00 1867809    /usr/lib/libcrypto.so.1.0.0
00439000-0044d000 rw-p 00172000 fd:00 1867809    /usr/lib/libcrypto.so.1.0.0
0044d000-00450000 rw-p 00000000 00:00 0
00450000-00457000 r-xp 00000000 fd:00 677864     /lib/libcrypt-2.12.so
00457000-00458000 r–p 00007000 fd:00 677864     /lib/libcrypt-2.12.so
00458000-00459000 rw-p 00008000 fd:00 677864     /lib/libcrypt-2.12.so
00459000-00480000 rw-p 00000000 00:00 0
00480000-0054a000 r-xp 00000000 fd:00 677879     /lib/libkrb5.so.3.3
0054a000-00550000 rw-p 000ca000 fd:00 677879     /lib/libkrb5.so.3.3
00550000-00595000 r-xp 00000000 fd:00 677863     /lib/libfreebl3.so
00595000-00596000 rw-p 00045000 fd:00 677863     /lib/libfreebl3.so
00596000-0059a000 rw-p 00000000 00:00 0
0059f000-005bd000 r-xp 00000000 fd:00 677835     /lib/ld-2.12.so
005bd000-005be000 r–p 0001d000 fd:00 677835     /lib/ld-2.12.so
005be000-005bf000 rw-p 0001e000 fd:00 677835     /lib/ld-2.12.so
005bf000-005c8000 r-xp 00000000 fd:00 677876     /lib/libkrb5support.so.0.1
005c8000-005c9000 rw-p 00008000 fd:00 677876     /lib/libkrb5support.so.0.1
005c9000-005e6000 r-xp 00000000 fd:00 677860     /lib/libselinux.so.1
005e6000-005e7000 r–p 0001c000 fd:00 677860     /lib/libselinux.so.1
005e7000-005e8000 rw-p 0001d000 fd:00 677860     /lib/libselinux.so.1
005f2000-00628000 r-xp 00000000 fd:00 1967263    /usr/libexec/postfix/pickup
00629000-0062a000 r–p 00036000 fd:00 1967263    /usr/libexec/postfix/pickup
0062a000-0062b000 rw-p 00037000 fd:00 1967263    /usr/libexec/postfix/pickup
0062b000-0062c000 rw-p 00000000 00:00 0
00634000-00678000 r-xp 00000000 fd:00 1867822    /usr/lib/libldap-2.4.so.2.5.2
00678000-00679000 rw-p 00044000 fd:00 1867822    /usr/lib/libldap-2.4.so.2.5.2
00679000-00685000 r-xp 00000000 fd:00 655000     /lib/libnss_files-2.12.so
00685000-00686000 r–p 0000b000 fd:00 655000     /lib/libnss_files-2.12.so
00686000-00687000 rw-p 0000c000 fd:00 655000     /lib/libnss_files-2.12.so
006c1000-006e2000 rw-p 00000000 00:00 0          [heap]
006f9000-0086c000 r-xp 00000000 fd:00 655098     /lib/libdb-4.7.so
0086c000-0086f000 rw-p 00173000 fd:00 655098     /lib/libdb-4.7.so
00888000-008a0000 r-xp 00000000 fd:00 1867808    /usr/lib/libsasl2.so.2.0.23
008a0000-008a1000 rw-p 00018000 fd:00 1867808    /usr/lib/libsasl2.so.2.0.23
009ff000-00a02000 r-xp 00000000 fd:00 677878     /lib/libcom_err.so.2.1
00a02000-00a03000 rw-p 00002000 fd:00 677878     /lib/libcom_err.so.2.1
00b40000-00b6f000 r-xp 00000000 fd:00 655464     /lib/libpcre.so.0.0.1
00b6f000-00b70000 rw-p 0002e000 fd:00 655464     /lib/libpcre.so.0.0.1
00bb5000-00bb7000 r-xp 00000000 fd:00 677875     /lib/libkeyutils.so.1.3
00bb7000-00bb8000 rw-p 00001000 fd:00 677875     /lib/libkeyutils.so.1.3
00c30000-00c56000 r-xp 00000000 fd:00 677877     /lib/libk5crypto.so.3.1
00c56000-00c57000 rw-p 00026000 fd:00 677877     /lib/libk5crypto.so.3.1
00c71000-00c72000 r-xp 00000000 00:00 0          [vdso]
00cac000-00ce2000 r-xp 00000000 fd:00 677880     /lib/libgssapi_krb5.so.2.2
00ce2000-00ce3000 rw-p 00036000 fd:00 677880     /lib/libgssapi_krb5.so.2.2
00cfb000-00d12000 r-xp 00000000 fd:00 677844     /lib/libpthread-2.12.so
00d12000-00d13000 r–p 00016000 fd:00 677844     /lib/libpthread-2.12.so
00d13000-00d14000 rw-p 00017000 fd:00 677844     /lib/libpthread-2.12.so
00d14000-00d16000 rw-p 00000000 00:00 0
00dab000-00dc1000 r-xp 00000000 fd:00 655424     /lib/libnsl-2.12.so
00dc1000-00dc2000 r–p 00016000 fd:00 655424     /lib/libnsl-2.12.so
00dc2000-00dc3000 rw-p 00017000 fd:00 655424     /lib/libnsl-2.12.so
00dc3000-00dc5000 rw-p 00000000 00:00 0
00e4f000-00ea2000 r-xp 00000000 fd:00 1867810    /usr/lib/libssl.so.1.0.0
00ea2000-00ea6000 rw-p 00052000 fd:00 1867810    /usr/lib/libssl.so.1.0.0
00efc000-00f09000 r-xp 00000000 fd:00 1843537    /usr/lib/liblber-2.4.so.2.5.2
00f09000-00f0a000 rw-p 0000c000 fd:00 1843537    /usr/lib/liblber-2.4.so.2.5.2
00f3b000-00f50000 r-xp 00000000 fd:00 677859     /lib/libresolv-2.12.so
00f50000-00f51000 r–p 00014000 fd:00 677859     /lib/libresolv-2.12.so
00f51000-00f52000 rw-p 00015000 fd:00 677859     /lib/libresolv-2.12.so
00f52000-00f54000 rw-p 00000000 00:00 0
b762c000-b7630000 rw-p 00000000 00:00 0
b7630000-b77b5000 r-xp 00000000 fd:00 677841     /lib/libc-2.12.so
b77b5000-b77b6000 —p 00185000 fd:00 677841     /lib/libc-2.12.so
b77b6000-b77b8000 r–p 00185000 fd:00 677841     /lib/libc-2.12.so
b77b8000-b77b9000 rw-p 00187000 fd:00 677841     /lib/libc-2.12.so
b77b9000-b77be000 rw-p 00000000 00:00 0
b77d3000-b77d4000 rw-p 00000000 00:00 0
bfdde000-bfdf3000 rw-p 00000000 00:00 0          [stack]

[H.323] TCS negotiation rule of HUAWEI

Encountered a capability negotiation issue when conferencing with a HUAWEI terminal.

It happened in a dual stream(H.239) capability negotiation.

Environments:
1. According to the H.245 TCS, both terminal have 1080P 60 fps capability.
2. The band width setting was extreme low.
3. HUAWEI sent a OLC request with 4CIF H.264 video.

Continue reading “[H.323] TCS negotiation rule of HUAWEI”

iPhone Companion App used to control Apple Watch unveiled with Novel Features

Some friends and former colleagues are focusing on wearable products developing, and invited me to participate with them. Got to admit, it’s intresting.

And here is a big news in the field of wearables from Apple. Continue reading “iPhone Companion App used to control Apple Watch unveiled with Novel Features”

An issue about HUAWEI TEx0 H.264 video capability calculating

Encounter a H.264 video capability issue while conferencing with HUAWEI TEx0 MT.
Both MTs(H800 and TEx0) have 1080P 60 fps H.264 video capability, but HUAWEI MT will send an OLC request with 1080P 30 fps when starting open the logical channel, while conferencing with Cisco MTs, this issue doesn’t happen.

H800 H.264 capabilityH.245 TCS from H800HUAWEI TEx0 H.264 capabilityH.245 TCS from TEx0OLC from HUAWEI TEx0H.245 OLC from HUAWEI TEx0

Checked the H.245 TCS of H800, found there was no CustomMaxFS value in it (only CustomMaxMBPS and CustomMaxBRandCPB),

  • H800 TCS: Level: 85, CustomMaxMBPS: 972 –> 1080P 60 fps
  • TEx0 TCS: Level: 71, CustomMaxMBPS: 983, CustomMaxFS: 32 –> 1080P 60 fps
  • OLC from TEx0: Level: 71, CustomMaxMBPS: 212, CustomMaxFS: 32 –> 1080P 30 fps

Conclusion:

HUAWEI TEx0 will use Level parameter only if there’s no CustomMaxFS.
In this case, we presented only H.264 Level, and CustomMaxMBPS, TEx0 will just ignore the CustomMaxMBPS parameter, that is to say it considered only the H.264 Level parameter, and the Level default capability is right 1080P 30 fps.

Solution:

Add CustomMaxFS parameter to H800’s TCS, everything works just fine.

[UniSVR] Taitronics 2014 Taipei

Oct. 6-9th we had participated in the Taitronics 2014 Electronic Show in Taipei. Taitronics is annual event that promotes the trade between TAIWAN (R.O.C.) and other countries. They Focused on the electronics sector with a special emphasis on the next era of IT generated by cloud computing. It was a pleasure meeting others in the industry, we had a lot of fun!
taitronicsmain
179208774

179208777
179208783

[UniSVR] Attending the 2015 CES at Las Vegas

CES2015_LARGE_unisvrAre you as excited as we are!!?? We can’t wait to show you what we have to offer!! Las Vegas has played host to the International CES since 1978. We are expecting another FULL HOUSE! Packed with new tech, crazy ideas and innovations, you better make sure that you are registered, hotels/shows booked and ready to rock the revenue!


Not too sure where/how to track us down at the CES event!? No worries, simply follow the link here:http://ces15.mapyourshow.com/6_0/exhibitor/exhibitor-details.cfm?ExhID=T0003005
or try out the new CES app to better navigate the showces2015map

CES2015