本文概述在 Mac 平台的 C 在用可执行程序或动态链接库 Breakpad 的方法。
如 Breakpad 入门 解释,整个 breakpad 包括在内的系统 C 访问程序 breakpad 客户端库还包括生成符号的工具 dump_syms 和将生成的 minidump 转为符号的 minidump_stackwalk 上传工具和工具 minidump 文件工具。本文将简要介绍这些二进制的所有编译和使用方法。
构建 Breadpad
这里说明基础 / 的 GN ninja 构建系统的方法。
通过 Git 下载 源代码。然后在 OpenRTCClient 根目录中执行以下命令 ./build_system/webrtc_pack mac debug crash_catch_system
,这将生成客户端静态库文件 ,其中包含 arm64 和 x64 两个 CPU 架构的静态库,同时还会生成用于生成符号文件和将生成的 minidump 它们位于符号工具中 。
把 Breakpad 集成到你的程序中
首先,配置你的施工过程 将目录地址添加到链接库文件的搜索路径中,并添加到链接中 库,把 链接到您的二进制文件,然后设置 include 路径包含 breakpad 源码树中的 目录。
Google breakpad 本身提供的 ExceptionHandler
虽然每个平台上的类接口差别不大,但也不完全一样。这给了我 breakpad 接入带来了一定的负担。
在 作者实现了一个简单的包装,即 open_rtc::InstallCrashHandler()
和 open_rtc::UninstallCrashHandler()
。包括崩溃处理器的头文件:
#include "client/crash_handler.h"
调用 open_rtc::InstallCrashHandler()
安装崩溃处理器。然后调用 open_rtc::UninstallCrashHandler()
在此之前,异常处理是激活的,所以在启动程序的过程中,你应该尽它 open_rtc::InstallCrashHandler()
,并使其尽可能接近关闭状态持激活。做任何有用的事情,open_rtc::InstallCrashHandler()
都需要一个可以写的 minidump 路径和回调函数接收已写入的相关路径 minidump 的信息:
#include <stdlib.h> #include "client/crash_handler.h" bool minidumpCallback(const char* dump_dir, const char* minidump_id, void* context, bool succeeded) { printf("Dump path: %s, minidump_id %s\n", dump_dir, minidump_id); return succeeded; } static void crashfunc() { volatile int* a = (int*)(NULL); *a = 1; } int main(int argc, const char *argv[]) { open_rtc::InstallCrashHandler(".", nullptr, minidumpCallback, nullptr); crashfunc(); return 0; }
本示例程序的编译和运行应在当前目录下生成 minidump 文件应在退出前打印 minidump 文件 id,id 即为 minidump 文件名,实际 minidump 文件有文件扩展名 “.dmp”。
也可直接使用 Google breakpad 提供的 ExceptionHandler
类来接入 breakpad。此时首先包括头文件:
#include "client/mac/handler/exception_handler.h"
然后创建 ExceptionHandler
类对象。ExceptionHandler
对象在整个生命周期中的异常处理是激活的,因此在程序启动过程中应尽早实例化,并尽可能接近关闭状态。做任何有用的事情,ExceptionHandler
构造函数需要一个可以写入的函数 minidump 路径和回调函数接收已写入的相关路径 minidump 的信息:
#include <stdio.h> #include "client/mac/handler/exception_handler.h" bool MinidumpCallback(const char* dump_dir, const char* minidump_id, void* context, bool succeeded) { printf("Dump path: %s, minidump_id %s\n", dump_dir, minidump_id); return succeeded; } int main(int argc, const char *argv[]) { google_breakpad::ExceptionHandler eh(".", NULL, MinidumpCallback, NULL, true, nullptr); crashfunc(); }
该程序的编译和操作与前一个程序相同。
按照 WebRTC 施工配置,USE_PROTECTED_ALLOCATIONS=1
宏定义是开放的 (webrtc/third_party/breakpad/BUILD.gn
)。但 breakpad 其中有以下代码段 (webrtc/third_party/breakpad/breakpad/src/client/mac/handler/exception_handler.cc
):
bool ExceptionHandler::InstallHandler() { // If a handler is already installed, something is really wrong. if (gProtectedData.handler != NULL) { return false; } if (!IsOutOfProcess()) { struct sigaction sa; memset(&sa, 0, sizeof(sa)); sigemptyset(&sa.sa_mask); sigaddset(&sa.sa_mask, SIGABRT); sa.sa_sigaction = ExceptionHandler::SignalHandler; sa.sa_flags = SA_SIGINFO; scoped_ptr<struct sigaction> old(new struct sigaction); if (sigaction(SIGABRT, &sa, old.get()) == -1) { return false; } old_handler_.swap(old); gProtectedData.handler = this; #if USE_PROTECTED_ALLOCATIONS assert(((size_t)(gProtectedData.protected_buffer) & PAGE_MASK) == 0); mprotect(gProtectedData.protected_buffer, PAGE_SIZE, PROT_READ); #endif } try { #if USE_PROTECTED_ALLOCATIONS previous_ = new (gBreakpadAllocator->Allocate(sizeof(ExceptionParameters)) ) ExceptionParameters(); #else previous_ = new ExceptionParamters();
#endif
}
catch (std::bad_alloc) {
return false;
}
这段代码在 ExceptionHandler
对象创建时执行。全局变量 gBreakpadAllocator
实际上是在 webrtc/third_party/breakpad/breakpad/src/client/mac/Framework/Breakpad.mm
初始化的,这段代码只有通过 BreakpadCreate()
在 OC 代码中接入 breakpad 客户端才会执行。这样在直接通过 breakpad 的 C++ 接口接入 breakpad 客户端时,全局变量 gBreakpadAllocator
值为空,进而导致 ExceptionHandler
对象创建时由于空指针而使程序崩溃。
在 项目中,通过把宏 USE_PROTECTED_ALLOCATIONS=1
的定义去掉来解决这个通过 C++ 接口接入 breakpad 客户端导致的问题。当然检查 gBreakpadAllocator
的值也可以解决这个问题。
为程序生成符号
如上面的说明,通过 构建 breakpad 会一并生成各种工具。这里以 的 smoke_test 为例说明这个过程。
构建 smoke_test
:
OpenRTCClient % ./build_system/webrtc_build build:smoke_test mac x64 debug
生成符号文件:
OpenRTCClient % ./build/mac/x64/debug/dump_syms -g build/mac/x64/debug/smoke_test.dSYM build/mac/x64/debug/smoke_test > build/mac/x64/debug/smoke_test.sym
所基于的 M98 版 WebRTC 代码的 webrtc/build/toolchain/apple/linker_driver.py
似乎有些问题,最终链接生成可执行文件时报类似如下这些 warning:
warning: (x86_64) could not find object file symbol for symbol __ZNSt3__17forwardIPN6webrtc20AudioEncoderG722ImplEEEOT_RNS_16remove_referenceIS4_E4typeE
warning: (x86_64) could not find object file symbol for symbol __ZNSt3__122__compressed_pair_elemIPN6webrtc12AudioEncoderELi0ELb0EEC2IPNS1_20AudioEncoderG722ImplEvEEOT_
warning: (x86_64) could not find object file symbol for symbol __ZNSt3__122__compressed_pair_elemINS_14default_deleteIN6webrtc12AudioEncoderEEELi1ELb1EEC2INS1_INS2_20AudioEncoderG722ImplEEEvEEOT_
warning: (x86_64) could not find object file symbol for symbol __ZNSt3__114default_deleteIN6webrtc12AudioEncoderEEC2INS1_20AudioEncoderG722ImplEEERKNS0_IT_EEPNS_9enable_ifIXsr14is_convertibleIPS6_PS2_EE5valueEvE4typeE
warning: (x86_64) could not find object file symbol for symbol __ZN6webrtc16AudioEncoderOpus11SdpToConfigERKNS_14SdpAudioFormatE
warning: (x86_64) could not find object file symbol for symbol __ZN6webrtc16AudioEncoderOpus16MakeAudioEncoderERKNS_22AudioEncoderOpusConfigEiN4absl8optionalINS_16AudioCodecPairIdEEE
warning: (x86_64) could not find object file symbol for symbol __ZN6webrtc16AudioEncoderOpus17QueryAudioEncoderERKNS_22AudioEncoderOpusConfigE
warning: (x86_64) could not find object file symbol for symbol __ZN6webrtc16AudioEncoderOpus23AppendSupportedEncodersEPNSt3__16vectorINS_14AudioCodecSpecENS1_9allocatorIS3_EEEE
. . . . . .
warning: (x86_64) could not find object file symbol for symbol _dav1d_itx_dsp_init_x86_8bpc
warning: (x86_64) could not find object file symbol for symbol _iclip_u8
warning: (x86_64) could not find object file symbol for symbol _dav1d_loop_filter_dsp_init_8bpc
warning: (x86_64) could not find object file symbol for symbol _dav1d_loop_filter_dsp_init_x86_8bpc
warning: (x86_64) could not find object file symbol for symbol _iclip_u8
warning: (x86_64) could not find object file symbol for symbol _dav1d_loop_restoration_dsp_init_8bpc
warning: (x86_64) could not find object file symbol for symbol _dav1d_loop_restoration_dsp_init_x86_8bpc
warning: (x86_64) could not find object file symbol for symbol _iclip_u8
warning: (x86_64) could not find object file symbol for symbol _dav1d_mc_dsp_init_8bpc
warning: (x86_64) could not find object file symbol for symbol _dav1d_mc_dsp_init_x86_8bpc
warning: (x86_64) could not find object file symbol for symbol _dav1d_get_cpu_flags_x86
build success
Chromium 的邮件列表中也有在讨论这个问题,macOS build warnings。这个问题似乎会导致生成符号文件失败:
smoke_test.dSYM/Contents/Resources/DWARF/smoke_test: the DIE at offset 0x134c917 has a DW_AT_specification attribute referring to the DIE at offset 0x134c8c6, which was not marked as a declaration
smoke_test.dSYM/Contents/Resources/DWARF/smoke_test: the DIE at offset 0x134ca01 has a DW_AT_specification attribute referring to the DIE at offset 0x134c9b1, which was not marked as a declaration
smoke_test.dSYM/Contents/Resources/DWARF/smoke_test: the DIE at offset 0x134caeb has a DW_AT_specification attribute referring to the DIE at offset 0x134ca9b, which was not marked as a declaration
smoke_test.dSYM/Contents/Resources/DWARF/smoke_test: the DIE at offset 0x134cc0b has a DW_AT_specification attribute referring to the DIE at offset 0x134cbc9, which was not marked as a declaration
Assertion failed: (it_debug == externs_.end() || (*it_debug)->address >= range.address + range.size), function AddFunction, file module.cc, line 175.
链接生成的二进制文件,导致 build/mac/x64/debug/dump_syms
崩溃执行失败。
在 项目中,通过把 webrtc/build/toolchain/apple/linker_driver.py
替换为 M88 的 webrtc/build/toolchain/mac/linker_driver.py
来解决这个问题。这样即可成功生成符号文件。
为了通过 minidump_stackwalk
工具使用这些符号文件,需要把它们放在一个特定的目录结构下。符号文件的第一行包含了生成这种目录结构所需的信息,比如(你的输出可能不太一样):
OpenRTCClient % ./build/mac/x64/debug/dump_syms -g build/mac/x64/debug/smoke_test.dSYM build/mac/x64/debug/smoke_test > build/mac/x64/debug/smoke_test.sym
OpenRTCClient % head build/mac/x64/debug/smoke_test.sym
MODULE mac x86_64 4C4C44D755553144A13B271E21EDCA370 smoke_test
FILE 0 /Users/henryhan/Projects/opensource/OpenRTCClient/build/mac/x64/debug/../../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CFBase.h
FILE 1 /Users/henryhan/Projects/opensource/OpenRTCClient/build/mac/x64/debug/../../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CFByteOrder.h
FILE 2 /Users/henryhan/Projects/opensource/OpenRTCClient/build/mac/x64/debug/../../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/_ctype.h
FILE 3 /Users/henryhan/Projects/opensource/OpenRTCClient/build/mac/x64/debug/../../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/_wctype.h
FILE 4 /Users/henryhan/Projects/opensource/OpenRTCClient/build/mac/x64/debug/../../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/architecture/byte_order.h
FILE 5 /Users/henryhan/Projects/opensource/OpenRTCClient/build/mac/x64/debug/../../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/dispatch/once.h
FILE 6 /Users/henryhan/Projects/opensource/OpenRTCClient/build/mac/x64/debug/../../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/dispatch/queue.h
FILE 7 /Users/henryhan/Projects/opensource/OpenRTCClient/build/mac/x64/debug/../../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/libkern/i386/_OSByteOrder.h
FILE 8 /Users/henryhan/Projects/opensource/OpenRTCClient/build/mac/x64/debug/../../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/math.h
OpenRTCClient % mkdir -p build/mac/x64/debug/symbols/smoke_test/4C4C44D755553144A13B271E21EDCA370
OpenRTCClient % mv build/mac/x64/debug/smoke_test.sym build/mac/x64/debug/symbols/smoke_test/4C4C44D755553144A13B271E21EDCA370
处理 minidump 生成栈追踪
这里先模拟一个崩溃的发生。在 项目的 smoke_test 中有几个测试用例可以产生崩溃:
OpenRTCClient % ./build/mac/x64/debug/smoke_test '--gtest_filter=CrashCatchTest.*' --gtest_also_run_disabled_tests
Note: Google Test filter = CrashCatchTest.*
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CrashCatchTest
[ RUN ] CrashCatchTest.DISABLED_crash_catch_common
Dump path: ~/OpenRTCClient/build/mac/x64/debug, minidump_id A25EA989-A9EC-400F-B44F-AF4B517101A1
崩溃产生时,会有一些捕获的崩溃的 minidump 文件相关的信息吐出来。根据这些信息我们可以知道,崩溃文件的完整路径为 ~/OpenRTCClient/build/mac/x64/debug/A25EA989-A9EC-400F-B44F-AF4B517101A1.dmp
。
Breakpad 包含了一个称为 minidump_stackwalk
的工具,它接收一个 minidump 及它对应的文本格式的符号,生成一个符号化的栈追踪。如果按照上面的指示编译了 Breakpad 源码,它应该位于 OpenRTCClient/build/mac/x64/debug
目录下。简单地把 minidump 和符号路径作为命令行参数传入:
OpenRTCClient % build/mac/x64/debug/minidump_stackwalk ./build/mac/x64/debug/A25EA989-A9EC-400F-B44F-AF4B517101A1.dmp ./build/mac/x64/debug/symbols
Operating system: Mac OS X
11.6.7 20G630
CPU: amd64
family 6 model 158 stepping 10
12 CPUs
GPU: UNKNOWN
Crash reason: EXC_BAD_ACCESS / KERN_INVALID_ADDRESS
Crash address: 0x0
Process uptime: 0 seconds
Thread 0 (crashed)
0 smoke_test!crashfunc() [crash_catch_test.cpp : 48 + 0x0]
rax = 0x0000000000000000 rdx = 0x00000000000fb340
rcx = 0x0000000000000000 rbx = 0x0000000000000000
rsi = 0x00000000d587bb72 rdi = 0x0000000107d11080
rbp = 0x00007ffeeebff1c0 rsp = 0x00007ffeeebff1c0
r8 = 0x00000000000000d7 r9 = 0x0000000000000008
r10 = 0x00007ff972a00000 r11 = 0x0000000000000000
r12 = 0x0000000000000000 r13 = 0x0000000000000000
r14 = 0x0000000000000000 r15 = 0x0000000000000000
rip = 0x000000010101744e
Found by: given as instruction pointer in context
1 smoke_test!CrashCatchTest_DISABLED_crash_catch_common_Test::TestBody() [crash_catch_test.cpp : 53 + 0x5]
rbp = 0x00007ffeeebff200 rsp = 0x00007ffeeebff1d0
rip = 0x00000001010173ff
Found by: previous frame's frame pointer
2 smoke_test!void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) [gtest.cc : 2631 + 0x2]
rbp = 0x00007ffeeebff260 rsp = 0x00007ffeeebff210
rip = 0x0000000101095e4b
Found by: previous frame's frame pointer
3 smoke_test!void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) [gtest.cc : 2686 + 0x15]
rbp = 0x00007ffeeebff2d0 rsp = 0x00007ffeeebff270
rip = 0x00000001010719c7
Found by: previous frame's frame pointer
4 smoke_test!testing::Test::Run() [gtest.cc : 2706 + 0x24]
rbp = 0x00007ffeeebff330 rsp = 0x00007ffeeebff2e0
rip = 0x0000000101071911
Found by: previous frame's frame pointer
5 smoke_test!testing::TestInfo::Run() [gtest.cc : 2885 + 0x5]
rbp = 0x00007ffeeebff3b0 rsp = 0x00007ffeeebff340
rip = 0x0000000101072455
Found by: previous frame's frame pointer
6 smoke_test!testing::TestSuite::Run() [gtest.cc : 3044 + 0x5]
rbp = 0x00007ffeeebff430 rsp = 0x00007ffeeebff3c0
rip = 0x00000001010732fb
Found by: previous frame's frame pointer
7 smoke_test!testing::internal::UnitTestImpl::RunAllTests() [gtest.cc : 5915 + 0x5]
rbp = 0x00007ffeeebff510 rsp = 0x00007ffeeebff440
rip = 0x000000010107f763
Found by: previous frame's frame pointer
8 smoke_test!bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) [gtest.cc : 2631 + 0x2]
rbp = 0x00007ffeeebff570 rsp = 0x00007ffeeebff520
rip = 0x000000010109bccb
Found by: previous frame's frame pointer
9 smoke_test!bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) [gtest.cc : 2686 + 0x15]
rbp = 0x00007ffeeebff5e0 rsp = 0x00007ffeeebff580
rip = 0x000000010107f277
Found by: previous frame's frame pointer
10 smoke_test!testing::UnitTest::Run() [gtest.cc : 5482 + 0x27]
rbp = 0x00007ffeeebff650 rsp = 0x00007ffeeebff5f0
rip = 0x000000010107f15f
Found by: previous frame's frame pointer
11 smoke_test!RUN_ALL_TESTS() [gtest.h : 2497 + 0x5]
rbp = 0x00007ffeeebff660 rsp = 0x00007ffeeebff660
rip = 0x000000010104ce41
Found by: previous frame's frame pointer
12 smoke_test!main [main.cpp : 12 + 0x5]
rbp = 0x00007ffeeebff690 rsp = 0x00007ffeeebff670
rip = 0x000000010104cdf2
Found by: previous frame's frame pointer
13 libdyld.dylib + 0x15f3d
rbp = 0x00007ffeeebff6a0 rsp = 0x00007ffeeebff6a0
rip = 0x00007fff2036bf3d
Found by: previous frame's frame pointer
Loaded modules:
0x101000000 - 0x104299fff smoke_test ??? (main)
0x7fff20088000 - 0x7fff20089fff libsystem_blocks.dylib ???
0x7fff2008a000 - 0x7fff200bffff libxpc.dylib ???
0x7fff200c0000 - 0x7fff200d7fff libsystem_trace.dylib ???
0x7fff200d8000 - 0x7fff20175fff libcorecrypto.dylib ???
0x7fff20176000 - 0x7fff201a2fff libsystem_malloc.dylib ???
0x7fff201a3000 - 0x7fff201e7fff libdispatch.dylib ???
0x7fff201e8000 - 0x7fff20221fff libobjc.A.dylib ???
它在 stderr 上产生详细输出,在 stdout 上产生堆栈跟踪,因此你可能需要重定向 stderr。
mac下利用Breakpad的dump文件进行调试
在 Linux 程序中使用 breakpad
breakpad的正确编译和常规用法
Breakpad在mac/ios上的跨平台的调用方式
[APM笔记]CLI Windows/macOS/Linux使用Breakpad