whoami

Lucas Jenß

cat /etc/motd

The Coding Journal ツ — Notes taken on an epic coding journey. Technical solutions, debugging notes, and practical guides from the trenches of software development.

Close-up of white dominoes with black dots standing on green felt surface, shallow depth of field, focused mood

ls -la ~/languages/

total 8
drwxr-xr-x
▶ PHP
▶ Ruby
▶ Scala
▶ C#
▶ JavaScript
▶ Objective-C
▶ Shell Scripting

ls -la ~/toolchain/

total 7
drwxr-xr-x
▶ Typo3
▶ Akka
▶ Capistrano
▶ Git
▶ MAMP
▶ Adobe Illustrator
▶ NSTrackingArea (Cocoa)

uname -a

platforms
drwxr-xr-x
▶ Mac OS X
▶ Unix

Tracking Down a Segmentation Fault in an Android NDK C++ Callback

A segfault deep inside native code is rarely convenient, and when it surfaces inside a callback invoked from a Java listener the diagnosis stretches across two runtimes. Native development on Android has matured, yet the mixed Java/JNI boundary still behaves like a shared workshop where every tool tossed across the table can bruise the other side. Bugs in this space look chaotic at first because their symptoms appear in Java frames while the root cause lives entirely in C++.

The most common culprit I have seen in client projects from Sydney and Melbourne consultancies is incorrect thread attachment around callbacks. Developers assume a JNI call dispatched from a worker thread automatically inherits a valid JNIEnv pointer, then crash when the garbage collector sweeps through a stale global reference. Add a vendored library compiled without stack-protector or thread-sanitizer flags and the report becomes harder to read still.

This note walks through the diagnostic flow I use whenever a callback segfaults a JNI-equipped Android service. It assumes a release build that crashes on certain devices but works elsewhere, which is the typical way these bugs surface after an APK ships to Australian testers on a Telstra-supplied handset.

Establishing a clean reproduction path

Before any of the theory matters, you need a test that fails in under a minute. I usually spin up an emulator image matching the slowest device profile my client targets — for many Australian teams that is an older Pixel shipped through JB Hi-Fi clearance stock — and run the offending flow with sanitiser-enabled native binaries attached. The ndk-build flags worth switching on early are APP_CFLAGS += -fsanitize=address,undefined -fno-omit-frame-pointer and a matching -fsanitize=address for shared libraries. AddressSanitizer typically catches the bug on the first or second iteration even when a stack trace looks clean.

When the crash refuses to appear in emulation, build artefacts run on a dedicated test rig. A reliable rig here means more than a developer's old laptop; a couple of our Brisbane partners run build agents under vSphere so the same image is reproducible across team members. Setting that up properly is its own rabbit hole, and the physical to VMware migration walkthrough covers the practical gotchas of moving physical dev boxes into a virtual cluster without losing native toolchain quirks.

A useful fallback when you cannot attach a debugger is to mark the callback entry with a synchronous native log call writing to a file in the app's private storage. If the write never appears, the callback is not being reached and the problem sits on the Java side. If the write appears and the crash follows, the suspect region is narrowed to one or two C++ functions.

Decoding the JNI threading rules

Android's JNI documentation is technically clear but easy to misread. Every thread that wants to touch Java objects or call back into Java must first call AttachCurrentThread. The Java main thread is already attached by the runtime, which is why naive callbacks that originate from that thread often work for weeks before a worker thread suddenly breaks the pattern. A callback scheduled through a pthread_create or a std::thread carries no JNIEnv cached, and invoking CallVoidMethod with a null JNIEnv produces the canonical segfault signature SIGSEGV at 0x0.

The second rule catches developers off guard: local references are scoped to the thread, not to the JNI frame in the way C++ locals are scoped. Pushing a jobject onto a global queue from thread A and reading it from thread B without NewGlobalRef is one of the classic corruption sources. Once you internalise these constraints the callback logic reduces to a few boilerplate helpers: attach_scope, env_guard, and a global reference table that you own for the lifetime of the parent object.

A subtle variant I have seen in Melbourne fintech apps concerns callback timers. Java code uses ScheduledExecutorService to invoke a Kotlin function that calls a native static, and the native static schedules a delayed C++ task. When the delayed task finally fires, it tries to reach a Kotlin object that has already been garbage-collected because the activity that owned it finished. Reaching beyond a WeakGlobalRef produces a random crash somewhere far from the real mistake.

Turning raw stack traces into actionable leads

A native crash report contains three layers: the Java exception thread, the signal handler metadata, and the actual C++ frames with their unresolved addresses. The middle layer tells you whether the segfault was a write to a protected page or a non-canonical address, which often separates use-after-free from null-deref. The bottom layer is the gold, but only after symbolication.

Tooling here is straightforward on macOS or Linux build hosts. Pull the app.so symbol file matching the exact build ID stored in the tombstone, feed it to addr2line -e lib/arm64-v8a/libyourlib.so -f -C 0xabc123. Doing this by hand becomes tedious; drop the addresses into a small shell script and you will trace the same five lines every time. For convenience, some teams I work with in Adelaide pre-stage a gradle task that runs ndk-stack against logcat output automatically after each instrumentation test run.

Watch out for optimisation. When callbacks are inlined and folded into the caller, the offset you symbolicate can land in an unrelated function. Recompile the affected translation unit with -O0 temporarily, or set CXXFLAGS += -fno-inline-functions for ndk-build. It costs build time but pays for itself the first time a frame points directly at the offending lambda.

Memory ownership around callback arguments

Most callback crashes trace back to lifetime assumptions that worked in desktop code but break on Android because the runtime enforces tighter guarantees. A common pattern: a native function copies a const std::string& into a std::function, returns, and the callback later dereferences memory that no longer exists. On Linux the heap allocator may keep the block alive long enough to hide the bug; on Android's jemalloc-derived allocator with low-memory reaping, the block vanishes almost immediately.

The cleanest fix I know is to make every asynchronous callback take shared ownership explicitly. Wrap the payload in std::shared_ptr on the boundary, capture the shared_ptr by value inside the lambda, and the segment violation disappears because no surviving reference to freed memory remains. A related pattern is to pass opaque handles (int64_t) into native code rather than raw pointers; the handle resolves against a thread-safe map and the lifetime stays obvious to every reader.

Asset paths are a quieter source of similar trouble. Mobile apps routinely pass image references between Kotlin and native rendering layers; if the asset catalogue lives in a configuration that is regenerated at every build, the URL or asset id captured at construct time may not match what the renderer finds at decode time. Some teams sidestep this entirely by doing the conversion in Java and feeding native code a stable byte array. The same asset pipelines that touch vector graphics in Android projects often depend on the same Adobe workflows covered in the Illustrator import walkthrough.

Replacing the broken callback with a vetted pattern

After the diagnosis settles on a culprit, the rewrite is usually short. Move the logic into a wrapper that owns a JavaVM* captured at JNI_OnLoad, attaches each worker thread once, and stashes its JNIEnv* in thread-local storage. The callback itself receives a pre-resolved jobject global reference plus whatever POD payload it needs; no Java interaction happens outside this controlled lane.

The minimum viable wrapper looks roughly like this in C++:

struct CallbackContext {
    JavaVM* vm = nullptr;
    jobject target = nullptr; // global ref
    void dispatch(int code) {
        JNIEnv* env = nullptr;
        if (vm->AttachCurrentThread(&env, nullptr) != JNI_OK) return;
        jclass cls = env->GetObjectClass(target);
        jmethodID m = env->GetMethodID(cls, "onNativeEvent", "(I)V");
        env->CallVoidMethod(target, m, code);
        env->DeleteLocalRef(cls);
        vm->DetachCurrentThread();
    }
};

Pair the wrapper with checks: every GetMethodID is guarded, every Call*Method is wrapped in ExceptionCheck, and every NewGlobalRef is balanced by a DeleteGlobalRef in the destructor. With these in place the callback stops behaving like a haunted staircase and starts behaving like an ordinary function. If the wider app makes heavy use of vector assets for its UI, keeping the rendering configuration consistent across the stack pays off later, and a refresher on SVG viewBox cropping clarifies a few of the dimensions involved.

Defensive habits that prevent the next regression

A team that learns one callback bug really has learned only the symptom. Locking down the patterns that produced it is what stops a repeat performance during the next refactor. The cheap tools for the job compare as follows.

Technique Setup cost Reveals best Watch out for
AddressSanitizer + UBSan Low (rebuild flag) Heap, stack, UB Slower runs, larger binaries
ndk-stack over logcat None Java/native mix Needs raw symbols present
addr2line script None Single offsets Manual, scales poorly
Verbose JNI tracing Low Attach/detach issues Log noise on hot paths
GC + headless emulator Medium Lifetime/garbage bugs Misses real device quirks

Pick at least two of the above for every release branch and wire them into the CI pipeline. AddressSanitizer belongs in the debug variant builds while ndk-stack runs against every nightly. The combination catches most regressions before code review surfaces them.

Habits worth adopting for every native callback

  • Add a JNI smoke test that exercises every callback path under the address and undefined-behaviour sanitizers in CI; ten-second builds are cheap insurance against ten-hour outages.
  • Lint native code with clang-tidy rules tuned for JNI: ban raw JNIEnv* outside the attach helper, and flag any capture of a jobject inside a non-Java lambda.
  • Keep a JNI_TRACE macro that logs entry and exit of every callback at verbose level; enabling it on demand reveals the call graph when a future bug appears.
  • Vet third-party libraries for thread-safety claims; an "MT-safe" badge on a Linux build does not necessarily translate to the Android Bionic runtime shipped on devices sold in Australia and overseas.
  • Run instrumentation tests on a low-memory emulator profile at least once per release cycle so lifetime bugs surface before a Telstra field test on a 2 GB handset does it for you.

Australian Consumer Law guarantees that software sold to consumers must be of acceptable quality, which includes crashes that arise in normal use. Treating each callback bug as a quality obligation, rather than a one-off, keeps the release notes honest and the support queue short.

If a segfault in a callback is currently blocking your release, open the codebase, mark the entry point with a synchronous log, and walk the steps above one at a time. The fix rarely takes more than an afternoon, and the test harness that survives the process will save the next person a week of guessing.


cat ~/interests.json

KeyValue
editorTerminal-first workflow
osMac OS X / Unix
vcsGit, distributed version control
deployCapistrano, cron automation
graphicsSVG, Adobe Illustrator troubleshooting
networkingIP validation, SSH, VPN

git log --oneline --reverse

2013-10-30

Solving SVG import issues in Adobe Illustrator CS6 and CC

When importing an SVG into Illustrator, the operation fails with an unknown error [CANT]. A workaround for this Adobe-side bug.

2013

Solving NDK build issues on OS X

Troubleshooting native development kit compilation problems on Mac OS X.

2013

Programmatically adding PHP generated TypoScript to the backend configuration

Integrating dynamically generated TypoScript into Typo3 backend setups using PHP.

2013

ArgumentError: Could not parse PKey: no start line

Debugging an SSH key parsing error encountered during deployment.

2011-08-04

Validating IP-Addresses in PHP

Using PHP filter functions with flags like FILTER_FLAG_IPV4 and FILTER_FLAG_IPV6, and understanding how filter_var handles reserved IP addresses.

2011-07-09

Cocoa: Using NSTrackingArea

A short tutorial on using Cocoa's NSTrackingArea to capture mouseEntered and mouseExited events.


cat ~/contact.txt

github: github.com/x3ro
stackoverflow: x3ro
coderwall: coderwall.com/x3ro
twitter: @x3rames