Cross Platform mdb_v8, was (No Post Mortem Core Dump Support in 4.0.0)

[yunong]

Hi, at Netflix we leverage https://github.com/joyent/mdb_v8 quite heavily in production as the primary tool used to debug memory leaks and analyze production issues. It is currently, the only tool that we've had success with when it comes to fixing memory leaks in production. We are able to take a Linux Node 0.10-0.12 core dump, and debug it with mdb on Illumos.

It looks like this facility no longer exists in 4.0.0. As it turns out -- this is rather unfortunate and is a blocker when it comes to moving our production Node deployments on to 4.0.0.

It looks like @tjfontaine has done the work sometime ago to migrate a subset of the functionality to LLDB here https://github.com/tjfontaine/lldb-v8

If mdb_v8.so is going to be no longer kept up to date with Node, can we extend lldb-v8 to port over the rest of mdb_v8? This would ensure that this kind of critical post mortem analysis is available in new releases of node going forward and also more broadly on other platforms. Specifically, commands like ::findjsobjects, ::jssource, ::jssonconstructor are critical to debugging our production stack.

Just to be clear, there are no other tools that exist for Node today that can successfully debug memory leaks on production instances in the way that mdb_v8 can.

cc @lucamaraschi @mikeal @indutny @davepacheco @misterdjules

[bnoordhuis]

I'm going to close this as it's basically a duplicate of #2517. I understand there is a CL under review (details in the other issue) that we can release in a point release once it lands upstream.

[mikeal]

We should rename this bug to port to lldb and reopen.

On Wednesday, September 9, 2015, Ben Noordhuis notifications@github.com
wrote:

I'm going to close this as it's basically a duplicate of #2517
#2517. I understand there is a CL
under review (details in the other issue) that we can release in a point
release once it lands upstream.

—
Reply to this email directly or view it on GitHub
#2764 (comment).

[lucamaraschi]

👍

[misterdjules]

@yunong With the changes in TritonDataCenter/mdb_v8#24, you should be able to load and inspect any core dump generated by node 4.0.0 and later. If you need help with that, please ping me. These changes will be released asap once they've been properly reviewed.

mdb_v8 commands that need to access functions' scopeinfo like ::jsclosure won't work properly, but these are new features that you might not be using now. Once the metadata needed to properly support these new commands land in the V8 version bundled with node, you'll also be able to use them. We're working to make that happen asap too.

In the future, we'll keep mdb_v8 updated to follow new versions of V8 that land in node, because we use mdb_v8 all the time. So while porting mdb_v8 features over to lldb-v8 is a great idea, it's a separate topic.

[davepacheco]

@mikeal @yunong I've started looking at library-izing mdb_v8 so that it can be used in a cross-platform debugger. It would be a shame to have two completely separate implementations of the same thing. I believe I've mentioned this in the postmortem WG, though that WG is still getting off the ground.

[yunong]

@davepacheco This sounds like a ripe topic for a quick WG meeting. @mikeal I don't have permissions to reopen this issue, would you mind re-opening?

[mikeal]

Re-opened now that the issue is re-scoped and no longer a duplicate.

[bnoordhuis]

But now it's an issue that properly belongs in https://github.com/nodejs/post-mortem. Please move it there.

[rvagg]

+1 to that, @nodejs/post-mortem can you take this please?

[misterdjules]

@yunong Just FYI, even if this issue has moved to another topic, TritonDataCenter/mdb_v8#33 applied on joyent/mdb_v8's master should allow you to have proper node v4.x support with mdb_v8. If you have some time to try it and encounter any issue, please let me know by filing issues in joyent/mdb_v8. Thank you!

[misterdjules]

@yunong @lucamaraschi mdb_v8 supports node v4.x and the upcoming v5.0 release as of version 1.1.1.