What is the most convenient way to read definitions of in-memory symbols when we don't have the source files? (Spelunking tools)

Link to the code on GitHub

---

I have been using this. It's mostly Leonid's code from the stackoverflow question you linked to, but it uses Definition instead of DownValues. Symbol names are printed without any context, but the full symbol name is put into a Tooltip so you can always find out what context a symbol is in.

Update

FullDefinition[symbol] claims to "print the definitions given for symbol, and all symbols on which these depend", but sometimes one wants to explore deeper than the first level of dependency. Here is a version of Spelunk which uses plain Definition instead of FullDefinition, but allows you to click on symbols in the definition to get their definition. So you can dig right down into the dependency chain.

Update 2

The code now copes with definitions containing strings with backticks in, and cases where Definition throws an error.

Also, it now works for symbols which have OwnValues, e.g. Internal`$VideoEncodings.

BeginPackage["Spelunk`"];

Spelunk::usage = "Spelunk[symbol]";

Begin["`Private`"];

defboxes[symbol_Symbol] := Hold[symbol] /. _[sym_] :>
        If[MemberQ[Attributes[sym], Locked], "Locked",
          Internal`InheritedBlock[{sym},
            Unprotect[sym]; ClearAttributes[sym, ReadProtected];
            Quiet@Check[ToBoxes[Definition@sym], "DefError"] /. 
            InterpretationBox[a_, b___] :> a ]];

defboxes[s_String] := defboxes[#] &@ToExpression[s, InputForm, Unevaluated]

prettyboxes[boxes_] := 
  boxes /. {" "} -> {"\n-----------\n"} //. {RowBox[{left___, ";", 
       next : Except["\n"], right___}] :> 
     RowBox[{left, ";", "\n", "\t", next, right}], 
    RowBox[{sc : ("Block" | "Module" | "With"), "[", 
       RowBox[{vars_, ",", body_}], "]"}] :> 
     RowBox[{sc, "[", RowBox[{vars, ",", "\n\t", body}], "]"}]};

fancydefinition[symbol_Symbol] :=
  Cell[BoxData[
    prettyboxes[
     defboxes[symbol] /. 
      s_String?(StringMatchQ[#, __ ~~ "`" ~~ __] &) :> 
       First@StringCases[s, 
         a : (__ ~~ "`" ~~ b__) :> processsymbol[a, b]]]], "Output", 
   Background -> RGBColor[1, 0.95, 0.9],
   CellGroupingRules->"OutputGrouping",
   GeneratedCell->True,
   CellAutoOverwrite->True,
   ShowAutoStyles->True,
   LanguageCategory->"Mathematica",
   FontWeight->"Bold"
];

processsymbol[a_, b_] := Module[{db},
  Which[
   ! StringFreeQ[a, "\""], a,
   ! StringFreeQ[a, "_"] || (db = defboxes[a]) === "Null", 
   TooltipBox[b, a],
   db === "Locked", TooltipBox[b, a <> "\nLocked Symbol"],
   db === "DefError", TooltipBox[b, a <> "\nError getting Definition"],
   True, ButtonBox[TooltipBox[b, a], ButtonFunction :> Spelunk@a, 
    BaseStyle -> {}, Evaluator -> Automatic]]]

Spelunk[symbol_Symbol] := CellPrint[fancydefinition[symbol]];

Spelunk[s_String] := CellPrint[fancydefinition[#] &@ToExpression[s, InputForm, Unevaluated]];

SetAttributes[{defboxes, fancydefinition, Spelunk}, HoldFirst] 

End[];

EndPackage[];

In version 10.1, I've built something like Spelunk into GeneralUtilities`.

To use it, run

Needs["GeneralUtilities`"]
PrintDefinitions[symbol];

This will pop up a window that shows all definitions of symbol. Here is a short summary of features:

• The window shows code cells containing all DownValues, OwnValues, UpValues, SubValues, and Attributes of a symbol.
• Most pathological kinds of StandardForm are avoided, so that Image, Graphics, Row, Column, SparseArray, etc. will show up as code, not UI elements. This is achieved via the PlainForm and CodeForm wrappers, which can also be found in GeneralUtilities`.
• Hyperlinks are be injected as appropriate. Click on a hyperlinked symbol to print its definitions in a new window.
• The CellContext of each code cell is set to reduce the amount of clutter from fully qualified private symbols. For symbols outside this context, the name is shown, the fully qualified name is tooltipped, and color coding is used: symbols in conventional private contexts like `Private` are brown/orange; system symbols are black; other symbols are dark gray if on the context path, otherwise light gray.

Here's a simple example of PrintDefinitions running on itself:

I can now offer a solution which leverages the full power of the code formatter, in its new, more robust form.

Load the formatter:

Import["https://raw.github.com/lshifr/CodeFormatter/master/CodeFormatter.m"]

Some examples:

CodeFormatterSpelunk[RunThrough]

CodeFormatterSpelunk[PacletManager`CreatePaclet]

In the last example, using MakeBoxes would produce a slightly better result:

CodeFormatterSpelunk[PacletManager`CreatePaclet, MakeBoxes]

CodeFormatterSpelunk[PacletManager`PackPaclet]