feat: Enhance GetSymbolsOverviewTool with nested symbol retrieval

[huhusmang]

Summary

Enhances the symbol overview functionality by adding support for nested symbol retrieval with intelligent filtering. This PR resolves the TODO comment in LanguageServerSymbolRetriever.get_symbol_overview() by refactoring the underlying API to support depth-based traversal and kind-based filtering, while maintaining backward compatibility.

Motivation & Problem

Current Limitations

1. Too shallow: get_symbols_overview only returned minimal information about top-level symbols (name_path and kind)
2. Missing context: Users couldn't see what methods exist within a class without making additional queries
3. Dead code: The SymbolOverviewElement dataclass was defined but never used outside its definition
4. Architecture issue: The tool would need to bypass the get_symbol_overview method to access nested symbols
5. Unresolved TODO: A TODO comment indicated the need for nested symbols and filtering

TODO Comment Context

# TODO: maybe include not just top-level symbols? We could filter by kind to exclude variables
#  The language server methods would need to be adjusted for this.

This TODO had two parts:

1. ✅ Include nested symbols (not just top-level)
2. ✅ Filter by kind to exclude noisy symbols like variables

Interestingly, the language server already returns complete symbol trees with children—the capability just wasn't exposed through the API!

Changes

1. Refactored LanguageServerSymbolRetriever.get_symbol_overview() (symbol.py)

Before:

@dataclass
class SymbolOverviewElement:  # Dead code - never used
    name_path: str
    kind: int

def get_symbol_overview(self, relative_path: str) -> dict[str, list[SymbolOverviewElement]]:
    # TODO: maybe include not just top-level symbols? We could filter by kind to exclude variables
    result[file_path] = [self.SymbolOverviewElement.from_symbol(...) for ...]

After:

def get_symbol_overview(
    self,
    relative_path: str,
    depth: int = 0,  # Backward compatible default
    include_kind: bool = True,
    include_location: bool = True,
    exclude_kinds: Sequence[SymbolKind] | None = None,
) -> dict[str, list[dict[str, Any]]]:
    """
    Get an overview of symbols in a file with optional depth for nested symbols.
    
    By default, excludes Variable symbols when depth > 0 to reduce noise.
    """
    # Smart default: exclude variables when retrieving nested symbols
    if exclude_kinds is None and depth > 0:
        exclude_kinds = [SymbolKind.Variable]
    
    # ... implementation with recursive filtering via _filter_children_by_kind()

2. Simplified GetSymbolsOverviewTool.apply() (symbol_tools.py)

Before (would need ~48 lines to support depth):

def apply(self, relative_path: str, max_answer_chars: int = -1):
    # Would need to bypass get_symbol_overview to access nested symbols
    # ... complex manual conversion logic

After:

def apply(
    self,
    relative_path: str,
    depth: int = 0,  # Backward compatible: default only top-level
    exclude_variable_symbols: bool = True,  # User-friendly parameter
    max_answer_chars: int = -1,
):
    from serena.symbol import SymbolKind
    
    # Validation...
    
    # Uses the proper API layer
    exclude_kinds = [SymbolKind.Variable] if exclude_variable_symbols else None
    overview_data = symbol_retriever.get_symbol_overview(
        relative_path=relative_path,
        depth=depth,
        include_kind=True,
        include_location=True,
        exclude_kinds=exclude_kinds,
    )
    result = overview_data[relative_path]
    
    return self._to_json(result)

Example Usage

Default behavior (backward compatible, depth=0)

# Returns only top-level symbols with full details
get_symbols_overview(relative_path="src/main.py")

Returns:

[
  {
    "name": "UserManager",
    "name_path": "UserManager",
    "kind": "Class",
    "location": {
      "line": 10,
      "column": 6,
      "relative_path": "src/main.py"
    },
    "body_location": {
      "start_line": 10,
      "end_line": 50
    }
  },
  {
    "name": "process_data",
    "name_path": "process_data",
    "kind": "Function",
    "location": {"line": 52, "column": 0},
    "body_location": {"start_line": 52, "end_line": 60}
  }
]

Get overview with methods (depth=1)

# Explicitly request nested symbols
get_symbols_overview(relative_path="src/main.py", depth=1)

Returns:

[
  {
    "name": "UserManager",
    "name_path": "UserManager",
    "kind": "Class",
    "location": {
      "line": 10,
      "column": 6,
      "relative_path": "src/main.py"
    },
    "body_location": {
      "start_line": 10,
      "end_line": 50
    },
    "children": [
      {
        "name": "__init__",
        "name_path": "UserManager/__init__",
        "kind": "Method",
        "location": {"line": 12, "column": 8},
        "body_location": {"start_line": 12, "end_line": 15}
      },
      {
        "name": "add_user",
        "name_path": "UserManager/add_user",
        "kind": "Method",
        "location": {"line": 17, "column": 8},
        "body_location": {"start_line": 17, "end_line": 25}
      }
    ]
  }
]

Note: Variable symbols are automatically filtered out by default when depth > 0.

[opcode81]

Thanks for this PR @huhusmang, but the approach you have taken is perhaps a bit complicated. With #827 I have provided an alternative way of allowing the depth to be specified while excluding low-level symbols.

[huhusmang]

Thanks for this PR @huhusmang, but the approach you have taken is perhaps a bit complicated. With #827 I have provided an alternative way of allowing the depth to be specified while excluding low-level symbols.

Thanks for the feedback! I've reviewed #827 and your approach is indeed cleaner - reusing to_dict() with a child_inclusion_predicate is more elegant than adding a separate method. The is_low_level() filter is also a nice addition to keep the output focused. Happy to close this PR in favor of yours.