Building a MongoDB Operations Manager with Python and Textual

Managing MongoDB operations in production environments can be challenging, especially when dealing with long-running queries that impact system performance. While MongoDB provides tools like mongosh and MongoDB Compass, we're developing a lightweight, customizable solution that aims to help engineering teams manage MongoDB operations more effectively. Today, I'll share our current progress building a Terminal User Interface (TUI) application.

The Problem

In production environments, MongoDB operations can sometimes go rogue - queries might run longer than expected, consume excessive resources, or get stuck in unexpected states. Traditional solutions often require:

1. Running multiple commands to gather operation information
2. Manual parsing of JSON output
3. Complex syntax for filtering and killing operations
4. Switching between different tools for monitoring and management

This creates friction in the incident response process and can lead to slower resolution times when dealing with database performance issues.

Current Implementation Status

While still in development, our tool currently provides:

• Basic operation monitoring with refresh capabilities
• Initial filtering options across operation attributes
• Operation selection and termination
• Core keyboard shortcuts for common actions
• Basic logging functionality

The interface is designed to be intuitive, though we're still gathering feedback and making improvements. Current features allow engineers to:

• View running operations in a sortable table
• Select operations using checkboxes
• Apply basic filters through input fields
• Kill operations with keyboard shortcuts

Known Limitations

Here are the limitations of the Close MongoDB Operations Manager in its current implementation:

• Testing: Add tests to the project.
• Releasing: Setup CI/CD do automatic releases.
• Error Handling: Basic error handling implemented, needs more robust coverage of edge cases.
• Performance: Not yet tested with large-scale deployments.
• Authentication: Basic authentication support, advanced security features still are not supported yet.

Roadmap

Our planned improvements include:

• Comprehensive error handling
• Extended test coverage
• Performance optimizations
• Advanced filtering capabilities
• Enhanced security features
• Automated testing with different MongoDB versions

The TUI

Here is how it looks:

Technical Implementation

The Power of Textual

One of the key decisions that made this project successful was choosing the Textual library for building the TUI. Textual is a modern Python framework for creating rich terminal applications, and it provided several advantages:

1. Declarative UI: Similar to modern web frameworks, Textual allows for declarative UI definition using Python classes and CSS-like styling:

def compose(self) -> ComposeResult:
    yield Header()
    with Container():
        yield FilterBar()
        with VerticalScroll():
            yield OperationsView()
    yield StatusBar()
    yield Footer()

2. CSS-like Styling: The ability to style components using familiar CSS syntax makes the UI customization intuitive:

DEFAULT_CSS = """
FilterBar {
    height: auto;
    layout: horizontal;
    background: $surface;
    border: solid $primary;
    padding: 1;
    margin: 0 1;
    width: 100%;
}
"""

3. Event-Driven Architecture: Textual's event system makes it easy to handle user interactions and update the UI accordingly:

def on_button_pressed(self, event: Button.Pressed) -> None:
    if event.button.id == "clear-filters":
        for input in self.query(".filter-input"):
            if isinstance(input, Input):
                input.value = ""
        self.post_message(FilterChanged({}))

MongoDB Integration

The application uses pymongo 4.9+ (MongoDB's native async Python driver) to interact with the database asynchronously, preventing UI freezes during operations. This is particularly important when dealing with large clusters or slow operations:

async def get_operations(self, filters: dict[str, str] | None = None) -> list[dict]:
    current_op_args = {
        "allUsers": True,
        "idleConnections": False,
        "idleCursors": False,
        "idleSessions": True,
        "localOps": False,
        "backtrace": False,
    }
    # ... implementation details

Engineering Benefits

The tool provides several key benefits for engineering teams:

1. Reduced Mean Time to Resolution (MTTR): Engineers can quickly identify and terminate problematic operations without needing to remember complex commands or syntax.

2. Safe Operation: Confirmation dialogs and clear visual feedback reduce the risk of accidentally killing the wrong operations.

Conclusion

Building the Close MongoDB Operations Manager has shown promise in how modern Python libraries like Textual can help create tools for complex operational tasks. While still in development, the combination of async MongoDB operations with a responsive TUI appears to be a viable approach for improving our database operations workflow.

We're continuing to test and refine the tool based on early feedback, and we expect to gain more insights as we expand its use across different scenarios. The source code is available for review and contributions, and we welcome feedback from the community to help shape its development.