The Outlier and Tree Structure Patterns

Most MongoDB collections have documents that follow a predictable size distribution. But occasionally you get outliers — documents that deviate dramatically from the norm. A social media post that goes viral might accumulate 50,000 comments while typical posts have 5–20. A product liked by a celebrity might have 10,000 reviews. Designing your schema around the average case while ignoring outliers leads to documents that eventually hit the 16 MB document size limit or cause memory pressure.

Before designing around outliers, identify whether they actually exist in your data. Use an aggregation pipeline to find documents with unusually large arrays. Set a threshold based on your expected normal range — if 99% of posts have fewer than 100 comments, documents with more than 1,000 comments are outliers worth handling specially.

The Outlier Pattern keeps the normal case fast by embedding arrays up to a threshold, and handles exceptional documents by setting an hasOverflow flag and storing the overflow items in a separate collection. Application code checks the flag — if false (the common case), it uses the embedded array. If true (the outlier case), it performs an additional query to the overflow collection.

Application code must handle the outlier flag explicitly. Most of the time, the flag is false and reads are fast. For outlier documents, perform the additional overflow query. This keeps the common path optimised while correctly handling exceptional cases without bloating normal documents or hitting the 16 MB limit.

Hierarchical data — product categories, organisational charts, file systems, comment threads — appears in almost every application. MongoDB does not have a native tree data type, so the structure must be modelled in the document schema. There are four common tree patterns, each optimised for different query access patterns: Parent References, Child References, Array of Ancestors, and Materialised Paths.

The Parent Reference pattern stores each node with a single parent field pointing to its parent's _id. Root nodes have parent: null. This is the simplest representation and mirrors how SQL nested-set or adjacency-list trees work. It is efficient for finding a node's direct parent or direct children, but requires recursive queries to traverse multiple levels.

The Array of Ancestors pattern stores the full path from root to the current node in an ancestors array. This makes it trivial to answer 'is X an ancestor of Y?' with a simple array membership check. It also makes it easy to find all descendants of a node — query for documents whose ancestors array contains that node's _id. The tradeoff is that moving a subtree requires updating all descendant documents.

The Materialised Path pattern stores the full path as a string (e.g., '/Electronics/Phones/Smartphones'). It enables prefix queries to find all nodes under a subtree, and regex queries to search within path segments. This pattern maps naturally to file system paths or URL hierarchies. It is efficient for both ancestor lookup and descendant enumeration, but can be fragile when nodes are renamed or moved.

Choose the tree pattern based on your most frequent query: Parent References — simple, good for traversal with application-side recursion; Child References — embed direct children array, fast for reading one level; Array of Ancestors — fast for ancestor lookup and subtree queries, expensive for moves; Materialised Paths — fast regex-based subtree queries, fragile on renames. Hybrid approaches (storing both parent and ancestors) trade write complexity for read speed.

The Child References pattern embeds an array of direct child _id values in each node document. This makes it fast to retrieve all children of a node in a single read (no separate query needed). It is ideal for trees that are read top-down frequently (e.g., a menu renders children immediately). The tradeoff is that the children array can grow large for wide nodes, and you cannot efficiently find a node's parent without an additional index or field.

MongoDB's $graphLookup aggregation stage recursively follows reference fields to traverse a tree or graph stored in any pattern. It returns all reachable nodes up to a specified depth. Use it with Parent References or Child References to traverse hierarchies without writing recursive application code. Specify maxDepth to prevent infinite loops in circular graphs.

Test your understanding of MongoDB & NoSQL Databases concepts from this lesson.

In this lesson you learned: the Outlier Pattern keeps normal documents lean by embedding arrays up to a threshold and using a hasOverflow flag to route exceptional documents to an overflow collection, tree structure patterns (Parent References, Array of Ancestors, Materialised Paths) each optimise for different query access patterns on hierarchical data, and $graphLookup recursively traverses references in the pipeline without application-side recursion. Next up we compare MongoDB against Redis for document vs key-value workloads.