Spring Data MongoDB provides support for the Aggregation Framework introduced to MongoDB in version 2.2.
For further information, see the full https://docs.mongodb.org/manual/aggregation/[reference documentation] of the aggregation framework and other data aggregation tools for MongoDB.
[[mongo.aggregation.basic-concepts]]
=== Basic Concepts
== Basic Concepts
The Aggregation Framework support in Spring Data MongoDB is based on the following key abstractions: `Aggregation`, `AggregationDefinition`, and `AggregationResults`.
Note that, if you provide an input class as the first parameter to the `newAggregation` method, the `MongoTemplate` derives the name of the input collection from this class. Otherwise, if you do not not specify an input class, you must provide the name of the input collection explicitly. If both an input class and an input collection are provided, the latter takes precedence.
@ -173,7 +173,7 @@ At the time of this writing, we provide support for the following Aggregation Op
@@ -173,7 +173,7 @@ At the time of this writing, we provide support for the following Aggregation Op
Note that the aggregation operations not listed here are currently not supported by Spring Data MongoDB. Comparison aggregation operators are expressed as `Criteria` expressions.
[[mongo.aggregation.projection]]
=== Projection Expressions
== Projection Expressions
Projection expressions are used to define the fields that are the outcome of a particular aggregation step. Projection expressions can be defined through the `project` method of the `Aggregation` class, either by passing a list of `String` objects or an aggregation framework `Fields` object. The projection can be extended with additional fields through a fluent API by using the `and(String)` method and aliased by using the `as(String)` method.
Note that you can also define fields with aliases by using the `Fields.field` static factory method of the aggregation framework, which you can then use to construct a new `Fields` instance. References to projected fields in later aggregation stages are valid only for the field names of included fields or their aliases (including newly defined fields and their aliases). Fields not included in the projection cannot be referenced in later aggregation stages. The following listings show examples of projection expression:
More examples for project operations can be found in the `AggregationTests` class. Note that further details regarding the projection expressions can be found in the https://docs.mongodb.org/manual/reference/operator/aggregation/project/#pipe._S_project[corresponding section] of the MongoDB Aggregation Framework reference documentation.
[[mongo.aggregation.facet]]
=== Faceted Classification
== Faceted Classification
As of Version 3.4, MongoDB supports faceted classification by using the Aggregation Framework. A faceted classification uses semantic categories (either general or subject-specific) that are combined to create the full classification entry. Documents flowing through the aggregation pipeline are classified into buckets. A multi-faceted classification enables various aggregations on the same set of input documents, without needing to retrieve the input documents multiple times.
[[buckets]]
==== Buckets
=== Buckets
Bucket operations categorize incoming documents into groups, called buckets, based on a specified expression and bucket boundaries. Bucket operations require a grouping field or a grouping expression. You can define them by using the `bucket()` and `bucketAuto()` methods of the `Aggregate` class. `BucketOperation` and `BucketAutoOperation` can expose accumulations based on aggregation expressions for input documents. You can extend the bucket operation with additional parameters through a fluent API by using the `with…()` methods and the `andOutput(String)` method. You can alias the operation by using the `as(String)` method. Each bucket is represented as a document in the output.
@ -263,7 +263,7 @@ Note that further details regarding bucket expressions can be found in the https
@@ -263,7 +263,7 @@ Note that further details regarding bucket expressions can be found in the https
https://docs.mongodb.org/manual/reference/operator/aggregation/bucketAuto/[`$bucketAuto` section] of the MongoDB Aggregation Framework reference documentation.
[[multi-faceted-aggregation]]
==== Multi-faceted Aggregation
=== Multi-faceted Aggregation
Multiple aggregation pipelines can be used to create multi-faceted aggregations that characterize data across multiple dimensions (or facets) within a single aggregation stage. Multi-faceted aggregations provide multiple filters and categorizations to guide data browsing and analysis. A common implementation of faceting is how many online retailers provide ways to narrow down search results by applying filters on product price, manufacturer, size, and other factors.
Note that further details regarding facet operation can be found in the https://docs.mongodb.org/manual/reference/operator/aggregation/facet/[`$facet` section] of the MongoDB Aggregation Framework reference documentation.
[[mongo.aggregation.sort-by-count]]
==== Sort By Count
=== Sort By Count
Sort by count operations group incoming documents based on the value of a specified expression, compute the count of documents in each distinct group, and sort the results by count. It offers a handy shortcut to apply sorting when using <<mongo.aggregation.facet>>. Sort by count operations require a grouping field or grouping expression. The following listing shows a sort by count example:
@ -315,12 +315,12 @@ A sort by count operation is equivalent to the following BSON (Binary JSON):
@@ -315,12 +315,12 @@ A sort by count operation is equivalent to the following BSON (Binary JSON):
----
[[mongo.aggregation.projection.expressions]]
==== Spring Expression Support in Projection Expressions
=== Spring Expression Support in Projection Expressions
We support the use of SpEL expressions in projection expressions through the `andExpression` method of the `ProjectionOperation` and `BucketOperation` classes. This feature lets you define the desired expression as a SpEL expression. On running a query, the SpEL expression is translated into a corresponding MongoDB projection expression part. This arrangement makes it much easier to express complex calculations.
[[complex-calculations-with-spel-expressions]]
===== Complex Calculations with SpEL expressions
==== Complex Calculations with SpEL expressions
Consider the following SpEL expression:
@ -389,12 +389,12 @@ In addition to the transformations shown in the preceding table, you can use sta
@@ -389,12 +389,12 @@ In addition to the transformations shown in the preceding table, you can use sta
----
[[mongo.aggregation.examples]]
==== Aggregation Framework Examples
=== Aggregation Framework Examples
The examples in this section demonstrate the usage patterns for the MongoDB Aggregation Framework with Spring Data MongoDB.
[[mongo.aggregation.examples.example1]]
===== Aggregation Framework Example 1
==== Aggregation Framework Example 1
In this introductory example, we want to aggregate a list of tags to get the occurrence count of a particular tag from a MongoDB collection (called `tags`) sorted by the occurrence count in descending order. This example demonstrates the usage of grouping, sorting, projections (selection), and unwinding (result splitting).
@ -435,7 +435,7 @@ The preceding listing uses the following algorithm:
@@ -435,7 +435,7 @@ The preceding listing uses the following algorithm:
Note that the input collection is explicitly specified as the `tags` parameter to the `aggregate` Method. If the name of the input collection is not specified explicitly, it is derived from the input class passed as the first parameter to the `newAggreation` method.
[[mongo.aggregation.examples.example2]]
===== Aggregation Framework Example 2
==== Aggregation Framework Example 2
This example is based on the https://docs.mongodb.org/manual/tutorial/aggregation-examples/#largest-and-smallest-cities-by-state[Largest and Smallest Cities by State] example from the MongoDB Aggregation Framework documentation. We added additional sorting to produce stable results with different MongoDB versions. Here we want to return the smallest and largest cities by population for each state by using the aggregation framework. This example demonstrates grouping, sorting, and projections (selection).
@ -501,7 +501,7 @@ The preceding listings use the following algorithm:
@@ -501,7 +501,7 @@ The preceding listings use the following algorithm:
Note that we derive the name of the input collection from the `ZipInfo` class passed as the first parameter to the `newAggregation` method.
[[mongo.aggregation.examples.example3]]
===== Aggregation Framework Example 3
==== Aggregation Framework Example 3
This example is based on the https://docs.mongodb.org/manual/tutorial/aggregation-examples/#states-with-populations-over-10-million[States with Populations Over 10 Million] example from the MongoDB Aggregation Framework documentation. We added additional sorting to produce stable results with different MongoDB versions. Here we want to return all states with a population greater than 10 million, using the aggregation framework. This example demonstrates grouping, sorting, and matching (filtering).
@ -537,7 +537,7 @@ The preceding listings use the following algorithm:
@@ -537,7 +537,7 @@ The preceding listings use the following algorithm:
Note that we derive the name of the input collection from the `ZipInfo` class passed as first parameter to the `newAggregation` method.
[[mongo.aggregation.examples.example4]]
===== Aggregation Framework Example 4
==== Aggregation Framework Example 4
This example demonstrates the use of simple arithmetic operations in the projection operation.
Note that we can also refer to other fields of the document within the SpEL expression.
[[mongo.aggregation.examples.example7]]
===== Aggregation Framework Example 7
==== Aggregation Framework Example 7
This example uses conditional projection. It is derived from the https://docs.mongodb.com/manual/reference/operator/aggregation/cond/[$cond reference documentation].
As of MongoDB 3.6, https://docs.mongodb.com/manual/changeStreams/[Change Streams] let applications get notified about changes without having to tail the oplog.
@ -13,7 +13,7 @@ changes from all collections within the database. When subscribing to a database
@@ -13,7 +13,7 @@ changes from all collections within the database. When subscribing to a database
In doubt, use `Document`.
[[change-streams-with-messagelistener]]
=== Change Streams with `MessageListener`
== Change Streams with `MessageListener`
Listening to a https://docs.mongodb.com/manual/tutorial/change-streams-example/[Change Stream by using a Sync Driver] creates a long running, blocking task that needs to be delegated to a separate component.
In this case, we need to first create a `MessageListenerContainer`, which will be the main entry point for running the specific `SubscriptionRequest` tasks.
@ -51,7 +51,7 @@ Please use `register(request, body, errorHandler)` to provide additional functio
@@ -51,7 +51,7 @@ Please use `register(request, body, errorHandler)` to provide additional functio
====
[[reactive-change-streams]]
=== Reactive Change Streams
== Reactive Change Streams
Subscribing to Change Streams with the reactive API is a more natural approach to work with streams. Still, the essential building blocks, such as `ChangeStreamOptions`, remain the same. The following example shows how to use Change Streams emitting ``ChangeStreamEvent``s:
The mapping framework does not have to store child objects embedded within the document.
You can also store them separately and use a `DBRef` to refer to that document.
@ -53,7 +53,7 @@ CAUTION: Lazy loading may require class proxies, that in turn, might need access
@@ -53,7 +53,7 @@ CAUTION: Lazy loading may require class proxies, that in turn, might need access
For those cases please consider falling back to an interface type (eg. switch from `ArrayList` to `List`) or provide the required `--add-opens` argument.
[[mapping-usage.document-references]]
=== Using Document References
= Using Document References
Using `@DocumentReference` offers a flexible way of referencing entities in MongoDB.
While the goal is the same as when using <<mapping-usage-references,DBRefs>>, the store representation is different.
MongoDB supports storing binary files inside its filesystem, GridFS. Spring Data MongoDB provides a `GridFsOperations` interface as well as the corresponding implementation, `GridFsTemplate`, to let you interact with the filesystem. You can set up a `GridFsTemplate` instance by handing it a `MongoDatabaseFactory` as well as a `MongoConverter`, as the following example shows:
Since Spring Data MongoDB 1.4, auditing can be enabled by annotating a configuration class with the `@EnableMongoAuditing` annotation, as the following example shows:
The most trivial way of influencing the mapping result is by specifying the desired native MongoDB target type via the
`@Field` annotation. This allows to work with non MongoDB types like `BigDecimal` in the domain model while persisting
@ -43,7 +43,7 @@ The `MappingMongoConverter` checks to see if any Spring converters can handle a
@@ -43,7 +43,7 @@ The `MappingMongoConverter` checks to see if any Spring converters can handle a
NOTE: For more information on the Spring type conversion service, see the reference docs link:{springDocsUrl}/core.html#validation[here].
[[mongo.custom-converters.writer]]
=== Saving by Using a Registered Spring Converter
== Saving by Using a Registered Spring Converter
The following example shows an implementation of the `Converter` that converts from a `Person` object to a `org.bson.Document`:
@ -66,7 +66,7 @@ public class PersonWriteConverter implements Converter<Person, Document> {
@@ -66,7 +66,7 @@ public class PersonWriteConverter implements Converter<Person, Document> {
----
[[mongo.custom-converters.reader]]
=== Reading by Using a Spring Converter
== Reading by Using a Spring Converter
The following example shows an implementation of a `Converter` that converts from a `Document` to a `Person` object:
@ -83,7 +83,7 @@ public class PersonReadConverter implements Converter<Document, Person> {
@@ -83,7 +83,7 @@ public class PersonReadConverter implements Converter<Document, Person> {
----
[[mongo.custom-converters.xml]]
=== Registering Spring Converters with the `MongoConverter`
== Registering Spring Converters with the `MongoConverter`
As of version 3.6, MongoDB supports collections that validate documents against a provided https://docs.mongodb.com/manual/core/schema-validation/#json-schema[JSON Schema].
The schema itself and both validation action and level can be defined when creating the collection, as the following example shows:
@ -399,7 +399,7 @@ public class EncryptionExtension implements EvaluationContextExtension {
@@ -399,7 +399,7 @@ public class EncryptionExtension implements EvaluationContextExtension {
====
[[mongo.jsonSchema.types]]
==== JSON Schema Types
== JSON Schema Types
The following table shows the supported JSON schema types:
While <<mongo.custom-converters, type-based conversion>> already offers ways to influence the conversion and representation of certain types within the target store, it has limitations when only certain values or properties of a particular type should be considered for conversion.
Property-based converters allow configuring conversion rules on a per-property basis, either declaratively (via `@ValueConverter`) or programmatically (by registering a `PropertyValueConverter` for a specific property).
@ -35,7 +35,7 @@ You can use `PropertyValueConverterFactory.beanFactoryAware(…)` to obtain a `P
@@ -35,7 +35,7 @@ You can use `PropertyValueConverterFactory.beanFactoryAware(…)` to obtain a `P
You can change the default behavior through `ConverterConfiguration`.
[[mongo.property-converters.declarative]]
=== Declarative Value Converter
== Declarative Value Converter
The most straight forward usage of a `PropertyValueConverter` is by annotating properties with the `@ValueConverter` annotation that defines the converter type:
@ -52,7 +52,7 @@ class Person {
@@ -52,7 +52,7 @@ class Person {
====
[[mongo.property-converters.programmatic]]
=== Programmatic Value Converter Registration
== Programmatic Value Converter Registration
Programmatic registration registers `PropertyValueConverter` instances for properties within an entity model by using a `PropertyValueConverterRegistrar`, as the following example shows.
The difference between declarative registration and programmatic registration is that programmatic registration happens entirely outside of the entity model.
WARNING: Dot notation (such as `registerConverter(Person.class, "address.street", …)`) for nagivating across properties into subdocuments is *not* supported when registering converters.
[[mongo.property-converters.value-conversions]]
=== MongoDB property value conversions
== MongoDB property value conversions
The preceding sections outlined the purpose an overall structure of `PropertyValueConverters`.
The repository layer offers means to interact with <<mongo.aggregation, the aggregation framework>> via annotated repository query methods.
Similar to the <<mongodb.repositories.queries.json-based, JSON based queries>>, you can define a pipeline using the `org.springframework.data.mongodb.repository.Aggregation` annotation.
The following example shows how to query by example when using a repository (of `Person` objects, in this case):
@ -73,7 +73,7 @@ Spring Data MongoDB provides support for the following matching options:
@@ -73,7 +73,7 @@ Spring Data MongoDB provides support for the following matching options:
|===
[[query-by-example.untyped]]
== Untyped Example
= Untyped Example
By default `Example` is strictly typed. This means that the mapped query has an included type match, restricting it to probe assignable types. For example, when sticking with the default type key (`_class`), the query has restrictions such as (`_class : { $in : [ com.acme.Person] }`).
// carry over the old bookmarks to prevent external links from failing
[[tailable-cursors]]
== [[mongo.reactive.repositories.infinite-streams]] Infinite Streams with Tailable Cursors
= [[mongo.reactive.repositories.infinite-streams]] Infinite Streams with Tailable Cursors
By default, MongoDB automatically closes a cursor when the client exhausts all results supplied by the cursor.
Closing a cursor on exhaustion turns a stream into a finite stream. For https://docs.mongodb.com/manual/core/capped-collections/[capped collections],
@ -14,7 +14,7 @@ reactive variant, as it is less resource-intensive. However, if you cannot use t
@@ -14,7 +14,7 @@ reactive variant, as it is less resource-intensive. However, if you cannot use t
concept that is already prevalent in the Spring ecosystem.
[[tailable-cursors.sync]]
=== Tailable Cursors with `MessageListener`
== Tailable Cursors with `MessageListener`
Listening to a capped collection using a Sync Driver creates a long running, blocking task that needs to be delegated to
a separate component. In this case, we need to first create a `MessageListenerContainer`, which will be the main entry point
Using tailable cursors with a reactive data types allows construction of infinite streams. A tailable cursor remains open until it is closed externally. It emits data as new documents arrive in a capped collection.
MongoDB 5.0 introduced https://docs.mongodb.com/manual/core/timeseries-collections/[Time Series] collections that are optimized to efficiently store documents over time such as measurements or events.
Those collections need to be created as such before inserting any data.
Unwrapped entities are used to design value objects in your Java domain model whose properties are flattened out into the parent's MongoDB Document.
[[unwrapped-entities.mapping]]
=== Unwrapped Types Mapping
== Unwrapped Types Mapping
Consider the following domain model where `User.name` is annotated with `@Unwrapped`.
The `@Unwrapped` annotation signals that all properties of `UserName` should be flattened out into the `user` document that owns the `name` property.
@ -53,7 +53,7 @@ However, those must not be, nor contain unwrapped fields themselves.
@@ -53,7 +53,7 @@ However, those must not be, nor contain unwrapped fields themselves.
====
[[unwrapped-entities.mapping.field-names]]
=== Unwrapped Types field names
== Unwrapped Types field names
A value object can be unwrapped multiple times by using the optional `prefix` attribute of the `@Unwrapped` annotation.
By dosing so the chosen prefix is prepended to each property or `@Field("…")` name in the unwrapped object.
@ -136,7 +136,7 @@ public class UserName {
@@ -136,7 +136,7 @@ public class UserName {
====
[[unwrapped-entities.queries]]
=== Query on Unwrapped Objects
== Query on Unwrapped Objects
Defining queries on unwrapped properties is possible on type- as well as field-level as the provided `Criteria` is matched against the domain type.
Prefixes and potential custom field names will be considered when rendering the actual query.
Fields of unwrapped objects can be used for sorting via their property path as shown in the sample below.
@ -205,7 +205,7 @@ Though possible, using the unwrapped object itself as sort criteria includes all
@@ -205,7 +205,7 @@ Though possible, using the unwrapped object itself as sort criteria includes all
====
[[unwrapped-entities.queries.project]]
==== Field projection on unwrapped objects
=== Field projection on unwrapped objects
Fields of unwrapped objects can be subject for projection either as a whole or via single fields as shown in the samples below.
Unwrapped objects can be used within an `Example` probe just as any other type.
Please review the <<query-by-example.running,Query By Example>> section, to learn more about this feature.
[[unwrapped-entities.queries.repository]]
==== Repository Queries on unwrapped objects.
=== Repository Queries on unwrapped objects.
The `Repository` abstraction allows deriving queries on fields of unwrapped objects as well as the entire object.
@ -284,7 +284,7 @@ Index creation for unwrapped objects is suspended even if the repository `create
@@ -284,7 +284,7 @@ Index creation for unwrapped objects is suspended even if the repository `create
====
[[unwrapped-entities.update]]
=== Update on Unwrapped Objects
== Update on Unwrapped Objects
Unwrapped objects can be updated as any other object that is part of the domain model.
The mapping layer takes care of flattening structures into their surroundings.