Sharding >
Sharded Cluster Administration

Sharded Cluster Administration¶

On this page

Shards
Config Servers
Sharded Cluster Operations and mongos Instances
Sharded Cluster Security Considerations

Sharding occurs within a sharded cluster. A sharded cluster consists of the following components:

Shards. Each shard is a separate mongod instance or replica set that holds a portion of the database collections.
Config servers. Each config server is a mongod instance that holds metadata about the cluster. The metadata maps chunks to shards.
mongos instances. The mongos instances route the reads and writes to the shards.

Shards¶

A shard is a container that holds a subset of a collection’s data. Each shard is either a single mongod instance or a replica set. In production, all shards should be replica sets.

Applications do not access the shards directly. Instead, the mongos instances routes reads and writes from applications to the shards.

Config Servers¶

Config servers maintain the shard metadata in a config database. The config database stores the relationship between chunks and where they reside within a sharded cluster. Without a config database, the mongos instances would be unable to route queries or write operations within the cluster.

Config servers do not run as replica sets. Instead, a cluster operates with a group of three config servers that use a two-phase commit process that ensures immediate consistency and reliability.

For testing purposes you may deploy a cluster with a single config server, but this is not recommended for production.

Warning

If your cluster has a single config server, this mongod is a single point of failure. If the instance is inaccessible the cluster is not accessible. If you cannot recover the data on a config server, the cluster will be inoperable.

Always use three config servers for production deployments.

The actual load on configuration servers is small because each mongos instance maintains a cached copy of the configuration database. MongoDB only writes data to the config server to:

create splits in existing chunks, which happens as data in existing chunks exceeds the maximum chunk size.
migrate a chunk between shards.

Additionally, all config servers must be available on initial setup of a sharded cluster, each mongos instance must be able to write to the config.version collection.

If one or two configuration instances become unavailable, the cluster’s metadata becomes read only. It is still possible to read and write data from the shards, but no chunk migrations or splits will occur until all three servers are accessible. At the same time, config server data is only read in the following situations:

A new mongos starts for the first time, or an existing mongos restarts.
After a chunk migration, the mongos instances update themselves with the new cluster metadata.

If all three config servers are inaccessible, you can continue to use the cluster as long as you don’t restart the mongos instances until after config servers are accessible again. If you restart the mongos instances and there are no accessible config servers, the mongos would be unable to direct queries or write operations to the cluster.

Because the configuration data is small relative to the amount of data stored in a cluster, the amount of activity is relatively low, and 100% up time is not required for a functioning sharded cluster. As a result, backing up the config servers is not difficult. Backups of config servers are critical as clusters become totally inoperable when you lose all configuration instances and data. Precautions to ensure that the config servers remain available and intact are critical.

Note

Configuration servers store metadata for a single sharded cluster. You must have a separate configuration server or servers for each cluster you administer.

Sharded Cluster Operations and `mongos` Instances¶

The mongos program provides a single unified interface to a sharded cluster for applications using MongoDB. Except for the selection of a shard key, application developers and administrators need not consider any of the internal details of sharding.

mongos caches data from the config server, and uses this to route operations from applications and clients to the mongod instances. mongos have no persistent state and consume minimal system resources.

The most common practice is to run mongos instances on the same systems as your application servers, but you can maintain mongos instances on the shards or on other dedicated resources.

Note

Changed in version 2.1.

Some aggregation operations using the aggregate command (i.e. db.collection.aggregate(),) will cause mongos instances to require more CPU resources than in previous versions. This modified performance profile may dictate alternate architecture decisions if you use the aggregation framework extensively in a sharded environment.

Automatic Operation and Query Routing with `mongos`¶

mongos uses information from config servers to route operations to the cluster as efficiently as possible. In general, operations in a sharded environment are either:

Targeted at a single shard or a limited group of shards based on the shard key.
Broadcast to all shards in the cluster that hold documents in a collection.

When possible you should design your operations to be as targeted as possible. Operations have the following targeting characteristics:

Query operations broadcast to all shards [1] unless the mongos can determine which shard or shard stores this data.

For queries that include the shard key, mongos can target the query at a specific shard or set of shards, if the portion of the shard key included in the query is a prefix of the shard key. For example, if the shard key is:
copy
```
{ a: 1, b: 1, c: 1 }
```
The mongos program can route queries that include the full shard key or either of the following shard key prefixes at a specific shard or set of shards:
copy
```
{ a: 1 }
{ a: 1, b: 1 }
```
Depending on the distribution of data in the cluster and the selectivity of the query, mongos may still have to contact multiple shards [2] to fulfill these queries.
All insert() operations target to one shard.
All single update() operations target to one shard. This includes upsert operations.
The mongos broadcasts multi-update operations to every shard.
The mongos broadcasts remove() operations to every shard unless the operation specifies the shard key in full.

While some operations must broadcast to all shards, you can improve performance by using as many targeted operations as possible by ensuring that your operations include the shard key.

[1]	If a shard does not store chunks from a given collection, queries for documents in that collection are not broadcast to that shard.

[2]	`mongos` will route some queries, even some that include the shard key, to all shards, if needed.

Sharded Query Response Process¶

To route a query to a cluster, mongos uses the following process:

Determine the list of shards that must receive the query.

In some cases, when the shard key or a prefix of the shard key is a part of the query, the mongos can route the query to a subset of the shards. Otherwise, the mongos must direct the query to all shards that hold documents for that collection.
Example

Given the following shard key:
copy
{ zipcode: 1, u_id: 1, c_date: 1 }
Depending on the distribution of chunks in the cluster, the mongos may be able to target the query at a subset of shards, if the query contains the following fields:
copy
{ zipcode: 1 } { zipcode: 1, u_id: 1 } { zipcode: 1, u_id: 1, c_date: 1 }
Establish a cursor on all targeted shards.

When the first batch of results returns from the cursors:
1. For query with sorted results (i.e. using cursor.sort()) the mongos instance performs a merge sort of all queries.
2. For a query with unsorted results, the mongos instance returns a result cursor that “round robins” results from all cursors on the shards.
  
  Changed in version 2.0.5: Before 2.0.5, the mongos exhausted each cursor, one by one.

Sharded Cluster Security Considerations¶

MongoDB controls access to sharded clusters with key files that store authentication credentials. The components of sharded clusters use the secret stored in the key files when authenticating to each other. Create key files and then point your mongos and mongod instances to the files, as described later in this section.

Beyond the auth mechanisms described in this section, always run your sharded clusters in trusted networking environments that limit access to the cluster with network rules. Your networking environments should enforce restrictions that ensure only known traffic reaches your mongos and mongod instances.

This section describes authentication specific to sharded clusters. For information on authentication across MongoDB, see Authentication.

Access Control Privileges in Sharded Clusters¶

In sharded clusters, MongoDB provides separate administrative privileges for the sharded cluster and for each shard. Beyond these administration privileges, privileges for sharded cluster deployments are functionally the same as any other MongoDB deployment. See, Authentication for more information.

For sharded clusters, MongoDB provides these separate administrative privileges:

Administrative privileges for the sharded cluster. These privileges provide read-and-write access to the config servers’ ‘admin. These users can run all administrative commands. Administrative privileges also give the user read-and-write access to all the cluster’s databases.

The credentials for administrative privileges on the cluster reside on the config servers. To receive admin access to the cluster, you must authenticate a session while connected to a mongos instance using the admin database.
Administrative privileges for the mongod instance, or replica set, that provides each individual shard. Each shard has its own admin database that stores administrative credentials and access for that shard only. These credentials are completely distinct from the cluster-wide administrative credentials.

As with all mongod instances, MongoDB provides two types of administrative privileges for a shard:
- Normal administrative privileges, which provide read-and-write access to the admin database and access to all administrative commands, and which provide read-and-write access to all other databases on that shard.
- Read-only administrative privileges, which provide read-only access to the admin database and to all other databases on that shard.

Also, as with all mongod instances, a MongoDB sharded cluster provides the following non-administrative user privileges:

Normal privileges, which provide read-and-write access to a specific database. Users with normal privilege can add users to the database.
Read-only privileges, which provide read-only access to a specific database.

For more information on privileges, see Authentication.

Enable Authentication in a Sharded Cluster¶

New in version 2.0: Support for authentication with sharded clusters.

To control access to a sharded cluster, create key files and then set the keyFile option on all components of the sharded cluster, including all mongos instances, all config server mongod instances, and all shard mongod instances. The content of the key file is arbitrary but must be the same on all cluster members.

To enable authentication, do the following:

Generate a key file to store authentication information, as described in the Generate a Key File section.
On each component in the sharded cluster, enable authentication by doing one of the following:
- In the configuration file, set the keyFile option to the key file’s path and then start the component, as in the following example:
  copy
  keyFile = /srv/mongodb/keyfile
- When starting the component, set --keyFile option, which is an option for both mongos instances and mongod instances. Set the --keyFile to the key file’s path.
Note

The keyFile setting implies auth, which means in most cases you do not need to set auth explicitly.
Add the first administrative user and then add subsequent users. See Add Users.

Access a Sharded Cluster with Authentication¶

To access a sharded cluster as an authenticated admin user, see Administrative Access in MongoDB.

To access a sharded cluster as an authenticated, non-admin user, see either of the following:

To terminate an authenticated session, see the logout command.

← Sharded Cluster Overview Sharded Cluster Architectures →