Application Performance Monitoring (APM)

Motor implements the same Command Monitoring and Topology Monitoring specifications as other MongoDB drivers. Therefore, you can register callbacks to be notified of every MongoDB query or command your program sends, and the server’s reply to each, as well as getting a notification whenever the driver checks a server’s status or detects a change in your replica set.

Motor wraps PyMongo, and it shares PyMongo’s API for monitoring. To receive notifications about events, you subclass one of PyMongo’s four listener classes, CommandListener, ServerListener, TopologyListener, or ServerHeartbeatListener.

Command Monitoring

Subclass CommandListener to be notified whenever a command starts, succeeds, or fails.

import logging
import sys

from pymongo import monitoring

logging.basicConfig(stream=sys.stdout, level=logging.INFO)


class CommandLogger(monitoring.CommandListener):
    def started(self, event):
        logging.info(
            f"Command {event.command_name} with request id "
            f"{event.request_id} started on server "
            f"{event.connection_id}"
        )

    def succeeded(self, event):
        logging.info(
            f"Command {event.command_name} with request id "
            f"{event.request_id} on server {event.connection_id} "
            f"succeeded in {event.duration_micros} "
            "microseconds"
        )

    def failed(self, event):
        logging.info(
            f"Command {event.command_name} with request id "
            f"{event.request_id} on server {event.connection_id} "
            f"failed in {event.duration_micros} "
            "microseconds"
        )


Register an instance of MyCommandLogger:

monitoring.register(CommandLogger())

You can register any number of listeners, of any of the four listener types.

Although you use only APIs from PyMongo’s monitoring module to configure monitoring, if you create a MotorClient its commands are monitored, the same as a PyMongo MongoClient.

from tornado import gen, ioloop

from motor import MotorClient

client = MotorClient()


async def do_insert():
    await client.test.collection.insert_one({"message": "hi!"})

    # For this example, wait 10 seconds for more monitoring events to fire.
    await gen.sleep(10)


ioloop.IOLoop.current().run_sync(do_insert)

This logs something like:

Command insert with request id 50073 started on server ('localhost', 27017)
Command insert with request id 50073 on server ('localhost', 27017)
succeeded in 362 microseconds

See PyMongo’s monitoring module for details about the event data your callbacks receive.

Server and Topology Monitoring

Subclass ServerListener to be notified whenever Motor detects a change in the state of a MongoDB server it is connected to.

class ServerLogger(monitoring.ServerListener):
    def opened(self, event):
        logging.info(f"Server {event.server_address} added to topology {event.topology_id}")

    def description_changed(self, event):
        previous_server_type = event.previous_description.server_type
        new_server_type = event.new_description.server_type
        if new_server_type != previous_server_type:
            logging.info(
                f"Server {event.server_address} changed type from "
                f"{event.previous_description.server_type_name} to "
                f"{event.new_description.server_type_name}"
            )

    def closed(self, event):
        logging.warning(f"Server {event.server_address} removed from topology {event.topology_id}")


monitoring.register(ServerLogger())

Subclass TopologyListener to be notified whenever Motor detects a change in the state of your server topology. Examples of such topology changes are a replica set failover, or if you are connected to several mongos servers and one becomes unavailable.

class TopologyLogger(monitoring.TopologyListener):
    def opened(self, event):
        logging.info(f"Topology with id {event.topology_id} opened")

    def description_changed(self, event):
        logging.info(f"Topology description updated for topology id {event.topology_id}")
        previous_topology_type = event.previous_description.topology_type
        new_topology_type = event.new_description.topology_type
        if new_topology_type != previous_topology_type:
            logging.info(
                f"Topology {event.topology_id} changed type from "
                f"{event.previous_description.topology_type_name} to "
                f"{event.new_description.topology_type_name}"
            )

    def closed(self, event):
        logging.info(f"Topology with id {event.topology_id} closed")


monitoring.register(TopologyLogger())

Motor monitors MongoDB servers with periodic checks called “heartbeats”. Subclass ServerHeartbeatListener to be notified whenever Motor begins a server check, and whenever a check succeeds or fails.

class HeartbeatLogger(monitoring.ServerHeartbeatListener):
    def started(self, event):
        logging.info(f"Heartbeat sent to server {event.connection_id}")

    def succeeded(self, event):
        logging.info(
            f"Heartbeat to server {event.connection_id} "
            "succeeded with reply "
            f"{event.reply.document}"
        )

    def failed(self, event):
        logging.warning(
            f"Heartbeat to server {event.connection_id} failed with error {event.reply}"
        )


monitoring.register(HeartbeatLogger())

Thread Safety

Watch out: Your listeners’ callbacks are executed on various background threads, not the main thread. To interact with Tornado or Motor from a listener callback, you must defer to the main thread using IOLoop.add_callback, which is the only thread-safe IOLoop method. Similarly, if you use asyncio instead of Tornado, defer your action to the main thread with call_soon_threadsafe(). There is probably no need to be concerned about this detail, however: logging is the only reasonable thing to do from a listener, and the Python logging module is thread-safe.

Further Information

See also: