M3 monorepo - Distributed TSDB, Aggregator and Query Engine, Prometheus Sidecar, Graphite Compatible, Metrics Platform

M3

Distributed TSDB and Query Engine, Prometheus Sidecar, Metrics Aggregator, and more. “More” now includes Graphite storage and query engine!

More information:

• Documentation
• Developer: Getting Started
• Slack (primary chat channel)
• Gitter (deprecated chat channel)
• Forum (Google Group)
• Twitter

Community meetings

M3 contributors and maintainers have monthly (every four weeks) meetings at 11:00 AM (USA Pacific) on Tuesday.

An initial agenda is posted to the community meeting shared Google doc a day before each meeting, and everyone is welcome to suggest additional topics or other agendas.

You can add the following public Google Calendar if you’re using Google Calendar (so that any changes are reflected in your own calendar when the invite is updated): https://calendar.google.com/calendar?cid=aWc5YWFxZWw0azFmMnRoZDA2dHBtZzZva2tAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ

For those not using Google Calendar, please download and import the following iCalendar (.ics) files to calendar system for the monthly invite: https://calendar.google.com/calendar/ical/ig9aaqel4k1f2thd06tpmg6okk%40group.calendar.google.com/public/basic.ics

Join Zoom Meeting: us04web.zoom.us/j/519434268

Test it out

The easiest way to testing out M3 is to follow one of the guides from the documentation. For a fully comprehensive getting started guide, see our single node how-to.

Starting a node

# to build a local m3dbnode process
make m3dbnode (note that we currently require at least Go 1.10 or higher)

# run it with the sample configuration
./bin/m3dbnode -f ./src/dbnode/config/m3dbnode-local-etcd.yml

To cross-compile and build for Linux AMD64 build with make m3dbnode-linux-amd64.

Creating a namespace to store metrics

curl -X POST http://localhost:7201/api/v1/database/create -d '{
  "type": "local",
  "namespaceName": "default",
  "retentionTime": "2h"
}'

Test RPC

To test out some of the functionality of M3DB there are some user friendly HTTP JSON APIs that you can use. These use the DB node cluster service endpoints.

Note: performance sensitive users are expected to use the more performant endpoints via either the Go src/dbnode/client/Session API, or the GRPC endpoints exposed via src/coordinator.

Write a datapoint

curl http://localhost:9003/writetagged -s -X POST -d '{
  "namespace": "default",
  "id": "foo",
  "tags": [
    {
      "name": "__name__",
      "value": "user_login"
    },
    {
      "name": "city",
      "value": "new_york"
    },
    {
      "name": "endpoint",
      "value": "/request"
    }
  ],
  "datapoint": {
    "timestamp":'"$(date +"%s")"',
    "value": 42.123456789
  }
}'

Query for reverse indexed time series data

curl http://localhost:9003/query -s -X POST -d '{
  "namespace": "default",
  "query": {
    "regexp": {
      "field": "city",
      "regexp": ".*"
    }
  },
  "rangeStart": 0,
  "rangeEnd":'"$(date +"%s")"'
}' | jq .

Building with Docker

A Dockerfile is included for both development and production deployment purposes. It uses a multi-stage build in order to produce a lightweight production image from a single Dockerfile. Accordingly, it requires Docker 17.05 or later to build.

docker build -f docker/m3dbnode/Dockerfile -t m3dbnode:$(git rev-parse head) .
docker run --name m3dbnode m3dbnode:$(git rev-parse head)

If you wish to build an image with the source code included you can stop the build after the builder stage:

docker build -f docker/m3dbnode/Dockerfile -t m3dbnode:$(git rev-parse head) --target builder .

Configuration

The default Docker image will start a single m3dbnode process with an embedded etcd instance to mimic a production environment. If you would like to further customize the configuration, you must provide your own and mount it into the container:

docker run --name m3dbnode -v /host/config.yml:/etc/m3dbnode/myconfig.yml m3dbnode:tag -f /etc/m3dbnode/myconfig.yml

This project is released under the Apache License, Version 2.0.