pax_global_header00006660000000000000000000000064127524760230014522gustar00rootroot0000000000000052 comment=b7c783d355de77a4429b9047d2c93d1e7f5464a9 trapperkeeper-metrics-0.4.2/000077500000000000000000000000001275247602300160425ustar00rootroot00000000000000trapperkeeper-metrics-0.4.2/.gitignore000066400000000000000000000002121275247602300200250ustar00rootroot00000000000000pom.xml pom.xml.asc *jar /lib/ /classes/ /target/ /checkouts/ .lein-deps-sum .lein-repl-history .lein-plugins/ .lein-failures .nrepl-port trapperkeeper-metrics-0.4.2/.travis.yml000066400000000000000000000007731275247602300201620ustar00rootroot00000000000000language: clojure lein: lein2 jdk: - oraclejdk7 - openjdk7 script: ./ext/travisci/test.sh notifications: email: false hipchat: rooms: secure: CfGS3yYsocLruSP0lRr9HFwzdZ1HFw4rFEVdJkP/i0i8aIPY3XB3vTQNxw/lIBVRDwWHaUfA+xnyK3itCxt0M0UtPDp1BkU6abLr8Apjet/nJJ2icBvQfnpx1hb6xBpoE69vpRiIVewoU69UPFjXdZd2D1BWX58tNbdV9CA8Ezw= template: - ! '%{repository}#%{build_number} (%{branch} - %{commit} : %{author}): %{message}' - ! 'Change view: %{compare_url}' - ! 'Build details: %{build_url}' trapperkeeper-metrics-0.4.2/CHANGELOG.md000066400000000000000000000024451275247602300176600ustar00rootroot00000000000000## 0.4.2 This is a bug fix release. * Don't require JMX to be enabled for the metrics endpoint to work. Now, if the `metrics-webservice` is added to the config, the `/metrics` endpoint will always be registered. ## 0.4.1 This is a maintenance release. * Bump puppetlabs/ring-middleware dependency from 0.3.1 to 1.0.0. ## 0.4.0 This is a feature release. * Add an `initialize-registry-settings` function to the MetricsService protocol. The implementation of this function in the trapperkeeper-metrics service in this repo is not yet implemented and currently just throws an error. ## 0.3.0 This is a minor feature, maintenance, and bugfix release. * Introduce i18n library and lay groundwork for future i18n work * Update project.clj to prefer explicit dependencies instead of implicit transitive dependencies to resolve conflicts * Extract common ring utils into puppetlabs/ring-middleware ## 0.2.0 This is a feature release. * Add the ability to configure multiple metrics registries * Add a new metrics-server service (ported from the PuppetDB `/metrics` API) for querying JMX metrics ## 0.1.2 This is a feature release. * [TK-252](https://tickets.puppetlabs.com/browse/TK-252) Always build a metrics registry, deprecate metrics.enabled setting * Add `mean-millis` and related utility fns trapperkeeper-metrics-0.4.2/CONTRIBUTING.md000066400000000000000000000010241275247602300202700ustar00rootroot00000000000000# How to contribute Third-party patches are essential for keeping Puppet Labs open-source projects great. We want to keep it as easy as possible to contribute changes that allow you to get the most out of our projects. There are a few guidelines that we need contributors to follow so that we can have a chance of keeping on top of things. For more info, see our canonical guide to contributing: [https://github.com/puppetlabs/puppet/blob/master/CONTRIBUTING.md](https://github.com/puppetlabs/puppet/blob/master/CONTRIBUTING.md) trapperkeeper-metrics-0.4.2/LICENSE000066400000000000000000000014011275247602300170430ustar00rootroot00000000000000 Trapperkeeper Metrics - A library to help make it easier to track metrics in other Trapperkeeper applications Copyright (C) 2005-2015 Puppet Labs Inc Puppet Labs can be contacted at: info@puppetlabs.com Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. trapperkeeper-metrics-0.4.2/README.md000066400000000000000000000031051275247602300173200ustar00rootroot00000000000000# `trapperkeeper-metrics` [![Build Status](https://travis-ci.org/puppetlabs/trapperkeeper-metrics.svg?branch=master)](https://travis-ci.org/puppetlabs/trapperkeeper-metrics) [![Clojars Project](http://clojars.org/puppetlabs/trapperkeeper-metrics/latest-version.svg)](http://clojars.org/puppetlabs/trapperkeeper-metrics) `trapperkeeper-metrics` is a library intended to help make it easier to track metrics in other Trapperkeeper applications. It includes: * a TK service that manages the life cycle of your metrics registry * config-driven control of metrics and metrics reporting * other miscellaneous utility functions for working with metrics For more detailed information (what this library does and doesn't do, more detailed tips on how to write code against it, future plans, etc.), check out the [documentation](./documentation/index.md). ## HTTP Metrics with `comidi` To get the most value out of this library, use it in concert with [comidi](https://github.com/puppetlabs/comidi) and [trapperkeeper-comidi-metrics](https://github.com/puppetlabs/trapperkeeper-comidi-metrics) (to take advantage of the built-in HTTP metrics; see the trapperkeeper-comidi-metrics docs) and the [Trapperkeeper Status Service](https://github.com/puppetlabs/trapperkeeper-status) (to expose the most useful metrics data from your app via HTTP). The `trapperkeeper-comidi-metrics` repo contains a working example app that illustrates how to tie everything together. #Support Please log tickets and issues at our [Jira Tracker](https://tickets.puppetlabs.com/issues/?jql=project%20%3D%20Trapperkeeper). trapperkeeper-metrics-0.4.2/dev-resources/000077500000000000000000000000001275247602300206305ustar00rootroot00000000000000trapperkeeper-metrics-0.4.2/dev-resources/logback-test.xml000066400000000000000000000005401275247602300237300ustar00rootroot00000000000000 %d %-5p [%c{2}] %m%n trapperkeeper-metrics-0.4.2/documentation/000077500000000000000000000000001275247602300207135ustar00rootroot00000000000000trapperkeeper-metrics-0.4.2/documentation/configuration.md000066400000000000000000000010651275247602300241060ustar00rootroot00000000000000# Configuring the `MetricsService` To configure the Metrics service, edit the `metrics.conf` file in your `conf.d` directory. #### `metrics.conf` Here is a sample config file that illustrates the available settings for metrics: ``` metrics: { # a server id that will be used as part of the namespace for metrics produced # by this server server-id: localhost # this section is used to enable/disable JMX reporting reporters: { # enable or disable JMX metrics reporter jmx: { enabled: true } } } ``` trapperkeeper-metrics-0.4.2/documentation/index.md000066400000000000000000000237041275247602300223520ustar00rootroot00000000000000# `trapperkeeper-metrics` Documentation ## What This Library Does The main purpose of this library is to provide some wrapper code around the [Dropwizard Metrics Java Library](https://dropwizard.github.io/metrics/3.1.0/), but there is some other functionality provided as well. Here are the major features available in `trapperkeeper-metrics`: * A `Trapperkeeper` service that handles life cycle management for objects from the Dropwizard Metrics library * Support for reading basic metrics configuration info from a `Trapperkeeper` config file, so that the metrics configuration syntax is consistent across all of your `Trapperkeeper` apps * Other utility functions for creating and interacting with metrics in your application For more detail on these features, read on. ### `MetricRegistry` Life Cycle The main entry point into the Dropwizard Metrics API is a class called [`MetricRegistry`](https://dropwizard.github.io/metrics/3.1.0/apidocs/com/codahale/metrics/MetricRegistry.html). This class requires some basic initialization, so `trapperkeeper-metrics` provides a Trapperkeeper service (`MetricsService`) that manages the life cycle of a `MetricRegistry`. The service includes a function, `get-metrics-registry`, so that all of your other Trapperkeeper services can access the registry and register new metrics with it. For example: ```clj (defservice my-service [[:MetricsService get-metrics-registry]] (init [this context] (let [metrics-registry (get-metrics-registry) my-metric-name (metrics/host-metric-name "localhost" "my-metric") my-timer (.timer metrics-registry my-metric-name)] (metrics/time! my-timer (do-some-work))) context)) ``` See the [source code for the sample app](https://github.com/puppetlabs/trapperkeeper-comidi-metrics/blob/master/dev/example/comidi_metrics_web_app.clj) for a working example. See the utility functions in the `trapperkeeper-metrics` [`puppetlabs.metrics`](../src/puppetlabs/metrics.clj) namespace for some helpers for constructing other kinds of metrics besides just `Timer`. See the [Dropwizard Metrics docs](https://dropwizard.github.io/metrics/3.1.0/) for more info about all of the available metrics types and their features. The `get-metrics-registry` also allows you to specify two additional fields, `registry-key` and `domain` which allow you to create other registries (besides the default given by `(get-metrics-registry)`) and allow you to configure the namespace of the reporter for that registry. ```clj (defservice my-service [[:MetricsService get-metrics-registry]] (init [this context] (let [default-metrics-registry (get-metrics-registry) my-metrics-registry (get-metrics-registry "my.metrics.domain") ;; This will create the metric ;; `my.metrics.domain:name=puppetlabs.localhost.my-metric` my-metric-name (metrics/host-metric-name "localhost" "my-metric") my-timer (.timer metrics-registry my-metric-name)] (assert (not= default-metrics-registry my-metrics-registry)) (metrics/time! my-timer (do-some-work))) context) (start [this context] ;; We can retrieve the same metrics-registry later. (let [my-metrics-registry (get-metrics-registry "my.metrics.domain")] (do-some-other-work my-metrics-registry)) context)) ``` ### Configuration & Reporters The `MetricsService` also provides a configuration syntax that users can use to configure the metrics for a running TK app. This means that all TK services can provide a consistent interface for interacting with metrics. For more specific details, see the [`MetricsService` configuration docs](../documentation/configuration.md). ### Utility Functions The [`puppetlabs.metrics`](../src/puppetlabs/metrics.clj) namespace contains some utility functions for working with metrics. See the source code and docstrings for more detail. Here are a few bits of basic info: `time!` is a macro that can be used to time some Clojure forms against an existing `Timer`. e.g.: ```clj (let [my-timer (.timer (get-metrics-registry) "my.metric.name")] (time! my-timer (do-some-work!) (do-some-more-work!))) ``` `host-metric-name` can be used to provide a qualified, namespaced metric name. For best results, it is advisable to use this function to create a name for each of your application's metrics; this will ensure that the metrics are namespaced consistently across services. It will also ensure that metrics are namespaced by hostname, which is critical when consolidating metrics data from multiple hosts/services. You can use the `server-id` value from the `MetricsService` configuration to get the appropriate hostname for your metric. (TODO: this part of the API needs to be fleshed out a bit further. We might want to have a more dynamic way to get the hostname/server-id rather than having to put it into the config file. We may want to tweak some other things as well. *In the interim*, it's probably a good idea to *make it clear in your application/service documentation* that the specific metric namespaces should not be considered part of a formal API and may change in subsequent releases.) `register` can be used to add a metric to an existing `MetricRegistry`. `ratio`, `metered-ratio`, and `gauge` can be used to construct other types of Metrics. ### Low-level HTTP API To enable the HTTP API for accessing individual metrics add the service to your `bootstrap.cfg` and configure the `web-router-service` accordingly: ``` web-router-service { "puppetlabs.trapperkeeper.services.metrics.metrics-service/metrics-webservice" : "/metrics" } ``` #### Listing available metrics ##### Request format To get a list of all available metric names: * Request `/metrics/v1/mbeans`. * Use a `GET` request. ##### Response format Responses return a JSON object mapping a string to a string: * The key is the name of a valid MBean. * The value is a URI to use for requesting that MBean's attributes. #### Retrieving multiple metrics ##### Request format To get a the attributes for multiple metrics at the same time: * Request `/metrics/v1/mbeans`. * Use a `POST` request. * Use a request body which is either a JSON object whose values are metric names, JSON array of metric names or a JSON string of a metric name. ##### Response format The response format, though always JSON, depends on the request format: * Requests with a JSON object will return the a JSON object where the values of the original object have been transformed into the Mbeans' attributes for the metric names. * Requests with a JSON array will return the a JSON array where the items of the original array have been transformed into the Mbeans' attributes for the metric names. * Requests with a JSON string will return the a JSON object of the Mbean's attributes for the given metric name. #### Retrieving an specific metric ##### Request format To get the attributes of a particular metric: * Request `/metrics/v1/mbeans/`, where `` is something that was returned in the list of available metrics specified above. * Use a `GET` request. ##### Response format Responses return a JSON object mapping strings to (strings/numbers/Booleans). For example, using `curl` from localhost: curl 'http://localhost:8080/metrics/v1/mbeans/java.lang:type=Memory' { "ObjectPendingFinalizationCount" : 0, "HeapMemoryUsage" : { "committed" : 807403520, "init" : 268435456, "max" : 3817865216, "used" : 129257096 }, "NonHeapMemoryUsage" : { "committed" : 85590016, "init" : 24576000, "max" : 184549376, "used" : 85364904 }, "Verbose" : false, "ObjectName" : "java.lang:type=Memory" } #### Alternatives Since we support sending the metrics data to JMX, there are several existing tools and approaches that can be used to read the data for individual metrics via JMX. JVisualVM is one example; see [pe-puppetserver-jruby-jmx-client](https://github.com/puppetlabs/pe-puppetserver-jruby-jmx-client) for an example of how to do this from a JRuby script. ## What This Library Does *Not* Do ## Notes for Developers For best results, use this library in combination with the [`comidi`](https://github.com/puppetlabs/comidi) library, and then take advantage of the `wrap-with-request-metrics` Ring middleware to track metrics about all HTTP requests made to your application. See the `comidi` docs for more info. ## In The Future There Will Be Robots Some ideas for things we might want to add/change in the future: ### metrics-clojure There is an existing clojure library that wraps Dropwizard Metrics: [`metrics-clojure`](https://github.com/sjl/metrics-clojure). At the time when we originally wrote our metrics code, this library was very out-of-date and didn't support some of the features we needed. It also seemed like a pretty thin facade around the Java library, and we decided that it wasn't worth adding an extra dependency. Since then, it's been updated and should be much more compatible with the more recent versions of Dropwizard Metrics. At a glance, it seems like it would be possible for other TK services to use `metrics-clojure` in combination with `trapperkeeper-metrics` as-is; it mostly just provides utility functions that should work fine with the `MetricRegistry` object surfaced by `trapperkeeper-metrics`. So, if you feel like the abstractions it provides over the Java library are worthwhile, try it out and let us know if something doesn't work properly. At some point in the future we may go ahead and add it as a direct dependency and refactor things in `trapperkeeper-metrics` to use it, but so far there hasn't been a hugely compelling reason to do so. ### Additional Facilities for Exposing Metrics There are a few other ideas floating around for how to make it easier for apps/services that are using `trapperkeeper-metrics` to expose the metrics info to end users. More utility functions for easier integration with status service, other tools for facilitating consumption / visualization / etc. Stay tuned. trapperkeeper-metrics-0.4.2/ext/000077500000000000000000000000001275247602300166425ustar00rootroot00000000000000trapperkeeper-metrics-0.4.2/ext/travisci/000077500000000000000000000000001275247602300204665ustar00rootroot00000000000000trapperkeeper-metrics-0.4.2/ext/travisci/test.sh000077500000000000000000000000301275247602300217750ustar00rootroot00000000000000#!/bin/bash lein2 test trapperkeeper-metrics-0.4.2/project.clj000066400000000000000000000054341275247602300202100ustar00rootroot00000000000000(def ks-version "1.3.0") (def tk-version "1.4.0") (defproject puppetlabs/trapperkeeper-metrics "0.4.2" :description "Trapperkeeper Metrics Service" :url "http://github.com/puppetlabs/trapperkeeper-metrics" :pedantic? :abort :dependencies [[org.clojure/clojure "1.7.0"] ;; begin version conflict resolution dependencies [clj-time "0.11.0"] [commons-codec "1.9"] [org.clojure/tools.macro "0.1.5"] [org.clojure/tools.reader "1.0.0-alpha1"] [prismatic/schema "1.1.0"] [slingshot "0.12.2"] [commons-io "2.4"] [ring/ring-servlet "1.4.0"] ;; end version conflict resolution dependencies [puppetlabs/kitchensink ~ks-version] [puppetlabs/trapperkeeper ~tk-version] [puppetlabs/ring-middleware "1.0.0"] [ring/ring-core "1.4.0"] [cheshire "5.6.1"] [org.clojure/java.jmx "0.3.1"] ;; ring-defaults brings in a bad, old version of the servlet-api, which ;; now has a new artifact name (javax.servlet/javax.servlet-api). If we ;; don't exclude the old one here, they'll both be brought in, and consumers ;; will be subject to the whims of which one shows up on the classpath first. ;; thus, we need to use exclusions here, even though we'd normally resolve ;; this type of thing by just specifying a fixed dependency version. [ring/ring-defaults "0.1.5" :exclusions [javax.servlet/servlet-api]] ;; Explicitly reference the correct servlet-api so that downstream ;; projects will always get it [javax.servlet/javax.servlet-api "3.1.0"] [org.clojure/tools.logging "0.3.1"] [org.slf4j/slf4j-api "1.7.13"] [io.dropwizard.metrics/metrics-core "3.1.2"] [puppetlabs/comidi "0.3.1"] [puppetlabs/i18n "0.4.1"]] :plugins [[puppetlabs/i18n "0.4.1"]] :deploy-repositories [["releases" {:url "https://clojars.org/repo" :username :env/clojars_jenkins_username :password :env/clojars_jenkins_password :sign-releases false}]] :profiles {:dev {:dependencies [[puppetlabs/http-client "0.5.0" :exclusions [commons-io]] [puppetlabs/trapperkeeper ~tk-version :classifier "test"] [puppetlabs/trapperkeeper-webserver-jetty9 "1.3.1" :exclusions [clj-time]] [puppetlabs/kitchensink ~ks-version :classifier "test"]]}}) trapperkeeper-metrics-0.4.2/src/000077500000000000000000000000001275247602300166315ustar00rootroot00000000000000trapperkeeper-metrics-0.4.2/src/puppetlabs/000077500000000000000000000000001275247602300210105ustar00rootroot00000000000000trapperkeeper-metrics-0.4.2/src/puppetlabs/metrics.clj000066400000000000000000000056131275247602300231550ustar00rootroot00000000000000;; Utility functions for working with the Metrics library (ns puppetlabs.metrics (:import (com.codahale.metrics MetricRegistry RatioGauge RatioGauge$Ratio Gauge Metric Metered Sampling Timer) (java.util.concurrent TimeUnit)) (:require [schema.core :as schema])) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; Public (schema/defn ^:alwyas-validate host-metric-name :- schema/Str "Given a hostname and a metric name, build a qualified metric name for use with Metrics." [hostname :- schema/Str metric-name :- schema/Str] (MetricRegistry/name "puppetlabs" (into-array String [hostname metric-name]))) (schema/defn ^:always-validate http-metric-name :- schema/Str "Given a hostname and a metric name, build a qualified http metric name for use with Metrics." [hostname :- schema/Str metric-name :- schema/Str] (MetricRegistry/name "puppetlabs" (into-array String [hostname "http" metric-name]))) (schema/defn ^:always-validate register :- Metric "Register a metric with a metrics registry, using the given metric name." [registry :- MetricRegistry metric-name :- schema/Str metric :- Metric] (.register registry metric-name metric)) (schema/defn mean :- Double "Given a Timer or Histogram object, get the current mean value." [sampling :- Sampling] (.. sampling getSnapshot getMean)) (schema/defn mean-millis :- Long "Given a Timer or Histogram object, get the mean sample time in milliseconds." [sampling :- Sampling] (.toMillis TimeUnit/NANOSECONDS (mean sampling))) (schema/defn mean-in-unit :- Long "Given a Timer or Histogram object, get the mean sample time in the specified time unit." [sampling :- Sampling time-unit :- TimeUnit] (.convert time-unit (mean sampling) TimeUnit/NANOSECONDS)) (schema/defn ^:always-validate ratio :- RatioGauge "Given two functions, return a Ratio metric whose value will be computed by calling the first function to retrieve the numerator and the second function to retrieve the denominator" [numerator-fn :- (schema/pred ifn?) denominator-fn :- (schema/pred ifn?)] (proxy [RatioGauge] [] (getRatio [] (RatioGauge$Ratio/of (numerator-fn) (denominator-fn))))) (schema/defn ^:always-validate metered-ratio :- RatioGauge "Given two Metered metrics, construct a Ratio metric whose numerator and denominator are computed by calling the `getCount` method of the Metered metrics." [numerator :- Metered denominator :- Metered] (ratio #(.getCount numerator) #(.getCount denominator))) (schema/defn ^:always-validate gauge :- Gauge "Returns a Gauge metric with an initial value" [value] (proxy [Gauge] [] (getValue [] value))) (defmacro time! "Times the body forms against the given Timer metric" [^Timer t & body] `(.time ~(vary-meta t assoc :tag `Timer) (proxy [Callable] [] (call [] (do ~@body))))) trapperkeeper-metrics-0.4.2/src/puppetlabs/trapperkeeper/000077500000000000000000000000001275247602300236615ustar00rootroot00000000000000trapperkeeper-metrics-0.4.2/src/puppetlabs/trapperkeeper/services/000077500000000000000000000000001275247602300255045ustar00rootroot00000000000000trapperkeeper-metrics-0.4.2/src/puppetlabs/trapperkeeper/services/metrics/000077500000000000000000000000001275247602300271525ustar00rootroot00000000000000trapperkeeper-metrics-0.4.2/src/puppetlabs/trapperkeeper/services/metrics/metrics_core.clj000066400000000000000000000113341275247602300323240ustar00rootroot00000000000000(ns puppetlabs.trapperkeeper.services.metrics.metrics-core (:import (com.codahale.metrics JmxReporter MetricRegistry) (com.fasterxml.jackson.core JsonParseException)) (:require [clojure.tools.logging :as log] [clojure.java.io :as io] [cheshire.core :as json] [schema.core :as schema] [ring.middleware.defaults :as ring-defaults] [puppetlabs.comidi :as comidi] [puppetlabs.ring-middleware.utils :as ringutils] [puppetlabs.trapperkeeper.services.metrics.metrics-utils :as metrics-utils] [puppetlabs.kitchensink.core :as ks] [puppetlabs.i18n.core :as i18n :refer [trs tru]])) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; Schemas (def JmxReporterConfig {:enabled schema/Bool}) (def ReportersConfig {(schema/optional-key :jmx) JmxReporterConfig}) (def MetricsConfig {:server-id schema/Str (schema/optional-key :enabled) schema/Bool (schema/optional-key :reporters) ReportersConfig}) (def RegistryContext {:registry (schema/maybe MetricRegistry) :jmx-reporter (schema/maybe JmxReporter)}) (def MetricsServiceContext {:registries (schema/atom {schema/Any RegistryContext})}) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; Private (schema/defn jmx-reporter :- JmxReporter [registry :- MetricRegistry domain :- (schema/maybe schema/Str)] (let [b (JmxReporter/forRegistry registry)] (when-let [^String d domain] (.inDomain b d)) (.build b))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; Public (schema/defn initialize :- RegistryContext [config :- MetricsConfig domain :- (schema/maybe schema/Str)] (let [jmx-config (get-in config [:reporters :jmx]) registry (MetricRegistry.)] (when (contains? config :enabled) (log/warn (format "%s %s" (trs "Metrics are now always enabled.") (trs "To suppress this warning remove metrics.enabled from your configuration.")))) {:registry registry :jmx-reporter (when (:enabled jmx-config) (doto ^JmxReporter (jmx-reporter registry domain) (.start)))})) (schema/defn get-or-initialize! :- RegistryContext [config :- MetricsConfig {:keys [registries]} :- MetricsServiceContext domain :- schema/Str] (if-let [metric-reg (get-in @registries [domain])] metric-reg (let [reg-context (initialize config domain)] (swap! registries assoc domain reg-context) reg-context))) (schema/defn stop :- RegistryContext [context :- RegistryContext] (if-let [jmx-reporter (:jmx-reporter context)] (.close jmx-reporter)) context) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; Comidi (defn build-handler [path] (comidi/routes->handler (comidi/wrap-routes (comidi/context path (comidi/context "/v1" (comidi/context "/mbeans" (comidi/GET "" [] (fn [req] (ringutils/json-response 200 (metrics-utils/mbean-names)))) (comidi/POST "" [] (fn [req] (try (let [metrics (with-open [reader (-> req :body io/reader)] (json/parse-stream reader true))] (cond (seq? metrics) (ringutils/json-response 200 (map metrics-utils/get-mbean metrics)) (string? metrics) (ringutils/json-response 200 (metrics-utils/get-mbean metrics)) (map? metrics) (ringutils/json-response 200 (ks/mapvals metrics-utils/get-mbean metrics)) :else (ringutils/json-response 400 (tru "metrics request must be a JSON array, string, or object")))) (catch JsonParseException e (ringutils/json-response 400 {:error (str e)}))))) (comidi/GET ["/" [#".*" :names]] [] (fn [{:keys [route-params] :as req}] (let [name (java.net.URLDecoder/decode (:names route-params))] (if-let [mbean (metrics-utils/get-mbean name)] (ringutils/json-response 200 mbean) (ringutils/json-response 404 (tru "No mbean ''{0}'' found" name))))))))) (comp i18n/locale-negotiator #(ring-defaults/wrap-defaults % ring-defaults/api-defaults))))) trapperkeeper-metrics-0.4.2/src/puppetlabs/trapperkeeper/services/metrics/metrics_service.clj000066400000000000000000000030471275247602300330360ustar00rootroot00000000000000(ns puppetlabs.trapperkeeper.services.metrics.metrics-service (:require [puppetlabs.trapperkeeper.core :as trapperkeeper] [puppetlabs.trapperkeeper.services.protocols.metrics :as metrics] [puppetlabs.trapperkeeper.services.metrics.metrics-core :as core] [puppetlabs.trapperkeeper.services :as tk-services])) (trapperkeeper/defservice metrics-service metrics/MetricsService [[:ConfigService get-in-config]] (init [this context] {:registries (atom {"default" (core/initialize (get-in-config [:metrics] {}) nil)})}) (stop [this context] (let [{:keys [registries] :as ctx} (tk-services/service-context this)] (doseq [[_ metrics-reg] @registries] (core/stop metrics-reg)) ctx)) (get-metrics-registry [this] (-> @(:registries (tk-services/service-context this)) (get "default") :registry)) (get-metrics-registry [this domain] (:registry (core/get-or-initialize! (get-in-config [:metrics] {}) (tk-services/service-context this) domain))) (initialize-registry-settings [this domain settings] (throw (RuntimeException. "`initialize-registry-settings` is not yet implemented for this service")))) (trapperkeeper/defservice metrics-webservice [[:ConfigService get-in-config] [:WebroutingService add-ring-handler get-route]] (init [this context] (add-ring-handler this (core/build-handler (get-route this))) context) (stop [this context] context)) trapperkeeper-metrics-0.4.2/src/puppetlabs/trapperkeeper/services/metrics/metrics_utils.clj000066400000000000000000000026411275247602300325350ustar00rootroot00000000000000(ns puppetlabs.trapperkeeper.services.metrics.metrics-utils (:require [cheshire.custom :refer [JSONable]] [clojure.java.jmx :as jmx] [puppetlabs.kitchensink.core :as ks])) (defn filter-mbean "Converts an mbean to a map. For attributes that can't be converted to JSON, return a string representation of the value." [mbean] {:post [(map? %)]} (->> mbean (ks/mapvals (fn [v] (cond ;; Nested structures should themselves be filtered (map? v) (filter-mbean v) (instance? java.util.HashMap v) (->> v (into {}) filter-mbean) (satisfies? JSONable v) v :else (str v)))))) (defn all-mbean-names "Return a seq of all mbeans names" [] {:post [(coll? %)]} (map str (jmx/mbean-names "*:*"))) (defn mbean-names "Return a map of mbean name to a link that will retrieve the attributes" [] (->> (all-mbean-names) (map (fn [mbean-name] [mbean-name (format "/mbeans/%s" (java.net.URLEncoder/encode mbean-name "UTF-8"))])) (into (sorted-map)))) (defn get-mbean "Returns the attributes of a given MBean" [name] (when (some #(= name %) (all-mbean-names)) (filter-mbean (jmx/mbean name)))) trapperkeeper-metrics-0.4.2/src/puppetlabs/trapperkeeper/services/protocols/000077500000000000000000000000001275247602300275305ustar00rootroot00000000000000trapperkeeper-metrics-0.4.2/src/puppetlabs/trapperkeeper/services/protocols/metrics.clj000066400000000000000000000014531275247602300316730ustar00rootroot00000000000000(ns puppetlabs.trapperkeeper.services.protocols.metrics) (defprotocol MetricsService "A service that tracks runtime metrics for the process." (get-metrics-registry [this] [this domain] "Provides access to a MetricsRegistry where `domain` is the string used to look up the registry. Specifing no `domain` will return the default MetricsRegistry. The `domain` is the name that will appear at the front of the JMX metric. For example in `foo:name=my-metric`, `foo` is the `domain`.") (initialize-registry-settings [this domain settings] "Allows for specifying settings for a metric registry reporter that don't go into a config file. Must be called during the 'init' phase of a service's lifecycle. Will error if called more than once per metric registry.")) trapperkeeper-metrics-0.4.2/test/000077500000000000000000000000001275247602300170215ustar00rootroot00000000000000trapperkeeper-metrics-0.4.2/test/puppetlabs/000077500000000000000000000000001275247602300212005ustar00rootroot00000000000000trapperkeeper-metrics-0.4.2/test/puppetlabs/metrics_test.clj000066400000000000000000000060631275247602300244040ustar00rootroot00000000000000(ns puppetlabs.metrics-test (:import (com.codahale.metrics MetricRegistry Meter Timer) (java.util.concurrent TimeUnit)) (:require [clojure.test :refer :all] [puppetlabs.metrics :refer :all] [schema.test :as schema-test])) (use-fixtures :once schema-test/validate-schemas) (deftest test-host-metric-name (testing "host-metric-name should create a metric name" (is (= "puppetlabs.localhost.foocount" (host-metric-name "localhost" "foocount"))))) (deftest test-http-metric-name (testing "http-metric-name should create a metric name" (is (= "puppetlabs.localhost.http.foocount" (http-metric-name "localhost" "foocount"))))) (deftest test-register (testing "register should add a metric to the registry" (let [registry (MetricRegistry.)] (register registry (host-metric-name "localhost" "foo") (gauge 2)) (let [gauges (.getGauges registry)] (is (= 1 (count gauges))) (is (= "puppetlabs.localhost.foo" (first (.keySet gauges)))))))) (deftest test-mean-utils (let [timer (Timer.)] (time! timer (Thread/sleep 1)) (is (= 1 (.getCount timer))) (let [elapsed-nanos (mean timer) elapsed-millis (mean-millis timer) elapsed-in-millis (mean-in-unit timer TimeUnit/MILLISECONDS) elapsed-in-nanos (mean-in-unit timer TimeUnit/NANOSECONDS)] (println "elapsed-in-millis:" elapsed-in-millis) (is (>= elapsed-millis 1)) (is (>= elapsed-in-millis 1)) (is (<= elapsed-millis 100)) (is (<= elapsed-in-millis 100)) (is (>= elapsed-nanos (* 1000 1000))) (is (>= elapsed-in-nanos (* 1000 1000))) (is (<= elapsed-nanos (* 1000 1000 100))) (is (<= elapsed-in-nanos (* 1000 1000 100)))))) (deftest test-ratio (testing "ratio should create a ratio metric" (let [numerator (atom 4) numerator-fn (fn [] @numerator) denominator-fn (constantly 2) ratio-metric (ratio numerator-fn denominator-fn)] (is (= 2.0 (.. ratio-metric getRatio getValue))) (reset! numerator 6) (is (= 3.0 (.. ratio-metric getRatio getValue)))))) (deftest test-metered-ratio (testing "metered-ratio builds a ratio metric from two counters" (let [numerator-meter (Meter.) denominator-timer (Timer.) ratio-metric (metered-ratio numerator-meter denominator-timer)] (dotimes [_ 2] (.update denominator-timer 0 TimeUnit/MILLISECONDS)) (dotimes [_ 4] (.mark numerator-meter)) (is (= 2.0 (.. ratio-metric getRatio getValue))) (dotimes [_ 2] (.mark numerator-meter)) (is (= 3.0 (.. ratio-metric getRatio getValue)))))) (deftest test-gauge (testing "gauge creates a gauge metric with an initial value" (let [gauge-metric (gauge 42)] (is (= 42 (.getValue gauge-metric)))))) (deftest test-time! (testing "time! will time the enclosed form" (let [timer (Timer.)] (time! timer (Thread/sleep 1)) (is (= 1 (.getCount timer))) (let [elapsed (mean-millis timer)] (is (>= elapsed 1)) (is (<= elapsed 100))))))trapperkeeper-metrics-0.4.2/test/puppetlabs/trapperkeeper/000077500000000000000000000000001275247602300240515ustar00rootroot00000000000000trapperkeeper-metrics-0.4.2/test/puppetlabs/trapperkeeper/services/000077500000000000000000000000001275247602300256745ustar00rootroot00000000000000trapperkeeper-metrics-0.4.2/test/puppetlabs/trapperkeeper/services/metrics/000077500000000000000000000000001275247602300273425ustar00rootroot00000000000000trapperkeeper-metrics-0.4.2/test/puppetlabs/trapperkeeper/services/metrics/metrics_core_test.clj000066400000000000000000000024051275247602300335520ustar00rootroot00000000000000(ns puppetlabs.trapperkeeper.services.metrics.metrics-core-test (:import (com.codahale.metrics MetricRegistry JmxReporter)) (:require [clojure.test :refer :all] [puppetlabs.trapperkeeper.testutils.logging :refer :all] [puppetlabs.trapperkeeper.services.metrics.metrics-core :refer :all] [schema.test :as schema-test])) (use-fixtures :once schema-test/validate-schemas) (deftest test-initialize (testing "it logs if :enabled is provided" (with-test-logging (let [context (initialize {:server-id "localhost" :enabled false} nil)] (is (logged? #"^Metrics are now always enabled." :warn)) (is (instance? MetricRegistry (:registry context)))))) (testing "initializes registry and adds to context" (let [context (initialize {:server-id "localhost"} "my.epic.domain")] (is (instance? MetricRegistry (:registry context))) (is (nil? (:jmx-reporter context))))) (testing "enables jmx reporter if configured to do so" (let [context (initialize {:server-id "localhost" :reporters {:jmx {:enabled true}}} "foo.bar.baz")] (is (instance? MetricRegistry (:registry context))) (is (instance? JmxReporter (:jmx-reporter context) ))))) trapperkeeper-metrics-0.4.2/test/puppetlabs/trapperkeeper/services/metrics/metrics_service_test.clj000066400000000000000000000150361275247602300342660ustar00rootroot00000000000000(ns puppetlabs.trapperkeeper.services.metrics.metrics-service-test (:import (com.codahale.metrics MetricRegistry)) (:require [clojure.test :refer :all] [cheshire.core :as json] [puppetlabs.http.client.sync :as http-client] [puppetlabs.metrics :as metrics] [puppetlabs.trapperkeeper.services.metrics.metrics-service :refer :all] [puppetlabs.trapperkeeper.services.protocols.metrics :as metrics-protocol] [schema.test :as schema-test] [puppetlabs.trapperkeeper.services.webrouting.webrouting-service :as webrouting-service] [puppetlabs.trapperkeeper.services.webserver.jetty9-service :as jetty9-service] [puppetlabs.trapperkeeper.testutils.bootstrap :refer [with-app-with-config]] [puppetlabs.trapperkeeper.app :as app] [puppetlabs.kitchensink.core :as ks])) (use-fixtures :once schema-test/validate-schemas) (defn parse-response ([resp] (parse-response resp false)) ([resp keywordize?] (-> resp :body slurp (json/parse-string keywordize?)))) (def metrics-service-config {:metrics {:server-id "localhost" :reporters {:jmx {:enabled true}}} :webserver {:port 8180 :host "0.0.0.0"} :web-router-service {:puppetlabs.trapperkeeper.services.metrics.metrics-service/metrics-webservice "/metrics"}}) (deftest test-metrics-service (testing "Can boot metrics service and access registry" (with-app-with-config app [jetty9-service/jetty9-service webrouting-service/webrouting-service metrics-service metrics-webservice] metrics-service-config (testing "metrics service functions" (let [svc (app/get-service app :MetricsService)] (testing "`get-metrics-registry` called without domain works" (is (instance? MetricRegistry (metrics-protocol/get-metrics-registry svc)))) (testing "`get-metrics-registry` called with domain works" (is (instance? MetricRegistry (metrics-protocol/get-metrics-registry svc "pl.foo.reg")))) (testing "`initialize-registry-settings` throws an error because it is not yet implemented" (is (thrown? RuntimeException (metrics-protocol/initialize-registry-settings svc "foo" {"foo" "bar"})))))) (testing "returns latest status for all services" (let [resp (http-client/get "http://localhost:8180/metrics/v1/mbeans") body (parse-response resp)] (is (= 200 (:status resp))) (doseq [[metric path] body :let [resp (http-client/get (str "http://localhost:8180/metrics/v1" path))]] (is (= 200 (:status resp)))))) (testing "register should add a metric to the registry" (let [svc (app/get-service app :MetricsService) registry (metrics-protocol/get-metrics-registry svc "pl.foo.reg")] (metrics/register registry (metrics/host-metric-name "localhost" "foo") (metrics/gauge 2)) (let [resp (http-client/get (java.net.URLDecoder/decode (str "http://localhost:8180/metrics/v1/mbeans/" "pl.foo.reg:name=puppetlabs.localhost.foo"))) body (parse-response resp)] (is (= 200 (:status resp))) (is (= {"Value" 2} body))))) (testing "querying multiple metrics via POST should work" (let [svc (app/get-service app :MetricsService) registry (metrics-protocol/get-metrics-registry svc "pl.other.reg")] (metrics/register registry (metrics/host-metric-name "localhost" "foo") (metrics/gauge 2)) (metrics/register registry (metrics/host-metric-name "localhost" "bar") (metrics/gauge 500)) (let [resp (http-client/post "http://localhost:8180/metrics/v1/mbeans" {:body (json/generate-string ["pl.other.reg:name=puppetlabs.localhost.foo" "pl.other.reg:name=puppetlabs.localhost.bar"])}) body (parse-response resp)] (is (= 200 (:status resp))) (is (= [{"Value" 2} {"Value" 500}] body))) (let [resp (http-client/post "http://localhost:8180/metrics/v1/mbeans" {:body (json/generate-string {:foo "pl.other.reg:name=puppetlabs.localhost.foo" :bar "pl.other.reg:name=puppetlabs.localhost.bar"})}) body (parse-response resp)] (is (= 200 (:status resp))) (is (= {"foo" {"Value" 2} "bar" {"Value" 500}} body))) (let [resp (http-client/post "http://localhost:8180/metrics/v1/mbeans" {:body (json/generate-string "pl.other.reg:name=puppetlabs.localhost.foo")}) body (parse-response resp)] (is (= 200 (:status resp))) (is (= {"Value" 2} body))) (let [resp (http-client/post "http://localhost:8180/metrics/v1/mbeans" {:body "{\"malformed json"}) body (slurp (:body resp))] (is (= 400 (:status resp))) (is (re-find #"Unexpected end-of-input" body)))))))) (deftest metrics-endpoint-with-jmx-disabled-test (testing "returns data for jvm even when jmx is not enabled" (let [config (assoc-in metrics-service-config [:metrics :reporters :jmx :enabled] false)] (with-app-with-config app [jetty9-service/jetty9-service webrouting-service/webrouting-service metrics-service metrics-webservice] config (testing "returns latest status for all services" (let [resp (http-client/get "http://localhost:8180/metrics/v1/mbeans") body (parse-response resp)] (is (= 200 (:status resp))) (is (not (empty? body))))) (testing "returns Memoory mbean information" (let [resp (http-client/get "http://localhost:8180/metrics/v1/mbeans/java.lang%3Atype%3DMemory") body (parse-response resp) heap-memory (get body "HeapMemoryUsage")] (is (= 200 (:status resp))) (is (= #{"committed" "init" "max" "used"} (ks/keyset heap-memory))) (is (every? #(< 0 %) (vals heap-memory)))))))))