toxiproxy-2.0.0/000077500000000000000000000000001270742716200136065ustar00rootroot00000000000000toxiproxy-2.0.0/.gitignore000066400000000000000000000001751270742716200156010ustar00rootroot00000000000000toxiproxy toxiproxy-server toxiproxy-cli cli toxiproxy.test testing testing.test cpu.out cover*.out coverage.html *.deb tmp/ toxiproxy-2.0.0/CHANGELOG.md000066400000000000000000000034721270742716200154250ustar00rootroot00000000000000# 2.0.0rc2 (Unreleased) * Add CLI (`toxiproxy-cli`) and rename server binary to `toxiproxy-server` #93 * Fix removing a timeout toxic causing API to hang #89 * API and client return toxics as array rather than a map of name to toxic #92 * Fix multiple latency toxics not accumulating #94 * Change default toxic name to `_` #96 * Nest toxic attributes rather than having a flat structure #98 # 2.0.0rc1 (Unreleased) * 2.0 RFC: #54 and PR #62 * Change toxic API endpoints to an Add/Update/Remove structure * Remove `enabled` field, and add `name` and `type` fields to toxics * Add global toxic fields to a wrapper struct * Chain toxics together dynamically instead of in a fixed length chain * Register toxics in `init()` functions instead of a hard-coded list * Clean up API error codes to make them more consistent * Move toxics to their own package to allow 3rd party toxics * Remove stream direction from API urls #73 * Add `toxicity` field for toxics #75 * Refactor Go client to make usage easier with 2.0 #76 * Make `ChanReader` in the `stream` package interruptible #77 * Define proxy buffer sizes per-toxic (Fixes #72) * Fix slicer toxic testing race condition #71 # 1.2.1 * Fix proxy name conflicts leaking an open port #69 # 1.2.0 * Add a Toxic and Toxics type for the Go client * Add `Dockerfile` * Fix latency toxic limiting bandwidth #67 * Add Slicer toxic # 1.1.0 * Remove /toxics endpoint in favour of /proxies * Add bandwidth toxic # 1.0.3 * Rename Go library package to Toxiproxy from Client * Fix latency toxic send to closed channel panic #46 * Fix latency toxic accumulating delay #47 # 1.0.2 * Added Toxic support to Go client # 1.0.1 * Various improvements to the documentation * Initial version of Go client * Fix toxic disabling bug #42 # 1.0.0 Initial public release. toxiproxy-2.0.0/CREATING_TOXICS.md000066400000000000000000000134461270742716200164250ustar00rootroot00000000000000# Creating custom toxics Creating a toxic is done by implementing the `Toxic` interface: ```go type Toxic interface { Pipe(*toxics.ToxicStub) } ``` The `Pipe()` function defines how data flows through the toxic, and is passed a `ToxicStub` to operate on. A `ToxicStub` stores the input and output channels for the toxic, as well as an interrupt channel that is used to pause operation of the toxic. The input and output channels in a `ToxicStub` send and receive `StreamChunk` structs, which are similar to network packets. A `StreamChunk` contains a `byte[]` of stream data, and a timestamp of when Toxiproxy received the data from the client or server. This is used instead of just a plain `byte[]` so that toxics like latency can find out how long a chunk of data has been waiting in the proxy. Toxics are registered in an `init()` function so that they can be used by the server: ```go func init() { toxics.Register("toxic_name", new(ExampleToxic)) } ``` In order to use your own toxics, you will need to compile your own binary. This can be done by copying [toxiproxy.go](https://github.com/Shopify/toxiproxy/blob/master/cmd/toxiproxy.go) into a new project and registering your toxic with the server. This will allow you to add toxics without having to make a full fork of the project. If you think your toxics will be useful to others, contribute them back with a Pull Request. An example project for building a separate binary can be found here: https://github.com/xthexder/toxic-example ## A basic toxic The most basic implementation of a toxic is the [noop toxic](https://github.com/Shopify/toxiproxy/blob/master/toxics/noop.go), which just passes data through without any modifications. ```go type NoopToxic struct{} func (t *NoopToxic) Pipe(stub *toxics.ToxicStub) { for { select { case <-stub.Interrupt: return case c := <-stub.Input: if c == nil { stub.Close() return } stub.Output <- c } } } ``` The above code reads from `stub.Input` in a loop, and passes the `StreamChunk` along to `stub.Output`. Since reading from `stub.Input` will block until a chunk is available, we need to check for interrupts as the same time. Toxics will be interrupted whenever they are being updated, or possibly removed. This can happen at any point within the `Pipe()` function, so all blocking operations (including sleep), should be interruptible. When an interrupt is received, the toxic should return from the `Pipe()` function after it has written any "in-flight" data back to `stub.Output`. It is important that all data read from `stub.Input` is passed along to `stub.Output`, otherwise the stream will be missing bytes and become corrupted. When an `end of stream` is reached, `stub.Input` will return a `nil` chunk. Whenever a nil chunk is returned, the toxic should call `Close()` on the stub, and return from `Pipe()`. ## Toxic configuration Toxic configuration information can be stored in the toxic struct. The toxic will be json encoded and decoded by the api, so all public fields will be api accessible. An example of a toxic that uses configuration values is the [latency toxic](https://github.com/Shopify/toxiproxy/blob/master/toxics/latency.go) ```go type LatencyToxic struct { Latency int64 `json:"latency"` Jitter int64 `json:"jitter"` } ``` These fields can be used inside the `Pipe()` function, but generally should not be written to from the toxic. A separate instance of the toxic exists for each connection through the proxy, and may be replaced when updated by the api. If state is required in your toxic, it is better to use a local variable at the top of `Pipe()`, since struct fields are not guaranteed to be persisted across interrupts. ## Toxic buffering By default, toxics are not buffered. This means that writes to `stub.Output` will block until either the endpoint or another toxic reads it. Since toxics are chained together, this means not reading from `stub.Input` will block other toxics (and endpoint writes) from operating. If this is not behavior you want your toxic to have, you can specify a buffer size for your toxic's input. The [latency toxic](https://github.com/Shopify/toxiproxy/blob/master/toxics/latency.go) uses this in order to prevent added latency from limiting the proxy bandwidth. Specifying a buffer size is done by implementing the `BufferedToxic` interface, which adds the `GetBufferSize()` function: ```go func (t *LatencyToxic) GetBufferSize() int { return 1024 } ``` The unit used by `GetBufferSize()` is `StreamChunk`s. Chunks are generally anywhere from 1 byte, up to 32KB, so keep this in mind when thinking about how much buffering you need, and how much memory you are comfortable with using. ## Using `io.Reader` and `io.Writer` If your toxic involves modifying the data going through a proxy, you can use the `ChanReader` and `ChanWriter` interfaces in the [stream package](https://github.com/Shopify/toxiproxy/tree/master/stream). These allow reading and writing from the input and output channels as you would a normal data stream such as a TCP socket. An implementation of the noop toxic above using the stream package would look something like this: ```go func (t *NoopToxic) Pipe(stub *toxics.ToxicStub) { buf := make([]byte, 32*1024) writer := stream.NewChanWriter(stub.Output) reader := stream.NewChanReader(stub.Input) reader.SetInterrupt(stub.Interrupt) for { n, err := reader.Read(buf) if err == stream.ErrInterrupted { writer.Write(buf[:n]) return } else if err == io.EOF { stub.Close() return } writer.Write(buf[:n]) } } ``` See https://github.com/xthexder/toxic-example/blob/master/http.go for a full example of using the stream package with Go's http package. toxiproxy-2.0.0/Dockerfile000066400000000000000000000005251270742716200156020ustar00rootroot00000000000000FROM golang:1.4 ADD . /app/src/github.com/Shopify/toxiproxy RUN cd /app/src/github.com/Shopify/toxiproxy && GOPATH=/app/src/github.com/Shopify/toxiproxy/Godeps/_workspace:/app go build -ldflags="-X github.com/Shopify/toxiproxy.Version $(cat VERSION)" -o /app/toxiproxy ./cmd EXPOSE 8474 ENTRYPOINT ["/app/toxiproxy"] CMD ["-host=0.0.0.0"] toxiproxy-2.0.0/LICENSE000066400000000000000000000020631270742716200146140ustar00rootroot00000000000000The MIT License (MIT) Copyright (c) 2014 Shopify Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. toxiproxy-2.0.0/Makefile000066400000000000000000000055351270742716200152560ustar00rootroot00000000000000SERVER_NAME=toxiproxy-server CLI_NAME=toxiproxy-cli VERSION=$(shell cat VERSION) DEB=pkg/toxiproxy_$(VERSION)_amd64.deb GODEP_PATH=$(shell pwd)/Godeps/_workspace ORIGINAL_PATH=$(shell echo $(GOPATH)) COMBINED_GOPATH=$(GODEP_PATH):$(ORIGINAL_PATH) .PHONY: packages deb test linux darwin windows build: GOPATH=$(COMBINED_GOPATH) go build -ldflags="-X github.com/Shopify/toxiproxy.Version=git-$(shell git rev-parse --short HEAD)" -o $(SERVER_NAME) ./cmd GOPATH=$(COMBINED_GOPATH) go build -ldflags="-X github.com/Shopify/toxiproxy.Version=git-$(shell git rev-parse --short HEAD)" -o $(CLI_NAME) ./cli all: deb linux darwin windows deb: $(DEB) darwin: tmp/build/$(SERVER_NAME)-darwin-amd64 tmp/build/$(CLI_NAME)-darwin-amd64 linux: tmp/build/$(SERVER_NAME)-linux-amd64 tmp/build/$(CLI_NAME)-linux-amd64 windows: tmp/build/$(SERVER_NAME)-windows-amd64.exe tmp/build/$(CLI_NAME)-windows-amd64.exe release: all docker clean: rm -f tmp/build/* rm -f $(SERVER_NAME) rm -f $(CLI_NAME) rm -f *.deb test: GOMAXPROCS=4 GOPATH=$(COMBINED_GOPATH) go test -v -race ./... tmp/build/$(SERVER_NAME)-linux-amd64: GOOS=linux GOARCH=amd64 GOPATH=$(COMBINED_GOPATH) go build -ldflags="-X github.com/Shopify/toxiproxy.Version=$(VERSION)" -o $(@) ./cmd tmp/build/$(SERVER_NAME)-darwin-amd64: GOOS=darwin GOARCH=amd64 GOPATH=$(COMBINED_GOPATH) go build -ldflags="-X github.com/Shopify/toxiproxy.Version=$(VERSION)" -o $(@) ./cmd tmp/build/$(SERVER_NAME)-windows-amd64.exe: GOOS=windows GOARCH=amd64 GOPATH=$(COMBINED_GOPATH) go build -ldflags="-X github.com/Shopify/toxiproxy.Version=$(VERSION)" -o $(@) ./cmd tmp/build/$(CLI_NAME)-linux-amd64: GOOS=linux GOARCH=amd64 GOPATH=$(COMBINED_GOPATH) go build -ldflags="-X github.com/Shopify/toxiproxy.Version=$(VERSION)" -o $(@) ./cli tmp/build/$(CLI_NAME)-darwin-amd64: GOOS=darwin GOARCH=amd64 GOPATH=$(COMBINED_GOPATH) go build -ldflags="-X github.com/Shopify/toxiproxy.Version=$(VERSION)" -o $(@) ./cli tmp/build/$(CLI_NAME)-windows-amd64.exe: GOOS=windows GOARCH=amd64 GOPATH=$(COMBINED_GOPATH) go build -ldflags="-X github.com/Shopify/toxiproxy.Version=$(VERSION)" -o $(@) ./cli $(DEB): tmp/build/$(SERVER_NAME)-linux-amd64 tmp/build/$(CLI_NAME)-linux-amd64 fpm -t deb \ -s dir \ -p tmp/build/ \ --name "toxiproxy" \ --version $(VERSION) \ --license MIT \ --no-depends \ --no-auto-depends \ --architecture amd64 \ --maintainer "Simon Eskildsen " \ --description "TCP proxy to simulate network and system conditions" \ --url "https://github.com/Shopify/toxiproxy" \ $(word 1,$^)=/usr/bin/$(SERVER_NAME) \ $(word 2,$^)=/usr/bin/$(CLI_NAME) \ ./share/toxiproxy.conf=/etc/init/toxiproxy.conf docker: docker build --tag="shopify/toxiproxy:$(VERSION)" . docker tag -f shopify/toxiproxy:$(VERSION) shopify/toxiproxy:latest docker push shopify/toxiproxy:$(VERSION) docker push shopify/toxiproxy:latest toxiproxy-2.0.0/README.md000066400000000000000000000417431270742716200150760ustar00rootroot00000000000000# Toxiproxy ![](http://i.imgur.com/sOaNw0o.png) Toxiproxy is a framework for simulating network conditions. It's made specifically to work in testing, CI and development environments, supporting deterministic tampering with connections, but with support for randomized chaos and customization. **Toxiproxy is the tool you need to prove with tests that your application doesn't have single points of failure.** We've been successfully using it in all development and test environments at Shopify since October, 2014. See our [blog post][blog] on resiliency for more information. Toxiproxy usage consists of two parts. A TCP proxy written in Go (what this repository contains) and a client communicating with the proxy over HTTP. You configure your application to make all test connections go through Toxiproxy and can then manipulate their health via HTTP. See [Usage](#usage) below on how to set up your project. For example, to add 1000ms of latency to the response of MySQL from the [Ruby client](https://github.com/Shopify/toxiproxy-ruby): ```ruby Toxiproxy[:mysql_master].downstream(:latency, latency: 1000).apply do Shop.first # this takes at least 1s end ``` To take down all Redis instances: ```ruby Toxiproxy[/redis/].down do Shop.first # this will throw an exception end ``` While the examples in this README are currently in Ruby, there's nothing stopping you from creating a client in any other language (see [Clients](#clients)). ## Table of Contents 1. [Why yet another chaotic TCP proxy?](#why-yet-another-chaotic-tcp-proxy) 2. [Clients](#clients) 3. [Example](#example) 4. [Usage](#usage) 1. [Installing](#1-installing-toxiproxy) 1. [Upgrading from 1.x](#upgrading-from-toxiproxy-1x) 2. [Populating](#2-populating-toxiproxy) 3. [Using](#3-using-toxiproxy) 5. [Toxics](#toxics) 1. [Latency](#latency) 2. [Down](#down) 3. [Bandwidth](#bandwidth) 4. [Slow close](#slow_close) 5. [Timeout](#timeout) 6. [Slicer](#slicer) 6. [HTTP API](#http-api) 1. [Proxy fields](#proxy-fields) 2. [Toxic fields](#toxic-fields) 3. [Endpoints](#endpoints) 7. [CLI example](#cli-example) 8. [FAQ](#frequently-asked-questions) 9. [Development](#development) ## Why yet another chaotic TCP proxy? The existing ones we found didn't provide the kind of dynamic API we needed for integration and unit testing. Linux tools like `nc` and so on are not cross-platform and require root, which makes them problematic in test, development and CI environments. ## Clients * [toxiproxy-ruby](https://github.com/Shopify/toxiproxy-ruby) * [toxiproxy-go](https://github.com/Shopify/toxiproxy/tree/master/client) * [toxiproxy.net](https://github.com/mdevilliers/Toxiproxy.Net) * [toxiproxy-php-client](https://github.com/ihsw/toxiproxy-php-client) * [toxiproxy-node](https://github.com/dlion/toxiproxy-node) * [toxiproxy-java](https://github.com/trekawek/toxiproxy-java) ## Example Let's walk through an example with a Rails application. Note that Toxiproxy is in no way tied to Ruby, it's just been our first use case and it's currently the only language that has a client. You can see the full example at [Sirupsen/toxiproxy-rails-example](https://github.com/Sirupsen/toxiproxy-rails-example). To get started right away, jump down to [Usage](#usage). For our popular blog, for some reason we're storing the tags for our posts in Redis and the posts themselves in MySQL. We might have a `Post` class that includes some methods to manipulate tags in a [Redis set](http://redis.io/commands#set): ```ruby class Post < ActiveRecord::Base # Return an Array of all the tags. def tags TagRedis.smembers(tag_key) end # Add a tag to the post. def add_tag(tag) TagRedis.sadd(tag_key, tag) end # Remove a tag from the post. def remove_tag(tag) TagRedis.srem(tag_key, tag) end # Return the key in Redis for the set of tags for the post. def tag_key "post:tags:#{self.id}" end end ``` We've decided that erroring while writing to the tag data store (adding/removing) is OK. However, if the tag data store is down, we should be able to see the post with no tags. We could simply rescue the `Redis::CannotConnectError` around the `SMEMBERS` Redis call in the `tags` method. Let's use Toxiproxy to test that. Since we've already installed Toxiproxy and it's running on our machine, we can skip to step 2. This is where we need to make sure Toxiproxy has a mapping for Redis tags. To `config/boot.rb` (before any connection is made) we add: ```ruby require 'toxiproxy' Toxiproxy.populate([ { name: "toxiproxy_test_redis_tags", listen: "127.0.0.1:22222", upstream: "127.0.0.1:6379" } ]) ``` Then in `config/environments/test.rb` we set the `TagRedis` to be a Redis client that connects to Redis through Toxiproxy by adding this line: ```ruby TagRedis = Redis.new(port: 22222) ``` All calls in the test environment now go through Toxiproxy. That means we can add a unit test where we simulate a failure: ```ruby test "should return empty array when tag redis is down when listing tags" do @post.add_tag "mammals" # Take down all Redises in Toxiproxy Toxiproxy[/redis/].down do assert_equal [], @post.tags end end ``` The test fails with `Redis::CannotConnectError`. Perfect! Toxiproxy took down the Redis successfully for the duration of the closure. Let's fix the `tags` method to be resilient: ```ruby def tags TagRedis.smembers(tag_key) rescue Redis::CannotConnectError [] end ``` The tests pass! We now have a unit test that proves fetching the tags when Redis is down returns an empty array, instead of throwing an exception. For full coverage you should also write an integration test that wraps fetching the entire blog post page when Redis is down. Full example application is at [Sirupsen/toxiproxy-rails-example](https://github.com/Sirupsen/toxiproxy-rails-example). ## Usage Configuring a project to use Toxiproxy consists of four steps: 1. Installing Toxiproxy 2. Populating Toxiproxy 3. Using Toxiproxy ### 1. Installing Toxiproxy **Linux** See [`Releases`](https://github.com/Shopify/toxiproxy/releases) for the latest binaries and system packages for your architecture. **Ubuntu** ```bash $ wget -O toxiproxy-1.2.1.deb https://github.com/Shopify/toxiproxy/releases/download/v1.2.1/toxiproxy_1.2.1_amd64.deb $ sudo dpkg -i toxiproxy-1.2.1.deb $ sudo service toxiproxy start ``` **OS X** ```bash $ brew tap shopify/shopify $ brew install toxiproxy ``` **Windows** Toxiproxy for Windows is available for download at https://github.com/Shopify/toxiproxy/releases/download/v1.2.1/toxiproxy-windows-amd64.exe **Docker** Toxiproxy is available on [Docker Hub](https://hub.docker.com/r/shopify/toxiproxy/). ```bash $ docker pull shopify/toxiproxy $ docker run -it shopify/toxiproxy ``` **Source** If you have Go installed, you can build Toxiproxy from source using the make file: ```bash $ make build $ ./toxiproxy-server ``` #### Upgrading from Toxiproxy 1.x In Toxiproxy 2.0 several changes were made to the API that make it incompatible with version 1.x. In order to use version 2.x of the Toxiproxy server, you will need to make sure your client library supports the same version. You can check which version of Toxiproxy you are running by looking at the `/version` endpoint. See the documentation for your client library for specific library changes. Detailed changes for the Toxiproxy server can been found in [CHANGELOG.md](https://github.com/Shopify/toxiproxy/blob/master/CHANGELOG.md). ### 2. Populating Toxiproxy When your application boots, it needs to make sure that Toxiproxy knows which endpoints to proxy where. The main parameters are: name, address for Toxiproxy to **listen** on and the address of the upstream. Some client libraries have helpers for this task, which is essentially just making sure each proxy in a list is created. Example from the Ruby client: ```ruby # Make sure `shopify_test_redis_master` and `shopify_test_mysql_master` are # present in Toxiproxy Toxiproxy.populate([ { name: "shopify_test_redis_master", listen: "127.0.0.1:22220", upstream: "127.0.0.1:6379" }, { name: "shopify_test_mysql_master", listen: "127.0.0.1:24220", upstream: "127.0.0.1:3306" } ]) ``` This code needs to run as early in boot as possible, before any code establishes a connection through Toxiproxy. Please check your client library for documentation on the population helpers. Alternatively use the CLI to create proxies, e.g.: ```bash toxiproxy-cli create shopify_test_redis_master -l localhost:26379 -u localhost:6379 ``` We recommend a naming such as the above: `___`. This makes sure there are no clashes between applications using the same Toxiproxy. For large application we recommend storing the Toxiproxy configurations in a separate configuration file. We use `config/toxiproxy.json`. Use ports outside the ephemeral port range to avoid random port conflicts. It's `32,768` to `61,000` on Linux by default, see `/proc/sys/net/ipv4/ip_local_port_range`. ### 3. Using Toxiproxy To use Toxiproxy, you now need to configure your application to connect through Toxiproxy. Continuing with our example from step two, we can configure our Redis client to connect through Toxiproxy: ```ruby # old straight to redis redis = Redis.new(port: 6380) # new through toxiproxy redis = Redis.new(port: 22220) ``` Now you can tamper with it through the Toxiproxy API. In Ruby: ```ruby redis = Redis.new(port: 22220) Toxiproxy[:shopify_test_redis_master].downstream(:latency, latency: 1000).apply do redis.get("test") # will take 1s end ``` Or via the CLI: ```bash toxiproxy-cli toxic add shopify_test_redis_master -t latency -a latency=1000 ``` Please consult your respective client library on usage. ### Toxics Toxics manipulate the pipe between the client and upstream. They can be added and removed from proxies using the [HTTP api](#http-api). Each toxic has its own parameters to change how it affects the proxy links. For documentation on implementing custom toxics, see [CREATING_TOXICS.md](https://github.com/Shopify/toxiproxy/blob/master/CREATING_TOXICS.md) #### latency Add a delay to all data going through the proxy. The delay is equal to `latency` +/- `jitter`. Attributes: - `latency`: time in milliseconds - `jitter`: time in milliseconds #### down Bringing a service down is not technically a toxic in the implementation of Toxiproxy. This is done by `POST`ing to `/proxies/{proxy}` and setting the `enabled` field to `false`. #### bandwidth Limit a connection to a maximum number of kilobytes per second. Attributes: - `rate`: rate in KB/s #### slow_close Delay the TCP socket from closing until `delay` has elapsed. Attributes: - `delay`: time in milliseconds #### timeout Stops all data from getting through, and closes the connection after `timeout`. If `timeout` is 0, the connection won't close, and data will be delayed until the toxic is removed. Attributes: - `timeout`: time in milliseconds #### slicer Slices TCP data up into small bits, optionally adding a delay between each sliced "packet". Attributes: - `average_size`: size in bytes of an average packet - `size_variation`: variation in bytes of an average packet (should be smaller than average_size) - `delay`: time in microseconds to delay each packet by ### HTTP API All communication with the Toxiproxy daemon from the client happens through the HTTP interface, which is described here. Toxiproxy listens for HTTP on port **8474**. #### Proxy fields: - `name`: proxy name (string) - `listen`: listen address (string) - `upstream`: proxy upstream address (string) - `enabled`: true/false (defaults to true on creation) To change a proxy's name, it must be deleted and recreated. Changing the `listen` or `upstream` fields will restart the proxy and drop any active connections. If `listen` is specified with a port of 0, toxiproxy will pick an ephemeral port. The `listen` field in the response will be updated with the actual port. If you change `enabled` to `false`, it will take down the proxy. You can switch it back to `true` to reenable it. #### Toxic fields: - `name`: toxic name (string, defaults to `_`) - `type`: toxic type (string) - `stream`: link direction to affect (defaults to `downstream`) - `toxicity`: probability of the toxic being applied to a link (defaults to 1.0, 100%) - `attributes`: a map of toxic-specific attributes See [Toxics](#toxics) for toxic-specific attributes. The `stream` direction must be either `upstream` or `downstream`. `upstream` applies the toxic on the `client -> server` connection, while `downstream` applies the toxic on the `server -> client` connection. This can be used to modify requests and responses separately. #### Endpoints All endpoints are JSON. - **GET /proxies** - List existing proxies and their toxics - **POST /proxies** - Create a new proxy - **GET /proxies/{proxy}** - Show the proxy with all its active toxics - **POST /proxies/{proxy}** - Update a proxy's fields - **DELETE /proxies/{proxy}** - Delete an existing proxy - **GET /proxies/{proxy}/toxics** - List active toxics - **GET /proxies/{proxy}/toxics/{toxic}** - Get an active toxic's fields - **POST /proxies/{proxy}/toxics/{toxic}** - Update an active toxic - **DELETE /proxies/{proxy}/toxics/{toxic}** - Remove an active toxic - **POST /reset** - Enable all proxies and remove all active toxics - **GET /version** - Returns the server version number ### CLI Example ```bash $ toxiproxy-cli create redis -l localhost:26379 -u localhost:6379 Created new proxy redis $ toxiproxy-cli list Listen Upstream Name Enabled Toxics ====================================================================== 127.0.0.1:26379 localhost:6379 redis true None Hint: inspect toxics with `toxiproxy-client inspect ` ``` ```bash $ redis-cli -p 26379 127.0.0.1:26379> SET omg pandas OK 127.0.0.1:26379> GET omg "pandas" ``` ```bash $ toxiproxy-cli toxic add redis -t latency -a latency=1000 Added downstream latency toxic 'latency_downstream' on proxy 'redis' ``` ```bash $ redis-cli -p 26379 127.0.0.1:26379> GET omg "pandas" (1.00s) 127.0.0.1:26379> DEL omg (integer) 1 (1.00s) ``` ```bash $ toxiproxy-cli toxic remove redis -n latency_downstream Removed toxic 'latency_downstream' on proxy 'redis' ``` ```bash $ redis-cli -p 26379 127.0.0.1:26379> GET omg (nil) ``` ```bash $ toxiproxy-cli delete redis Deleted proxy redis ``` ```bash $ redis-cli -p 26379 Could not connect to Redis at 127.0.0.1:26379: Connection refused ``` ### Frequently Asked Questions **How fast is Toxiproxy?** The speed of Toxiproxy depends largely on your hardware, but you can expect a latency of *< 100µs* when no toxics are enabled. When running with `GOMAXPROCS=4` on a Macbook Pro we acheived *~1000MB/s* throughput, and as high as *2400MB/s* on a higher end desktop. Basically, you can expect Toxiproxy to move data around at least as fast the app you're testing. **Can Toxiproxy do randomized testing?** Many of the available toxics can be configured to have randomness, such as `jitter` in the `latency` toxic. There is also a global `toxicity` parameter that specifies the percentage of connections a toxic will affect. This is most useful for things like the `timeout` toxic, which would allow X% of connections to timeout. **I am not seeing my Toxiproxy actions reflected for MySQL**. MySQL will prefer the local Unix domain socket for some clients, no matter which port you pass it if the host is set to `localhost`. Configure your MySQL server to not create a socket, and use `127.0.0.1` as the host. Remember to remove the old socket after you restart the server. **Toxiproxy causes intermittent connection failures**. Use ports outside the ephemeral port range to avoid random port conflicts. It's `32,768` to `61,000` on Linux by default, see `/proc/sys/net/ipv4/ip_local_port_range`. **Should I run a Toxiproxy for each application?** No, we recommend using the same Toxiproxy for all applications. To distinguish between services we recommend naming your proxies with the scheme: `___`. For example, `shopify_test_redis_master` or `shopify_development_mysql_1`. ### Development * `make`. Build a toxiproxy development binary for the current platform. * `make all`. Build Toxiproxy binaries and packages for all platforms. Requires to have Go compiled with cross compilation enabled on Linux and Darwin (amd64) as well as [`fpm`](https://github.com/jordansissel/fpm) in your `$PATH` to build the Debian package. * `make test`. Run the Toxiproxy tests. * `make darwin`. Build binary for Darwin. * `make linux`. Build binary for Linux. * `make windows`. Build binary for Windows. ### Release 1. Update `CHANGELOG.md` 2. Bump `VERSION` 3. Change versions in `README.md` 4. Commit 5. Tag 6. `make release` to create binaries, packages and push new Docker image 7. Create [Github draft release](https://github.com/Shopify/toxiproxy/releases/new) against new tag and upload binaries and Debian package 8. [Bump version for Homebrew](https://github.com/Shopify/homebrew-shopify/blob/master/toxiproxy.rb#L9) [blog]: https://engineering.shopify.com/17489072-building-and-testing-resilient-ruby-on-rails-applications toxiproxy-2.0.0/VERSION000066400000000000000000000000061270742716200146520ustar00rootroot000000000000002.0.0 toxiproxy-2.0.0/api.go000066400000000000000000000235311270742716200147120ustar00rootroot00000000000000package toxiproxy import ( "encoding/json" "fmt" "log" "net" "net/http" "github.com/Shopify/toxiproxy/toxics" "github.com/Sirupsen/logrus" "github.com/gorilla/mux" ) type ApiServer struct { Collection *ProxyCollection } func NewServer() *ApiServer { return &ApiServer{ Collection: NewProxyCollection(), } } func (server *ApiServer) Listen(host string, port string) { r := mux.NewRouter() r.HandleFunc("/reset", server.ResetState).Methods("POST") r.HandleFunc("/proxies", server.ProxyIndex).Methods("GET") r.HandleFunc("/proxies", server.ProxyCreate).Methods("POST") r.HandleFunc("/proxies/{proxy}", server.ProxyShow).Methods("GET") r.HandleFunc("/proxies/{proxy}", server.ProxyUpdate).Methods("POST") r.HandleFunc("/proxies/{proxy}", server.ProxyDelete).Methods("DELETE") r.HandleFunc("/proxies/{proxy}/toxics", server.ToxicIndex).Methods("GET") r.HandleFunc("/proxies/{proxy}/toxics", server.ToxicCreate).Methods("POST") r.HandleFunc("/proxies/{proxy}/toxics/{toxic}", server.ToxicShow).Methods("GET") r.HandleFunc("/proxies/{proxy}/toxics/{toxic}", server.ToxicUpdate).Methods("POST") r.HandleFunc("/proxies/{proxy}/toxics/{toxic}", server.ToxicDelete).Methods("DELETE") r.HandleFunc("/version", server.Version).Methods("GET") http.Handle("/", r) logrus.WithFields(logrus.Fields{ "host": host, "port": port, "version": Version, }).Info("API HTTP server starting") err := http.ListenAndServe(net.JoinHostPort(host, port), nil) if err != nil { log.Fatal("ListenAndServe: ", err) } } func (server *ApiServer) ProxyIndex(response http.ResponseWriter, request *http.Request) { proxies := server.Collection.Proxies() marshalData := make(map[string]interface{}, len(proxies)) for name, proxy := range proxies { marshalData[name] = proxyWithToxics(proxy) } data, err := json.Marshal(marshalData) if apiError(response, err) { return } response.Header().Set("Content-Type", "application/json") _, err = response.Write(data) if err != nil { logrus.Warn("ProxyIndex: Failed to write response to client", err) } } func (server *ApiServer) ResetState(response http.ResponseWriter, request *http.Request) { proxies := server.Collection.Proxies() for _, proxy := range proxies { err := proxy.Start() if err != ErrProxyAlreadyStarted && apiError(response, err) { return } proxy.Toxics.ResetToxics() } response.WriteHeader(http.StatusNoContent) _, err := response.Write(nil) if err != nil { logrus.Warn("ResetState: Failed to write headers to client", err) } } func (server *ApiServer) ProxyCreate(response http.ResponseWriter, request *http.Request) { // Default fields to enable the proxy right away input := Proxy{Enabled: true} err := json.NewDecoder(request.Body).Decode(&input) if apiError(response, joinError(err, ErrBadRequestBody)) { return } if len(input.Name) < 1 { apiError(response, joinError(fmt.Errorf("name"), ErrMissingField)) return } if len(input.Upstream) < 1 { apiError(response, joinError(fmt.Errorf("upstream"), ErrMissingField)) return } proxy := NewProxy() proxy.Name = input.Name proxy.Listen = input.Listen proxy.Upstream = input.Upstream err = server.Collection.Add(proxy, input.Enabled) if apiError(response, err) { return } data, err := json.Marshal(proxyWithToxics(proxy)) if apiError(response, err) { return } response.Header().Set("Content-Type", "application/json") response.WriteHeader(http.StatusCreated) _, err = response.Write(data) if err != nil { logrus.Warn("ProxyCreate: Failed to write response to client", err) } } func (server *ApiServer) ProxyShow(response http.ResponseWriter, request *http.Request) { vars := mux.Vars(request) proxy, err := server.Collection.Get(vars["proxy"]) if apiError(response, err) { return } data, err := json.Marshal(proxyWithToxics(proxy)) if apiError(response, err) { return } response.Header().Set("Content-Type", "application/json") _, err = response.Write(data) if err != nil { logrus.Warn("ProxyShow: Failed to write response to client", err) } } func (server *ApiServer) ProxyUpdate(response http.ResponseWriter, request *http.Request) { vars := mux.Vars(request) proxy, err := server.Collection.Get(vars["proxy"]) if apiError(response, err) { return } // Default fields are the same as existing proxy input := Proxy{Listen: proxy.Listen, Upstream: proxy.Upstream, Enabled: proxy.Enabled} err = json.NewDecoder(request.Body).Decode(&input) if apiError(response, joinError(err, ErrBadRequestBody)) { return } err = proxy.Update(&input) if apiError(response, err) { return } data, err := json.Marshal(proxyWithToxics(proxy)) if apiError(response, err) { return } response.Header().Set("Content-Type", "application/json") _, err = response.Write(data) if err != nil { logrus.Warn("ProxyUpdate: Failed to write response to client", err) } } func (server *ApiServer) ProxyDelete(response http.ResponseWriter, request *http.Request) { vars := mux.Vars(request) err := server.Collection.Remove(vars["proxy"]) if apiError(response, err) { return } response.WriteHeader(http.StatusNoContent) _, err = response.Write(nil) if err != nil { logrus.Warn("ProxyDelete: Failed to write headers to client", err) } } func (server *ApiServer) ToxicIndex(response http.ResponseWriter, request *http.Request) { vars := mux.Vars(request) proxy, err := server.Collection.Get(vars["proxy"]) if apiError(response, err) { return } toxics := proxy.Toxics.GetToxicArray() data, err := json.Marshal(toxics) if apiError(response, err) { return } response.Header().Set("Content-Type", "application/json") _, err = response.Write(data) if err != nil { logrus.Warn("ToxicIndex: Failed to write response to client", err) } } func (server *ApiServer) ToxicCreate(response http.ResponseWriter, request *http.Request) { vars := mux.Vars(request) proxy, err := server.Collection.Get(vars["proxy"]) if apiError(response, err) { return } toxic, err := proxy.Toxics.AddToxicJson(request.Body) if apiError(response, err) { return } data, err := json.Marshal(toxic) if apiError(response, err) { return } response.Header().Set("Content-Type", "application/json") _, err = response.Write(data) if err != nil { logrus.Warn("ToxicCreate: Failed to write response to client", err) } } func (server *ApiServer) ToxicShow(response http.ResponseWriter, request *http.Request) { vars := mux.Vars(request) proxy, err := server.Collection.Get(vars["proxy"]) if apiError(response, err) { return } toxic := proxy.Toxics.GetToxic(vars["toxic"]) if toxic == nil { apiError(response, ErrToxicNotFound) return } data, err := json.Marshal(toxic) if apiError(response, err) { return } response.Header().Set("Content-Type", "application/json") _, err = response.Write(data) if err != nil { logrus.Warn("ToxicShow: Failed to write response to client", err) } } func (server *ApiServer) ToxicUpdate(response http.ResponseWriter, request *http.Request) { vars := mux.Vars(request) proxy, err := server.Collection.Get(vars["proxy"]) if apiError(response, err) { return } toxic, err := proxy.Toxics.UpdateToxicJson(vars["toxic"], request.Body) if apiError(response, err) { return } data, err := json.Marshal(toxic) if apiError(response, err) { return } response.Header().Set("Content-Type", "application/json") _, err = response.Write(data) if err != nil { logrus.Warn("ToxicUpdate: Failed to write response to client", err) } } func (server *ApiServer) ToxicDelete(response http.ResponseWriter, request *http.Request) { vars := mux.Vars(request) proxy, err := server.Collection.Get(vars["proxy"]) if apiError(response, err) { return } err = proxy.Toxics.RemoveToxic(vars["toxic"]) if apiError(response, err) { return } response.WriteHeader(http.StatusNoContent) _, err = response.Write(nil) if err != nil { logrus.Warn("ToxicDelete: Failed to write headers to client", err) } } func (server *ApiServer) Version(response http.ResponseWriter, request *http.Request) { response.Header().Set("Content-Type", "text/plain") _, err := response.Write([]byte(Version)) if err != nil { logrus.Warn("Version: Failed to write response to client", err) } } type ApiError struct { Message string `json:"title"` StatusCode int `json:"status"` } func (e *ApiError) Error() string { return e.Message } func newError(msg string, status int) *ApiError { return &ApiError{msg, status} } func joinError(err error, wrapper *ApiError) *ApiError { if err != nil { return &ApiError{wrapper.Message + ": " + err.Error(), wrapper.StatusCode} } return nil } var ( ErrBadRequestBody = newError("bad request body", http.StatusBadRequest) ErrMissingField = newError("missing required field", http.StatusBadRequest) ErrProxyNotFound = newError("proxy not found", http.StatusNotFound) ErrProxyAlreadyExists = newError("proxy already exists", http.StatusConflict) ErrInvalidStream = newError("stream was invalid, can be either upstream or downstream", http.StatusBadRequest) ErrInvalidToxicType = newError("invalid toxic type", http.StatusBadRequest) ErrToxicAlreadyExists = newError("toxic already exists", http.StatusConflict) ErrToxicNotFound = newError("toxic not found", http.StatusNotFound) ) func apiError(resp http.ResponseWriter, err error) bool { obj, ok := err.(*ApiError) if !ok && err != nil { logrus.Warn("Error did not include status code:", err) obj = &ApiError{err.Error(), http.StatusInternalServerError} } if obj == nil { return false } data, err2 := json.Marshal(obj) if err2 != nil { logrus.Warn("Error json encoding error (╯°□°）╯︵ ┻━┻", err2) } resp.Header().Set("Content-Type", "application/json") http.Error(resp, string(data), obj.StatusCode) return true } func proxyWithToxics(proxy *Proxy) (result struct { *Proxy Toxics []toxics.Toxic `json:"toxics"` }) { result.Proxy = proxy result.Toxics = proxy.Toxics.GetToxicArray() return } toxiproxy-2.0.0/api_test.go000066400000000000000000000514671270742716200157620ustar00rootroot00000000000000package toxiproxy_test import ( "io/ioutil" "net/http" "testing" "time" "github.com/Shopify/toxiproxy" tclient "github.com/Shopify/toxiproxy/client" ) var testServer *toxiproxy.ApiServer var client = tclient.NewClient("http://127.0.0.1:8475") func WithServer(t *testing.T, f func(string)) { // Make sure only one server is running at a time. Apparently there's no clean // way to shut it down between each test run. if testServer == nil { testServer = toxiproxy.NewServer() go testServer.Listen("localhost", "8475") // Allow server to start. There's no clean way to know when it listens. time.Sleep(50 * time.Millisecond) } defer func() { err := testServer.Collection.Clear() if err != nil { t.Error("Failed to clear collection", err) } }() f("http://localhost:8475") } func TestIndexWithNoProxies(t *testing.T) { WithServer(t, func(addr string) { client := tclient.NewClient(addr) proxies, err := client.Proxies() if err != nil { t.Fatal("Failed getting proxies:", err) } if len(proxies) > 0 { t.Fatal("Expected no proxies, got:", proxies) } }) } func TestCreateProxyBlankName(t *testing.T) { WithServer(t, func(addr string) { _, err := client.CreateProxy("", "", "") if err == nil { t.Fatal("Expected error creating proxy, got nil") } else if err.Error() != "Create: HTTP 400: missing required field: name" { t.Fatal("Expected different error creating proxy:", err) } }) } func TestCreateProxyBlankUpstream(t *testing.T) { WithServer(t, func(addr string) { _, err := client.CreateProxy("test", "", "") if err == nil { t.Fatal("Expected error creating proxy, got nil") } else if err.Error() != "Create: HTTP 400: missing required field: upstream" { t.Fatal("Expected different error creating proxy:", err) } }) } func TestListingProxies(t *testing.T) { WithServer(t, func(addr string) { _, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } proxies, err := client.Proxies() if err != nil { t.Fatal("Error listing proxies:", err) } if len(proxies) == 0 { t.Fatal("Expected new proxy in list") } proxy, ok := proxies["mysql_master"] if !ok { t.Fatal("Expected to see mysql_master proxy in list") } if proxy.Name != "mysql_master" || proxy.Listen != "127.0.0.1:3310" || proxy.Upstream != "localhost:20001" { t.Fatalf("Unexpected proxy metadata: %s, %s, %s", proxy.Name, proxy.Listen, proxy.Upstream) } AssertToxicExists(t, proxy.ActiveToxics, "latency", "", "", false) }) } func TestCreateAndGetProxy(t *testing.T) { WithServer(t, func(addr string) { _, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } proxy, err := client.Proxy("mysql_master") if err != nil { t.Fatal("Unable to retriecve proxy:", err) } if proxy.Name != "mysql_master" || proxy.Listen != "127.0.0.1:3310" || proxy.Upstream != "localhost:20001" || !proxy.Enabled { t.Fatalf("Unexpected proxy metadata: %s, %s, %s, %v", proxy.Name, proxy.Listen, proxy.Upstream, proxy.Enabled) } AssertToxicExists(t, proxy.ActiveToxics, "latency", "", "", false) }) } func TestCreateProxyWithSave(t *testing.T) { WithServer(t, func(addr string) { testProxy := client.NewProxy() testProxy.Name = "mysql_master" testProxy.Listen = "localhost:3310" testProxy.Upstream = "localhost:20001" testProxy.Enabled = true err := testProxy.Save() if err != nil { t.Fatal("Unable to create proxy:", err) } proxy, err := client.Proxy("mysql_master") if err != nil { t.Fatal("Unable to retriecve proxy:", err) } if proxy.Name != "mysql_master" || proxy.Listen != "127.0.0.1:3310" || proxy.Upstream != "localhost:20001" || !proxy.Enabled { t.Fatalf("Unexpected proxy metadata: %s, %s, %s, %v", proxy.Name, proxy.Listen, proxy.Upstream, proxy.Enabled) } AssertProxyUp(t, proxy.Listen, true) }) } func TestCreateDisabledProxy(t *testing.T) { WithServer(t, func(addr string) { disabledProxy := client.NewProxy() disabledProxy.Name = "mysql_master" disabledProxy.Listen = "localhost:3310" disabledProxy.Upstream = "localhost:20001" err := disabledProxy.Save() if err != nil { t.Fatal("Unable to create proxy:", err) } proxy, err := client.Proxy("mysql_master") if err != nil { t.Fatal("Unable to retriecve proxy:", err) } if proxy.Name != "mysql_master" || proxy.Listen != "localhost:3310" || proxy.Upstream != "localhost:20001" || proxy.Enabled { t.Fatalf("Unexpected proxy metadata: %s, %s, %s, %v", proxy.Name, proxy.Listen, proxy.Upstream, proxy.Enabled) } AssertProxyUp(t, proxy.Listen, false) }) } func TestCreateDisabledProxyAndEnable(t *testing.T) { WithServer(t, func(addr string) { disabledProxy := client.NewProxy() disabledProxy.Name = "mysql_master" disabledProxy.Listen = "localhost:3310" disabledProxy.Upstream = "localhost:20001" err := disabledProxy.Save() if err != nil { t.Fatal("Unable to create proxy:", err) } proxy, err := client.Proxy("mysql_master") if err != nil { t.Fatal("Unable to retriecve proxy:", err) } if proxy.Name != "mysql_master" || proxy.Listen != "localhost:3310" || proxy.Upstream != "localhost:20001" || proxy.Enabled { t.Fatalf("Unexpected proxy metadata: %s, %s, %s, %v", proxy.Name, proxy.Listen, proxy.Upstream, proxy.Enabled) } proxy.Enabled = true err = proxy.Save() if err != nil { t.Fatal("Failed to update proxy:", err) } AssertProxyUp(t, proxy.Listen, true) proxy.Enabled = false err = proxy.Save() if err != nil { t.Fatal("Failed to update proxy:", err) } AssertProxyUp(t, proxy.Listen, false) }) } func TestDeleteProxy(t *testing.T) { WithServer(t, func(addr string) { testProxy, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } proxies, err := client.Proxies() if err != nil { t.Fatal("Error listing proxies:", err) } if len(proxies) == 0 { t.Fatal("Expected new proxy in list") } AssertProxyUp(t, testProxy.Listen, true) err = testProxy.Delete() if err != nil { t.Fatal("Failed deleting proxy:", err) } AssertProxyUp(t, testProxy.Listen, false) proxies, err = client.Proxies() if err != nil { t.Fatal("Error listing proxies:", err) } if len(proxies) > 0 { t.Fatal("Expected proxy to be deleted from list") } err = testProxy.Delete() if err == nil { t.Fatal("Proxy did not result in not found.") } else if err.Error() != "Delete: HTTP 404: proxy not found" { t.Fatal("Incorrect error removing proxy:", err) } }) } func TestCreateProxyPortConflict(t *testing.T) { WithServer(t, func(addr string) { testProxy, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } _, err = client.CreateProxy("test", "localhost:3310", "localhost:20001") if err == nil { t.Fatal("Proxy did not result in conflict.") } else if err.Error() != "Create: HTTP 500: listen tcp 127.0.0.1:3310: bind: address already in use" { t.Fatal("Incorrect error adding proxy:", err) } err = testProxy.Delete() if err != nil { t.Fatal("Unable to delete proxy:", err) } _, err = client.CreateProxy("test", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } }) } func TestCreateProxyNameConflict(t *testing.T) { WithServer(t, func(addr string) { testProxy, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } _, err = client.CreateProxy("mysql_master", "localhost:3311", "localhost:20001") if err == nil { t.Fatal("Proxy did not result in conflict.") } else if err.Error() != "Create: HTTP 409: proxy already exists" { t.Fatal("Incorrect error adding proxy:", err) } err = testProxy.Delete() if err != nil { t.Fatal("Unable to delete proxy:", err) } _, err = client.CreateProxy("mysql_master", "localhost:3311", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } }) } func TestResetState(t *testing.T) { WithServer(t, func(addr string) { testProxy, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } latency, err := testProxy.AddToxic("", "latency", "downstream", 1, tclient.Attributes{ "latency": 100, "jitter": 10, }) if err != nil { t.Fatal("Error setting toxic:", err) } if latency.Attributes["latency"] != 100.0 || latency.Attributes["jitter"] != 10.0 { t.Fatal("Latency toxic did not start up with correct settings") } err = client.ResetState() if err != nil { t.Fatal("unable to reset state:", err) } proxies, err := client.Proxies() if err != nil { t.Fatal("Error listing proxies:", err) } proxy, ok := proxies["mysql_master"] if !ok { t.Fatal("Expected proxy to still exist") } if !proxy.Enabled { t.Fatal("Expected proxy to be enabled") } toxics, err := proxy.Toxics() if err != nil { t.Fatal("Error requesting toxics:", err) } AssertToxicExists(t, toxics, "latency", "", "", false) AssertProxyUp(t, proxy.Listen, true) }) } func TestListingToxics(t *testing.T) { WithServer(t, func(addr string) { testProxy, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } toxics, err := testProxy.Toxics() if err != nil { t.Fatal("Error returning toxics:", err) } AssertToxicExists(t, toxics, "latency", "", "", false) }) } func TestAddToxic(t *testing.T) { WithServer(t, func(addr string) { testProxy, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } latency, err := testProxy.AddToxic("foobar", "latency", "downstream", 1, tclient.Attributes{ "latency": 100, "jitter": 10, }) if err != nil { t.Fatal("Error setting toxic:", err) } if latency.Attributes["latency"] != 100.0 || latency.Attributes["jitter"] != 10.0 { t.Fatal("Latency toxic did not start up with correct settings") } toxics, err := testProxy.Toxics() if err != nil { t.Fatal("Error returning toxics:", err) } toxic := AssertToxicExists(t, toxics, "foobar", "latency", "downstream", true) if toxic.Toxicity != 1.0 || toxic.Attributes["latency"] != 100.0 || toxic.Attributes["jitter"] != 10.0 { t.Fatal("Toxic was not read back correctly:", toxic) } }) } func TestAddMultipleToxics(t *testing.T) { WithServer(t, func(addr string) { testProxy, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } _, err = testProxy.AddToxic("latency1", "latency", "downstream", 1, nil) if err != nil { t.Fatal("Error setting toxic:", err) } _, err = testProxy.AddToxic("latency2", "latency", "downstream", 1, nil) if err != nil { t.Fatal("Error setting toxic:", err) } toxics, err := testProxy.Toxics() if err != nil { t.Fatal("Error returning toxics:", err) } AssertToxicExists(t, toxics, "latency1", "latency", "downstream", true) toxic := AssertToxicExists(t, toxics, "latency2", "latency", "downstream", true) if toxic.Toxicity != 1.0 || toxic.Attributes["latency"] != 0.0 || toxic.Attributes["jitter"] != 0.0 { t.Fatal("Toxic was not read back correctly:", toxic) } AssertToxicExists(t, toxics, "latency1", "", "upstream", false) AssertToxicExists(t, toxics, "latency2", "", "upstream", false) }) } func TestAddConflictingToxic(t *testing.T) { WithServer(t, func(addr string) { testProxy, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } _, err = testProxy.AddToxic("foobar", "latency", "downstream", 1, nil) if err != nil { t.Fatal("Error setting toxic:", err) } _, err = testProxy.AddToxic("foobar", "slow_close", "downstream", 1, nil) if err == nil { t.Fatal("Toxic did not result in conflict.") } else if err.Error() != "AddToxic: HTTP 409: toxic already exists" { t.Fatal("Incorrect error setting toxic:", err) } toxics, err := testProxy.Toxics() if err != nil { t.Fatal("Error returning toxics:", err) } toxic := AssertToxicExists(t, toxics, "foobar", "latency", "downstream", true) if toxic.Toxicity != 1.0 || toxic.Attributes["latency"] != 0.0 || toxic.Attributes["jitter"] != 0.0 { t.Fatal("Toxic was not read back correctly:", toxic) } AssertToxicExists(t, toxics, "foobar", "", "upstream", false) }) } func TestAddConflictingToxicsMultistream(t *testing.T) { WithServer(t, func(addr string) { testProxy, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } _, err = testProxy.AddToxic("foobar", "latency", "upstream", 1, nil) if err != nil { t.Fatal("Error setting toxic:", err) } _, err = testProxy.AddToxic("foobar", "latency", "downstream", 1, nil) if err == nil { t.Fatal("Toxic did not result in conflict.") } else if err.Error() != "AddToxic: HTTP 409: toxic already exists" { t.Fatal("Incorrect error setting toxic:", err) } toxics, err := testProxy.Toxics() if err != nil { t.Fatal("Error returning toxics:", err) } toxic := AssertToxicExists(t, toxics, "foobar", "latency", "upstream", true) if toxic.Toxicity != 1.0 || toxic.Attributes["latency"] != 0.0 || toxic.Attributes["jitter"] != 0.0 { t.Fatal("Toxic was not read back correctly:", toxic) } AssertToxicExists(t, toxics, "foobar", "", "downstream", false) }) } func TestAddConflictingToxicsMultistreamDefaults(t *testing.T) { WithServer(t, func(addr string) { testProxy, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } _, err = testProxy.AddToxic("", "latency", "upstream", 1, nil) if err != nil { t.Fatal("Error setting toxic:", err) } _, err = testProxy.AddToxic("", "latency", "downstream", 1, nil) if err != nil { t.Fatal("Error setting toxic:", err) } toxics, err := testProxy.Toxics() if err != nil { t.Fatal("Error returning toxics:", err) } toxic := AssertToxicExists(t, toxics, "latency_upstream", "latency", "upstream", true) if toxic.Toxicity != 1.0 || toxic.Attributes["latency"] != 0.0 || toxic.Attributes["jitter"] != 0.0 { t.Fatal("Toxic was not read back correctly:", toxic) } toxic = AssertToxicExists(t, toxics, "latency_downstream", "latency", "downstream", true) if toxic.Toxicity != 1.0 || toxic.Attributes["latency"] != 0.0 || toxic.Attributes["jitter"] != 0.0 { t.Fatal("Toxic was not read back correctly:", toxic) } }) } func TestAddToxicWithToxicity(t *testing.T) { WithServer(t, func(addr string) { testProxy, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } latency, err := testProxy.AddToxic("", "latency", "downstream", 0.2, tclient.Attributes{ "latency": 100, "jitter": 10, }) if err != nil { t.Fatal("Error setting toxic:", err) } if latency.Toxicity != 0.2 || latency.Attributes["latency"] != 100.0 || latency.Attributes["jitter"] != 10.0 { t.Fatal("Latency toxic did not start up with correct settings:", latency) } toxics, err := testProxy.Toxics() if err != nil { t.Fatal("Error returning toxics:", err) } toxic := AssertToxicExists(t, toxics, "latency_downstream", "latency", "downstream", true) if toxic.Toxicity != 0.2 || toxic.Attributes["latency"] != 100.0 || toxic.Attributes["jitter"] != 10.0 { t.Fatal("Toxic was not read back correctly:", toxic) } }) } func TestAddNoop(t *testing.T) { WithServer(t, func(addr string) { testProxy, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } noop, err := testProxy.AddToxic("foobar", "noop", "", 1, nil) if err != nil { t.Fatal("Error setting toxic:", err) } if noop.Toxicity != 1.0 || noop.Name != "foobar" || noop.Type != "noop" || noop.Stream != "downstream" { t.Fatal("Noop toxic did not start up with correct settings:", noop) } toxics, err := testProxy.Toxics() if err != nil { t.Fatal("Error returning toxics:", err) } toxic := AssertToxicExists(t, toxics, "foobar", "noop", "downstream", true) if toxic.Toxicity != 1.0 { t.Fatal("Toxic was not read back correctly:", toxic) } }) } func TestUpdateToxics(t *testing.T) { WithServer(t, func(addr string) { testProxy, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } latency, err := testProxy.AddToxic("", "latency", "downstream", -1, tclient.Attributes{ "latency": 100, "jitter": 10, }) if err != nil { t.Fatal("Error setting toxic:", err) } if latency.Toxicity != 1.0 || latency.Attributes["latency"] != 100.0 || latency.Attributes["jitter"] != 10.0 { t.Fatal("Latency toxic did not start up with correct settings:", latency) } latency, err = testProxy.UpdateToxic("latency_downstream", 0.5, tclient.Attributes{ "latency": 1000, }) if err != nil { t.Fatal("Error setting toxic:", err) } if latency.Toxicity != 0.5 || latency.Attributes["latency"] != 1000.0 || latency.Attributes["jitter"] != 10.0 { t.Fatal("Latency toxic did not get updated with the correct settings:", latency) } latency, err = testProxy.UpdateToxic("latency_downstream", -1, tclient.Attributes{ "latency": 500, }) if err != nil { t.Fatal("Error setting toxic:", err) } if latency.Toxicity != 0.5 || latency.Attributes["latency"] != 500.0 || latency.Attributes["jitter"] != 10.0 { t.Fatal("Latency toxic did not get updated with the correct settings:", latency) } toxics, err := testProxy.Toxics() if err != nil { t.Fatal("Error returning toxics:", err) } toxic := AssertToxicExists(t, toxics, "latency_downstream", "latency", "downstream", true) if toxic.Toxicity != 0.5 || toxic.Attributes["latency"] != 500.0 || toxic.Attributes["jitter"] != 10.0 { t.Fatal("Toxic was not read back correctly:", toxic) } }) } func TestRemoveToxic(t *testing.T) { WithServer(t, func(addr string) { testProxy, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } _, err = testProxy.AddToxic("", "latency", "downstream", 1, nil) if err != nil { t.Fatal("Error setting toxic:", err) } toxics, err := testProxy.Toxics() if err != nil { t.Fatal("Error returning toxics:", err) } toxic := AssertToxicExists(t, toxics, "latency_downstream", "latency", "downstream", true) if toxic.Toxicity != 1.0 || toxic.Attributes["latency"] != 0.0 || toxic.Attributes["jitter"] != 0.0 { t.Fatal("Toxic was not read back correctly:", toxic) } err = testProxy.RemoveToxic("latency_downstream") if err != nil { t.Fatal("Error removing toxic:", err) } toxics, err = testProxy.Toxics() if err != nil { t.Fatal("Error returning toxics:", err) } AssertToxicExists(t, toxics, "latency_downstream", "", "", false) }) } func TestVersionEndpointReturnsVersion(t *testing.T) { WithServer(t, func(addr string) { resp, err := http.Get(addr + "/version") if err != nil { t.Fatal("Failed to get index", err) } body, err := ioutil.ReadAll(resp.Body) if err != nil { t.Fatal("Unable to read body from response") } if string(body) != toxiproxy.Version { t.Fatal("Expected to return Version from /version, got:", string(body)) } }) } func TestInvalidStream(t *testing.T) { WithServer(t, func(addr string) { testProxy, err := client.CreateProxy("mysql_master", "localhost:3310", "localhost:20001") if err != nil { t.Fatal("Unable to create proxy:", err) } _, err = testProxy.AddToxic("", "latency", "walrustream", 1, nil) if err == nil { t.Fatal("Error setting toxic:", err) } }) } func AssertToxicExists(t *testing.T, toxics tclient.Toxics, name, typeName, stream string, exists bool) *tclient.Toxic { var toxic *tclient.Toxic var actualType, actualStream string for _, tox := range toxics { if name == tox.Name { toxic = &tox actualType = tox.Type actualStream = tox.Stream } } if exists { if toxic == nil { t.Fatalf("Expected to see %s toxic in list", name) } else if actualType != typeName { t.Fatalf("Expected %s to be of type %s, found %s", name, typeName, actualType) } else if actualStream != stream { t.Fatalf("Expected %s to be in stream %s, found %s", name, stream, actualStream) } } else if toxic != nil && actualStream == stream { t.Fatalf("Expected %s toxic to be missing from list, found type %s", name, actualType) } return toxic } toxiproxy-2.0.0/circle.yml000066400000000000000000000002661270742716200155760ustar00rootroot00000000000000# Don't let Circle try to do anything clever with this Go project. We isolate # with godep to avoid `go get` going nuts. dependencies: override: test: override: - make test toxiproxy-2.0.0/cli/000077500000000000000000000000001270742716200143555ustar00rootroot00000000000000toxiproxy-2.0.0/cli/cli.go000066400000000000000000000327631270742716200154660ustar00rootroot00000000000000package main import ( "sort" "strconv" "strings" toxiproxyServer "github.com/Shopify/toxiproxy" "github.com/Shopify/toxiproxy/client" "github.com/codegangsta/cli" "fmt" "os" ) const ( redColor = "\x1b[31m" greenColor = "\x1b[32m" yellowColor = "\x1b[33m" blueColor = "\x1b[34m" cyanColor = "\x1b[36m" purpleColor = "\x1b[35m" grayColor = "\x1b[37m" noColor = "\x1b[0m" ) var toxicDescription = ` Default Toxics: latency: delay all data +/- jitter latency=,jitter= bandwidth: limit to max kb/s rate= slow_close: delay from closing delay= timeout: stop all data and close after timeout timeout= slicer: slice data into bits with optional delay average_size=,size_variation=,delay= toxic add: usage: toxiproxy-cli add --type --toxicName \ --attributes --upstream --downstream example: toxiproxy-cli toxic add myProxy -t latency -n myToxic -f latency=100,jitter=50 toxic update: usage: toxiproxy-cli update --toxicName \ --attributes example: toxiproxy-cli toxic update myProxy -n myToxic -f jitter=25 toxic delete: usage: toxiproxy-cli update --toxicName example: toxiproxy-cli toxic delete myProxy -n myToxic ` func main() { toxiproxyClient := toxiproxy.NewClient("http://localhost:8474") app := cli.NewApp() app.Name = "toxiproxy-cli" app.Version = toxiproxyServer.Version app.Usage = "Simulate network and system conditions" app.Commands = []cli.Command{ { Name: "list", Usage: "list all proxies\n\tusage: 'toxiproxy-cli list'\n", Aliases: []string{"l", "li", "ls"}, Action: withToxi(list, toxiproxyClient), }, { Name: "inspect", Aliases: []string{"i", "ins"}, Usage: "inspect a single proxy\n\tusage: 'toxiproxy-cli inspect '\n", Action: withToxi(inspect, toxiproxyClient), }, { Name: "create", Usage: "create a new proxy\n\tusage: 'toxiproxy-cli create --listen --upstream '\n", Aliases: []string{"c", "new"}, Flags: []cli.Flag{ cli.StringFlag{ Name: "listen, l", Usage: "proxy will listen on this address", }, cli.StringFlag{ Name: "upstream, u", Usage: "proxy will forward to this address", }, }, Action: withToxi(create, toxiproxyClient), }, { Name: "toggle", Usage: "\ttoggle enabled status on a proxy\n\t\tusage: 'toxiproxy-cli toggle '\n", Aliases: []string{"tog"}, Action: withToxi(toggle, toxiproxyClient), }, { Name: "delete", Usage: "\tdelete a proxy\n\t\tusage: 'toxiproxy-cli delete '\n", Aliases: []string{"d"}, Action: withToxi(delete, toxiproxyClient), }, { Name: "toxic", Aliases: []string{"t"}, Usage: "\tadd, remove or update a toxic\n\t\tusage: see 'toxiproxy-cli toxic'\n", Description: toxicDescription, Subcommands: []cli.Command{ { Name: "add", Aliases: []string{"a"}, Usage: "add a new toxic", ArgsUsage: "", Flags: []cli.Flag{ cli.StringFlag{ Name: "toxicName, n", Usage: "name of the toxic", }, cli.StringFlag{ Name: "type, t", Usage: "type of toxic", }, cli.StringFlag{ Name: "toxicity, tox", Usage: "toxicity of toxic", }, cli.StringFlag{ Name: "attributes, a", Usage: "comma seperated key=value toxic attributes", }, cli.BoolFlag{ Name: "upstream, u", Usage: "add toxic to upstream", }, cli.BoolFlag{ Name: "downstream, d", Usage: "add toxic to downstream", }, }, Action: withToxi(addToxic, toxiproxyClient), }, { Name: "update", Aliases: []string{"u"}, Usage: "update an enabled toxic", ArgsUsage: "", Flags: []cli.Flag{ cli.StringFlag{ Name: "toxicName, n", Usage: "name of the toxic", }, cli.StringFlag{ Name: "toxicity, tox", Usage: "toxicity of toxic", }, cli.StringFlag{ Name: "attributes, a", Usage: "comma seperated key=value toxic attributes", }, }, Action: withToxi(updateToxic, toxiproxyClient), }, { Name: "remove", Aliases: []string{"r", "delete", "d"}, Usage: "remove an enabled toxic", ArgsUsage: "", Flags: []cli.Flag{ cli.StringFlag{ Name: "toxicName, n", Usage: "name of the toxic", }, }, Action: withToxi(removeToxic, toxiproxyClient), }, }, }, } app.Run(os.Args) } type toxiAction func(*cli.Context, *toxiproxy.Client) func withToxi(f toxiAction, t *toxiproxy.Client) func(*cli.Context) { return func(c *cli.Context) { f(c, t) } } func list(c *cli.Context, t *toxiproxy.Client) { proxies, err := t.Proxies() if err != nil { fatalf("Failed to retrieve proxies: %s", err) } var proxyNames []string for proxyName := range proxies { proxyNames = append(proxyNames, proxyName) } sort.Strings(proxyNames) fmt.Fprintf(os.Stderr, "%sListen\t\t%sUpstream\t%sName\t%sEnabled\t%sToxics\n%s", blueColor, yellowColor, greenColor, purpleColor, redColor, noColor) fmt.Fprintf(os.Stderr, "%s======================================================================\n", noColor) if len(proxyNames) == 0 { fmt.Printf("%sno proxies\n\n%s", redColor, noColor) hint("create a proxy with `toxiproxy-cli create`") return } for _, proxyName := range proxyNames { proxy := proxies[proxyName] numToxics := strconv.Itoa(len(proxy.ActiveToxics)) if numToxics == "0" { numToxics = "None" } fmt.Printf("%s%s\t%s%s\t%s%s\t%s%v\t%s%s%s\n", blueColor, proxy.Listen, yellowColor, proxy.Upstream, enabledColor(proxy.Enabled), proxy.Name, purpleColor, proxy.Enabled, redColor, numToxics, noColor) } fmt.Println() hint("inspect toxics with `toxiproxy-cli inspect `") } func inspect(c *cli.Context, t *toxiproxy.Client) { proxyName := c.Args().First() if proxyName == "" { cli.ShowSubcommandHelp(c) fatalf("Proxy name is required as the first argument.\n") } proxy, err := t.Proxy(proxyName) if err != nil { fatalf("Failed to retrieve proxy %s: %s\n", proxyName, err.Error()) } fmt.Printf("%sName: %s%s\t", purpleColor, noColor, proxy.Name) fmt.Printf("%sListen: %s%s\t", blueColor, noColor, proxy.Listen) fmt.Printf("%sUpstream: %s%s\n", yellowColor, noColor, proxy.Upstream) fmt.Printf("%s======================================================================\n", noColor) splitToxics := func(toxics toxiproxy.Toxics) (toxiproxy.Toxics, toxiproxy.Toxics) { upstream := make(toxiproxy.Toxics, 0) downstream := make(toxiproxy.Toxics, 0) for _, toxic := range toxics { if toxic.Stream == "upstream" { upstream = append(upstream, toxic) } else { downstream = append(downstream, toxic) } } return upstream, downstream } if len(proxy.ActiveToxics) == 0 { fmt.Printf("%sProxy has no toxics enabled.\n%s", redColor, noColor) } else { up, down := splitToxics(proxy.ActiveToxics) listToxics(up, "Upstream") fmt.Println() listToxics(down, "Downstream") } fmt.Println() hint("add a toxic with `toxiproxy-cli toxic add`") } func toggle(c *cli.Context, t *toxiproxy.Client) { proxyName := c.Args().First() if proxyName == "" { cli.ShowSubcommandHelp(c) fatalf("Proxy name is required as the first argument.\n") } proxy, err := t.Proxy(proxyName) if err != nil { fatalf("Failed to retrieve proxy %s: %s\n", proxyName, err.Error()) } proxy.Enabled = !proxy.Enabled err = proxy.Save() if err != nil { fatalf("Failed to toggle proxy %s: %s\n", proxyName, err.Error()) } fmt.Printf("Proxy %s%s%s is now %s%s%s\n", enabledColor(proxy.Enabled), proxyName, noColor, enabledColor(proxy.Enabled), enabledText(proxy.Enabled), noColor) } func create(c *cli.Context, t *toxiproxy.Client) { proxyName := c.Args().First() if proxyName == "" { cli.ShowSubcommandHelp(c) fatalf("Proxy name is required as the first argument.\n") } listen := getArgOrFail(c, "listen") upstream := getArgOrFail(c, "upstream") _, err := t.CreateProxy(proxyName, listen, upstream) if err != nil { fatalf("Failed to create proxy: %s\n", err.Error()) } fmt.Printf("Created new proxy %s\n", proxyName) } func delete(c *cli.Context, t *toxiproxy.Client) { proxyName := c.Args().First() if proxyName == "" { cli.ShowSubcommandHelp(c) fatalf("Proxy name is required as the first argument.\n") } p, err := t.Proxy(proxyName) if err != nil { fatalf("Failed to retrieve proxy %s: %s\n", proxyName, err.Error()) } err = p.Delete() if err != nil { fatalf("Failed to delete proxy: %s\n", err.Error()) } fmt.Printf("Deleted proxy %s\n", proxyName) } func parseToxicity(c *cli.Context, defaultToxicity float32) float32 { var toxicity = defaultToxicity toxicityString := c.String("toxicity") if toxicityString != "" { tox, err := strconv.ParseFloat(toxicityString, 32) if err != nil || tox > 1 || tox < 0 { fatalf("toxicity should be a float between 0 and 1.\n") } toxicity = float32(tox) } return toxicity } func addToxic(c *cli.Context, t *toxiproxy.Client) { proxyName := c.Args().First() if proxyName == "" { cli.ShowSubcommandHelp(c) fatalf("Proxy name is required as the first argument.\n") } toxicName := c.String("toxicName") toxicType := getArgOrFail(c, "type") toxicAttributes := getArgOrFail(c, "attributes") upstream := c.Bool("upstream") downstream := c.Bool("downstream") toxicity := parseToxicity(c, 1.0) attributes := parseAttributes(toxicAttributes) p, err := t.Proxy(proxyName) if err != nil { fatalf("Failed to retrieve proxy %s: %s\n", proxyName, err.Error()) } addToxic := func(stream string) { t, err := p.AddToxic(toxicName, toxicType, stream, toxicity, attributes) if err != nil { fatalf("Failed to add toxic: %s\n", err.Error()) } toxicName = t.Name fmt.Printf("Added %s %s toxic '%s' on proxy '%s'\n", stream, toxicType, toxicName, proxyName) } if upstream { addToxic("upstream") } // Default to downstream. if downstream || (!downstream && !upstream) { addToxic("downstream") } } func updateToxic(c *cli.Context, t *toxiproxy.Client) { proxyName := c.Args().First() if proxyName == "" { cli.ShowSubcommandHelp(c) fatalf("Proxy name is required as the first argument.\n") } toxicName := getArgOrFail(c, "toxicName") toxicAttributes := getArgOrFail(c, "attributes") attributes := parseAttributes(toxicAttributes) p, err := t.Proxy(proxyName) if err != nil { fatalf("Failed to retrieve proxy %s: %s\n", proxyName, err.Error()) } toxicity := parseToxicity(c, -1) _, err = p.UpdateToxic(toxicName, toxicity, attributes) if err != nil { fatalf("Failed to update toxic: %s\n", err.Error()) } fmt.Printf("Updated toxic '%s' on proxy '%s'\n", toxicName, proxyName) } func removeToxic(c *cli.Context, t *toxiproxy.Client) { proxyName := c.Args().First() if proxyName == "" { cli.ShowSubcommandHelp(c) fatalf("Proxy name is required as the first argument.\n") } toxicName := getArgOrFail(c, "toxicName") p, err := t.Proxy(proxyName) if err != nil { fatalf("Failed to retrieve proxy %s: %s\n", proxyName, err.Error()) } err = p.RemoveToxic(toxicName) if err != nil { fatalf("Failed to remove toxic: %s\n", err.Error()) } fmt.Printf("Removed toxic '%s' on proxy '%s'\n", toxicName, proxyName) } func parseAttributes(raw string) toxiproxy.Attributes { parsed := map[string]interface{}{} keyValues := strings.Split(raw, ",") for _, keyValue := range keyValues { kv := strings.SplitN(keyValue, "=", 2) if len(kv) != 2 { fatalf("Attributes should be in format key1=value1,key2=value2\n") } value, err := strconv.Atoi(kv[1]) if err != nil { fatalf("Toxic attributes was expected to be an integer.\n") } parsed[kv[0]] = value } return parsed } func enabledColor(enabled bool) string { if enabled { return greenColor } return redColor } func enabledText(enabled bool) string { if enabled { return "enabled" } return "disabled" } type attribute struct { key string value interface{} } type attributeList []attribute func (a attributeList) Len() int { return len(a) } func (a attributeList) Less(i, j int) bool { return a[i].key < a[j].key } func (a attributeList) Swap(i, j int) { a[i], a[j] = a[j], a[i] } func sortedAttributes(attrs toxiproxy.Attributes) attributeList { li := make(attributeList, 0, len(attrs)) for k, v := range attrs { li = append(li, attribute{k, v.(float64)}) } sort.Sort(li) return li } func listToxics(toxics toxiproxy.Toxics, stream string) { fmt.Printf("%s%s toxics:\n%s", greenColor, stream, noColor) if len(toxics) == 0 { fmt.Printf("%sProxy has no %s toxics enabled.\n%s", redColor, stream, noColor) return } for _, t := range toxics { fmt.Printf("%s%s:%s ", redColor, t.Name, noColor) fmt.Printf("type=%s ", t.Type) fmt.Printf("stream=%s ", t.Stream) fmt.Printf("toxicity=%.2f ", t.Toxicity) fmt.Printf("attributes=[") sorted := sortedAttributes(t.Attributes) for _, a := range sorted { fmt.Printf(" %s=", a.key) fmt.Print(a.value) } fmt.Printf(" ]\n") } } func getArgOrFail(c *cli.Context, name string) string { arg := c.String(name) if arg == "" { cli.ShowSubcommandHelp(c) fatalf("Required argument '%s' was empty.\n", name) } return arg } func hint(m string) { fmt.Printf("%sHint: %s\n", noColor, m) } func fatalf(m string, args ...interface{}) { fmt.Printf(m, args...) os.Exit(1) } toxiproxy-2.0.0/client/000077500000000000000000000000001270742716200150645ustar00rootroot00000000000000toxiproxy-2.0.0/client/README.md000066400000000000000000000105261270742716200163470ustar00rootroot00000000000000# toxiproxy-go This is the Go client library for the [Toxiproxy](https://github.com/shopify/toxiproxy) API. Please read the [usage section in the Toxiproxy README](https://github.com/shopify/toxiproxy#usage) before attempting to use the client. This client is compatible with Toxiproxy 2.x, for the latest 1.x client see [v1.2.1](https://github.com/Shopify/toxiproxy/tree/v1.2.1/client). ## Changes in Toxiproxy-go Client 2.x In order to make use of the 2.0 api, and to make usage a little easier, the client api has changed: - `client.NewProxy()` no longer accepts a proxy as an argument. - `proxy.Create()` is removed in favour of using `proxy.Save()`. - Proxies can be created in a single call using `client.CreateProxy()`. - `proxy.Disable()` and `proxy.Enable()` have been added to simplify taking down a proxy. - `proxy.ToxicsUpstream` and `proxy.ToxicsDownstream` have been merged into a single `ActiveToxics` list. - `proxy.Toxics()`` no longer requires a direction to be specified, and will return toxics for both directions. - `proxy.SetToxic()` has been replaced by `proxy.AddToxic()`, `proxy.UpdateToxic()`, and `proxy.RemoveToxic()`. ## Usage For detailed API docs please [see the Godoc documentation](http://godoc.org/github.com/Shopify/toxiproxy/client). First import toxiproxy and create a new client: ```go import "github.com/Shopify/toxiproxy/client" client := toxiproxy.NewClient("localhost:8474") ``` You can then create a new proxy using the client: ```go proxy := client.CreateProxy("redis", "localhost:26379", "localhost:6379") ``` For large amounts of proxies, they can also be created using a configuration file: ```go var config []toxiproxy.Proxy data, _ := ioutil.ReadFile("config.json") json.Unmarshal(data, &config) proxies, err = client.Populate(config) ``` ```json [{ "name": "redis", "listen": "localhost:26379", "upstream": "localhost:6379" }] ``` Toxics can be added as follows: ```go // Add 1s latency to 100% of downstream connections proxy.AddToxic("latency_down", "latency", "downstream", 1.0, toxiproxy.Attributes{ "latency": 1000, }) // Change downstream latency to add 100ms of jitter proxy.UpdateToxic("latency_down", 1.0, toxiproxy.Attributes{ "jitter": 100, }) // Remove the latency toxic proxy.RemoveToxic("latency_down") ``` The proxy can be taken down using `Disable()`: ```go proxy.Disable() ``` When a proxy is no longer needed, it can be cleaned up with `Delete()`: ```go proxy.Delete() ``` ## Full Example ```go import ( "net/http" "testing" "time" "github.com/Shopify/toxiproxy/client" "github.com/garyburd/redigo/redis" ) var toxiClient *toxiproxy.Client var proxies map[string]*toxiproxy.Proxy func init() { var err error toxiClient = toxiproxy.NewClient("localhost:8474") proxies, err = toxiClient.Populate([]toxiproxy.Proxy{{ Name: "redis", Listen: "localhost:26379", Upstream: "localhost:6379", }}) if err != nil { panic(err) } // Alternatively, create the proxies manually with // toxiClient.CreateProxy("redis", "localhost:26379", "localhost:6379") } func TestRedisBackendDown(t *testing.T) { proxies["redis"].Disable() defer proxies["redis"].Enable() // Test that redis is down _, err := redis.Dial("tcp", ":26379") if err == nil { t.Fatal("Connection to redis did not fail") } } func TestRedisBackendSlow(t *testing.T) { proxies["redis"].AddToxic("", "latency", "", 1, toxiproxy.Attributes{ "latency": 1000, }) defer proxies["redis"].RemoveToxic("latency_downstream") // Test that redis is slow start := time.Now() conn, err := redis.Dial("tcp", ":26379") if err != nil { t.Fatal("Connection to redis failed", err) } _, err = conn.Do("GET", "test") if err != nil { t.Fatal("Redis command failed", err) } else if time.Since(start) < 900*time.Millisecond { t.Fatal("Redis command did not take long enough:", time.Since(start)) } } func TestEphemeralProxy(t *testing.T) { proxy, _ := toxiClient.CreateProxy("test", "", "google.com:80") defer proxy.Delete() // Test connection through proxy.Listen resp, err := http.Get("http://" + proxy.Listen) if err != nil { t.Fatal(err) } else if resp.StatusCode != 200 { t.Fatal("Proxy to google failed:", resp.StatusCode) } } ``` toxiproxy-2.0.0/client/client.go000066400000000000000000000217561270742716200167040ustar00rootroot00000000000000// Package Toxiproxy provides a client wrapper around the Toxiproxy HTTP API for // testing the resiliency of Go applications. // // For use with Toxiproxy 2.x package toxiproxy import ( "bytes" "encoding/json" "fmt" "net/http" "strings" ) // Client holds information about where to connect to Toxiproxy. type Client struct { endpoint string } type Attributes map[string]interface{} type Toxic struct { Name string `json:"name"` Type string `json:"type"` Stream string `json:"stream,omitempty"` Toxicity float32 `json:"toxicity"` Attributes Attributes `json:"attributes"` } type Toxics []Toxic // Proxy represents a Proxy. type Proxy struct { Name string `json:"name"` // The name of the proxy Listen string `json:"listen"` // The address the proxy listens on Upstream string `json:"upstream"` // The upstream address to proxy to Enabled bool `json:"enabled"` // Whether the proxy is enabled ActiveToxics Toxics `json:"toxics"` // The toxics active on this proxy client *Client created bool // True if this proxy exists on the server } // NewClient creates a new client which provides the base of all communication // with Toxiproxy. Endpoint is the address to the proxy (e.g. localhost:8474 if // not overriden) func NewClient(endpoint string) *Client { if !strings.HasPrefix(endpoint, "http://") { endpoint = "http://" + endpoint } return &Client{endpoint: endpoint} } // Proxies returns a map with all the proxies and their toxics. func (client *Client) Proxies() (map[string]*Proxy, error) { resp, err := http.Get(client.endpoint + "/proxies") if err != nil { return nil, err } err = checkError(resp, http.StatusOK, "Proxies") if err != nil { return nil, err } proxies := make(map[string]*Proxy) err = json.NewDecoder(resp.Body).Decode(&proxies) if err != nil { return nil, err } for _, proxy := range proxies { proxy.client = client proxy.created = true } return proxies, nil } // Generates a new uncommitted proxy instance. In order to use the result, the // proxy fields will need to be set and have `Save()` called. func (client *Client) NewProxy() *Proxy { return &Proxy{ client: client, } } // CreateProxy instantiates a new proxy and starts listening on the specified address. // This is an alias for `NewProxy()` + `proxy.Save()` func (client *Client) CreateProxy(name, listen, upstream string) (*Proxy, error) { proxy := &Proxy{ Name: name, Listen: listen, Upstream: upstream, Enabled: true, client: client, } err := proxy.Save() if err != nil { return nil, err } return proxy, nil } // Proxy returns a proxy by name. func (client *Client) Proxy(name string) (*Proxy, error) { // TODO url encode resp, err := http.Get(client.endpoint + "/proxies/" + name) if err != nil { return nil, err } err = checkError(resp, http.StatusOK, "Proxy") if err != nil { return nil, err } proxy := new(Proxy) err = json.NewDecoder(resp.Body).Decode(proxy) if err != nil { return nil, err } proxy.client = client proxy.created = true return proxy, nil } // Create a list of proxies using a configuration list. If a proxy already exists, it will be replaced // with the specified configuration. For large amounts of proxies, `config` can be loaded from a file. func (client *Client) Populate(config []Proxy) (map[string]*Proxy, error) { proxies := make(map[string]*Proxy, len(config)) for _, proxy := range config { existing, err := client.Proxy(proxy.Name) if err != nil && err.Error() != "Proxy: HTTP 404: proxy not found" { return nil, err } else if existing != nil && (existing.Listen != proxy.Listen || existing.Upstream != proxy.Upstream) { existing.Delete() } proxies[proxy.Name], err = client.CreateProxy(proxy.Name, proxy.Listen, proxy.Upstream) if err != nil { return nil, err } } return proxies, nil } // Save saves changes to a proxy such as its enabled status or upstream port. func (proxy *Proxy) Save() error { request, err := json.Marshal(proxy) if err != nil { return err } var resp *http.Response if proxy.created { resp, err = http.Post(proxy.client.endpoint+"/proxies/"+proxy.Name, "text/plain", bytes.NewReader(request)) } else { resp, err = http.Post(proxy.client.endpoint+"/proxies", "application/json", bytes.NewReader(request)) } if err != nil { return err } if proxy.created { err = checkError(resp, http.StatusOK, "Save") } else { err = checkError(resp, http.StatusCreated, "Create") } if err != nil { return err } err = json.NewDecoder(resp.Body).Decode(proxy) if err != nil { return err } proxy.created = true return nil } // Enable a proxy again after it has been disabled. func (proxy *Proxy) Enable() error { proxy.Enabled = true return proxy.Save() } // Disable a proxy so that no connections can pass through. This will drop all active connections. func (proxy *Proxy) Disable() error { proxy.Enabled = false return proxy.Save() } // Delete a proxy complete and close all existing connections through it. All information about // the proxy such as listen port and active toxics will be deleted as well. If you just wish to // stop and later enable a proxy, use `Enable()` and `Disable()`. func (proxy *Proxy) Delete() error { httpClient := &http.Client{} req, err := http.NewRequest("DELETE", proxy.client.endpoint+"/proxies/"+proxy.Name, nil) if err != nil { return err } resp, err := httpClient.Do(req) if err != nil { return err } return checkError(resp, http.StatusNoContent, "Delete") } // Toxics returns a map of all the active toxics and their attributes. func (proxy *Proxy) Toxics() (Toxics, error) { resp, err := http.Get(proxy.client.endpoint + "/proxies/" + proxy.Name + "/toxics") if err != nil { return nil, err } err = checkError(resp, http.StatusOK, "Toxics") if err != nil { return nil, err } toxics := make(Toxics, 0) err = json.NewDecoder(resp.Body).Decode(&toxics) if err != nil { return nil, err } return toxics, nil } // AddToxic adds a toxic to the given stream direction. // If a name is not specified, it will default to _. // If a stream is not specified, it will default to downstream. // See https://github.com/Shopify/toxiproxy#toxics for a list of all Toxic types. func (proxy *Proxy) AddToxic(name, typeName, stream string, toxicity float32, attrs Attributes) (*Toxic, error) { toxic := Toxic{name, typeName, stream, toxicity, attrs} if toxic.Toxicity == -1 { toxic.Toxicity = 1 // Just to be consistent with a toxicity of -1 using the default } request, err := json.Marshal(&toxic) if err != nil { return nil, err } resp, err := http.Post(proxy.client.endpoint+"/proxies/"+proxy.Name+"/toxics", "application/json", bytes.NewReader(request)) if err != nil { return nil, err } err = checkError(resp, http.StatusOK, "AddToxic") if err != nil { return nil, err } result := &Toxic{} err = json.NewDecoder(resp.Body).Decode(result) if err != nil { return nil, err } return result, nil } // UpdateToxic sets the parameters for an existing toxic with the given name. // If toxicity is set to -1, the current value will be used. func (proxy *Proxy) UpdateToxic(name string, toxicity float32, attrs Attributes) (*Toxic, error) { toxic := map[string]interface{}{ "attributes": attrs, } if toxicity != -1 { toxic["toxicity"] = toxicity } request, err := json.Marshal(&toxic) if err != nil { return nil, err } resp, err := http.Post(proxy.client.endpoint+"/proxies/"+proxy.Name+"/toxics/"+name, "application/json", bytes.NewReader(request)) if err != nil { return nil, err } err = checkError(resp, http.StatusOK, "UpdateToxic") if err != nil { return nil, err } result := &Toxic{} err = json.NewDecoder(resp.Body).Decode(result) if err != nil { return nil, err } return result, nil } // RemoveToxic renives the toxic with the given name. func (proxy *Proxy) RemoveToxic(name string) error { httpClient := &http.Client{} req, err := http.NewRequest("DELETE", proxy.client.endpoint+"/proxies/"+proxy.Name+"/toxics/"+name, nil) if err != nil { return err } resp, err := httpClient.Do(req) if err != nil { return err } return checkError(resp, http.StatusNoContent, "RemoveToxic") } // ResetState resets the state of all proxies and toxics in Toxiproxy. func (client *Client) ResetState() error { resp, err := http.Post(client.endpoint+"/reset", "text/plain", bytes.NewReader([]byte{})) if err != nil { return err } return checkError(resp, http.StatusNoContent, "ResetState") } type ApiError struct { Title string `json:"title"` Status int `json:"status"` } func (err *ApiError) Error() string { return fmt.Sprintf("HTTP %d: %s", err.Status, err.Title) } func checkError(resp *http.Response, expectedCode int, caller string) error { if resp.StatusCode != expectedCode { apiError := new(ApiError) err := json.NewDecoder(resp.Body).Decode(apiError) if err != nil { apiError.Title = fmt.Sprintf("Unexpected response code, expected %d", expectedCode) apiError.Status = resp.StatusCode } return fmt.Errorf("%s: %v", caller, apiError) } return nil } toxiproxy-2.0.0/cmd/000077500000000000000000000000001270742716200143515ustar00rootroot00000000000000toxiproxy-2.0.0/cmd/toxiproxy.go000066400000000000000000000007701270742716200167710ustar00rootroot00000000000000package main import ( "flag" "math/rand" "time" "github.com/Shopify/toxiproxy" ) var host string var port string func init() { flag.StringVar(&host, "host", "localhost", "Host for toxiproxy's API to listen on") flag.StringVar(&port, "port", "8474", "Port for toxiproxy's API to listen on") seed := flag.Int64("seed", time.Now().UTC().UnixNano(), "Seed for randomizing toxics with") flag.Parse() rand.Seed(*seed) } func main() { server := toxiproxy.NewServer() server.Listen(host, port) } toxiproxy-2.0.0/link.go000066400000000000000000000104301270742716200150700ustar00rootroot00000000000000package toxiproxy import ( "io" "github.com/Shopify/toxiproxy/stream" "github.com/Shopify/toxiproxy/toxics" "github.com/Sirupsen/logrus" ) // ToxicLinks are single direction pipelines that connects an input and output via // a chain of toxics. The chain always starts with a NoopToxic, and toxics are added // and removed as they are enabled/disabled. New toxics are always added to the end // of the chain. // // NoopToxic LatencyToxic // v v // Input > ToxicStub > ToxicStub > Output // type ToxicLink struct { stubs []*toxics.ToxicStub proxy *Proxy toxics *ToxicCollection input *stream.ChanWriter output *stream.ChanReader direction stream.Direction } func NewToxicLink(proxy *Proxy, collection *ToxicCollection, direction stream.Direction) *ToxicLink { link := &ToxicLink{ stubs: make([]*toxics.ToxicStub, len(collection.chain[direction]), cap(collection.chain[direction])), proxy: proxy, toxics: collection, direction: direction, } // Initialize the link with ToxicStubs last := make(chan *stream.StreamChunk) // The first toxic is always a noop link.input = stream.NewChanWriter(last) for i := 0; i < len(link.stubs); i++ { var next chan *stream.StreamChunk if i+1 < len(link.stubs) { next = make(chan *stream.StreamChunk, link.toxics.chain[direction][i+1].BufferSize) } else { next = make(chan *stream.StreamChunk) } link.stubs[i] = toxics.NewToxicStub(last, next) last = next } link.output = stream.NewChanReader(last) return link } // Start the link with the specified toxics func (link *ToxicLink) Start(name string, source io.Reader, dest io.WriteCloser) { go func() { bytes, err := io.Copy(link.input, source) if err != nil { logrus.WithFields(logrus.Fields{ "name": link.proxy.Name, "bytes": bytes, "err": err, }).Warn("Source terminated") } link.input.Close() }() for i, toxic := range link.toxics.chain[link.direction] { go link.stubs[i].Run(toxic) } go func() { bytes, err := io.Copy(dest, link.output) if err != nil { logrus.WithFields(logrus.Fields{ "name": link.proxy.Name, "bytes": bytes, "err": err, }).Warn("Destination terminated") } dest.Close() link.toxics.RemoveLink(name) link.proxy.RemoveConnection(name) }() } // Add a toxic to the end of the chain. func (link *ToxicLink) AddToxic(toxic *toxics.ToxicWrapper) { i := len(link.stubs) newin := make(chan *stream.StreamChunk, toxic.BufferSize) link.stubs = append(link.stubs, toxics.NewToxicStub(newin, link.stubs[i-1].Output)) // Interrupt the last toxic so that we don't have a race when moving channels if link.stubs[i-1].InterruptToxic() { link.stubs[i-1].Output = newin go link.stubs[i].Run(toxic) go link.stubs[i-1].Run(link.toxics.chain[link.direction][i-1]) } else { // This link is already closed, make sure the new toxic matches link.stubs[i].Output = newin // The real output is already closed, close this instead link.stubs[i].Close() } } // Update an existing toxic in the chain. func (link *ToxicLink) UpdateToxic(toxic *toxics.ToxicWrapper) { if link.stubs[toxic.Index].InterruptToxic() { go link.stubs[toxic.Index].Run(toxic) } } // Remove an existing toxic from the chain. func (link *ToxicLink) RemoveToxic(toxic *toxics.ToxicWrapper) { i := toxic.Index if link.stubs[i].InterruptToxic() { stop := make(chan bool) // Interrupt the previous toxic to update its output go func() { stop <- link.stubs[i-1].InterruptToxic() }() // Unblock the previous toxic if it is trying to flush // If the previous toxic is closed, continue flusing until we reach the end. interrupted := false stopped := false for !interrupted { select { case interrupted = <-stop: stopped = true case tmp := <-link.stubs[i].Input: if tmp == nil { link.stubs[i].Close() if !stopped { <-stop } return } link.stubs[i].Output <- tmp } } // Empty the toxic's buffer if necessary for len(link.stubs[i].Input) > 0 { tmp := <-link.stubs[i].Input if tmp == nil { link.stubs[i].Close() return } link.stubs[i].Output <- tmp } link.stubs[i-1].Output = link.stubs[i].Output link.stubs = append(link.stubs[:i], link.stubs[i+1:]...) go link.stubs[i-1].Run(link.toxics.chain[link.direction][i-1]) } } toxiproxy-2.0.0/link_test.go000066400000000000000000000133501270742716200161330ustar00rootroot00000000000000package toxiproxy import ( "encoding/binary" "io" "testing" "github.com/Shopify/toxiproxy/stream" "github.com/Shopify/toxiproxy/toxics" ) func TestToxicsAreLoaded(t *testing.T) { if toxics.Count() < 1 { t.Fatal("No toxics loaded!") } } func TestStubInitializaation(t *testing.T) { collection := NewToxicCollection(nil) link := NewToxicLink(nil, collection, stream.Downstream) if len(link.stubs) != 1 { t.Fatalf("Link created with wrong number of stubs: %d != 1", len(link.stubs)) } else if cap(link.stubs) != toxics.Count()+1 { t.Fatalf("Link created with wrong capacity: %d != %d", cap(link.stubs), toxics.Count()+1) } else if cap(link.stubs[0].Input) != 0 { t.Fatalf("Noop buffer was not initialized as 0: %d", cap(link.stubs[0].Input)) } else if cap(link.stubs[0].Output) != 0 { t.Fatalf("Link output buffer was not initialized as 0: %d", cap(link.stubs[0].Output)) } } func TestStubInitializaationWithToxics(t *testing.T) { collection := NewToxicCollection(nil) collection.chainAddToxic(&toxics.ToxicWrapper{ Toxic: new(toxics.LatencyToxic), Type: "latency", Direction: stream.Downstream, BufferSize: 1024, Toxicity: 1, }) collection.chainAddToxic(&toxics.ToxicWrapper{ Toxic: new(toxics.BandwidthToxic), Type: "bandwidth", Direction: stream.Downstream, Toxicity: 1, }) link := NewToxicLink(nil, collection, stream.Downstream) if len(link.stubs) != 3 { t.Fatalf("Link created with wrong number of stubs: %d != 3", len(link.stubs)) } else if cap(link.stubs) != toxics.Count()+1 { t.Fatalf("Link created with wrong capacity: %d != %d", cap(link.stubs), toxics.Count()+1) } else if cap(link.stubs[len(link.stubs)-1].Output) != 0 { t.Fatalf("Link output buffer was not initialized as 0: %d", cap(link.stubs[0].Output)) } for i, toxic := range collection.chain[stream.Downstream] { if cap(link.stubs[i].Input) != toxic.BufferSize { t.Fatalf("%s buffer was not initialized as %d: %d", toxic.Type, toxic.BufferSize, cap(link.stubs[i].Input)) } } } func TestAddRemoveStubs(t *testing.T) { collection := NewToxicCollection(nil) link := NewToxicLink(nil, collection, stream.Downstream) go link.stubs[0].Run(collection.chain[stream.Downstream][0]) collection.links["test"] = link // Add stubs collection.chainAddToxic(&toxics.ToxicWrapper{ Toxic: new(toxics.LatencyToxic), Type: "latency", Direction: stream.Downstream, BufferSize: 1024, Toxicity: 1, }) toxic := &toxics.ToxicWrapper{ Toxic: new(toxics.BandwidthToxic), Type: "bandwidth", Direction: stream.Downstream, BufferSize: 2048, Toxicity: 1, } collection.chainAddToxic(toxic) if cap(link.stubs[len(link.stubs)-1].Output) != 0 { t.Fatalf("Link output buffer was not initialized as 0: %d", cap(link.stubs[0].Output)) } for i, toxic := range collection.chain[stream.Downstream] { if cap(link.stubs[i].Input) != toxic.BufferSize { t.Fatalf("%s buffer was not initialized as %d: %d", toxic.Type, toxic.BufferSize, cap(link.stubs[i].Input)) } } // Remove stubs collection.chainRemoveToxic(toxic) if cap(link.stubs[len(link.stubs)-1].Output) != 0 { t.Fatalf("Link output buffer was not initialized as 0: %d", cap(link.stubs[0].Output)) } for i, toxic := range collection.chain[stream.Downstream] { if cap(link.stubs[i].Input) != toxic.BufferSize { t.Fatalf("%s buffer was not initialized as %d: %d", toxic.Type, toxic.BufferSize, cap(link.stubs[i].Input)) } } } func TestNoDataDropped(t *testing.T) { collection := NewToxicCollection(nil) link := NewToxicLink(nil, collection, stream.Downstream) go link.stubs[0].Run(collection.chain[stream.Downstream][0]) collection.links["test"] = link toxic := &toxics.ToxicWrapper{ Toxic: &toxics.LatencyToxic{ Latency: 1000, }, Type: "latency", Direction: stream.Downstream, BufferSize: 1024, Toxicity: 1, } done := make(chan struct{}) defer close(done) go func() { for i := 0; i < 64*1024; i++ { buf := make([]byte, 2) binary.BigEndian.PutUint16(buf, uint16(i)) link.input.Write(buf) } link.input.Close() }() go func() { for { select { case <-done: return default: collection.chainAddToxic(toxic) collection.chainRemoveToxic(toxic) } } }() buf := make([]byte, 2) for i := 0; i < 64*1024; i++ { n, err := link.output.Read(buf) if n != 2 || err != nil { t.Fatalf("Read failed: %d %v", n, err) } else { val := binary.BigEndian.Uint16(buf) if val != uint16(i) { t.Fatalf("Read incorrect bytes: %v != %d", val, i) } } } n, err := link.output.Read(buf) if n != 0 || err != io.EOF { t.Fatalf("Expected EOF: %d %v", n, err) } } func TestToxicity(t *testing.T) { collection := NewToxicCollection(nil) link := NewToxicLink(nil, collection, stream.Downstream) go link.stubs[0].Run(collection.chain[stream.Downstream][0]) collection.links["test"] = link toxic := &toxics.ToxicWrapper{ Toxic: new(toxics.TimeoutToxic), Name: "timeout1", Type: "timeout", Direction: stream.Downstream, Toxicity: 0, } collection.chainAddToxic(toxic) // Toxic should be a Noop because of toxicity n, err := link.input.Write([]byte{42}) if n != 1 || err != nil { t.Fatalf("Write failed: %d %v", n, err) } buf := make([]byte, 2) n, err = link.output.Read(buf) if n != 1 || err != nil { t.Fatalf("Read failed: %d %v", n, err) } else if buf[0] != 42 { t.Fatalf("Read wrong byte: %x", buf[0]) } toxic.Toxicity = 1 toxic.Toxic.(*toxics.TimeoutToxic).Timeout = 100 collection.chainUpdateToxic(toxic) // Toxic should timeout after 100ms n, err = link.input.Write([]byte{42}) if n != 1 || err != nil { t.Fatalf("Write failed: %d %v", n, err) } n, err = link.output.Read(buf) if n != 0 || err != io.EOF { t.Fatalf("Read did not get EOF: %d %v", n, err) } } toxiproxy-2.0.0/proxy.go000066400000000000000000000120551270742716200153210ustar00rootroot00000000000000package toxiproxy import ( "errors" "sync" "github.com/Shopify/toxiproxy/stream" "github.com/Sirupsen/logrus" "gopkg.in/tomb.v1" "net" ) // Proxy represents the proxy in its entirity with all its links. The main // responsibility of Proxy is to accept new client and create Links between the // client and upstream. // // Client <-> toxiproxy <-> Upstream // type Proxy struct { sync.Mutex Name string `json:"name"` Listen string `json:"listen"` Upstream string `json:"upstream"` Enabled bool `json:"enabled"` started chan error tomb tomb.Tomb connections ConnectionList Toxics *ToxicCollection `json:"-"` } type ConnectionList struct { list map[string]net.Conn lock sync.Mutex } func (c *ConnectionList) Lock() { c.lock.Lock() } func (c *ConnectionList) Unlock() { c.lock.Unlock() } var ErrProxyAlreadyStarted = errors.New("Proxy already started") func NewProxy() *Proxy { proxy := &Proxy{ started: make(chan error), connections: ConnectionList{list: make(map[string]net.Conn)}, } proxy.Toxics = NewToxicCollection(proxy) return proxy } func (proxy *Proxy) Start() error { proxy.Lock() defer proxy.Unlock() return start(proxy) } func (proxy *Proxy) Update(input *Proxy) error { proxy.Lock() defer proxy.Unlock() if input.Listen != proxy.Listen || input.Upstream != proxy.Upstream { stop(proxy) proxy.Listen = input.Listen proxy.Upstream = input.Upstream } if input.Enabled != proxy.Enabled { if input.Enabled { return start(proxy) } stop(proxy) } return nil } func (proxy *Proxy) Stop() { proxy.Lock() defer proxy.Unlock() stop(proxy) } // server runs the Proxy server, accepting new clients and creating Links to // connect them to upstreams. func (proxy *Proxy) server() { ln, err := net.Listen("tcp", proxy.Listen) if err != nil { proxy.started <- err return } proxy.Listen = ln.Addr().String() proxy.started <- nil logrus.WithFields(logrus.Fields{ "name": proxy.Name, "proxy": proxy.Listen, "upstream": proxy.Upstream, }).Info("Started proxy") acceptTomb := tomb.Tomb{} defer acceptTomb.Done() // This channel is to kill the blocking Accept() call below by closing the // net.Listener. go func() { <-proxy.tomb.Dying() // Notify ln.Accept() that the shutdown was safe acceptTomb.Killf("Shutting down from stop()") // Unblock ln.Accept() err := ln.Close() if err != nil { logrus.WithFields(logrus.Fields{ "proxy": proxy.Name, "listen": proxy.Listen, "err": err, }).Warn("Attempted to close an already closed proxy server") } // Wait for the accept loop to finish processing acceptTomb.Wait() proxy.tomb.Done() }() for { client, err := ln.Accept() if err != nil { // This is to confirm we're being shut down in a legit way. Unfortunately, // Go doesn't export the error when it's closed from Close() so we have to // sync up with a channel here. // // See http://zhen.org/blog/graceful-shutdown-of-go-net-dot-listeners/ select { case <-acceptTomb.Dying(): default: logrus.WithFields(logrus.Fields{ "proxy": proxy.Name, "listen": proxy.Listen, "err": err, }).Warn("Error while accepting client") } return } logrus.WithFields(logrus.Fields{ "name": proxy.Name, "client": client.RemoteAddr(), "proxy": proxy.Listen, "upstream": proxy.Upstream, }).Info("Accepted client") upstream, err := net.Dial("tcp", proxy.Upstream) if err != nil { logrus.WithFields(logrus.Fields{ "name": proxy.Name, "client": client.RemoteAddr(), "proxy": proxy.Listen, "upstream": proxy.Upstream, }).Error("Unable to open connection to upstream") client.Close() continue } name := client.RemoteAddr().String() proxy.connections.Lock() proxy.connections.list[name+"upstream"] = upstream proxy.connections.list[name+"downstream"] = client proxy.connections.Unlock() proxy.Toxics.StartLink(name+"upstream", client, upstream, stream.Upstream) proxy.Toxics.StartLink(name+"downstream", upstream, client, stream.Downstream) } } func (proxy *Proxy) RemoveConnection(name string) { proxy.connections.Lock() defer proxy.connections.Unlock() delete(proxy.connections.list, name) } // Starts a proxy, assumes the lock has already been taken func start(proxy *Proxy) error { if proxy.Enabled { return ErrProxyAlreadyStarted } proxy.tomb = tomb.Tomb{} // Reset tomb, from previous starts/stops go proxy.server() err := <-proxy.started // Only enable the proxy if it successfully started proxy.Enabled = err == nil return err } // Stops a proxy, assumes the lock has already been taken func stop(proxy *Proxy) { if !proxy.Enabled { return } proxy.Enabled = false proxy.tomb.Killf("Shutting down from stop()") proxy.tomb.Wait() // Wait until we stop accepting new connections proxy.connections.Lock() defer proxy.connections.Unlock() for _, conn := range proxy.connections.list { conn.Close() } logrus.WithFields(logrus.Fields{ "name": proxy.Name, "proxy": proxy.Listen, "upstream": proxy.Upstream, }).Info("Terminated proxy") } toxiproxy-2.0.0/proxy_collection.go000066400000000000000000000040431270742716200175320ustar00rootroot00000000000000package toxiproxy import "sync" // ProxyCollection is a collection of proxies. It's the interface for anything // to add and remove proxies from the toxiproxy instance. It's responsibilty is // to maintain the integrity of the proxy set, by guarding for things such as // duplicate names. type ProxyCollection struct { sync.RWMutex proxies map[string]*Proxy } func NewProxyCollection() *ProxyCollection { return &ProxyCollection{ proxies: make(map[string]*Proxy), } } func (collection *ProxyCollection) Add(proxy *Proxy, start bool) error { collection.Lock() defer collection.Unlock() if _, exists := collection.proxies[proxy.Name]; exists { return ErrProxyAlreadyExists } if start { err := proxy.Start() if err != nil { return err } } collection.proxies[proxy.Name] = proxy return nil } func (collection *ProxyCollection) Proxies() map[string]*Proxy { collection.RLock() defer collection.RUnlock() // Copy the map since using the existing one isn't thread-safe proxies := make(map[string]*Proxy, len(collection.proxies)) for k, v := range collection.proxies { proxies[k] = v } return proxies } func (collection *ProxyCollection) Get(name string) (*Proxy, error) { collection.RLock() defer collection.RUnlock() return collection.getByName(name) } func (collection *ProxyCollection) Remove(name string) error { collection.Lock() defer collection.Unlock() proxy, err := collection.getByName(name) if err != nil { return err } proxy.Stop() delete(collection.proxies, proxy.Name) return nil } func (collection *ProxyCollection) Clear() error { collection.Lock() defer collection.Unlock() for _, proxy := range collection.proxies { proxy.Stop() delete(collection.proxies, proxy.Name) } return nil } // getByName returns a proxy by its name. Its used from #remove and #get. // It assumes the lock has already been acquired. func (collection *ProxyCollection) getByName(name string) (*Proxy, error) { proxy, exists := collection.proxies[name] if !exists { return nil, ErrProxyNotFound } return proxy, nil } toxiproxy-2.0.0/proxy_collection_test.go000066400000000000000000000055541270742716200206010ustar00rootroot00000000000000package toxiproxy_test import ( "bytes" "net" "testing" "github.com/Shopify/toxiproxy" ) func TestAddProxyToCollection(t *testing.T) { collection := toxiproxy.NewProxyCollection() proxy := NewTestProxy("test", "localhost:20000") if _, err := collection.Get(proxy.Name); err == nil { t.Error("Expected proxies to be empty") } err := collection.Add(proxy, false) if err != nil { t.Error("Expected to be able to add first proxy to collection") } if _, err := collection.Get(proxy.Name); err != nil { t.Error("Expected proxy to be added to map") } } func TestAddTwoProxiesToCollection(t *testing.T) { collection := toxiproxy.NewProxyCollection() proxy := NewTestProxy("test", "localhost:20000") err := collection.Add(proxy, false) if err != nil { t.Error("Expected to be able to add first proxy to collection") } err = collection.Add(proxy, false) if err == nil { t.Error("Expected to not be able to add proxy with same name") } } func TestListProxies(t *testing.T) { collection := toxiproxy.NewProxyCollection() proxy := NewTestProxy("test", "localhost:20000") err := collection.Add(proxy, false) if err != nil { t.Error("Expected to be able to add first proxy to collection") } proxies := collection.Proxies() proxy, ok := proxies[proxy.Name] if !ok { t.Error("Expected to be able to see existing proxy") } else if proxy.Enabled { t.Error("Expected proxy not to be running") } } func TestAddProxyAndStart(t *testing.T) { collection := toxiproxy.NewProxyCollection() proxy := NewTestProxy("test", "localhost:20000") err := collection.Add(proxy, true) if err != nil { t.Error("Expected to be able to add proxy to collection:", err) } proxies := collection.Proxies() proxy, ok := proxies[proxy.Name] if !ok { t.Error("Expected to be able to see existing proxy") } else if !proxy.Enabled { t.Error("Expected proxy to be running") } } func TestAddAndRemoveProxyFromCollection(t *testing.T) { WithTCPProxy(t, func(conn net.Conn, response chan []byte, proxy *toxiproxy.Proxy) { collection := toxiproxy.NewProxyCollection() if _, err := collection.Get(proxy.Name); err == nil { t.Error("Expected proxies to be empty") } err := collection.Add(proxy, false) if err != nil { t.Error("Expected to be able to add first proxy to collection") } if _, err := collection.Get(proxy.Name); err != nil { t.Error("Expected proxy to be added to map") } msg := []byte("go away") _, err = conn.Write(msg) if err != nil { t.Error("Failed writing to socket to shut down server") } conn.Close() resp := <-response if !bytes.Equal(resp, msg) { t.Error("Server didn't read bytes from client") } err = collection.Remove(proxy.Name) if err != nil { t.Error("Expected to remove proxy from collection") } if _, err := collection.Get(proxy.Name); err == nil { t.Error("Expected proxies to be empty") } }) } toxiproxy-2.0.0/proxy_test.go000066400000000000000000000157751270742716200163740ustar00rootroot00000000000000package toxiproxy_test import ( "bytes" "encoding/hex" "io" "io/ioutil" "net" "testing" "time" "github.com/Shopify/toxiproxy" "github.com/Sirupsen/logrus" "gopkg.in/tomb.v1" ) func init() { logrus.SetLevel(logrus.FatalLevel) } func NewTestProxy(name, upstream string) *toxiproxy.Proxy { proxy := toxiproxy.NewProxy() proxy.Name = name proxy.Listen = "localhost:0" proxy.Upstream = upstream return proxy } func WithTCPServer(t *testing.T, f func(string, chan []byte)) { ln, err := net.Listen("tcp", "localhost:0") if err != nil { t.Fatal("Failed to create TCP server", err) } defer ln.Close() response := make(chan []byte, 1) tomb := tomb.Tomb{} go func() { defer tomb.Done() src, err := ln.Accept() if err != nil { select { case <-tomb.Dying(): default: t.Fatal("Failed to accept client") } return } ln.Close() val, err := ioutil.ReadAll(src) if err != nil { t.Fatal("Failed to read from client") } response <- val }() f(ln.Addr().String(), response) tomb.Killf("Function body finished") ln.Close() tomb.Wait() close(response) } func TestSimpleServer(t *testing.T) { WithTCPServer(t, func(addr string, response chan []byte) { conn, err := net.Dial("tcp", addr) if err != nil { t.Error("Unable to dial TCP server", err) } msg := []byte("hello world") _, err = conn.Write(msg) if err != nil { t.Error("Failed writing to TCP server", err) } err = conn.Close() if err != nil { t.Error("Failed to close TCP connection", err) } resp := <-response if !bytes.Equal(resp, msg) { t.Error("Server didn't read bytes from client") } }) } func WithTCPProxy(t *testing.T, f func(proxy net.Conn, response chan []byte, proxyServer *toxiproxy.Proxy)) { WithTCPServer(t, func(upstream string, response chan []byte) { proxy := NewTestProxy("test", upstream) proxy.Start() conn := AssertProxyUp(t, proxy.Listen, true) f(conn, response, proxy) proxy.Stop() }) } func TestProxySimpleMessage(t *testing.T) { WithTCPProxy(t, func(conn net.Conn, response chan []byte, proxy *toxiproxy.Proxy) { msg := []byte("hello world") _, err := conn.Write(msg) if err != nil { t.Error("Failed writing to TCP server", err) } err = conn.Close() if err != nil { t.Error("Failed to close TCP connection", err) } resp := <-response if !bytes.Equal(resp, msg) { t.Error("Server didn't read correct bytes from client", resp) } }) } func TestProxyToDownUpstream(t *testing.T) { proxy := NewTestProxy("test", "localhost:20009") proxy.Start() conn := AssertProxyUp(t, proxy.Listen, true) // Check to make sure the connection is closed conn.SetReadDeadline(time.Now().Add(100 * time.Millisecond)) _, err := conn.Read(make([]byte, 1)) if err != io.EOF { t.Error("Proxy did not close connection when upstream down", err) } proxy.Stop() } func TestProxyBigMessage(t *testing.T) { WithTCPProxy(t, func(conn net.Conn, response chan []byte, proxy *toxiproxy.Proxy) { buf := make([]byte, 32*1024) msg := make([]byte, len(buf)*2) hex.Encode(msg, buf) _, err := conn.Write(msg) if err != nil { t.Error("Failed writing to TCP server", err) } err = conn.Close() if err != nil { t.Error("Failed to close TCP connection", err) } resp := <-response if !bytes.Equal(resp, msg) { t.Error("Server didn't read correct bytes from client", resp) } }) } func TestProxyTwoPartMessage(t *testing.T) { WithTCPProxy(t, func(conn net.Conn, response chan []byte, proxy *toxiproxy.Proxy) { msg1 := []byte("hello world") msg2 := []byte("hello world") _, err := conn.Write(msg1) if err != nil { t.Error("Failed writing to TCP server", err) } _, err = conn.Write(msg2) if err != nil { t.Error("Failed writing to TCP server", err) } err = conn.Close() if err != nil { t.Error("Failed to close TCP connection", err) } msg1 = append(msg1, msg2...) resp := <-response if !bytes.Equal(resp, msg1) { t.Error("Server didn't read correct bytes from client", resp) } }) } func TestClosingProxyMultipleTimes(t *testing.T) { WithTCPProxy(t, func(conn net.Conn, response chan []byte, proxy *toxiproxy.Proxy) { proxy.Stop() proxy.Stop() proxy.Stop() }) } func TestStartTwoProxiesOnSameAddress(t *testing.T) { WithTCPProxy(t, func(conn net.Conn, response chan []byte, proxy *toxiproxy.Proxy) { proxy2 := NewTestProxy("proxy_2", "localhost:3306") proxy2.Listen = proxy.Listen if err := proxy2.Start(); err == nil { t.Fatal("Expected an err back from start") } }) } func TestStopProxyBeforeStarting(t *testing.T) { WithTCPServer(t, func(upstream string, response chan []byte) { proxy := NewTestProxy("test", upstream) AssertProxyUp(t, proxy.Listen, false) proxy.Stop() err := proxy.Start() if err != nil { t.Error("Proxy failed to start", err) } err = proxy.Start() if err != toxiproxy.ErrProxyAlreadyStarted { t.Error("Proxy did not fail to start when already started", err) } AssertProxyUp(t, proxy.Listen, true) proxy.Stop() AssertProxyUp(t, proxy.Listen, false) }) } func TestProxyUpdate(t *testing.T) { WithTCPServer(t, func(upstream string, response chan []byte) { proxy := NewTestProxy("test", upstream) err := proxy.Start() if err != nil { t.Error("Proxy failed to start", err) } AssertProxyUp(t, proxy.Listen, true) before := proxy.Listen input := &toxiproxy.Proxy{Listen: "localhost:0", Upstream: proxy.Upstream, Enabled: true} err = proxy.Update(input) if err != nil { t.Error("Failed to update proxy", err) } if proxy.Listen == before || proxy.Listen == input.Listen { t.Errorf("Proxy update didn't change listen address: %s to %s", before, proxy.Listen) } AssertProxyUp(t, proxy.Listen, true) input.Listen = proxy.Listen err = proxy.Update(input) if err != nil { t.Error("Failed to update proxy", err) } AssertProxyUp(t, proxy.Listen, true) input.Enabled = false err = proxy.Update(input) if err != nil { t.Error("Failed to update proxy", err) } AssertProxyUp(t, proxy.Listen, false) }) } func TestRestartFailedToStartProxy(t *testing.T) { WithTCPServer(t, func(upstream string, response chan []byte) { proxy := NewTestProxy("test", upstream) conflict := NewTestProxy("test2", upstream) err := conflict.Start() if err != nil { t.Error("Proxy failed to start", err) } AssertProxyUp(t, conflict.Listen, true) proxy.Listen = conflict.Listen err = proxy.Start() if err == nil || err == toxiproxy.ErrProxyAlreadyStarted { t.Error("Proxy started when it should have conflicted") } conflict.Stop() AssertProxyUp(t, conflict.Listen, false) err = proxy.Start() if err != nil { t.Error("Proxy failed to start after conflict went away", err) } AssertProxyUp(t, proxy.Listen, true) proxy.Stop() AssertProxyUp(t, proxy.Listen, false) }) } func AssertProxyUp(t *testing.T, addr string, up bool) net.Conn { conn, err := net.Dial("tcp", addr) if err != nil && up { t.Error("Expected proxy to be up", err) } else if err == nil && !up { t.Error("Expected proxy to be down") } return conn } toxiproxy-2.0.0/share/000077500000000000000000000000001270742716200147105ustar00rootroot00000000000000toxiproxy-2.0.0/share/toxiproxy.conf000066400000000000000000000004231270742716200176430ustar00rootroot00000000000000description "TCP proxy to simulate network and system conditions" author "Simon Eskildsen & Jacob Wirth" start on startup stop on shutdown env HOST="localhost" env PORT="8474" env BINARY="/usr/bin/toxiproxy-server" script exec $BINARY -port $PORT -host $HOST end script toxiproxy-2.0.0/stream/000077500000000000000000000000001270742716200151015ustar00rootroot00000000000000toxiproxy-2.0.0/stream/io_chan.go000066400000000000000000000047171270742716200170410ustar00rootroot00000000000000package stream import ( "fmt" "io" "time" ) type Direction uint8 const ( Upstream Direction = iota Downstream NumDirections ) // Stores a slice of bytes with its receive timestmap type StreamChunk struct { Data []byte Timestamp time.Time } // Implements the io.WriteCloser interface for a chan []byte type ChanWriter struct { output chan<- *StreamChunk } func NewChanWriter(output chan<- *StreamChunk) *ChanWriter { return &ChanWriter{output} } // Write `buf` as a StreamChunk to the channel. The full buffer is always written, and error // will always be nil. Calling `Write()` after closing the channel will panic. func (c *ChanWriter) Write(buf []byte) (int, error) { packet := &StreamChunk{make([]byte, len(buf)), time.Now()} copy(packet.Data, buf) // Make a copy before sending it to the channel c.output <- packet return len(buf), nil } // Close the output channel func (c *ChanWriter) Close() error { close(c.output) return nil } // Implements the io.Reader interface for a chan []byte type ChanReader struct { input <-chan *StreamChunk interrupt <-chan struct{} buffer []byte } var ErrInterrupted = fmt.Errorf("read interrupted by channel") func NewChanReader(input <-chan *StreamChunk) *ChanReader { return &ChanReader{input, make(chan struct{}), []byte{}} } // Specify a channel that can interrupt a read if it is blocking. func (c *ChanReader) SetInterrupt(interrupt <-chan struct{}) { c.interrupt = interrupt } // Read from the channel into `out`. This will block until data is available, // and can be interrupted with a channel using `SetInterrupt()`. If the read // was interrupted, `ErrInterrupted` will be returned. func (c *ChanReader) Read(out []byte) (int, error) { if c.buffer == nil { return 0, io.EOF } n := copy(out, c.buffer) c.buffer = c.buffer[n:] if len(out) <= len(c.buffer) { return n, nil } else if n > 0 { // We have some data to return, so make the channel read optional select { case p := <-c.input: if p == nil { // Stream was closed c.buffer = nil if n > 0 { return n, nil } return 0, io.EOF } n2 := copy(out[n:], p.Data) c.buffer = p.Data[n2:] return n + n2, nil default: return n, nil } } var p *StreamChunk select { case p = <-c.input: case <-c.interrupt: c.buffer = c.buffer[:0] return n, ErrInterrupted } if p == nil { // Stream was closed c.buffer = nil return 0, io.EOF } n2 := copy(out[n:], p.Data) c.buffer = p.Data[n2:] return n + n2, nil } toxiproxy-2.0.0/stream/io_chan_test.go000066400000000000000000000120331270742716200200660ustar00rootroot00000000000000package stream import ( "bytes" "io" "testing" "time" ) func TestBasicReadWrite(t *testing.T) { send := []byte("hello world") c := make(chan *StreamChunk) writer := NewChanWriter(c) reader := NewChanReader(c) go writer.Write(send) buf := make([]byte, len(send)) n, err := reader.Read(buf) if n != len(send) { t.Fatalf("Read wrong number of bytes: %d expected %d", n, len(send)) } if err != nil { t.Fatal("Couldn't read from stream", err) } if !bytes.Equal(buf, send) { t.Fatal("Got wrong message from stream", string(buf)) } writer.Close() n, err = reader.Read(buf) if err != io.EOF { t.Fatal("Read returned wrong error after close:", err) } if n != 0 { t.Fatalf("Read still returned data after close: %d bytes", n) } } func TestReadMoreThanWrite(t *testing.T) { send := []byte("hello world") c := make(chan *StreamChunk) writer := NewChanWriter(c) reader := NewChanReader(c) go writer.Write(send) buf := make([]byte, len(send)+10) n, err := reader.Read(buf) if n != len(send) { t.Fatalf("Read wrong number of bytes: %d expected %d", n, len(send)) } if err != nil { t.Fatal("Couldn't read from stream", err) } if !bytes.Equal(buf[:n], send) { t.Fatal("Got wrong message from stream", string(buf[:n])) } writer.Close() n, err = reader.Read(buf) if err != io.EOF { t.Fatal("Read returned wrong error after close:", err) } if n != 0 { t.Fatalf("Read still returned data after close: %d bytes", n) } } func TestReadLessThanWrite(t *testing.T) { send := []byte("hello world") c := make(chan *StreamChunk) writer := NewChanWriter(c) reader := NewChanReader(c) go writer.Write(send) buf := make([]byte, 6) n, err := reader.Read(buf) if n != len(buf) { t.Fatalf("Read wrong number of bytes: %d expected %d", n, len(buf)) } if err != nil { t.Fatal("Couldn't read from stream", err) } if !bytes.Equal(buf, send[:len(buf)]) { t.Fatal("Got wrong message from stream", string(buf)) } writer.Close() n, err = reader.Read(buf) if n != len(send)-len(buf) { t.Fatalf("Read wrong number of bytes: %d expected %d", n, len(send)-len(buf)) } if err != nil { t.Fatal("Couldn't read from stream", err) } if !bytes.Equal(buf[:n], send[len(buf):]) { t.Fatal("Got wrong message from stream", string(buf[:n])) } n, err = reader.Read(buf) if err != io.EOF { t.Fatal("Read returned wrong error after close:", err) } if n != 0 { t.Fatalf("Read still returned data after close: %d bytes", n) } } func TestMultiReadWrite(t *testing.T) { send := []byte("hello world, this message is longer") c := make(chan *StreamChunk) writer := NewChanWriter(c) reader := NewChanReader(c) go func() { writer.Write(send[:9]) writer.Write(send[9:19]) writer.Write(send[19:]) writer.Close() }() buf := make([]byte, 10) for read := 0; read < len(send); { n, err := reader.Read(buf) if err != nil { t.Fatal("Couldn't read from stream", err, n) } if !bytes.Equal(buf[:n], send[read:read+n]) { t.Fatal("Got wrong message from stream", string(buf)) } read += n } n, err := reader.Read(buf) if err != io.EOF { t.Fatal("Read returned wrong error after close:", err, string(buf[:n])) } if !bytes.Equal(buf[:n], send[len(send)-n:]) { t.Fatal("Got wrong message from stream", string(buf[:n])) } } func TestMultiWriteWithCopy(t *testing.T) { send := []byte("hello world, this message is longer") c := make(chan *StreamChunk) writer := NewChanWriter(c) reader := NewChanReader(c) go func() { writer.Write(send[:9]) writer.Write(send[9:19]) writer.Write(send[19:]) writer.Close() }() buf := new(bytes.Buffer) n, err := io.Copy(buf, reader) if int(n) != len(send) { t.Fatalf("Read wrong number of bytes: %d expected %d", n, len(send)) } if err != nil { t.Fatal("Couldn't read from stream", err) } if !bytes.Equal(buf.Bytes(), send) { t.Fatal("Got wrong message from stream", buf.String()) } } func TestReadInterrupt(t *testing.T) { send := []byte("hello world") c := make(chan *StreamChunk) interrupt := make(chan struct{}) writer := NewChanWriter(c) reader := NewChanReader(c) reader.SetInterrupt(interrupt) go writer.Write(send) buf := make([]byte, len(send)) n, err := reader.Read(buf) if n != len(send) { t.Fatalf("Read wrong number of bytes: %d expected %d", n, len(send)) } if err != nil { t.Fatal("Couldn't read from stream", err) } if !bytes.Equal(buf, send) { t.Fatal("Got wrong message from stream", string(buf)) } // Try interrupting the stream mid-read go func() { time.Sleep(50 * time.Millisecond) interrupt <- struct{}{} }() n, err = reader.Read(buf) if err != ErrInterrupted { t.Fatal("Read returned wrong error after interrupt:", err) } if n != 0 { t.Fatalf("Read still returned data after interrput: %d bytes", n) } // Try writing again after the channel was interrupted go writer.Write(send) n, err = reader.Read(buf) if n != len(send) { t.Fatalf("Read wrong number of bytes: %d expected %d", n, len(send)) } if err != nil { t.Fatal("Couldn't read from stream", err) } if !bytes.Equal(buf, send) { t.Fatal("Got wrong message from stream", string(buf)) } } toxiproxy-2.0.0/testing/000077500000000000000000000000001270742716200152635ustar00rootroot00000000000000toxiproxy-2.0.0/testing/benchmark_test.go000066400000000000000000000045041270742716200206060ustar00rootroot00000000000000package main import ( "io/ioutil" "net/http" "testing" ) // Benchmark numbers: // // Toxiproxy 1.1 // // 1x Toxic Types: // BenchmarkDirect 3000 588148 ns/op // BenchmarkProxy 2000 999949 ns/op // BenchmarkDirectSmall 5000 291324 ns/op // BenchmarkProxySmall 3000 504501 ns/op // // 10x Toxic Types: // BenchmarkDirect 3000 599519 ns/op // BenchmarkProxy 2000 1044746 ns/op // BenchmarkDirectSmall 5000 280713 ns/op // BenchmarkProxySmall 3000 574816 ns/op // // Toxiproxy 2.0 // // No Enabled Toxics: // BenchmarkDirect 2000 597998 ns/op // BenchmarkProxy 2000 964510 ns/op // BenchmarkDirectSmall 10000 287448 ns/op // BenchmarkProxySmall 5000 560694 ns/op // Test the backend server directly, use 64k random endpoint func BenchmarkDirect(b *testing.B) { client := http.Client{} for i := 0; i < b.N; i++ { resp, err := client.Get("http://localhost:20002/test1") if err != nil { b.Fatal(err) } _, err = ioutil.ReadAll(resp.Body) if err != nil { b.Fatal(err) } resp.Body.Close() } } // Test the backend through toxiproxy, use 64k random endpoint func BenchmarkProxy(b *testing.B) { client := http.Client{} for i := 0; i < b.N; i++ { resp, err := client.Get("http://localhost:20000/test1") if err != nil { b.Fatal(err) } _, err = ioutil.ReadAll(resp.Body) if err != nil { b.Fatal(err) } resp.Body.Close() } } // Test the backend server directly, use "hello world" endpoint func BenchmarkDirectSmall(b *testing.B) { client := http.Client{} for i := 0; i < b.N; i++ { resp, err := client.Get("http://localhost:20002/test2") if err != nil { b.Fatal(err) } _, err = ioutil.ReadAll(resp.Body) if err != nil { b.Fatal(err) } resp.Body.Close() } } // Test the backend through toxiproxy, use "hello world" endpoint func BenchmarkProxySmall(b *testing.B) { client := http.Client{} for i := 0; i < b.N; i++ { resp, err := client.Get("http://localhost:20000/test2") if err != nil { b.Fatal(err) } _, err = ioutil.ReadAll(resp.Body) if err != nil { b.Fatal(err) } resp.Body.Close() } } toxiproxy-2.0.0/testing/endpoint.go000066400000000000000000000014041270742716200174310ustar00rootroot00000000000000package main import ( "encoding/hex" "fmt" "net/http" ) var stuff []byte var out []byte var out2 []byte func handler1(w http.ResponseWriter, r *http.Request) { n, err := w.Write(out) if n != len(out) { fmt.Println("Short write!") } if err != nil { fmt.Println(err) } } func handler2(w http.ResponseWriter, r *http.Request) { n, err := w.Write(out2) if n != len(out2) { fmt.Println("Short write!") } if err != nil { fmt.Println(err) } } func main() { stuff = make([]byte, 32*1024) out = make([]byte, len(stuff)*2) out2 = []byte("hello world") for i := 0; i < len(stuff); i++ { stuff[i] = byte(i % 256) } hex.Encode(out, stuff) http.HandleFunc("/test1", handler1) http.HandleFunc("/test2", handler2) http.ListenAndServe(":20002", nil) } toxiproxy-2.0.0/toxic_collection.go000066400000000000000000000134031270742716200174770ustar00rootroot00000000000000package toxiproxy import ( "bytes" "encoding/json" "fmt" "io" "strings" "sync" "github.com/Shopify/toxiproxy/stream" "github.com/Shopify/toxiproxy/toxics" ) type ToxicCollection struct { sync.Mutex noop *toxics.ToxicWrapper proxy *Proxy chain [][]*toxics.ToxicWrapper toxics [][]*toxics.ToxicWrapper links map[string]*ToxicLink } func NewToxicCollection(proxy *Proxy) *ToxicCollection { collection := &ToxicCollection{ noop: &toxics.ToxicWrapper{ Toxic: new(toxics.NoopToxic), Type: "noop", }, proxy: proxy, chain: make([][]*toxics.ToxicWrapper, stream.NumDirections), toxics: make([][]*toxics.ToxicWrapper, stream.NumDirections), links: make(map[string]*ToxicLink), } for dir := range collection.chain { collection.chain[dir] = make([]*toxics.ToxicWrapper, 1, toxics.Count()+1) collection.chain[dir][0] = collection.noop collection.toxics[dir] = make([]*toxics.ToxicWrapper, 0, toxics.Count()) } return collection } func (c *ToxicCollection) ResetToxics() { c.Lock() defer c.Unlock() for dir := range c.toxics { for _, toxic := range c.toxics[dir] { // TODO do this in bulk c.chainRemoveToxic(toxic) } c.toxics[dir] = c.toxics[dir][:0] } } func (c *ToxicCollection) GetToxic(name string) *toxics.ToxicWrapper { c.Lock() defer c.Unlock() for dir := range c.toxics { for _, toxic := range c.toxics[dir] { if toxic.Name == name { return toxic } } } return nil } func (c *ToxicCollection) GetToxicArray() []toxics.Toxic { c.Lock() defer c.Unlock() result := make([]toxics.Toxic, 0) for dir := range c.toxics { for _, toxic := range c.toxics[dir] { result = append(result, toxic) } } return result } func (c *ToxicCollection) AddToxicJson(data io.Reader) (*toxics.ToxicWrapper, error) { c.Lock() defer c.Unlock() var buffer bytes.Buffer // Default to a downstream toxic with a toxicity of 1. wrapper := &toxics.ToxicWrapper{ Stream: "downstream", Toxicity: 1.0, Toxic: new(toxics.NoopToxic), } err := json.NewDecoder(io.TeeReader(data, &buffer)).Decode(wrapper) if err != nil { return nil, joinError(err, ErrBadRequestBody) } switch strings.ToLower(wrapper.Stream) { case "downstream": wrapper.Direction = stream.Downstream case "upstream": wrapper.Direction = stream.Upstream default: return nil, ErrInvalidStream } if wrapper.Name == "" { wrapper.Name = fmt.Sprintf("%s_%s", wrapper.Type, wrapper.Stream) } if toxics.New(wrapper) == nil { return nil, ErrInvalidToxicType } for dir := range c.toxics { for _, toxic := range c.toxics[dir] { if toxic.Name == wrapper.Name { return nil, ErrToxicAlreadyExists } } } // Parse attributes because we now know the toxics type. attrs := &struct { Attributes interface{} `json:"attributes"` }{ wrapper.Toxic, } err = json.NewDecoder(&buffer).Decode(attrs) if err != nil { return nil, joinError(err, ErrBadRequestBody) } c.toxics[wrapper.Direction] = append(c.toxics[wrapper.Direction], wrapper) c.chainAddToxic(wrapper) return wrapper, nil } func (c *ToxicCollection) UpdateToxicJson(name string, data io.Reader) (*toxics.ToxicWrapper, error) { c.Lock() defer c.Unlock() for dir := range c.toxics { for _, toxic := range c.toxics[dir] { if toxic.Name == name { attrs := &struct { Attributes interface{} `json:"attributes"` Toxicity float32 `json:"toxicity"` }{ toxic.Toxic, toxic.Toxicity, } err := json.NewDecoder(data).Decode(attrs) if err != nil { return nil, joinError(err, ErrBadRequestBody) } toxic.Toxicity = attrs.Toxicity c.chainUpdateToxic(toxic) return toxic, nil } } } return nil, ErrToxicNotFound } func (c *ToxicCollection) RemoveToxic(name string) error { c.Lock() defer c.Unlock() for dir := range c.toxics { for index, toxic := range c.toxics[dir] { if toxic.Name == name { c.toxics[dir] = append(c.toxics[dir][:index], c.toxics[dir][index+1:]...) c.chainRemoveToxic(toxic) return nil } } } return ErrToxicNotFound } func (c *ToxicCollection) StartLink(name string, input io.Reader, output io.WriteCloser, direction stream.Direction) { c.Lock() defer c.Unlock() link := NewToxicLink(c.proxy, c, direction) link.Start(name, input, output) c.links[name] = link } func (c *ToxicCollection) RemoveLink(name string) { c.Lock() defer c.Unlock() delete(c.links, name) } // All following functions assume the lock is already grabbed func (c *ToxicCollection) chainAddToxic(toxic *toxics.ToxicWrapper) { dir := toxic.Direction toxic.Index = len(c.chain[dir]) c.chain[dir] = append(c.chain[dir], toxic) // Asynchronously add the toxic to each link group := sync.WaitGroup{} for _, link := range c.links { if link.direction == dir { group.Add(1) go func(link *ToxicLink) { defer group.Done() link.AddToxic(toxic) }(link) } } group.Wait() } func (c *ToxicCollection) chainUpdateToxic(toxic *toxics.ToxicWrapper) { c.chain[toxic.Direction][toxic.Index] = toxic // Asynchronously update the toxic in each link group := sync.WaitGroup{} for _, link := range c.links { if link.direction == toxic.Direction { group.Add(1) go func(link *ToxicLink) { defer group.Done() link.UpdateToxic(toxic) }(link) } } group.Wait() } func (c *ToxicCollection) chainRemoveToxic(toxic *toxics.ToxicWrapper) { dir := toxic.Direction c.chain[dir] = append(c.chain[dir][:toxic.Index], c.chain[dir][toxic.Index+1:]...) for i := toxic.Index; i < len(c.chain[dir]); i++ { c.chain[dir][i].Index = i } // Asynchronously remove the toxic from each link group := sync.WaitGroup{} for _, link := range c.links { if link.direction == dir { group.Add(1) go func(link *ToxicLink) { defer group.Done() link.RemoveToxic(toxic) }(link) } } group.Wait() toxic.Index = -1 } toxiproxy-2.0.0/toxics/000077500000000000000000000000001270742716200151175ustar00rootroot00000000000000toxiproxy-2.0.0/toxics/bandwidth.go000066400000000000000000000025351270742716200174170ustar00rootroot00000000000000package toxics import ( "time" "github.com/Shopify/toxiproxy/stream" ) // The BandwidthToxic passes data through at a limited rate type BandwidthToxic struct { // Rate in KB/s Rate int64 `json:"rate"` } func (t *BandwidthToxic) Pipe(stub *ToxicStub) { var sleep time.Duration = 0 for { select { case <-stub.Interrupt: return case p := <-stub.Input: if p == nil { stub.Close() return } if t.Rate <= 0 { sleep = 0 } else { sleep += time.Duration(len(p.Data)) * time.Millisecond / time.Duration(t.Rate) } // If the rate is low enough, split the packet up and send in 100 millisecond intervals for int64(len(p.Data)) > t.Rate*100 { select { case <-time.After(100 * time.Millisecond): stub.Output <- &stream.StreamChunk{p.Data[:t.Rate*100], p.Timestamp} p.Data = p.Data[t.Rate*100:] sleep -= 100 * time.Millisecond case <-stub.Interrupt: stub.Output <- p // Don't drop any data on the floor return } } start := time.Now() select { case <-time.After(sleep): // time.After only seems to have ~1ms prevision, so offset the next sleep by the error sleep -= time.Since(start) stub.Output <- p case <-stub.Interrupt: stub.Output <- p // Don't drop any data on the floor return } } } } func init() { Register("bandwidth", new(BandwidthToxic)) } toxiproxy-2.0.0/toxics/bandwidth_test.go000066400000000000000000000050051270742716200204510ustar00rootroot00000000000000package toxics_test import ( "bytes" "io" "net" "strings" "testing" "time" "github.com/Shopify/toxiproxy/toxics" ) func TestBandwidthToxic(t *testing.T) { ln, err := net.Listen("tcp", "localhost:0") if err != nil { t.Fatal("Failed to create TCP server", err) } defer ln.Close() proxy := NewTestProxy("test", ln.Addr().String()) proxy.Start() defer proxy.Stop() serverConnRecv := make(chan net.Conn) go func() { conn, err := ln.Accept() if err != nil { t.Error("Unable to accept TCP connection", err) } serverConnRecv <- conn }() conn, err := net.Dial("tcp", proxy.Listen) if err != nil { t.Error("Unable to dial TCP server", err) } serverConn := <-serverConnRecv rate := 1000 // 1MB/s proxy.Toxics.AddToxicJson(ToxicToJson(t, "", "bandwidth", "upstream", &toxics.BandwidthToxic{Rate: int64(rate)})) buf := []byte(strings.Repeat("hello world ", 40000)) // 480KB go func() { n, err := conn.Write(buf) conn.Close() if n != len(buf) || err != nil { t.Errorf("Failed to write buffer: (%d == %d) %v", n, len(buf), err) } }() buf2 := make([]byte, len(buf)) start := time.Now() _, err = io.ReadAtLeast(serverConn, buf2, len(buf2)) if err != nil { t.Errorf("Proxy read failed: %v", err) } else if bytes.Compare(buf, buf2) != 0 { t.Errorf("Server did not read correct buffer from client!") } AssertDeltaTime(t, "Bandwidth", time.Since(start), time.Duration(len(buf))*time.Second/time.Duration(rate*1000), 10*time.Millisecond, ) } func BenchmarkBandwidthToxic100MB(b *testing.B) { ln, err := net.Listen("tcp", "localhost:0") if err != nil { b.Fatal("Failed to create TCP server", err) } defer ln.Close() proxy := NewTestProxy("test", ln.Addr().String()) proxy.Start() defer proxy.Stop() buf := []byte(strings.Repeat("hello world ", 1000)) go func() { conn, err := ln.Accept() if err != nil { b.Error("Unable to accept TCP connection", err) } buf2 := make([]byte, len(buf)) for err == nil { _, err = conn.Read(buf2) } }() conn, err := net.Dial("tcp", proxy.Listen) if err != nil { b.Error("Unable to dial TCP server", err) } proxy.Toxics.AddToxicJson(ToxicToJson(nil, "", "bandwidth", "upstream", &toxics.BandwidthToxic{Rate: 100 * 1000})) b.SetBytes(int64(len(buf))) b.ReportAllocs() b.ResetTimer() for i := 0; i < b.N; i++ { n, err := conn.Write(buf) if err != nil || n != len(buf) { b.Errorf("%v, %d == %d", err, n, len(buf)) break } } err = conn.Close() if err != nil { b.Error("Failed to close TCP connection", err) } } toxiproxy-2.0.0/toxics/latency.go000066400000000000000000000021041270742716200171020ustar00rootroot00000000000000package toxics import ( "math/rand" "time" ) // The LatencyToxic passes data through with the a delay of latency +/- jitter added. type LatencyToxic struct { // Times in milliseconds Latency int64 `json:"latency"` Jitter int64 `json:"jitter"` } func (t *LatencyToxic) GetBufferSize() int { return 1024 } func (t *LatencyToxic) delay() time.Duration { // Delay = t.Latency +/- t.Jitter delay := t.Latency jitter := int64(t.Jitter) if jitter > 0 { delay += rand.Int63n(jitter*2) - jitter } return time.Duration(delay) * time.Millisecond } func (t *LatencyToxic) Pipe(stub *ToxicStub) { for { select { case <-stub.Interrupt: return case c := <-stub.Input: if c == nil { stub.Close() return } sleep := t.delay() - time.Since(c.Timestamp) select { case <-time.After(sleep): c.Timestamp = c.Timestamp.Add(sleep) stub.Output <- c case <-stub.Interrupt: // Exit fast without applying latency. stub.Output <- c // Don't drop any data on the floor return } } } } func init() { Register("latency", new(LatencyToxic)) } toxiproxy-2.0.0/toxics/latency_test.go000066400000000000000000000146011270742716200201460ustar00rootroot00000000000000package toxics_test import ( "bufio" "bytes" "io" "net" "strconv" "strings" "testing" "time" "github.com/Shopify/toxiproxy" "github.com/Shopify/toxiproxy/toxics" ) func AssertDeltaTime(t *testing.T, message string, actual, expected, delta time.Duration) { diff := actual - expected if diff < 0 { diff *= -1 } if diff > delta { t.Errorf("[%s] Time was more than %v off: got %v expected %v", message, delta, actual, expected) } else { t.Logf("[%s] Time was correct: %v (expected %v)", message, actual, expected) } } func DoLatencyTest(t *testing.T, upLatency, downLatency *toxics.LatencyToxic) { WithEchoProxy(t, func(conn net.Conn, response chan []byte, proxy *toxiproxy.Proxy) { if upLatency == nil { upLatency = &toxics.LatencyToxic{} } else { _, err := proxy.Toxics.AddToxicJson(ToxicToJson(t, "latency_up", "latency", "upstream", upLatency)) if err != nil { t.Error("AddToxicJson returned error:", err) } } if downLatency == nil { downLatency = &toxics.LatencyToxic{} } else { _, err := proxy.Toxics.AddToxicJson(ToxicToJson(t, "latency_down", "latency", "downstream", downLatency)) if err != nil { t.Error("AddToxicJson returned error:", err) } } t.Logf("Using latency: Up: %dms +/- %dms, Down: %dms +/- %dms", upLatency.Latency, upLatency.Jitter, downLatency.Latency, downLatency.Jitter) msg := []byte("hello world " + strings.Repeat("a", 32*1024) + "\n") timer := time.Now() _, err := conn.Write(msg) if err != nil { t.Error("Failed writing to TCP server", err) } resp := <-response if !bytes.Equal(resp, msg) { t.Error("Server didn't read correct bytes from client:", string(resp)) } AssertDeltaTime(t, "Server read", time.Since(timer), time.Duration(upLatency.Latency)*time.Millisecond, time.Duration(upLatency.Jitter+10)*time.Millisecond, ) timer2 := time.Now() scan := bufio.NewScanner(conn) if scan.Scan() { resp = append(scan.Bytes(), '\n') if !bytes.Equal(resp, msg) { t.Error("Client didn't read correct bytes from server:", string(resp)) } } AssertDeltaTime(t, "Client read", time.Since(timer2), time.Duration(downLatency.Latency)*time.Millisecond, time.Duration(downLatency.Jitter+10)*time.Millisecond, ) AssertDeltaTime(t, "Round trip", time.Since(timer), time.Duration(upLatency.Latency+downLatency.Latency)*time.Millisecond, time.Duration(upLatency.Jitter+downLatency.Jitter+10)*time.Millisecond, ) proxy.Toxics.RemoveToxic("latency_up") proxy.Toxics.RemoveToxic("latency_down") err = conn.Close() if err != nil { t.Error("Failed to close TCP connection", err) } }) } func TestUpstreamLatency(t *testing.T) { DoLatencyTest(t, &toxics.LatencyToxic{Latency: 100}, nil) } func TestDownstreamLatency(t *testing.T) { DoLatencyTest(t, nil, &toxics.LatencyToxic{Latency: 100}) } func TestFullstreamLatencyEven(t *testing.T) { DoLatencyTest(t, &toxics.LatencyToxic{Latency: 100}, &toxics.LatencyToxic{Latency: 100}) } func TestFullstreamLatencyBiasUp(t *testing.T) { DoLatencyTest(t, &toxics.LatencyToxic{Latency: 1000}, &toxics.LatencyToxic{Latency: 100}) } func TestFullstreamLatencyBiasDown(t *testing.T) { DoLatencyTest(t, &toxics.LatencyToxic{Latency: 100}, &toxics.LatencyToxic{Latency: 1000}) } func TestZeroLatency(t *testing.T) { DoLatencyTest(t, &toxics.LatencyToxic{Latency: 0}, &toxics.LatencyToxic{Latency: 0}) } func TestLatencyToxicCloseRace(t *testing.T) { ln, err := net.Listen("tcp", "localhost:0") if err != nil { t.Fatal("Failed to create TCP server", err) } defer ln.Close() proxy := NewTestProxy("test", ln.Addr().String()) proxy.Start() defer proxy.Stop() go func() { for { _, err := ln.Accept() if err != nil { return } } }() // Check for potential race conditions when interrupting toxics for i := 0; i < 1000; i++ { proxy.Toxics.AddToxicJson(ToxicToJson(t, "", "latency", "upstream", &toxics.LatencyToxic{Latency: 10})) conn, err := net.Dial("tcp", proxy.Listen) if err != nil { t.Error("Unable to dial TCP server", err) } conn.Write([]byte("hello")) conn.Close() proxy.Toxics.RemoveToxic("latency") } } func TestTwoLatencyToxics(t *testing.T) { WithEchoProxy(t, func(conn net.Conn, response chan []byte, proxy *toxiproxy.Proxy) { toxics := []*toxics.LatencyToxic{&toxics.LatencyToxic{Latency: 500}, &toxics.LatencyToxic{Latency: 500}} for i, toxic := range toxics { _, err := proxy.Toxics.AddToxicJson(ToxicToJson(t, "latency_"+strconv.Itoa(i), "latency", "upstream", toxic)) if err != nil { t.Error("AddToxicJson returned error:", err) } } msg := []byte("hello world " + strings.Repeat("a", 32*1024) + "\n") timer := time.Now() _, err := conn.Write(msg) if err != nil { t.Error("Failed writing to TCP server", err) } resp := <-response if !bytes.Equal(resp, msg) { t.Error("Server didn't read correct bytes from client:", string(resp)) } AssertDeltaTime(t, "Upstream two latency toxics", time.Since(timer), time.Duration(1000)*time.Millisecond, time.Duration(10)*time.Millisecond, ) for i := range toxics { proxy.Toxics.RemoveToxic("latency_" + strconv.Itoa(i)) } err = conn.Close() if err != nil { t.Error("Failed to close TCP connection", err) } }) } func TestLatencyToxicBandwidth(t *testing.T) { ln, err := net.Listen("tcp", "localhost:0") if err != nil { t.Fatal("Failed to create TCP server", err) } defer ln.Close() proxy := NewTestProxy("test", ln.Addr().String()) proxy.Start() defer proxy.Stop() buf := []byte(strings.Repeat("hello world ", 1000)) go func() { conn, err := ln.Accept() if err != nil { t.Error("Unable to accept TCP connection", err) } for err == nil { _, err = conn.Write(buf) } }() conn, err := net.Dial("tcp", proxy.Listen) if err != nil { t.Error("Unable to dial TCP server", err) } proxy.Toxics.AddToxicJson(ToxicToJson(t, "", "latency", "", &toxics.LatencyToxic{Latency: 100})) time.Sleep(150 * time.Millisecond) // Wait for latency toxic buf2 := make([]byte, len(buf)) start := time.Now() count := 0 for i := 0; i < 100; i++ { n, err := io.ReadFull(conn, buf2) count += n if err != nil { t.Error(err) break } } // Assert the transfer was at least 100MB/s AssertDeltaTime(t, "Latency toxic bandwidth", time.Since(start), 0, time.Duration(count/100000)*time.Millisecond) err = conn.Close() if err != nil { t.Error("Failed to close TCP connection", err) } } toxiproxy-2.0.0/toxics/noop.go000066400000000000000000000005521270742716200164230ustar00rootroot00000000000000package toxics // The NoopToxic passes all data through without any toxic effects. type NoopToxic struct{} func (t *NoopToxic) Pipe(stub *ToxicStub) { for { select { case <-stub.Interrupt: return case c := <-stub.Input: if c == nil { stub.Close() return } stub.Output <- c } } } func init() { Register("noop", new(NoopToxic)) } toxiproxy-2.0.0/toxics/slicer.go000066400000000000000000000042321270742716200167300ustar00rootroot00000000000000package toxics import ( "math/rand" "time" "github.com/Shopify/toxiproxy/stream" ) // The SlicerToxic slices data into multiple smaller packets // to simulate real-world TCP behaviour. type SlicerToxic struct { // Average number of bytes to slice at AverageSize int `json:"average_size"` // +/- bytes to vary sliced amounts. Must be less than // the average size SizeVariation int `json:"size_variation"` // Microseconds to delay each packet. May be useful since there's // usually some kind of buffering of network data Delay int `json:"delay"` } // Returns a list of chunk offsets to slice up a packet of the // given total size. For example, for a size of 100, output might be: // // []int{0, 18, 18, 43, 43, 67, 67, 77, 77, 100} // ^---^ ^----^ ^----^ ^----^ ^-----^ // // This tries to get fairly evenly-varying chunks (no tendency // to have a small/large chunk at the start/end). func (t *SlicerToxic) chunk(start int, end int) []int { // Base case: // If the size is within the random varation, _or already // less than the average size_, just return it. // Otherwise split the chunk in about two, and recurse. if (end-start)-t.AverageSize <= t.SizeVariation { return []int{start, end} } // +1 in the size variation to offset favoring of smaller // numbers by integer division mid := start + (end-start)/2 + (rand.Intn(t.SizeVariation*2) - t.SizeVariation) + rand.Intn(1) left := t.chunk(start, mid) right := t.chunk(mid, end) return append(left, right...) } func (t *SlicerToxic) Pipe(stub *ToxicStub) { for { select { case <-stub.Interrupt: return case c := <-stub.Input: if c == nil { stub.Close() return } chunks := t.chunk(0, len(c.Data)) for i := 1; i < len(chunks); i += 2 { stub.Output <- &stream.StreamChunk{ Data: c.Data[chunks[i-1]:chunks[i]], Timestamp: c.Timestamp, } select { case <-stub.Interrupt: stub.Output <- &stream.StreamChunk{ Data: c.Data[chunks[i]:], Timestamp: c.Timestamp, } return case <-time.After(time.Duration(t.Delay) * time.Microsecond): } } } } } func init() { Register("slicer", new(SlicerToxic)) } toxiproxy-2.0.0/toxics/slicer_test.go000066400000000000000000000021301270742716200177620ustar00rootroot00000000000000package toxics_test import ( "bytes" "strings" "testing" "time" "github.com/Shopify/toxiproxy/stream" "github.com/Shopify/toxiproxy/toxics" ) func TestSlicerToxic(t *testing.T) { data := []byte(strings.Repeat("hello world ", 40000)) // 480 kb slicer := &toxics.SlicerToxic{AverageSize: 1024, SizeVariation: 512, Delay: 10} input := make(chan *stream.StreamChunk) output := make(chan *stream.StreamChunk) stub := toxics.NewToxicStub(input, output) done := make(chan bool) go func() { slicer.Pipe(stub) done <- true }() defer func() { close(input) for { select { case <-done: return case <-output: } } }() input <- &stream.StreamChunk{Data: data} buf := make([]byte, 0, len(data)) reads := 0 L: for { select { case c := <-output: reads++ buf = append(buf, c.Data...) case <-time.After(10 * time.Millisecond): break L } } if reads < 480/2 || reads > 480/2+480 { t.Errorf("Expected to read about 480 times, but read %d times.", reads) } if bytes.Compare(buf, data) != 0 { t.Errorf("Server did not read correct buffer from client!") } } toxiproxy-2.0.0/toxics/slow_close.go000066400000000000000000000011441270742716200176170ustar00rootroot00000000000000package toxics import "time" // The SlowCloseToxic stops the TCP connection from closing until after a delay. type SlowCloseToxic struct { // Times in milliseconds Delay int64 `json:"delay"` } func (t *SlowCloseToxic) Pipe(stub *ToxicStub) { for { select { case <-stub.Interrupt: return case c := <-stub.Input: if c == nil { delay := time.Duration(t.Delay) * time.Millisecond select { case <-time.After(delay): stub.Close() return case <-stub.Interrupt: return } } stub.Output <- c } } } func init() { Register("slow_close", new(SlowCloseToxic)) } toxiproxy-2.0.0/toxics/timeout.go000066400000000000000000000011551270742716200171360ustar00rootroot00000000000000package toxics import "time" // The TimeoutToxic stops any data from flowing through, and will close the connection after a timeout. // If the timeout is set to 0, then the connection will not be closed. type TimeoutToxic struct { // Times in milliseconds Timeout int64 `json:"timeout"` } func (t *TimeoutToxic) Pipe(stub *ToxicStub) { timeout := time.Duration(t.Timeout) * time.Millisecond if timeout > 0 { select { case <-time.After(timeout): stub.Close() return case <-stub.Interrupt: return } } else { <-stub.Interrupt return } } func init() { Register("timeout", new(TimeoutToxic)) } toxiproxy-2.0.0/toxics/toxic.go000066400000000000000000000061611270742716200166000ustar00rootroot00000000000000package toxics import ( "math/rand" "reflect" "sync" "github.com/Shopify/toxiproxy/stream" ) // A Toxic is something that can be attatched to a link to modify the way // data can be passed through (for example, by adding latency) // // Toxic // v // Client <-> ToxicStub <-> Upstream // // Toxic's work in a pipeline fashion, and can be chained together // with channels. The toxic itself only defines the settings and // Pipe() function definition, and uses the ToxicStub struct to store // per-connection information. This allows the same toxic to be used // for multiple connections. type Toxic interface { // Defines how packets flow through a ToxicStub. Pipe() blocks until the link is closed or interrupted. Pipe(*ToxicStub) } type BufferedToxic interface { // Defines the size of buffer this toxic should use GetBufferSize() int } type ToxicWrapper struct { Toxic `json:"attributes"` Name string `json:"name"` Type string `json:"type"` Stream string `json:"stream"` Toxicity float32 `json:"toxicity"` Direction stream.Direction `json:"-"` Index int `json:"-"` BufferSize int `json:"-"` } type ToxicStub struct { Input <-chan *stream.StreamChunk Output chan<- *stream.StreamChunk Interrupt chan struct{} running chan struct{} closed chan struct{} } func NewToxicStub(input <-chan *stream.StreamChunk, output chan<- *stream.StreamChunk) *ToxicStub { return &ToxicStub{ Interrupt: make(chan struct{}), closed: make(chan struct{}), Input: input, Output: output, } } // Begin running a toxic on this stub, can be interrupted. // Runs a noop toxic randomly depending on toxicity func (s *ToxicStub) Run(toxic *ToxicWrapper) { s.running = make(chan struct{}) defer close(s.running) if rand.Float32() < toxic.Toxicity { toxic.Pipe(s) } else { new(NoopToxic).Pipe(s) } } // Interrupt the flow of data so that the toxic controlling the stub can be replaced. // Returns true if the stream was successfully interrupted, or false if the stream is closed. func (s *ToxicStub) InterruptToxic() bool { select { case <-s.closed: return false case s.Interrupt <- struct{}{}: <-s.running // Wait for the running toxic to exit return true } } func (s *ToxicStub) Close() { close(s.closed) close(s.Output) } var ToxicRegistry map[string]Toxic var registryMutex sync.RWMutex func Register(typeName string, toxic Toxic) { registryMutex.Lock() defer registryMutex.Unlock() if ToxicRegistry == nil { ToxicRegistry = make(map[string]Toxic) } ToxicRegistry[typeName] = toxic } func New(wrapper *ToxicWrapper) Toxic { registryMutex.RLock() defer registryMutex.RUnlock() orig, ok := ToxicRegistry[wrapper.Type] if !ok { return nil } wrapper.Toxic = reflect.New(reflect.TypeOf(orig).Elem()).Interface().(Toxic) if buffered, ok := wrapper.Toxic.(BufferedToxic); ok { wrapper.BufferSize = buffered.GetBufferSize() } else { wrapper.BufferSize = 0 } return wrapper.Toxic } func Count() int { registryMutex.RLock() defer registryMutex.RUnlock() return len(ToxicRegistry) } toxiproxy-2.0.0/toxics/toxic_test.go000066400000000000000000000154761270742716200176500ustar00rootroot00000000000000package toxics_test import ( "bufio" "bytes" "encoding/json" "io" "net" "strings" "testing" "time" "github.com/Shopify/toxiproxy" "github.com/Shopify/toxiproxy/toxics" "github.com/Sirupsen/logrus" "gopkg.in/tomb.v1" ) func init() { logrus.SetLevel(logrus.FatalLevel) } func NewTestProxy(name, upstream string) *toxiproxy.Proxy { proxy := toxiproxy.NewProxy() proxy.Name = name proxy.Listen = "localhost:0" proxy.Upstream = upstream return proxy } func WithEchoServer(t *testing.T, f func(string, chan []byte)) { ln, err := net.Listen("tcp", "localhost:0") if err != nil { t.Fatal("Failed to create TCP server", err) } defer ln.Close() response := make(chan []byte, 1) tomb := tomb.Tomb{} go func() { defer tomb.Done() src, err := ln.Accept() if err != nil { select { case <-tomb.Dying(): default: t.Fatal("Failed to accept client") } return } ln.Close() scan := bufio.NewScanner(src) if scan.Scan() { received := append(scan.Bytes(), '\n') response <- received src.Write(received) } }() f(ln.Addr().String(), response) tomb.Killf("Function body finished") ln.Close() tomb.Wait() close(response) } func WithEchoProxy(t *testing.T, f func(proxy net.Conn, response chan []byte, proxyServer *toxiproxy.Proxy)) { WithEchoServer(t, func(upstream string, response chan []byte) { proxy := NewTestProxy("test", upstream) proxy.Start() conn, err := net.Dial("tcp", proxy.Listen) if err != nil { t.Error("Unable to dial TCP server", err) } f(conn, response, proxy) proxy.Stop() }) } func ToxicToJson(t *testing.T, name, typeName, stream string, toxic toxics.Toxic) io.Reader { data := map[string]interface{}{ "name": name, "type": typeName, "stream": stream, "attributes": toxic, } request, err := json.Marshal(data) if err != nil { t.Errorf("Failed to marshal toxic for api (1): %v", toxic) } return bytes.NewReader(request) } func AssertEchoResponse(t *testing.T, client, server net.Conn) { msg := []byte("hello world\n") _, err := client.Write(msg) if err != nil { t.Error("Failed writing to TCP server", err) } scan := bufio.NewScanner(server) if !scan.Scan() { t.Error("Client unexpectedly closed connection") } resp := append(scan.Bytes(), '\n') if !bytes.Equal(resp, msg) { t.Error("Server didn't read correct bytes from client:", string(resp)) } _, err = server.Write(resp) if err != nil { t.Error("Failed writing to TCP client", err) } scan = bufio.NewScanner(client) if !scan.Scan() { t.Error("Server unexpectedly closed connection") } resp = append(scan.Bytes(), '\n') if !bytes.Equal(resp, msg) { t.Error("Client didn't read correct bytes from server:", string(resp)) } } func TestPersistentConnections(t *testing.T) { ln, err := net.Listen("tcp", "localhost:0") if err != nil { t.Fatal("Failed to create TCP server", err) } defer ln.Close() proxy := NewTestProxy("test", ln.Addr().String()) proxy.Start() defer proxy.Stop() serverConnRecv := make(chan net.Conn) go func() { conn, err := ln.Accept() if err != nil { t.Error("Unable to accept TCP connection", err) } serverConnRecv <- conn }() conn, err := net.Dial("tcp", proxy.Listen) if err != nil { t.Error("Unable to dial TCP server", err) } serverConn := <-serverConnRecv proxy.Toxics.AddToxicJson(ToxicToJson(t, "noop_up", "noop", "upstream", &toxics.NoopToxic{})) proxy.Toxics.AddToxicJson(ToxicToJson(t, "noop_down", "noop", "downstream", &toxics.NoopToxic{})) AssertEchoResponse(t, conn, serverConn) proxy.Toxics.ResetToxics() AssertEchoResponse(t, conn, serverConn) proxy.Toxics.ResetToxics() AssertEchoResponse(t, conn, serverConn) err = conn.Close() if err != nil { t.Error("Failed to close TCP connection", err) } } func TestToxicAddRemove(t *testing.T) { ln, err := net.Listen("tcp", "localhost:0") if err != nil { t.Fatal("Failed to create TCP server", err) } defer ln.Close() proxy := NewTestProxy("test", ln.Addr().String()) proxy.Start() defer proxy.Stop() serverConnRecv := make(chan net.Conn) go func() { conn, err := ln.Accept() if err != nil { t.Error("Unable to accept TCP connection", err) } serverConnRecv <- conn }() conn, err := net.Dial("tcp", proxy.Listen) if err != nil { t.Error("Unable to dial TCP server", err) } serverConn := <-serverConnRecv running := make(chan struct{}) go func() { enabled := false for { select { case <-running: return default: if enabled { proxy.Toxics.AddToxicJson(ToxicToJson(t, "noop_up", "noop", "upstream", &toxics.NoopToxic{})) proxy.Toxics.RemoveToxic("noop_down") } else { proxy.Toxics.RemoveToxic("noop_up") proxy.Toxics.AddToxicJson(ToxicToJson(t, "noop_down", "noop", "downstream", &toxics.NoopToxic{})) } enabled = !enabled } } }() for i := 0; i < 100; i++ { AssertEchoResponse(t, conn, serverConn) } close(running) err = conn.Close() if err != nil { t.Error("Failed to close TCP connection", err) } } func TestProxyLatency(t *testing.T) { ln, err := net.Listen("tcp", "localhost:0") if err != nil { t.Fatal("Failed to create TCP server", err) } defer ln.Close() proxy := NewTestProxy("test", ln.Addr().String()) proxy.Start() defer proxy.Stop() serverConnRecv := make(chan net.Conn) go func() { conn, err := ln.Accept() if err != nil { t.Error("Unable to accept TCP connection", err) } serverConnRecv <- conn }() conn, err := net.Dial("tcp", proxy.Listen) if err != nil { t.Error("Unable to dial TCP server", err) } serverConn := <-serverConnRecv start := time.Now() for i := 0; i < 100; i++ { AssertEchoResponse(t, conn, serverConn) } latency := time.Since(start) / 200 if latency > 300*time.Microsecond { t.Errorf("Average proxy latency > 300µs (%v)", latency) } else { t.Logf("Average proxy latency: %v", latency) } err = conn.Close() if err != nil { t.Error("Failed to close TCP connection", err) } } func BenchmarkProxyBandwidth(b *testing.B) { ln, err := net.Listen("tcp", "localhost:0") if err != nil { b.Fatal("Failed to create TCP server", err) } defer ln.Close() proxy := NewTestProxy("test", ln.Addr().String()) proxy.Start() defer proxy.Stop() buf := []byte(strings.Repeat("hello world ", 1000)) go func() { conn, err := ln.Accept() if err != nil { b.Error("Unable to accept TCP connection", err) } buf2 := make([]byte, len(buf)) for err == nil { _, err = conn.Read(buf2) } }() conn, err := net.Dial("tcp", proxy.Listen) if err != nil { b.Error("Unable to dial TCP server", err) } b.SetBytes(int64(len(buf))) b.ReportAllocs() b.ResetTimer() for i := 0; i < b.N; i++ { n, err := conn.Write(buf) if err != nil || n != len(buf) { b.Errorf("%v, %d == %d", err, n, len(buf)) break } } err = conn.Close() if err != nil { b.Error("Failed to close TCP connection", err) } } toxiproxy-2.0.0/version.go000066400000000000000000000000471270742716200156230ustar00rootroot00000000000000package toxiproxy var Version = "git"