NAME¶
Perlbal::Plugin::Throttle - Perlbal plugin that throttles connections from hosts
that connect too frequently.
SYNOPSIS¶
# in perlbal.conf
LOAD Throttle
CREATE POOL web
POOL web ADD 10.0.0.1:80
CREATE SERVICE throttler
SET role = reverse_proxy
SET listen = 0.0.0.0:80
SET pool = web
# adjust throttler aggressiveness
SET initial_delay = 10
SET max_delay = 60
SET throttle_threshold_seconds = 3
SET max_concurrent = 2
SET ban_threshold = 4
SET ban_expiration = 180
# limit which requests are throttled
SET path_regex = ^/webapp/
SET method_regex = ^GET$
# allow or ban specific addresses or range (requires Net::CIDR::Lite)
SET whitelist_file = conf/whitelist.txt
SET blacklist_file = conf/blacklist.txt
# granular logging (requires Perlbal::Plugin::Syslogger)
SET log_events = ban,unban,throttled,banned
SET log_only = false
# share state between perlbals (requires Cache::Memcached::Async)
SET memcached_servers = 10.0.2.1:11211,10.0.2.2:11211
SET memcached_async_clients = 4
SET instance_name = mywebapp
SET plugins = Throttle
ENABLE throttler
DESCRIPTION¶
This plugin intercepts HTTP requests to a Perlbal service and slows or drops
connections from IP addresses which are determined to be connecting too fast.
BEHAVIOR¶
An IP address address may be in one of four states depending on its recent
activity; that state determines how new requests from the IP are handled:
- •
- allowed
An IP begins in the allowed state. When a request is received from an
IP in this state, the request is handled immediately and the IP enters the
probation state.
- •
- probation
If no requests are received from an IP in the probation state for
throttle_threshold_seconds, it returns to the allowed state.
When a new request is received from an IP in the probation state, the
IP enters the throttled state and is assigned a delay
property initially equal to initial_delay. Connection to a backend
is postponed for delay seconds while perlbal continues to work. If
the connection is still open after the delay, the request is then handled
normally. A dropped connection does not change the IP's delay
value.
- •
- throttled
If no requests are received from an IP in the throttled state for
delay seconds, it returns to the probation state.
When a new request is received from an IP in the throttled state, its
violations property is incremented, and its delay property
is doubled (up to a maximum of max_delay). The request is postponed
for the new value of delay.
Only after the most recently created connection from a given IP exits the
throttled state do violations and delay reset to 0.
Furthermore, if the violations exceeds ban_threshold, the
connection is closed and the IP moves to the banned state.
IPs in the throttled state may have no more than
max_concurrent connections being delayed at once. Any additional
requests received in that circumstance are sent a "503 Too many
connections" response. Long-running requests which have already been
connected to a backend do not count towards this limit.
- •
- banned
New connections from IPs in the banned state are immediately closed with a
403 error response.
An IP leaves the banned state after ban_expiration seconds
have elapsed.
FEATURES¶
- •
- IP whitelist
Connections from IPs/CIDRs listed in the file specified by
whitelist_file are always allowed.
- •
- IP blacklist
Connections from IPs/CIDRs listed in the file specified by
blacklist_file immediately sent a "403 Forbidden"
response.
- •
- Flexible attack response
For services where throttling should not normally be enabled, use the
default_action tunable. When default_action is set to
"allow", new connections from non-white/blacklisted IPs will not
be throttled.
Furthermore, if throttling should only apply to specific clients, set
blacklist_action to "throttle". Blacklisted connections
will then be throttled instead of denied.
- •
- Dynamic configuration
Most service tunables may be updated from the management port, after which
the new values will be respected (although see "CAVEATS"). To
reload the whitelist and blacklist files, issue the throttle reload
whitelist or throttle reload blacklist command to the
service.
- •
- Path specificity
Throttling may be restricted to URI paths matching the path_regex
regex.
- •
- External shared state
The plugin stores state which IPs have been seen in a memcached(1)
instance. This allows many throttlers to share their state and also
minimizes memory use within the perlbal. If state exceeds the capacity of
the memcacheds, the least-recently seen IPs will be forgotten, effectively
resetting them to the allowed state.
Orthogonally, multiple throttlers which need to share memcacheds but not
state may specify distinct instance_name values.
- •
- Logging
If Perlbal::Plugin::Syslogger is installed and registered with the service,
Throttle can use it to send syslog messages regarding actions that are
taken. Granular control for which events are logged is available via the
log_events parameter. log_events is composed of one or more
of the following events, separated by commas:
- •
- ban
Log when a temporary local ban is added for an IP address.
- •
- unban
Log when a temporary local ban is removed for an IP address.
- •
- whitelisted
Log when a request is allowed because the source IP is on the
whitelist.
- •
- blacklisted
Log when a request is denied or throttled because the source IP is on the
blacklist.
- •
- banned
Log when a request is denied because the source IP is on the temporary ban
list for connecting excessively.
- •
- concurrent
Log when a request is denied because the source IP has too many open
connections waiting to be unthrottled.
- •
- throttled
Log when a request is throttled because the source IP was not on the
whitelist or blacklist.
- •
- all
Enables all the above logging options.
- •
- none
Disables all the above logging options.
CAVEATS¶
- •
- Dynamic configuration changes
Changes to certain service tunables will not be noticed until the
throttle reload config management command is issued. These
include log_events, path_regex, and method_regex).
Changes to certain other tunables will not be respected after the plugin has
been registered. These include memcached_servers and
memcached_async_clients.
- •
- List loading is blocking
The throttle reload whitelist and throttle reload blacklist
management commands load the whitelist and blacklist files synchronously,
which will cause the perlbal to hang until it completes.
- •
- Redirects
If a handled request returns a 30x response code and the redirect URI is
also throttled, then the client's attempt to follow the redirect will
necessarily be delayed by initial_delay. Fixing this would require
that the plugin inspect the HTTP response headers, which would incur a lot
of overhead. To workaround, try to have your backend not return 30x's if
both the original and redirect URI are proxied by the same throttler
instance (yes, this is difficult for the case where a backend 302s to add
a trailing / to a directory).
OPTIONAL DEPENDENCIES¶
- •
- Cache::Memcached::Async
Required for memcached support. This is the supported way to share state
between different perlbal instances.
- •
- Net::CIDR::Lite
Required for blacklist/whitelist support.
- •
- Perlbal::Plugin::Syslogger
Required for event logging support.
SEE ALSO¶
- •
- List of tunables in Throttle.pm.
TODO¶
- •
- Fix white/blacklist loading
Load CIDR lists asynchronously (perhaps in the manner of
Perlbal::Pool::_load_nodefile_async).
AUTHOR¶
Adam Thomason, <athomason@cpan.org>
COPYRIGHT AND LICENSE¶
Copyright (C) 2007-2011 by Say Media Inc, <cpan@sixapart.com>
This library is free software; you can redistribute it and/or modify it under
the same terms as Perl itself, either Perl version 5.8.6 or, at your option,
any later version of Perl 5 you may have available.