NAME¶
gdnsd-plugin-geoip - gdnsd meta-plugin for GSLB + failover via MaxMind's GeoIP
databases
SYNOPSIS¶
Minimal example gdnsd config file using this plugin:
plugins => { geoip => {
maps => {
my_prod_map => {
geoip_db => GeoIPCity.dat,
datacenters => [dc-03, dc-02, dc-01, dc-fail],
map => {
EU => {
DE => [dc-03, dc-01, dc-fail],
CH => [dc-01, dc-03, dc-fail]
},
NA => { MX => [dc-02, dc-fail] }
}
},
my_auto_map => {
geoip_db => GeoIPCityv6.dat,
datacenters => [dc1, dc2],
auto_dc_coords => {
dc1 => [ 38.9, -77 ],
dc2 => [ 50.1, 8.7 ],
}
}
},
resources => {
prod_www => {
map => my_prod_map
service_types => up
dcmap => {
dc-01 => 192.0.2.1,
dc-02 => { lb01 => 192.0.2.2, lb02 => 192.0.2.3 },
dc-03 => [ 192.0.2.4, 192.0.2.5, 192.0.2.6 ],
dc-fail => last.resort.cname.example.net.
}
}
corp_www => {
map => my_auto_map
dcmap => {
dc1 => 192.0.2.100,
dc2 => 192.0.2.101
}
}
}
}}
Example zonefile RRs:
www 600 DYNA geoip!prod_www
www-dc01 600 DYNA geoip!prod_www/dc-01
www.corp 600 DYNA geoip!corp_www
DESCRIPTION¶
gdnsd-plugin-geoip uses MaxMind's GeoIP binary databases to map address
and CNAME results based on geography and monitored service availability. It
fully supports both IPv6 and the emerging edns-client-subnet standard. If a
request contains the edns-client-subnet option with a source netmask greater
than zero, the edns-client-subnet information will be used instead of the
source IP of the request (the IP of the querying cache).
This plugin can operate in an automatic distance-based mode (using City-level
coordinate information rather than an external file and a Region-level db). It
can also operate coordinate-free and rely on the user to configure a
hierarchical map of cascading default user-location-to-datacenter mappings,
starting at the continent level.
The two modes can also be effectively mixed at geographic boundaries.
For each "map" you define (which maps geographic location codes to
preference-ordered lists of your datacenter locations), this plugin merges all
of the raw GeoIP subnets into the largest possible supernets which contain
identical responses in your configuration. These in turn are used to set
larger edns-client-subnet scope masks than you'd see simply returning raw
GeoIP results.
The documentation for
gdnsd-plugin-metafo(8) is required reading for
understanding the geoip plugin documentation here. The geoip plugin is an
exact superset of the metafo plugin, and re-uses almost all of the metafo
plugin's source code. Metafo does failover along a single, global, ordered
list of datacenters. What plugin_geoip adds on top of the functionality of
metafo is the ability to have the order of the datacenter failover list become
dynamic per-request based on geographic hints derived from the client's
network address.
FILE LOCATIONS¶
The configuration of this plugin can reference several external configuration
and/or data files. By default, all files referenced in this plugin's
configuration are loaded from the
geoip subdirectory of the daemon's
configuration directory (default
/etc/gdnsd). You can load from other
locations by specifying absolute file paths.
CONFIGURATION - TOP-LEVEL¶
The top level of the geoip plugin's configuration (i.e. "plugins => {
geoip => { ... } }") supports only two special keys, both of which are
required and expanded upon in detail in the next two sections:
"maps", and "resources". The "maps" section
defines one or more named mappings of location information from GeoIP binary
databases to ordered subsets of datacenter names. The "resources"
section defines one or more named resources, each of which references one of
the named maps and resolves datacenter names to specific sets of addresses or
CNAMEs.
Any other keys present at this level will be inherited down inside of each
per-resource hash inside the "resources" stanza, acting as
per-resource defaults for anything not defined explicitly there.
CONFIGURATION - MAPS¶
The "maps" stanza supports one special configuration key at the top
level:
"city_region_names = region_codes.csv"¶
String, filename, optional. GeoIP City databases use FIPS 10-4 codes for the
names of Regions outside of the US and Canada, and two-letter national alpha
codes within the US and Canada. For example the Geneve region of Switzerland
is identified as 07 in the database. By default you would have to use these
relatively confusing region codes in your hierarchical maps that make use of
Region-level information (e.g. "EU => { CH => { 07 => { Geneva
=> [ ... ] } } } }". If this option is specified, it points to a text
file that maps these FIPS codes to canonical, memorable full names for clearer
map configuration (e.g. "EU => { CH => { Geneve => { Geneva
=> [ ... ] } } } }". Note that while older versions of this data did
not map the US/Canadian two-letter alpha codes, newer versions do (e.g. TX
-> Texas).
This setting does not affect the GeoIP "Region" -format databases,
which have no region codes outside of the US and Canada, and always need the
two-letter alpha codes in the map.
The file format is a simple subset of the CSV format with 3 fields: ISO 3166-1
country code, FIPS 10-4 region code (or two-letter alpha in US/Canada), and
the region name in double-quotes. It is recommended you download this file
directly from MaxMind's reference copy in this format. As of this writing, it
is available from them at the following URL:
<
http://www.maxmind.com/download/geoip/misc/region_codes.csv>.
CONFIGURATION - PER-MAP¶
All other "maps"-level configuration keys are the names of the maps
you choose to define. A map, conceptually, is a mapping between geography
and/or network topology to varying ordered datacenter sub-sets. The value of
each named map must be a hash, and the following configuration keys apply
within:
"geoip_db = GeoIPv6.dat"¶
String, filename, optional. This is the filename of one of the supported MaxMind
GeoIP database types. It will be reloaded at runtime (without any significant
query interruptions) if a change to the database file is detected.
"geoip_db_v4_overlay = GeoIP.dat"¶
String, pathname, optional. This specifies an optional IPv4-level GeoIP database
to overwrite the IPv4 sub-space of the IPv6 data loaded from
"geoip_db". It must be a V4-format database, and
"geoip_db" must be defined as a V6-format database. In all other
respects, it is similar to "geoip_db".
As of this writing, MaxMind doesn't sell a commercial GeoIPv6 database. What
they offer are free IPv6 GeoLite database downloads, which include the IPv4
subset in the less-accurate GeoLite form. This option allows you to use these
GeoLitev6 databases for IPv6 coverage, and then overlay your paid commercial
GeoIPv4 data on top for more accurate IPv4 results.
"datacenters = [ one, two, three, ... ]"¶
Array of strings, required. This is the total set of datacenter names used by
this map. You must define at least one datacenter name (although 2 or more
would be infinitely more useful). At this time, there is a maximum limit of
254 datacenter names per map, although this could be raised if anyone requires
it. The order specified here is the fallback default result ordering in
various default cases (e.g. if no explicit top-level map default list is
given).
"city_no_region = true"¶
Boolean, default "false". If this key is set to "true" and
"geoip_db" references a City-level database, the Region-level
information within it will be completely ignored for mapping purposes. Your
hierarchical map structure will now be "continent => country =>
city" rather than "continent => country => region =>
city".
"nets = { ... }"¶
Key-value hash, optional (see below for alternate form). If specified, the
contents should be key-value pairs of "network/netmask" mapped to a
datacenter name (or an array of datacenter names). Any network-to-datacenter
mappings specified here will override mappings determined via GeoIP. Note that
it is illegal to specify networks in the IPv4-like subspaces of IPv6 other
than v4compat, but it is legal to specify actual IPv4 networks (which are
treated identically to v4compat). See the section on IPv4 Compatible Addresses
later in this document for more details. The order of the networks is
unimportant; they will always be sorted and inserted such that an entry which
is a subnet of another entry is not obliterated by the parent supernet.
nets => {
10.0.0.0/8 => [ dc1, dc2 ],
192.0.2.128/25 => dc3
2001:DB8::/32 => [ dc4, dc5, dc6 ],
}
In the case that one entry is a subnet of another with a different result
dclist, the entries are merged correctly such that the supernet surrounds the
subnet. In the case of an exact duplicate entry (or an effective one, after
merging smaller subnets) with a different dclist, it is arbitrary which one
"wins" and the condition is warned about. If you care about this
case, you should sanitize your nets data beforehand with an external tool
and/or parse for the warning message in log outputs.
"nets = nets_file_name"¶
String pathname, optional. A variant of the above, but the contents of the
key-value hash are loaded from the named external file. This makes life easier
for external tools and scripts generating large sets of nets entries (e.g.
from BGP data). The file will be monitored for changes and reloaded at runtime
much like the GeoIP databases.
"map = { ... }"¶
Key-value hash, optional. This is the heart of a named map which uses GeoIP: the
map itself, which maps places to ordered lists of datacenters. It requires
that "geoip_db" is also specified, and makes no sense without it.
This is a nested key-value hash. At each level, the keys are location codes
(continent, country, region, or city information depending on depth), and the
values are either an ordered datacenter array (e.g. "[ dc03, dc01, dc04
]"), or a sub-hash containing a deeper level of distinction. At each
layer, a special key named "default" is available, which sets the
default for everything within the current scope. The top-level default itself
defaults to the ordered list from "datacenters" in the normal case.
If the entire "map" stanza is missing or empty, you just get the
default behavior of "default". A datacenter array can also be empty,
which implies that this location is mapped to receive no response data (the
server will still respond to the query, and will not issue an NXDOMAIN. It
will simply be a NODATA/NOERROR response like you'd get if there were no
records of this type, but could be records of other types for the same name).
The meaningful location keys at the top level are continent codes, of which
there are primarily seven in MaxMind's databases: "AF" for Africa,
"AS" for Asia, "NA" for North America, "SA" for
South America, "EU" for Europe, "OC" for Oceania, and
"AN" for Antarctica. There is also an eighth continent-level code
which is, literally, "--". This is a sort of fallback "no
information available" continent code, and it contains the special
country codes "A1", "A2", "O1", and
"--", which represent Anonymous Proxies, Satellite Providers, Other,
and Unknown, respsectively.
The next layer (the sub-hash beneath any continent code) maps ISO-3166-1
2-letter country codes, which as with continents can map directly to
datacenters, or to yet another recursive layer.
The next two layers deep are for Region and City level information, only
available from the Region and City type databases. The Region database type
only provides region information for the US and Canada, using the standard
local 2-letter abbrevations (e.g. AB for Alberta, OK for Oklahama). The City
databases use those same region abbrevations for the US and Canada, but use
either FIPS 10-4 2-letter codes or full region names for the rest of the
world's regions (as detailed earlier in, and controlled by, the
"city_region_names" option).
The actual City names at the final layer appear to be encoded using some form of
ISO8859-1 and/or CP1252 character set in the databases themselves, and your
map entries will have to match byte-for-byte in the case of non-ASCII
characters. May come up with a better solution for this down the road.
There is also one other special key (aside from "default") available
at all levels of the map hierarchy, a boolean named "skip_level",
default "false". If set within the hierarchical "map" at
any layer, it causes the next layer of detail to be skipped for this portion
of the map. For example, setting this at the very top layer would mean that
the top layer would contain country-level codes directly, without an enclosing
continent-level hierarchy. Setting it within a country would mean that city
names are used directly within that country, without an intervening layer of
region names. This option is not aware of the "city_no_region"
option, so e.g. setting that option and specifying "skip_level" at
the country-level would result in no further information being available
within that country (as "skip_level" would skip the remaining layer
of city data).
CONFIGURATION - MAPS - CITY AUTO MODE¶
"City-auto-mode" is a special mode of operation that automatically
maps out the world to your datacenters based on coordinate math, so that you
don't have to manually construct a complex hierarchical "map". It
can still be mixed with "map" of course, allowing you to use
auto-mode for only select geographic areas if you wish (or disabling it for
select areas by specifying manual lists). The key parameter is
"auto_dc_coords", which enables city-auto-mode.
- "auto_dc_coords = { ... }"
- Key-value hash, optional. If this option is specified, the whole map's
basic mode of operation changes to "city-auto-mode". The
contents of the hash are a key for each datacenter named in
"datacenters", with their values set to an array of "[lat,
lon]" in decimal degree units. When city-auto-mode is enabled by
this, the following configuration-validation changes occur from the
default, static-mapping mode: the loaded GeoIP database(s) are required be
City-level databases, and the special keyword "auto" becomes a
legal "datacenter list" in the "map" stanza.
With city-auto-mode enabled, the top-level map "default" defaults
to "auto", but can be overridden with a manual list. For any
location that maps to "auto", the coordinates specified here in
"auto_dc_coords" will be compared with the coordinates from the
City-level database(s) to determine an automatic distance-sorted
datacenter list.
If you omit one or more defined datacenters from the coordinate list in
"auto_dc_coords", those datacenters will not be used in
automatic results, but will still be available for manual use via
"map" and/or "nets" entries.
- "auto_dc_limit = N"
- Unsigned integer, optional, default 3. When city-auto-mode is in effect,
this is the upper length limit for auto-generated lists. 3 is a reasonable
default even if you have a considerably longer set of datacenters, as this
provides a primary as well as two fallbacks. Raising this to a large
number in the presence of a long datacenter list will cause the set of
unique result datacenter lists to increase rapidly, and thus reduce the
optimization of the final result database for edns-client-subnet purposes.
It's really not worth raising this value in almost any case, unless you
really need to handle more than 3 random datacenters going offline at the
same time and still have clients fail elsewhere. The value zero is treated
as unlimited (highly un-recommended).
Under city-auto-mode, when the top-level default is (explicitly or implicitly)
"auto", there is still a fallback static ordering which is the whole
ordered "datacenters" list, which is the normal static default
"default" when not in city-auto-mode. This fallback is used when no
location information is available at all (e.g. IPv6 client vs IPv4 GeoIP DB,
Anonymous Proxies, etc).
MAP TESTING¶
A binary program "gdnsd_geoip_test" is included. This can be used
directly from the commandline, parses the relevant bits of your gdnsd config
file for geoip map info, and then provides datacenter list results for IP
address + map combinations supplied by the user. Useful for debugging your
maps and testing the mapping of client IPs. It has a separate manpage
gdnsd_geoip_test(1).
CONFIGURATION - RESOURCES¶
Resource-level configuration within the "resources" stanza is nearly
identical to the resources configuration of the metafo plugin, with all of the
same basic behaviors about synthesizing or directly referencing the
configuration of other plugins per-datacenter. The only real difference is
that metafo's per-resource "datacenters" array is replaced with
"map => mapname", which references one of the maps defined in the
"maps" stanza, described in detail earlier. The set of defined
datacenters in the "dcmap" stanza must match the total set of
datacenters defined by the referenced map.
Both of the meta-plugins ("metafo" and "geoip") can
reference their own as well as each others' resources by direct reference
within a "dcmap", so long as a resource does not directly refer to
itself. This allows plugin-layering configurations such as geoip -> metafo
-> weighted, or metafo -> geoip -> multifo, or even metafo ->
metafo -> simplefo, etc.
Bear in mind that once you begin using inter-meta-plugin references, you could
create a reference loop. gdnsd does not currently detect or prevent such
loops, and they will cause complete runtime failure when queried, probably by
running out of stack space during recursion.
Additionally, "geoip" can synthesize configuration for
"metafo" resources, but the reverse does not hold;
"metafo" cannot synthesize configuration for "geoip"
resources.
IPv4 Compatible Addresses¶
This plugin knows of five different relatively-trivial ways to map IPv4
addresses into the IPv6 address space. These are shown below, with
"NNNN:NNNN" in place of the copied IPv4 address bytes:
::NNNN:NNNN/96 # v4compat - canonical form for this plugin
::FFFF:NNNN:NNNN/96 # v4mapped
::FFFF:0:NNNN:NNNN/96 # SIIT
2001::NNNN:NNNN/32 # Teredo (NNNN:NNNN is xor'd with FFFF:FFFF)
2002:NNNN:NNNN::/16 # 6to4
All of this plugin's internal lookup databases are IPv6 databases, and any
IPv4-like information is always stored in the v4compat space within these
databases. When doing runtime lookups all other v4-like addresses (raw IPv4
addresses, v4mapped, SIIT, Teredo, and 6to4) are converted to the canonical
v4compat IPv6 representation before querying the internal databases. The other
representations (v4mapped, SIIT, Teredo, 6to4) are Undefined internally, and
will never be referenced at lookup-time due to the v4compat conversion
mentioned earlier.
The "nets" stanza is not allowed to specify entries in the four
undefined v4-like IPv6 spaces (those other than v4compat). Specify those
networks as normal IPv4 networks or v4compat networks instead. Legitimate IPv6
"nets" entries which happen to be a supernet of any v4-like spaces
will *not* unduly affect v4-like lookups. There is no functional difference
between v4compat and native v4 forms in "nets", e.g.
"192.0.2.0/24" and "::C000:0200/120" are completely
identical.
GeoIP databases that are natively IPv4-only get all of their data loaded into
the v4compat space only. For IPv6 GeoIP databases, by default we load the
v4compat space directly (which is where MaxMind stores IPv4 data in their IPv6
databases), but ignore the v4mapped/SIIT/Teredo/6to4 spaces (some of which are
empty in MaxMind's databases, and some of which simply alias the v4compat
space). When using an IPv6 GeoIP database combined with an IPv4 GeoIP overlay
(geoip_db_v4_overlay config), the v4compat space of the IPv6 database is also
ignored on loading, and the direct IPv4 data from the IPv4 databasee takes its
place.
ANOTHER CONFIG EXAMPLE¶
A relatively-maximal example config, showing the interaction of valid
"maps" and "resources" sections:
service_types => {
xmpp_svc => { plugin => "tcp_connect", ... }
www_svc => { plugin => "http_status", ... }
}
plugins => {
geoip => {
maps => {
city_region_names => fips_include,
my_prod_map => {
geoip_db => GeoIPCityv6.dat,
geoip_db_v4_overlay => GeoIPCity.dat,
city_no_region => false, # default
datacenters => [us-01, de-01, sg-01],
map => {
# Hierarchy is Continent -> Country -> Region -> City
NA => {
US => {
skip_level => 1, # skip past region level
Dallas => [sg-01],
}
}
SA => [us-01, sg-01, de-01],
EU => {
default => [eu-01, us-01, sg-01],
CH => {
Geneve => {
Geneva => [sg-01],
}
}
}
AF => [eu-01, us-01, sg-01],
AS => [sg-01, eu-01, us-01],
OC => [sg-01, us-01, eu-01],
}
nets => {
10.0.0.0/8 => [ eu-01 ],
2001:DB8::/32 => [ us-01 ],
}
}
my_auto_map => {
geoip_db => GeoIPCityv6.dat,
geoip_db_v4_overlay => GeoIPCity.dat,
datacenters => [us-01, de-01, sg-01],
auto_dc_coords => {
us-01 => [ 38.9, -77 ],
de-01 => [ 50.1, 8.7 ],
sg-01 => [ 1.3, 103.9 ],
}
}
}
resources => {
prod_app => {
map => my_auto_map
# these two are inherited multifo config keys
# for all of the dcmap below:
service_types => [www_svc, xmpp_svc],
up_thresh => 0.4,
dcmap => {
us-01 => {
lb01 => 192.0.2.1,
lb02 => 192.0.2.2,
lb03 => 192.0.2.3,
lb01.v6 => 2001:DB8::1,
lb02.v6 => 2001:DB8::2,
lb03.v6 => 2001:DB8::3,
},
sg-01 => {
lb01 => 192.0.2.4,
lb02 => 192.0.2.5,
lb03 => 192.0.2.6,
lb01.v6 => 2001:DB8::4,
lb02.v6 => 2001:DB8::5,
lb03.v6 => 2001:DB8::6,
},
de-01 => {
lb01 => 192.0.2.7,
lb02 => 192.0.2.8,
lb03 => 192.0.2.9,
lb01.v6 => 2001:DB8::7,
lb02.v6 => 2001:DB8::8,
lb03.v6 => 2001:DB8::9,
},
}
},
prod_cdn => {
map => my_prod_map,
dcmap => {
us-01 => us-cdn-provider.example.com.
sg-01 => asia-cdn-provider.example.com.
de-01 => europe-cdn-provider.example.com.
}
}
}
}
}
Example zonefile RRs:
app 600 DYNA geoip!prod_app
app.us 600 DYNA geoip!prod_app/us-01
app.sg 600 DYNA geoip!prod_app/sg-01
app.de 600 DYNA geoip!prod_app/de-01
content 600 DYNC geoip!prod_cdn
plugins => {
geoip => {
maps => {
auto_map => {
geoip_db => GeoIPCityv6.dat,
datacenters => [dc1, dc2, dc3, dc4],
auto_dc_coords => {
dc1 => [ 38.9, -77 ],
dc2 => [ 50.1, 8.7 ],
dc3 => [ 20.2, 88.9 ],
dc4 => [ 39.0, -20 ],
},
# only fail through the nearest 2 before giving up:
auto_dc_limit => 2,
}
},
resources => {
www_real => {
map => my_auto_map,
service_types => [ http, xmpp ],
dcmap => {
dc1 => 192.0.2.100,
dc2 => 192.0.2.101,
dc3 => 192.0.2.102,
dc4 => 192.0.2.103
}
}
}
},
metafo => {
resources => {
www => {
datacenters => [ real, backup ],
dcmap => {
real => %geoip!www_real,
backup => backup-host.example.net.
}
}
}
}
}
And in the zonefile:
; This tries through the closest 2/4 datacenters to
; the client from the geoip map, and if both of
; those are down it returns a CNAME to backup-host.example.net.
; for a downtime message or something:
www DYNC metafo!www
SEE ALSO¶
gdnsd-plugin-metafo(8),
gdnsd_geoip_test(1),
gdnsd.config(5),
gdnsd.zonefile(5),
gdnsd(8)
The gdnsd manual.
COPYRIGHT AND LICENSE¶
Copyright (c) 2012 Brandon L Black <blblack@gmail.com>
This file is part of gdnsd.
gdnsd is free software: you can redistribute it and/or modify it under the terms
of the GNU General Public License as published by the Free Software
Foundation, either version 3 of the License, or (at your option) any later
version.
gdnsd is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with
gdnsd. If not, see <
http://www.gnu.org/licenses/>.