NgxExport.Tools.Resolve

DNS resolve utilities from the more extra tools collection for nginx-haskell-module.

Dynamic upstreams in Nginx

With Nginx module nginx-upconf-module, it is possible to update servers inside upstreams dynamically. The module requires an agent to update a bound variable with upstreams layout and also signal that the variable has been altered. This module is such an agent. It updates the variable with the upstreams layout in service collectUpstreams and signals about this in service callback signalUpconf. Collecting upstreams encompasses DNS queries of A and SRV records. The queries are configured independently for each managed upstream. With both A and SRV queries, the module allows configuration of complex hierarchies of priorities given that compound upstream containers named upstrands are in use (they are implemented in nginx-combined-upstreams-module).

Additionally, the module exports a number of functions and data types which implement service collectUpstreams.

In the following example, we are going to extract IP addresses from an SRV record for _http._tcp.mycompany.com to inhabit upstream utest.

File test_tools_extra_prometheus.hs

module TestToolsExtraResolve where

import NgxExport.Tools.Resolve ()

The file does not contain any significant declarations as we are going to use only the exporters.

File nginx.conf

user                    nginx;
worker_processes        4;

events {
    worker_connections  1024;
}

http {
    default_type        application/octet-stream;
    sendfile            on;

    upstream utest {
        zone utest 64k;
        upconf_round_robin;
        server localhost:9000;
    }

    haskell load /var/lib/nginx/test_tools_extra_resolve.so;

    haskell_run_service simpleService_collectUpstreams $hs_upstreams
        'Conf { upstreams =
                    [UData { uQuery =
                                 QuerySRV
                                     (Name "_http._tcp.mycompany.com")
                                         (SinglePriority "utest")
                           , uMaxFails = 1
                           , uFailTimeout = 10
                           }
                    ]
              , maxWait = Sec 300
              , waitOnException = Sec 2
              , responseTimeout = Unset
              }';

    haskell_service_var_ignore_empty $hs_upstreams;
    haskell_service_var_in_shm upstreams 64k /tmp $hs_upstreams;

    haskell_service_var_update_callback simpleService_signalUpconf $hs_upstreams
        '["http://127.0.0.1:8010/upconf"]';

    server {
        listen          localhost:8010;
        server_name     main;

        location /upconf {
            upconf $hs_upstreams;

            allow 127.0.0.1;
            deny  all;
        }

        location /upstreams {
            default_type application/json;
            echo $hs_upstreams;

            allow 127.0.0.1;
            deny  all;
        }

        location / {
            proxy_pass http://utest;
        }
    }

    server {
        listen          localhost:9000;
        server_name     backend9000;

        location / {
            echo_status 503;
            echo "Not configured";
        }
    }
}

At the start of Nginx, upstream utest contains a statically declared server which reports Not configured, but so soon as service collectUpstreams collects servers for the upstream in variable $hs_upstreams, and then the upconf module gets notified about this via callback signalUpconf, the upstream gets inhabited by the collected servers. Notice that signalUpconf accepts a list of URLs which means that it can broadcast collected servers to multiple upconf endpoints listening on this or other hosts.

The upstream contents will be re-checked within the time interval of (1 or waitOnException, maxWait). Particularly, if an exception happens during the collection of the servers, then the service will restart in waitOnException. If there were no exceptions and the smallest value of TTL calculated from all collected servers does not exceed the value of maxWait, then the service will restart in this time.

Too big response times may also cause exceptions during the collection of the servers. The timeout is defined by the value of responseTimeout. In our example, the timeout is not set.

Notice that we used QuerySRV and SinglePriority "utest". The latter means that all collected servers will inhabit upstream utest regardless of their priority values. To distribute collected servers among a number of upstreams, we can use PriorityList.

File nginx.conf: collect upstreams with PriorityList

    haskell_run_service simpleService_collectUpstreams $hs_upstreams
        'Conf { upstreams =
                    [UData { uQuery =
                                 QuerySRV
                                     (Name "_http._tcp.mycompany.com")
                                         (PriorityList ["utest", "utest1"])
                           , uMaxFails = 1
                           , uFailTimeout = 10
                           }
                    ]
              , maxWait = Sec 300
              , waitOnException = Sec 2
              , responseTimeout = Unset
              }';

With this configuration, servers with the highest priority will inhabit upstream utest, while servers with the less priority will inhabit upstream utest1. Upstream utest1 must also be managed by the upconf module. Generally, given the number of upstreams in the priority list is \(N\) and the number of all variations of server priorities collected in the response is \(M\), and \(N\) is less than \(M\), then remaining \(M - N\) servers with the lowest priorities won't be used in the upstreams at all, otherwise, if \(N\) is greater than \(M\), then remaining \(N - M\) upstreams at the end of the priority list will contain the same servers of the lowest priority.

Upstreams in the priority list can be put inside of an upstrand to form the main and the backup layers of servers.

File nginx.conf: upstrand utest

    upstream utest1 {
        zone utest1 64k;
        upconf_round_robin;
        server localhost:9000;
    }

    upstrand utest {
        upstream utest;
        upstream utest1;
        order per_request;
        next_upstream_statuses error timeout 5xx;
        next_upstream_timeout 60s;
    }

File nginx.conf: location /upstrand

        location /upstrand {
            proxy_pass http://$upstrand_utest;
        }

Exported type declarations

type UName = Text

Upstream name.

type SAddress = Text

Domain name or IP address with or without port.

data UQuery

DNS query model of the upstream(s).

There are 4 ways to get the list of server addresses:

• query A records on a weighted list of domain names,
• the same as the previous, but distribute collected servers among a list of upstreams according to the weights set in the name list,
• query an SRV record on a single service name and then query A records on the collected list of domain names,
• the same as the previous, but distribute collected servers among a list of upstreams according to the collected priorities.

Particularly, in SRV queries priorities are taken from values of srvPriority, the less this value the higher the priority. In A queries priorities are taken from WeightedList, the higher the weight the higher the priority.

Weights of individual servers depend on both priority policy and type of the query:

• single priority, A query: weights are taken from WeightedList,
• priority list, A query: no weights are specified as the values from WeightedList are used for the priority list parameterization,
• single priority, SRV query: no weights are specified as it is not clear how to choose them correctly from the two parameters srvPriority and srvWeight,
• priority list, SRV query: weights are taken from srvWeight.

Names in the QueryA name list may contain suffix :port (a port number) which is ignored in collectA and only appended to values of sAddr collected by collectServerData.

Constructors

QueryA NameList UNamePriorityPolicy

Query A records

QuerySRV Name UNamePriorityPolicy

Query an SRV record

Instances

Read UQuery

Defined in NgxExport.Tools.Resolve

Methods

readsPrec :: Int -> ReadS UQuery

readList :: ReadS [UQuery]

readPrec :: ReadPrec UQuery

readListPrec :: ReadPrec [UQuery]

data WeightedList a

Weighted list.

A list of elements optionally annotated by weight values.

Constructors

Singleton a

List with a single element

PlainList [a]

Plain list without weights

WeightedList [(a, Word)]

Weighted list

Instances

Foldable WeightedList

Defined in NgxExport.Tools.Resolve

Methods

fold :: Monoid m => WeightedList m -> m

foldMap :: Monoid m => (a -> m) -> WeightedList a -> m

foldMap' :: Monoid m => (a -> m) -> WeightedList a -> m

foldr :: (a -> b -> b) -> b -> WeightedList a -> b

foldr' :: (a -> b -> b) -> b -> WeightedList a -> b

foldl :: (b -> a -> b) -> b -> WeightedList a -> b

foldl' :: (b -> a -> b) -> b -> WeightedList a -> b

foldr1 :: (a -> a -> a) -> WeightedList a -> a

foldl1 :: (a -> a -> a) -> WeightedList a -> a

toList :: WeightedList a -> [a]

null :: WeightedList a -> Bool

length :: WeightedList a -> Int

elem :: Eq a => a -> WeightedList a -> Bool

maximum :: Ord a => WeightedList a -> a

minimum :: Ord a => WeightedList a -> a

sum :: Num a => WeightedList a -> a

product :: Num a => WeightedList a -> a

Read a => Read (WeightedList a)

Defined in NgxExport.Tools.Resolve

Methods

readsPrec :: Int -> ReadS (WeightedList a)

readList :: ReadS [WeightedList a]

readPrec :: ReadPrec (WeightedList a)

readListPrec :: ReadPrec [WeightedList a]

type NameList = WeightedList Name

Weighted list of domain names.

data PriorityPolicy a

Priority policy.

Specifies how to distribute collected items by priorities. In particular, PriorityPolicy UName specifies how to distribute collected servers among the given upstreams.

Constructors

SinglePriority a

All items go to a single element

PriorityList [a]

Distribute items by priorities

Instances

Read a => Read (PriorityPolicy a)

Defined in NgxExport.Tools.Resolve

Methods

readsPrec :: Int -> ReadS (PriorityPolicy a)

readList :: ReadS [PriorityPolicy a]

readPrec :: ReadPrec (PriorityPolicy a)

readListPrec :: ReadPrec [PriorityPolicy a]

type UNamePriorityPolicy = PriorityPolicy UName

Priority policy of upstream names.

data UData

Upstream configuration.

Includes DNS query model and parameters for Nginx server description. Values of uMaxFails and uFailTimeout get assigned to each collected server as max_fails and fail_timeout respectively. Note that setting of parameters max_conns, backup and down is not supported.

Constructors

UData

Fields

• uQuery :: UQuery

  DNS query model

• uMaxFails :: Int

  max_fails

• uFailTimeout :: Int

  fail_timeout

Instances

Read UData

Defined in NgxExport.Tools.Resolve

Methods

readsPrec :: Int -> ReadS UData

readList :: ReadS [UData]

readPrec :: ReadPrec UData

readListPrec :: ReadPrec [UData]

data ServerData

Server data.

The fields map exactly to parameters from Nginx server description.

Constructors

ServerData

Fields

• sAddr :: SAddress

  Server address

• sHost :: SAddress

  Server host name

• sWeight :: Maybe Int

  weight

• sMaxFails :: Maybe Int

  max_fails

• sFailTimeout :: Maybe Int

  fail_timeout

Instances

ToJSON ServerData

Defined in NgxExport.Tools.Resolve

Methods

toJSON :: ServerData -> Value

toEncoding :: ServerData -> Encoding

toJSONList :: [ServerData] -> Value

toEncodingList :: [ServerData] -> Encoding

omitField :: ServerData -> Bool

Show ServerData

Defined in NgxExport.Tools.Resolve

Methods

showsPrec :: Int -> ServerData -> ShowS

show :: ServerData -> String

showList :: [ServerData] -> ShowS

Eq ServerData

Defined in NgxExport.Tools.Resolve

Methods

(==) :: ServerData -> ServerData -> Bool

(/=) :: ServerData -> ServerData -> Bool

Ord ServerData

Defined in NgxExport.Tools.Resolve

Methods

compare :: ServerData -> ServerData -> Ordering

(<) :: ServerData -> ServerData -> Bool

(<=) :: ServerData -> ServerData -> Bool

(>) :: ServerData -> ServerData -> Bool

(>=) :: ServerData -> ServerData -> Bool

max :: ServerData -> ServerData -> ServerData

min :: ServerData -> ServerData -> ServerData

type CollectedServerData = Map UName [ServerData]

Collected server data.

Exported functions

collectA

Arguments

:: TTL

Fallback TTL value

-> Name

Domain name

-> IO (TTL, [IPv4])

Queries an A record for the given domain name.

Returns a list of IP addresses and the minimum value of their TTLs. If the list is empty, then the returned TTL value gets taken from the first argument.

collectSRV

Arguments

:: TTL

Fallback TTL value

-> Name

Service name

-> IO (TTL, [SRV (Name, IPv4)])

Queries an SRV record for the given service name.

After getting the SRV record, runs collectA for each collected element.

Returns a list of pairs (Domain name, IP address) wrapped in an SRV container and the minimum value of their TTLs. If the list is empty, then the returned TTL value gets taken from the first argument. Note that trailing dots in the collected domain names (as in www.mycompany.com.) get removed in the returned list.

collectServerData

Arguments

:: TTL

Fallback TTL value

-> UData

Upstream configuration

-> IO (TTL, CollectedServerData)

Collects server data for the given upstream configuration.

Returns the collected server data and the minimum value of all the collected TTLs. If this TTL value, having been converted into a TimeInterval, is not bigger than maxWait, then it defines in how long time service collectUpstreams, which calls this function, will restart again.