NgxExport.Tools.Resolve

DNS resolve utilities from the more extra tools collection for nginx-haskell-module.

Dynamic upstreams in Nginx

With Nginx module nginx-upconf-module, it is possible to update servers inside upstreams dynamically. The module requires an agent to update a bound variable with upstreams layout and also signal that the variable has been altered. This module is such an agent. It updates the variable with the upstreams layout in service collectUpstreams and signals about this in service callback signalUpconf. Collecting upstreams encompasses DNS queries of A and SRV records. The queries are configured independently for each managed upstream. With SRV queries, the module allows configuration of complex hierarchies of priorities given that compound upstream containers named upstrands are in use (they are implemented in nginx-combined-upstreams-module).

Additionally, the module exports a number of functions and data types which implement service collectUpstreams.

In the following example, we are going to extract IP addresses from an SRV record for _http._tcp.mycompany.com to inhabit upstream utest.

File test_tools_extra_prometheus.hs

module TestToolsExtraResolve where

import NgxExport.Tools.Resolve ()

The file does not contain any significant declarations as we are going to use only the exporters.

File nginx.conf

user                    nginx;
worker_processes        4;

events {
    worker_connections  1024;
}

http {
    default_type        application/octet-stream;
    sendfile            on;

    upstream utest {
        zone utest 64k;
        upconf_round_robin;
        server localhost:9000;
    }

    haskell load /var/lib/nginx/test_tools_extra_resolve.so;

    haskell_run_service simpleService_collectUpstreams $hs_upstreams
        'Conf { upstreams =
                    [UData { uQuery =
                                 QuerySRV
                                     (Name "_http._tcp.mycompany.com")
                                         (SinglePriority "utest")
                           , uMaxFails = 0
                           , uFailTimeout = 10
                           }
                    ]
              , maxWait = Sec 300
              , waitOnException = Sec 2
              }';

    haskell_service_var_ignore_empty $hs_upstreams;
    haskell_service_var_in_shm upstreams 64k /tmp $hs_upstreams;

    haskell_service_var_update_callback simpleService_signalUpconf $hs_upstreams
        'Upconf { upconfAddr = ("/upconf", "127.0.0.1:8010")
                }';

    server {
        listen          localhost:8010;
        server_name     main;

        location /upconf {
            upconf $hs_upstreams;

            allow 127.0.0.1;
            deny  all;
        }

        location /upstreams {
            default_type application/json;
            echo $hs_upstreams;

            allow 127.0.0.1;
            deny  all;
        }

        location / {
            proxy_pass http://utest;
        }
    }

    server {
        listen          localhost:9000;
        server_name     backend9000;

        location / {
            echo_status 503;
            echo "Not configured";
        }
    }
}

At the start of Nginx, upstream utest contains a statically declared server which reports Not configured, but so soon as service collectUpstreams collects servers for the upstream in variable $hs_upstreams, and then the upconf module gets notified about this via callback signalUpconf, the upstream gets inhabited by the collected servers. The upstream contents will be re-checked within the time interval of (1 or waitOnException, maxWait). Particularly, if an exception happens during the collection of the servers, then the service will restart in waitOnException. If there were no exceptions and the smallest value of TTL calculated from all collected servers does not exceed the value of maxWait, then the service will restart in this time.

Notice that we used QuerySRV and SinglePriority "utest". The latter means that all collected servers will inhabit upstream utest regardless of their priority values. To distribute collected servers among a number of upstreams, we can use PriorityList.

File nginx.conf: collect upstreams with PriorityList

    haskell_run_service simpleService_collectUpstreams $hs_upstreams
        'Conf { upstreams =
                    [UData { uQuery =
                                 QuerySRV
                                     (Name "_http._tcp.mycompany.com")
                                         (PriorityList ["utest", "utest1"])
                           , uMaxFails = 0
                           , uFailTimeout = 10
                           }
                    ]
              , maxWait = Sec 300
              , waitOnException = Sec 2
              }';

With this configuration, servers with the highest priority will inhabit upstream utest, while servers with lesser priorities will inhabit upstream utest1. Upstream utest1 must also be managed by the upconf module. The priority list may contain more than two upstreams, in which case upstreams at the beginning of the list will take higher priorities found in the collected servers, while the last upstream will take the remainder of the priorities.

Upstreams in the priority list can be put inside of an upstrand to form the main and the backup layers of servers.

File nginx.conf: upstrand utest

    upstream utest1 {
        zone utest1 64k;
        upconf_round_robin;
        server localhost:9000;
    }

    upstrand utest {
        upstream utest;
        upstream utest1;
        order per_request;
        next_upstream_statuses error timeout 5xx;
        next_upstream_timeout 60s;
    }

File nginx.conf: location upstrand

        location /upstrand {
            proxy_pass http://$upstrand_utest;
        }

Exported type declarations

type UName = Text

Upstream name.

type SAddress = Text

Domain name or IP address with or without port.

data UQuery

DNS query model of the upstream(s).

There are 3 ways to get the list of server addresses:

• query A records for a list of domain names,
• query an SRV record for a single service name and then query A records for the collected list of domain names,
• the same as the previous, but distribute collected servers among a list of upstreams according to the collected priorities.

Constructors

QueryA [Name] UName

Query A records

QuerySRV Name PriorityList

Query an SRV record

Instances

Read UQuery

Defined in NgxExport.Tools.Resolve

Methods

readsPrec :: Int -> ReadS UQuery

readList :: ReadS [UQuery]

readPrec :: ReadPrec UQuery

readListPrec :: ReadPrec [UQuery]

data PriorityList

Specifies how to distribute collected servers among the given upstreams.

Constructors

SinglePriority UName

All servers to one upstream

PriorityList [UName]

Distribute servers by priorities

Instances

Read PriorityList

Defined in NgxExport.Tools.Resolve

Methods

readsPrec :: Int -> ReadS PriorityList

readList :: ReadS [PriorityList]

readPrec :: ReadPrec PriorityList

readListPrec :: ReadPrec [PriorityList]

data UData

Upstream configuration.

Includes DNS query model and data for Nginx server description.

Constructors

UData

Fields

• uQuery :: UQuery

  DNS query model

• uMaxFails :: Int

  maxFails

• uFailTimeout :: Int

  failTimeout

Instances

Read UData

Defined in NgxExport.Tools.Resolve

Methods

readsPrec :: Int -> ReadS UData

readList :: ReadS [UData]

readPrec :: ReadPrec UData

readListPrec :: ReadPrec [UData]

data ServerData

Server data.

The fields map exactly to data from Nginx server description.

Constructors

ServerData

Fields

• sAddr :: SAddress

  Server address

• sWeight :: Maybe Int

  weight

• sMaxFails :: Maybe Int

  maxFails

• sFailTimeout :: Maybe Int

  failTimeout

Instances

FromJSON ServerData

Defined in NgxExport.Tools.Resolve

Methods

parseJSON :: Value -> Parser ServerData

parseJSONList :: Value -> Parser [ServerData]

ToJSON ServerData

Defined in NgxExport.Tools.Resolve

Methods

toJSON :: ServerData -> Value

toEncoding :: ServerData -> Encoding

toJSONList :: [ServerData] -> Value

toEncodingList :: [ServerData] -> Encoding

Show ServerData

Defined in NgxExport.Tools.Resolve

Methods

showsPrec :: Int -> ServerData -> ShowS

show :: ServerData -> String

showList :: [ServerData] -> ShowS

Eq ServerData

Defined in NgxExport.Tools.Resolve

Methods

(==) :: ServerData -> ServerData -> Bool

(/=) :: ServerData -> ServerData -> Bool

Ord ServerData

Defined in NgxExport.Tools.Resolve

Methods

compare :: ServerData -> ServerData -> Ordering

(<) :: ServerData -> ServerData -> Bool

(<=) :: ServerData -> ServerData -> Bool

(>) :: ServerData -> ServerData -> Bool

(>=) :: ServerData -> ServerData -> Bool

max :: ServerData -> ServerData -> ServerData

min :: ServerData -> ServerData -> ServerData

type CollectedServerDataGen a = (a, Map UName [ServerData])

Generic type to collect and store server data.

Type a is instantiated either by TTL (to collect) or TimeInterval (to store).

type CollectedServerData = CollectedServerDataGen TTL

Collected server data.

The first element of the tuple gets transformed into the time interval before the next run of the collectUpstreams service. The second element contains the collected data.

Exported functions

collectA

Arguments

:: TTL

Fallback TTL value

-> Name

Domain name

-> IO (TTL, [IPv4])

Queries an A record for the given domain name.

Returns a list of IP addresses and the minimum value of their TTLs. If the list is empty, then the returned TTL value gets taken from the first argument.

collectSRV

Arguments

:: TTL

Fallback TTL value

-> Name

Service name

-> IO (TTL, [SRV IPv4])

Queries an SRV record for the given service name.

After getting the SRV record, runs collectA for each collected element.

Returns a list of IP addresses wrapped in SRV container and the minimum value of their TTLs. If the list is empty, then the returned TTL value gets taken from the first argument.

collectServerData

Arguments

:: TTL

Fallback TTL value

-> UData

Upstream configuration

-> IO CollectedServerData

Collects server data for the given upstream configuration.