3. Using Stork

This section describes how to use general features available in Stork. For DHCP-specific features, see DHCP. For DNS-specific features, see DNS.

To connect to Stork, use a web browser and connect to port 8080 on the Stork server machine. If stork-server is running on a localhost, it can be reached by navigating to http://localhost:8080.

3.1. Dashboard

The main Stork page presents a dashboard. It contains two expandable panels: including DHCP and DNS data respectively. I also includes an events panel listing the events pertaining to the monitored servers.

3.1.1. The DHCP Panel

The DHCP panel includes two sections: one for DHCPv4 and one for DHCPv6. Each section contains three kinds of information:

  • a list of up to five subnets with the highest pool utilization

  • a list of up to five shared networks with the highest pool utilization

  • statistics about DHCP

The Service Status section in the DHCP panel lists all monitored DHCP servers. The information shown includes the hostname, the version, the daemon name, the daemon running on the host, its communication status, the average number of ACKs sent by the daemon over the previous 15 minutes and 24 hours, its High Availability (HA) state, whether an HA failure has been detected, and the host’s running uptime. More details about the meaning of these status indicators is included in the DHCP section.

3.1.2. The DNS Panel

The Service Status section in the DNS panel lists all monitored DNS servers. Similarly, to the DHCP panel, it includes the hostname, the daemon name, the communication status with the daemon, and the daemon uptime. It also includes the following indicators for each DNS daemon:

  • Zone Fetch Status: whether or not the zones have been fetched from the server into the Stork server’s database.

  • Zone Configs Count: the number of different zone configurations in the server (if the same zone name appears in multiple views, it is counted multiple times).

  • Distinct Zones: the number of different zones in the server (if the same zone name appears in multiple views, it is counted only once).

  • Builtin Zones: the number of distinct builtin zones in the server. Builtin zones are special zones generated by BIND 9.

More information about these indicators can be found in the DNS section.

3.1.3. The Events Panel

The Events panel presents the list of the most recent events captured by the Stork server. There are three event urgency levels: info, warning, and error. Events pertaining to particular entities, e.g. machines or daemons, provide a link to a web page containing information about the given object. Users with super-admin permissions can clear the events (though this leaves behind an event describing their action).

3.2. Managing Users

A default administrator account is created when Stork is initially installed. It can be used to sign in to the system via the web interface, with the username admin and password admin.

To see a list of existing users, click on the Configuration menu and choose Users. There will be at least one user, admin.

To add a new user, click Create User Account. A new tab opens to specify the new account parameters. Some fields have specific restrictions:

  • The username can consist of only letters, numbers, and underscores (_).

  • The e-mail field is optional, but if specified, it must be a well-formed e-mail address.

  • The firstname and lastname fields are mandatory for the user accounts managed by Stork. They are optional for authentication methods provided by hooks.

  • The password can consist of uppercase and lowercase letters A-Z, numbers 0-9, and any ASCII special characters or whitespace characters; it must be at least twelve characters long; it must include at least one uppercase letter; it must include at least one lowercase letter; it must include at least one digit; it must include at least one special character or whitespace character.

Currently, each user is associated with one of the three predefined groups (roles), which are super-admin, admin or read-only; one of these must be selected when a user account is created. Both super-admin and admin types of users can view Stork status screens, edit interval and reporting configuration settings, and add/edit/remove Kea host reservations, subnets, shared networks etc.

Only super-admin users can perform following actions:

  • create and manage user accounts

  • add/remove/authorize machines for monitoring

  • clear the event log

  • read access point keys of monitored BIND9 servers

  • read/regenerate Stork server token

  • read secrets in configs of monitored Kea servers.

read-only group users can only have read access to the system components and REST API endpoints. Users that belong to this group cannot perform Create, Update nor Delete actions.

Once the new user account information has been specified and all requirements are met, the Save button becomes active and the new account can be enabled.

3.2.1. Changing a User Password

An initial password is assigned by the administrator when a user account is created; by default, each user must change their password when first logging into the system. After that, each user has the possibility to change their password whenever they need to. To change the password, click on the Profile menu and choose Settings to display the user profile information. Click on Change password in the menu bar on the left and specify the current password in the first input box. The new password must be entered and confirmed in the second and third input boxes, and must meet the password requirements specified in the previous section. When all entered data is valid, the Save button is activated to change the password.

3.3. Runtime Configuration Settings

The Stork server is started with the command-line switches and/or environment variables controlling some of its behavior. However, the server also exposes other configuration options only available at runtime from the web UI. To access these options, select Configuration from the menu and choose Settings. There are four classes of settings available: Security, Automatic software updates checking, Intervals, and Grafana.

Security settings currently contain only one option, which controls whether the machine registration REST API endpoint is enabled. New machines connect to this endpoint when they begin the registration. To avoid malicious attempts to register fake machines in the Stork server, it is practical to uncheck the Enable machine registration option when no new registrations are expected. The option can be re-enabled at any time when new registrations are required. Unchecking the option does not affect the ability to re-register existing machines.

Automatic software updates checking setting controls whether Stork should automatically check for software updates available for Kea, BIND 9 and Stork itself. To be able to do that, Stork server downloads a JSON file with the latest software releases metadata. This feature can be disabled using this setting. When disabled, all the feedback displayed regarding software versions detected by Stork will be generated based on offline built-in information stored as a JSON file.

Note

The offline JSON file location is under path /etc/stork/versions.json in the Stork server installation location. If the Automatic software updates checking feature is disabled for any reason, the offline JSON file content can be updated manually. It is suggested to update it with the content published under https://www.isc.org/versions.json URL. The content of this file is maintained by ISC and is kept up-to-date with current Stork, Kea and BIND 9 releases on a “best-effort” basis.

Intervals settings specify the configuration of “pullers.” A puller is a mechanism in Stork that triggers a specific action at the specified interval. Each puller has its own specific action and interval. The puller interval is specified in seconds and designates the time period between the completion of the previously invoked action and the beginning of the next invocation of this action. For example, if the Kea Hosts Puller Interval is set to 10 seconds and it takes five seconds to pull the hosts information, the time period between the starts of the two consecutive attempts to pull the hosts information is 15 seconds.

The pull time varies between deployments and depends on the amount of information pulled, network congestion, and other factors. The interval setting guarantees that there is a constant idle time between any consecutive attempts.

The Grafana setting allows to specify the URL of the Grafana instance used with Stork.

3.4. Connecting and Monitoring Machines

3.4.1. Monitoring a Machine

Monitoring of registered machines is accomplished via the Services menu, under Machines. A list of currently registered machines is displayed, with multiple pages available if needed.

A filtering mechanism that acts as an omnibox is available. Via a typed string, Stork can search for an address, agent version, hostname, OS, platform, OS version, kernel version, kernel architecture, virtualization system, or host ID field.

The state of a machine can be inspected by clicking its hostname; a new tab opens with the machine’s details. Multiple tabs can be open at the same time, and clicking Refresh updates the available information.

The machine state can also be refreshed via the Action menu. On the Machines list, each machine has its own menu; click on the triple-lines button at the right side and choose the Refresh option.

3.4.2. Disconnecting From a Machine

To stop monitoring a machine, go to the Machines list, find the machine to stop monitoring, click on the triple-lines button at the right side, and choose Delete. This terminates the connection between the Stork server and the agent running on the machine, and the server no longer monitors that machine; however, the stork-agent process continues running. Complete shutdown of the stork-agent process must be done manually, e.g. by connecting to the machine using SSH and stopping the agent there. For example, when the Stork agent has been installed from packages, run:

$ sudo systemctl stop isc-stork-agent

Alternatively:

$ sudo killall -9 stork-agent

3.4.3. Dumping Diagnostic Information Into a File

It is sometimes difficult or impossible to diagnose issues without seeing the actual logs, database contents, and configuration files. Gathering such information can be challenging for a user because it requires looking into many places like databases, remote machine logs, etc.

Stork makes it convenient for users to gather diagnostic information from the selected machines with a single click. Navigate to the Machines page where all monitored machines are listed, click on the Action button for a selected machine, and choose the Dump Troubleshooting Data option. Alternatively, navigate to the selected machine’s page and click on the Dump Troubleshooting Data button at the bottom of the page. In both cases, the Stork server automatically gathers useful diagnostics information and offers it for download as a tar.gz file. The downloaded package contains configurations, log tails, stork-server settings, warning and error-level events, high-availability service states, etc.

Note

Stork sanitizes passwords and other sensitive information when it creates the package.

The tarball can be easily sent via email or attached to a bug report.

3.4.4. Communication Status With the Monitored Machines

The communication status with the monitored agents and daemons. To see the detailed status for all daemons on a single page, navigate to Monitoring and then Communication. If this page shows no communication issues, all connected systems are online. If there are issues, the page lists a hierarchical view of the Stork agents, Kea Control Agents, and the daemons, highlighting any for which communication failures have occurred. The communication failures may be caused by a process failure (e.g., a Stork agent failure) or a machine failure. With a process failure, it is possible that other daemons are still running, but the lack of agent connectivity may cause an inaccurate status to be reported. With a machine failure, all processes on the culprit machine are down. The Stork server tries to provide accurate data about the states of all processes, but some information may be unavailable.

3.5. Monitoring Daemons

3.5.1. Daemon Status

Kea DHCP daemons discovered on connected machines are listed via the top-level menu bar, under Machines. The list view includes the daemon version, daemon status, and some machine details. The Action button is also available, to refresh the information about the daemons.

Several daemons may be presented in the Daemons column. The daemons are not listed for the DNS servers because each DNS server has only one daemon.

The listed Kea daemons are those that Stork finds in the Kea CA configuration file. A warning is displayed for any daemons from the CA configuration file that are not running. When the Kea installation is simply using the default CA configuration file, which includes configuration of daemons that are never intended to be launched, it is recommended to remove (or comment out) those configurations to eliminate unwanted warnings from Stork about inactive daemons.

Stork uses rndc to retrieve the BIND 9 daemon’s status. It looks for the controls statement in the configuration file, and uses the first listed control point to monitor the daemon. The statistics-channels must be configured to allow fetching BIND 9 statistics (and exporting them to Prometheus), and fetch a list of zones.

PowerDNS servers to be monitored by Stork are required to have the webserver enabled in their configuration file. See the PowerDNS documentation for more details. Stork agent uses the webserver to retrieve the PowerDNS server’s status information, a list of zones.

Stork’s zone viewer allows for browsing the zone contents (RRs) for the selected zone types. Internally, the zone contents are fetched from the DNS server using zone transfer (AXFR). For that reason, the servers must be properly configured to allow zone transfer to the Stork agent. The detailed requirements for the DNS servers configurations to properly work with the Stork agent can be found in the DNS section.

Furthermore, the Stork agent can be used as a Prometheus exporter if named is built with json-c, because Stork gathers statistics via the JSON statistics API. The named.conf file must have statistics-channel configured; the exporter queries the first listed channel. Stork is able to export the most metrics if zone-statistics is set to full in the named.conf configuration.

3.5.2. Friendly Daemon Labels

Every daemon connected to Stork is assigned a label. For example, if a Kea DHCPv4 daemon runs on the machine abc.example.org, this daemon’s label is DHCPv4@abc.example.org. Similarly, if a BIND 9 daemon runs on the machine with the address 192.0.2.3, the resulting label is named@192.0.2.3.

The daemon labels allow the user to distinguish daemons in the dashboard, daemons list, events panel, and other views. However, the default labels may become lengthy when machine names consist of fully qualified domain names (FQDNs). When machines’ IP addresses are used instead of FQDNs, the daemon labels are less meaningful for someone not familiar with addressing in the managed network.

3.6. The Events Page

The Events page presents a list of all events. It allows events to be filtered by:

  • urgency level

  • machine

  • daemon name (dhcp4, dhcp6, named, etc.)

  • the user who caused a given event (available only to users in the super-admin group).

3.7. The Software Versions Page

The Software Versions page, which can be found under the Monitoring -> Software versions menu, provides information about the Kea, Stork, and BIND 9 software versions currently running on monitored machines. It consists of two main parts, described below.

3.7.1. Summary of ISC Software Versions Detected by Stork

Stork can identify the ISC software used on all authorized machines and check whether those software packages are up-to-date. The summary table indicates whether there are software updates available for any of the versions that are running, with messages that show how critical those updates are. The table also includes whether the machine’s Stork agent version matches the Stork server version. Stork server’s version is also checked. If an update is available, the notification message is displayed right above the Summary table.

Note

The version of the Stork server and all Stork agents should match; e.g. if the Stork server version is 2.0.0, all Stork agents should also be version 2.0.0.

For each machine where the Kea server is found, Stork also checks whether all the Kea daemons use matching versions.

Note

If the Kea server has more than one daemon active, they should all use the same version; e.g. if the Kea server has active daemons DHCPv4, DHCPv6 and DDNS, and the DHCPv4 daemon is version 2.6.1, all other Kea daemons (DHCPv6 and DDNS) should be version 2.6.1.

The table includes color-coded notices about the importance of upgrading the Kea, BIND 9, or Stork software, based on the software version checks performed. The summary table groups the machines by severity and sorts them in descending order.

ISC advises reviewing the summaries for machines with red and yellow severity and updating those software versions.

3.7.2. Kea, BIND 9, and Stork Current Releases

These tables show the currently available versions of ISC’s Kea, BIND 9, and Stork software. There are links to the software documentation and release notes, as well as to packages and tarball downloads. The table also indicates the version release dates and an EoL (End-of-Life) date for stable releases.

The tables may include different types of releases described with the following terms:

  • Development - These releases introduce new and updated features and may not be backward-compatible with their immediate predecessor. Development versions are suitable for those interested in experimenting with and providing feedback to ISC but are not recommended for production use.

  • Stable - These versions are fully supported and meant for production use.

  • ESV (only for the BIND 9 Extended Support version) - These versions are suitable for those needing long-term stability.

Note

For details about ISC’s Software Support Policy and Version Numbering, please refer to this KB article.

3.7.3. Data Source

The information about ISC software releases shown on the Software Versions page may come from different sources:

  • Online JSON file - A JSON file available online. This data is intended to be always up-to-date. It is updated with every Stork, Kea or BIND 9 release.

  • Offline JSON file - This data is updated with every Stork release. Of course, the more time has passed since a given release date, the more outdated this data may be. ISC advises regularly checking the ISC software download page for up-to-date information. Please note that the date this data was generated is displayed in the top notification message. The date is also displayed in messages in the Summary column of the Summary of ISC software versions detected by Stork table.

Note

When the Offline JSON file is the source of the data, the stable BIND 9 version should be verified; the BIND 9 team usually issues stable releases every month. To check the latest release, visit the BIND download page.

Note

Stork server tries to retrieve the data from the Online source first. If for any reason this data cannot be retrieved, there is a fallback mechanism that reads the Offline JSON file.

3.7.4. The Version Status Icon

There are many places in the Stork UI where either the Kea, BIND 9, or Stork agent version is displayed, e.g., the Services -> Machines list, the Services -> Kea Daemons list, etc. Next to the displayed software version, there is an icon with feedback about the version. Hovering the mouse over the icon displays a tooltip with full feedback about the version. Clicking on the icon leads to the Software Versions page.