BIND 9 - 2022 Retrospective

BIND 9 Development

BIND 9.18 was released at the beginning of 2022. This new major branch completed the transition to the new networking code. For a full list of added, removed, and changed features, see the release notes for BIND 9.18, but below are a few highlights for your enjoyment.

BIND 9.18 was declared to be an Extended Support Version at the end of 2022: we encourage our users to upgrade to the latest BIND 9.18 release.

In 2022, the BIND 9 team grew - Tony Finch (formerly of the University of Cambridge) started working for ISC.

Encrypted Protocols

BIND 9.18 added support for the new encrypted DNS protocols:

• DNS over TLS (RFC 7858)
• XFR over TLS (RFC 9013)
• DNS over HTTPS/2 (RFC 8484)

TLS certificate management infrastructure functionality and corresponding configuration options were added to BIND. The networking code for DNS over TCP and DNS over TLS has been unified in the latest development releases, significantly reducing the amount of networking code.

We have had relatively little user feedback on encrypted DNS in general, as usage and interest have been low. This, coupled with the recent refactoring, means there is a somewhat increased risk of undiscovered bugs in this area.

Improved Memory Management

BIND 9.18 now natively supports the jemalloc memory allocator, which works better with BIND’s DNS workloads, bringing reduced memory fragmentation and better performance.

Finishing the transition to the new networking code also halved the number of the threads needed to run named. All these changes allowed us to reduce BIND 9.18’s memory usage to BIND 9.11 levels, while improving both authoritative and recursive performance. See our earlier blog post on this topic for details and data.

Improved Performance

BIND 9.16 used the new networking code on the “server side” while still using the older networking code for outgoing DNS queries. Completing the conversion to the new networking code in BIND 9.18 improved recursive performance, particularly in cold-cache scenarios.

BIND’s DNS name decompression and compression code were rewritten to be significantly faster. The improved decompression code is in BIND 9.18 and the new compression code will be in BIND 9.20. The new compression code can reduce the CPU time for outgoing zone transfers by 30%. Alongside the name compression improvements, we also rewrote the way BIND does case-insensitive DNS name comparisons to make it more efficient while reducing code duplication.

Security

In 2022, BIND 9.16 had six CVEs and BIND 9.18 had 10 CVEs. Details can be found in the BIND 9 Security Vulnerability Matrix in the ISC Knowledgebase. We changed our vulnerability scoring process slightly, dropping the temporal metric from our scoring guidelines, because we found it had not made the scores any more accurate or useful in the case of DNS.

Future Development

The ISC team continues to work on refactoring the multithreading design. The ultimate goal is to keep as much data as possible local to the I/O event loop in each thread to reduce contention between the threads, and only share the data structures that have to be shared - like the zone database, cache, or address database (ADB). Everything else (networking) has strong affinity to the thread that initiated the request.

More major refactoring is happening “under the hood” in 2023. BIND’s underlying database for storing zone records and cache uses a modified version of a red-black tree. Tony Finch has been working to adapt a qp-trie (https://dotat.at/prog/qp/) optimized for storing DNS names in BIND, to replace the venerable red-black tree in upcoming releases. We expect it will reduce memory usage and reduce the need for multithreaded locking.

We plan to add PROXYv2 support in one of the upcoming development releases (currently for -S edition only). This should be of interest to operators employing sophisticated forwarding architectures. Some of the RFC 8914 Extended Errors were added in 2022, with a few more still to come.

BIND 9 Quality Assurance - Michał Kępień

ISC’s BIND 9 QA team grew by one full-time engineer and one part-time intern in 2023, for a total of four full-time staff and our intern.

• We have revised our security incident handling and code-signing processes to bring outdated procedures up-to-date and simplify and automate operations where possible. See this blog post for the code-signing update.
• ISC released 34 BIND open source releases (plus the associated private -S versions) in 2022. Each of these had a corresponding Docker, Debian, Ubuntu, and Fedora version, making a total of 170 packages published. A few of the older versions also had MS Windows binaries as well. We are continuing to automate as many release-related tasks as possible because of the high volume of releases.
• The BIND 9 QA team triaged and investigated many bug reports, including a few that led to very long threads … see this blog post on a jemalloc issue!
• We performed the usual tedious work on bringing the OS versions used in our CI up to date (this was often non-trivial!). This also included moving away from CentOS due to that project’s fate.
• We automated configuration grammar processing as part of a significant update to the BIND ARM, which added hyperlinks to every command and significantly overhauled coverage of DNSSEC. The idea for the hyperlinking, and most of the new introductory text and images, were contributed by Ron Aitchison, author of the popular reference work “DNS and BIND for Rocket Scientists.”
• The QA team prepared images for testing BIND 9 on FIPS-ready platforms (this is still a work-in-progress as far as actual testing goes - but we needed to lay some groundwork for that).
• We finished (for the time being) work on improving the resolver performance benchmarking environment (more reliable provisioning, more readable charts/results and the codebase is better organized and easier to maintain).
• The QA team added new checks for automatic detection of leaking memory that is not tracked by named’s internal memory contexts (https://gitlab.isc.org/isc-projects/bind9/-/issues/3054).
• We made improvements to ISC’s Perflab test tool (https://github.com/isc-projects/perflab/pulls?q=is%3Apr+author%3Apspacek).
• Petr Spaček gave a talk at the July 2022 DNS-OARC workshop on the Performance Effects of DNSSEC Validation. The upshot was that “DNSSEC validation has a negligible impact on server latency, bandwidth usage, CPU time consumption, and the number of OS sockets. The only area of our tests in which we saw a significant impact from DNSSEC validation was memory consumption, which increased by about 10% with validation.”

Contributions to Community Projects

In addition to their work for ISC, BIND 9 team members contributed to a number of community projects.

• Ondřej Surý:

  • OpenSSL 3.x: helped set up the PKCS#11 Provider: https://github.com/latchset/pkcs11-provider/graphs/contributors
  • libuv: fixed libuv recvmmsg() with MUSL libc: https://github.com/libuv/libuv/pull/3419

• Michał Kępień:

  • Wireshark: implemented displaying NSEC3 Next Hashed Owner Name field in Base32-encoded form: https://gitlab.com/wireshark/wireshark/-/merge_requests/620
  • Linux kernel: implemented a new ioctl for reading the contents of flash memory devices, to address the deficiencies in the previously available ioctls: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=095bb6e44eb17da2cf95dbde9c83b44664a493f5

• Petr Špaček:

  • dnsperf: implemented the ability to send arbitrary queries, fast: https://github.com/DNS-OARC/dnsperf/pull/201; implemented the option to NOT log various annoying high-frequency messages like “unexpected message ID”: https://github.com/DNS-OARC/dnsperf/pull/196
  • python-gitlab: improvements for retry logic - retries when a GitLab resource is locked: https://github.com/python-gitlab/python-gitlab/pull/2326

• Tom Krizek:

  • DNS Shotgun: made various improvements in charts and PCAP processing: https://gitlab.nic.cz/knot/shotgun/-/merge_requests?scope=all&state=merged&author_username=isc-tkrizek
  • respdiff-rs: implemented a CLI interface to allow for future extensions: https://gitlab.nic.cz/knot/respdiff-rs/-/merge_requests/8

• Michal Nowak:

  • Fixed software collections for Fedora 35+: https://github.com/sclorg/scl-utils/pull/43
  • Reported a broken libuv 1.43.0 tarball: https://github.com/libuv/libuv/issues/3401
  • Worked with the Mock community to implement Oracle Linux 9 support: https://github.com/rpm-software-management/mock/pull/963
  • Worked with the packager of experimental GCC RPMs to fix a packaging issue: https://github.com/jwakely/pkg-gcc-latest/issues/8

• Artem Boldariev:

  • Commented on the design for QUIC support in OpenSSL (https://github.com/openssl/openssl/pull/17185)
  • Updated his MSI installer for GNU Emacs on Windows (https://github.com/arbv/emacs-msi-installer)

2022 Board or Committee Involvement

• Petr Špaček:
  • Became a DNS-OARC Program Committee member.
  • Continues sitting on IETF’s ICANN Technical Liaison Group (although the group is dormant and had literally zero requests in the past year)
  • Made minor contributions to the IETF’s standardization of catalog zones
  • Performed substantial work on RFC 9276: https://datatracker.ietf.org/doc/html/rfc9276