Several DNS Applications, in this case, Microsoft DNS support EDNS0 which extends query and response datagrams. When upgrading your DNS infrastructure to Windows 2008 R2 from earlier versions or other versions of DNS, you may notice some peculiar behavior. You may notice that certain hostnames may not be resolvable any longer from these new DNS servers. However, other hostnames appear to resolve just fine.
EDNS, or Extension mechanisms for DNS, also known as EDNS0, is a specification for expanding the size of several parameters of the Domain Name System (DNS) protocol. What you will see using a protocol analyzer is that the DNS packets will contain an OPT (optional) resource record which contains the extra parameters.
These optional records may be inserted in the communications between DNS nodes to mark a data transfer using EDNS. Older clients without EDNS support simply ignore the new record type. DNS resolvers should only send EDNS requests to a DNS server if they are prepared to accept an EDNS response.
The cause of this issue is most likely due to your perimeter firewall’s inability to allow the passage of these packets. Some firewalls are hardwired to expect that DNS/UDP datagrams will always be at most 512 bytes long, an expectation that is incorrect and will simply discard any DNS/UDP datagrams that are longer. If the firewall drops or discards these packets, query resolution will fail as the back-end queries time out without receiving responses.
The easiest way to resolve this issue is to configure your DNS server as a “forwarder”. If your DNS server is configured to forward to your ISP’s DNS server(s) or a public DNS server, this issue is immediately mitigated. Another method that can be used, if forwarding is not an option, is to disable EDNS0 on your DNS server(s). This option, in my opinion, is a temporary one and should not be considered as the first option.
To temporarily resolve this issue while you upgrade your firewall’s firmware and/or replace it, you can disable EDNS0 on your Windows DNS servers.
To disable EDNS0, you can make the changes from the command prompt, or by editing the registry directly.
Command prompt (no restart is required):
dnscmd /config /EnableEDNSProbes 0 (The value of “0” disables EDNS0 and the value of “1” enables it)
Registry (requires the DNS service to be restarted):
Create a DWORD called EnableEDNSProbes and set to 0 in HKLM\SYSTEM\CurrentControlSet\services\DNS\Parameters
This summary does not necessarily encourage you to configure your DNS servers in this manner. The intention is only to provide a temporary solution until you are ready to support the DNS extensions.
