I was having dinner with some of the IT guys from a client I work for and I brought up the suggestion that they do MAC filtering for all of their network devices as an added security measure. The only problem was they were using Windows Server 2003 for the DHCP server which, natively, does not support MAC filtering. That's where the DHCP Server Callout DLL comes in.

This is a DLL that was created by the Microsoft DHCP team to allow access to certain parts of DHCP that were not before. In this case, MAC filtering.

Why filter MAC addresses

The idea of DHCP MAC filtering is that when a foreign system tries to connect to your network, they are not given an IP address unless their network card is on the list of allowed systems. In order for them to get on the network, they have to see a member of the IT department.

This protects a guest from accidentally spreading infections of spyware, viruses, or trojans not to mention it helps the IT department keep track of who and what goes on the network. [Please notice how I say accidentally because MAC spoofing would easily circumvent this security measure]

Install The Callout DLL

Overview:

• Install the DLL
• Create the necessary registry keys
• Populate the list of allowed or denied MAC addresses
• Restart the DHCP

Download and Install

1. Download and install the files: Download
   The MACFilterCallout.dll was installed to %SystemRoot%\system32 along with a file named SetupDHCPMacFilter.rtf.This includes very basic instructions.
2. Run the MacFilterCallout.msi and go through the steps to install it. All this does is extract the two files to your %systemroot%\system32\ folder.

Create the registry keys:

Choose one of two ways:
Option 1: Manually create the following registry keys:

Key Name	Key Type	Description
CalloutDlls	REG_MULTI_SZ	The location of the MacFilterCallout.dll
CalloutEnabled	DWORD	0 = Disable MacFilterCallout 1 = Enable MacFilterCallout
CalloutErrorLogFile	REG_MULTI_SZ	Log path. If this registry key is not specified, callout dll will output errors %WINDIR%\System32\Log.txt
CalloutInfoLogFile	REG_MULTI_SZ	Info log path. If this key is not present, no information messages will be logged.
CalloutMACAddressListFile	REG_MULTI_SZ	This is the name and location of the MAC filtering list you're going to be creating next.

Option 2: Merge the keys that I've made for you: Download

Download the file above, extract the contents, and merge the registry file that I created for you.

Here are the values the .REG file contains. Make sure they match up to your environment.

Key Name	Value
CalloutDlls	C:\windows\system32\MacFilterCallout.dll
CalloutEnabled	1
CalloutErrorLogFile	C:\windows\system32\MacFilterCallout.log
CalloutInfoLogFile	C:\windows\system32\MacFilterCalloutInfo.log
CalloutMACAddressListFile	C:\windows\system32\MAClist.txt

NOTE: If you are not using C:\windows as your windows directory, you will have to edit the registry to fit your system.

Create the MAC list

As I showed above, the key CalloutMACAddressListFile points to a location where you need to create a specially formatted text file that contacts which MAC addresses to filter.You can only choose to allow a certain set of MAC's or DENY them. Here is the format of that file:

MAC_ACTION = {ALLOW / DENY} 0cdc1c6d1266 05dc2de01023 ...

Note: You must include the { }'s around either the ALLOW or DENY action

Help Populating the MAC list

If you are going to use the ALLOW action you're most likely going to want to find all of the valid MAC addresses on the network. Here are some suggestions for ways you can do this:

• Nmap + ARP- with the command nmap -PR 192.168.0.0/24 or whatever your network is, it will do an arp scan of the network. Then doing an "arp -a > arptable.txt" gives you tab delimited file perfect for opening as a spread sheet and extracting the list of MAC addresses you need to use
• DHCP logs - use your existing DHCP server logs [usually under c:\windows\system32\dhcp\] to find all the MAC addresses in the last week.
• Switch logs - if you have a good enough switch, it will keep track of which MAC addresses are using the devices.

Note: Obviously be careful how you create this list. If the CEO of the company has a laptop that you happened to forget to put onto the allow list, he may not be happy with your new security measure.

Want to know what the MSI Installer REALLY does?

One major pet peeve of mine is when you download a program and it installs without telling you what it did. That's how this MSI works. Here's what it does:

• copies the dll to %system%/system32/
• copies the rtf to %system%/system32/
• registers BOTH the dll and the rtf as shared DLL's
• adds some interesting registry keys like on named "CompleteMacLevel" that I don't know what it does.
  

I know that may not help anything but it makes me feel a little better.

External Links

http://blogs.technet.com/teamdhcp/archive/2007/10/03/dhcp-server-callout-dll-for-mac-address-based-filtering.aspx - DHCP server team's blog with the original article