By default, when your code unloads the DLL (with __FUnloadDelayLoadedDLL), the delay-loaded imports remain in the IAT table. However, if you use the /delay:unload switch on the linker command line, the helper function will support the explicit unloading of the DLL, resetting the IAT to its original form; the now-invalid pointers will be overwritten.
See Unloading a Delay-Loaded DLL for more information.