If you believe the security of the device you are whitelisting is in good shape this shouldn't be a concern, but any compromise on the whitelisted device/network will not be caught by the IDS at that point due to the whitelisting. So it is "safe" so long as the device or network you're whitelisting is "safe" but I do recommend monitoring reports to look for changes in behavior as these could point out compromised environments that wouldn't be caught otherwise due to the whitelisting.
Kyle Kerst
IT Coordinator
SmarterTools Inc.
www.smartertools.com