Baylessfamily.org

Today I learned (well last night anyway) that Fortinet has a problem with ‘Dialup’ VPN in the Fortigate firewalls running FortiOS 4(MR3 Patch 10). It may exist in other versions but the 3 firewalls I tested with that release (the latest) all show the same issue.

When creating an IPSec VPN, if you give the profile name something longer than about 8-10 characters (I was trying one about 14 characters), the VPN will be created successfully and everything will seem fine until you try to connect with the FortiClient VPN software. The connection attempt starts and you can see in the firewall logs that phase 1 negotiation begins. Then… nothing.

The client sits waiting, sometimes up to 5 or even 10 minutes without progress and the logs show nothing until:

Error negotiate Negotiate SA Error: No matching gateway for new phase 1 request.

The client spits out a generic message about something being wrong (Check your encryption key, addresses, authentication, whatever..).

After opening a ticket with Fortinet support and redoing the config I had done, the tech confirmed it did not work as it should. So we did some packet captures on the firewall and could see the initial connection to the firewall, a response from the firewall to the client and the client sending something back to the firewall, and then nothing. The firewall failed to respond to the client past the initial phase 1 negotiation.

So we turned on some debugging:

diagnose debug enable
diagnose debug application ike

This revealed the problem. When the firewall attempts to read from its list of configured VPN profiles, if the matching one has a name that is too long, the firewall spits out debug (but not a log entry) saying the name is too long and then just sits there.

Obviously a bug. Either the firewall should reject a profile name as too long, or it should allow longer names, or at least it should handle such an obvious design error with a usable log entry and should terminate the VPN connection immediately instead of leaving the client connection hanging until timeout.

Once we changed the VPN profile name to a few characters shorter, everything worked fine.