While using a VPN, you have probably come across references to IPSec and OpenVPN. What’s more, you have to choose one or the other. How do you decide which protocol suits your current needs and requirements better? Read this article to learn more about what OpenVPN and IPSec are, how they differ from each other, and their relative pros and cons.
What is a VPN?
A virtual private network (VPN) is a tunneled communication channel that can span either private and trusted or public and untrusted networks. VPN protocols often employ encryption techniques that secure the connection and protect it from eavesdropping or data manipulation when transferred over untrusted parts of the network (i.e. the Internet). One of VPNs advantages is their ability to interconnect separated and often geographically dispersed networks as if (from the point of view of the end user) there were no extra routers in between, thus allowing communication without the need for reconfiguring third-party networking devices. So you can think about VPN as a tunnel between two devices or locations. For example, it can allow remote access to a company's resources, it can provide a secure communications channel between company offices, or even allow interconnectivity with parts of private networks belonging to other companies.
This article focuses on encrypted types of VPN protocols.
OpenVPN—a definition
OpenVPN is an example of a virtual private network (VPN) protocol. It is an open source, free VPN protocol that can be installed as a software suite on all of the popular operating systems. It is most commonly used for client-to-site connectivity, such as employee remote access.
IPSec—a definition
"IPSec'' stands for Internet Protocol Security. IPSec is a framework of connected protocols that ensures data confidentiality. It protects data by encrypting packets before their transmission over a network. IPSec also cares about data integrity—it checks that the transmission does not influence the packets and does not change them. This VPN protocol provides data authentication at the IP layer, and can protect more than one data flow. IPSec also has an anti-replay feature that identifies and rejects replayed packets.
It is most commonly used for site-to-site connectivity, such as allowing communication between multiple offices belonging to the same company.
Are you wondering how IPSec works? Check out our previous article to get the details.
At first sight, OpenVPN and IPSec are similar—both were created to protect data. However, there are some differences too.
Differences between OpenVPN and IPSec
This is a short list of the most important differences you should be aware of.
- Installation process
OpenVPN requires extra software that the operating system (OS) usually does not have installed by default. On the other hand, IPSec is directly supported by many modern operating systems, such as Windows, Ubuntu, macOS, and Android, iOS/iPadOS and can be used out of the box.
- Security level
OpenVPN is famous for its limited number of vulnerabilities, and some describe it as the most secure protocol. It allows for choosing from multiple cipher suites, such as the trusted and well-established AES, and more modern options such as ChaCha. It also gives you the ability to select the tunneling protocol, with support for standards such as TLS 1.3
When it comes to IPSec with IKEv2, this also gives you the ability to choose from multiple cipher suites, although the spectrum is usually not so wide compared to OpenVPN. The reason for that may be the fact that IPSec is often accelerated in the hardware (especially on networking devices such as firewalls), which provides great performance, but at the cost of a limited number of supported encryption protocols.
- Firewall Ports
OpenVPN uses a chosen UDP or TCP port, allowing for flexible configuration choices. On the other hand, IPSec uses predefined communication channels, UDP 500 and UDP 4500, to establish the encrypted tunnel and ESP for the transmission of encrypted data.
The advantages and disadvantages of both solutions emerge from the list above. In the next section, we will explore them in more depth.
| IPSec | OpenVPN | |
|---|---|---|
| Easy to install | YES | depends on OS |
| Ease of management | complex | complex |
| Available documentation and guides | plentiful (especially for site-to-site scenarios) | plentiful (especially for client-to-site scenarios) |
| Authentication by password | YES | YES |
| Authorization by certificate | YES | YES |
| Authentication by server (e.g, LDAP, RADIUS, etc.) | YES (for client-to-site tunnels) | YES |
| Support for L3 unicast traffic | YES | YES |
| Support for L2 traffic | NO (requires additional tunneling protocols) | YES |
| Support for point-to-multipoint tunnels | YES | NO |
| Support for different transmission protocols | NO (easy to block on the FW) | YES (ability to run TCP/UDP on the port of choice) |
| Supported on networking devices | YES (popular solution) | very limited support |
| Dynamic routing in tunnel | YES | YES |
| Ability to set up a tunnel if one side’s private IP address is hidden behind NAT (NAT traversal) | YES | YES |
| Support for IPv6 | YES | YES |
Advantages of IPSec and OpenVPN
- IPSec benefits
Everything above, taken together, makes IPSec strong protection for a network. IPSec implementation in the firewall or router does not require changing any software on the user or server systems. IPSec is also preinstalled in the user system—this is a real benefit when working with remote employees with no entitlement to install software on their computers.
- OpenVPN advantages
With its 256-bit encryption keys and high-end ciphers, OpenVPN makes information takeover difficult. The ability to use any port on TCP or UDP, means a connection through OpenVPN will appear as a regular HTTPS and pass through the firewall without any problem.
OpenVPN software is free for download, but nowadays, a paid version—OpenVPN Access—is more widely promoted. This solution is compatible with devices produced by the most prominent companies on the market. Furthermore, if OpenVPN disconnects, it will pause the network until the connection can be repaired or reconfigured or maintain only inner company traffic (depending on the configuration).
Disadvantages of IPSec and OpenVPN
As we all know, every solution has its vulnerabilities and weaknesses. What disadvantages do you have to be aware of when choosing between OpenVPN and IPSec?
- Cons of IPSec
While IPSec is a well-established and commonly used protocol, the encrypted tunnel setup requires the appropriate configuration of multiple parameters, which may seem quite overwhelming for a new user. If the VPN is not working as expected, troubleshooting usually involves browsing through a large number of logs with complex terminology.
While IPSec has good support for client-to-site connectivity, it is not as flexible as OpenVPN.
- OpenVPN disadvantages
As in the case of IPSec, OpenVPN's configuration can be demanding and complex. A lot of manual settings and a wide variety of possible options require expert knowledge. An inappropriate setup can do more harm than good. For that reason, OpenVPN can be perceived as not a user-friendly solution. As mentioned before, the complexity of OpenVPN goes further and it is not preinstalled on any OS, so it requires configuration of third-party software.
WireGuard—an alternative for IPSec and OpenVPN?
If neither OpenVPN or IPSec wins you over, there is another option—the WireGuard. This solution is well known for its easy implementation, safety, cross-platforming, and high performance. It is a great choice when you want to quickly set up a secure connection without the need for complex configuration. WireGuard describes itself as a modern solution more useful than IPSec but has a severe disadvantage—very limited configuration options, making it not very friendly for enterprise environments.
There are also hybrid, alternative solutions like SoftEther, which support multiple types of tunneling protocols (it can even tunnel traffic in ICMP). Multiple commercial client-to-site options are on the market—they are also worth considering in specific cases.
Conclusion
When is it worth considering each of the mentioned solutions? IPSec is often used for site-to-site connections between network devices (mainly firewalls but can also be used between servers). It is common for encrypting transmissions between company branches and for communication with selected subnets/addresses in partner companies' networks. OpenVPN can be the best-fit solution when building a client-to-site connection (remote access to the company for users). On the other hand, if features like flexibility in authentication, ease in adding more users who can use a VPN, etc. are not the priority, and quick and easily set up encryption for a connection is the aim, WireGuard may be the answer.
This paper aims not to point to a single right choice, but rather to show these solutions in a wider context, with their pros and cons, sharing the necessary knowledge to make a wise and informed decision suited to your requirements and needs.
