IPsec Explained
IPsec is the protocol suite that encrypts and authenticates IP traffic — the engine inside most site-to-site VPNs. Three ideas unlock it: ESP does the protecting, tunnel vs transport decides how much is wrapped, and IKE negotiates the keys.
ESP vs AH — what protects the packet
ESP (Encapsulating Security Payload) encrypts and authenticates — it is what real deployments use. AH (Authentication Header) authenticates only, with no encryption, and breaks behind NAT — so in practice it is rare. Remember: ESP = confidentiality + integrity; AH = integrity only.
Tunnel vs transport mode
| Tunnel mode | Transport mode | |
|---|---|---|
| What is protected | The entire original packet (new IP header added) | Only the payload; original header kept |
| Hides internal IPs? | Yes | No |
| Typical use | Site-to-site VPN between gateways | Host-to-host protection |
Site-to-site VPNs use tunnel mode: the original packet — private addresses and all — is wrapped whole inside a new packet between the two gateways.
IKE — the two-phase key negotiation
- Phase 1 — the peers authenticate each other (pre-shared key or certificates) and build a secure management channel (the ISAKMP/IKE SA). Runs on UDP 500 (4500 behind NAT).
- Phase 2 — inside that channel they negotiate the actual IPsec SAs: which traffic to protect, with which ciphers, rekeying on schedule.
When a tunnel will not come up, troubleshoot in phase order: phase 1 mismatches (auth, ciphers, peer IPs) before phase 2 (interesting-traffic ACLs, transform sets).
Where you meet IPsec
Site-to-site tunnels between offices, remote-access clients, and cloud hybrid links (AWS/Azure VPN gateways are IPsec underneath). GRE-over-IPsec pairs routing flexibility with encryption — see GRE tunnels.
Frequently asked questions
What is the difference between ESP and AH?
ESP encrypts and authenticates traffic and works through NAT — it is what real deployments use. AH only authenticates (no encryption) and breaks behind NAT, so it is rarely deployed.
What is the difference between tunnel and transport mode?
Tunnel mode wraps the entire original packet in a new one (used gateway-to-gateway in site-to-site VPNs, hiding internal addresses). Transport mode protects only the payload and keeps the original header (host-to-host).
What does IKE do?
Internet Key Exchange authenticates the two peers and negotiates encryption keys — phase 1 builds a secure management channel, phase 2 negotiates the IPsec SAs that protect actual traffic.
What ports does IPsec use?
IKE uses UDP 500, switching to UDP 4500 for NAT traversal. ESP itself is IP protocol 50 (not a TCP/UDP port).
Related articles
Want hands-on training?
Learn this on real Cisco lab devices with placement support at Attila Technologies, Ahmedabad.