CalvinBackup/ctrld
1
0
Fork 0
mirror of https://github.com/Control-D-Inc/ctrld.git synced 2026-05-15 00:50:25 +02:00
Code Issues Packages Projects Releases Wiki Activity

ios support.

Browse Source
This commit is contained in:
Ginder Singh
2026-03-19 03:55:25 -04:00
parent ae37c56467
commit 593805bf6f
5 changed files with 186 additions and 366 deletions
Download Patch File Download Diff File Expand all files Collapse all files
cmd/ctrld_library/netstack/README.md
+137 -362
View File
@@ -1,428 +1,203 @@
# Netstack - Full Packet Capture for Mobile VPN
Complete TCP/UDP/DNS packet capture implementation using gVisor netstack for Android and iOS VPN apps.
Complete TCP/UDP/DNS packet capture implementation using gVisor netstack for Android and iOS.
## Overview
This module provides full packet capture capabilities for mobile VPN applications, handling:
Provides full packet capture for mobile VPN applications:
- **DNS filtering** through ControlD proxy
- **TCP forwarding** for HTTP/HTTPS and all TCP traffic
- **UDP forwarding** for games, video streaming, VoIP, etc.
- **Socket protection** to prevent routing loops on Android/iOS
- **TCP forwarding** for all TCP traffic
- **UDP forwarding** with session tracking
- **Socket protection** to prevent routing loops
- **QUIC blocking** for better content filtering
## Architecture
```
Mobile Apps (Browser, Games, etc)
Mobile Apps → VPN TUN Interface → PacketHandler → gVisor Netstack
↓
VPN TUN Interface
- Device: 10.0.0.2/24
- Gateway: 10.0.0.1
- DNS: 10.0.0.1 (ControlD)
├─→ DNS Filter (Port 53)
│ └─→ ControlD DNS Proxy
├─→ TCP Forwarder
│ └─→ net.Dial("tcp") + protect(fd)
└─→ UDP Forwarder
└─→ net.Dial("udp") + protect(fd)
↓
PacketHandler (Read/Write/Protect)
↓
gVisor Netstack (10.0.0.1)
├─→ DNS Filter (dest port 53)
│ ├─→ DNS Bridge (transaction ID tracking)
│ └─→ ControlD DNS Proxy (localhost:5354)
│ └─→ DoH Upstream (freedns.controld.com)
│ └─→ Protected Socket → Internet
├─→ TCP Forwarder (non-DNS TCP traffic)
│ ├─→ net.Dial("tcp", destination)
│ ├─→ protect(fd) BEFORE connect()
│ └─→ io.Copy() bidirectional
└─→ UDP Forwarder (non-DNS UDP traffic)
├─→ net.Dial("udp", destination)
├─→ protect(fd) BEFORE connect()
└─→ Session tracking (60s timeout)
↓
Real Network (WiFi/Cellular) - All Sockets Protected
Real Network (Protected Sockets)
```
## Key Components
## Components
### 1. DNS Filter (`dns_filter.go`)
- Detects DNS packets (UDP port 53)
- Extracts DNS query payload
- Sends to DNS bridge
- Builds DNS response packets
### DNS Filter (`dns_filter.go`)
Detects DNS packets on port 53 and routes to ControlD proxy.
### 2. DNS Bridge (`dns_bridge.go`)
- Transaction ID tracking
- Query/response matching
- 5-second timeout per query
- Channel-based communication
### DNS Bridge (`dns_bridge.go`)
Tracks DNS queries by transaction ID with 5-second timeout.
### 3. TCP Forwarder (`tcp_forwarder.go`)
- Uses gVisor's `tcp.NewForwarder()`
- Converts gVisor endpoints to Go `net.Conn`
- Dials regular TCP sockets (no root required)
- Protects sockets using `VpnService.protect()` callback
- Bidirectional `io.Copy()` for data forwarding
### TCP Forwarder (`tcp_forwarder.go`)
Forwards TCP connections using gVisor's `tcp.NewForwarder()`.
### 4. UDP Forwarder (`udp_forwarder.go`)
- Uses gVisor's `udp.NewForwarder()`
- Per-session connection tracking
- Dials regular UDP sockets (no root required)
- Protected sockets prevent routing loops
- 60-second idle timeout with automatic cleanup
### UDP Forwarder (`udp_forwarder.go`)
Forwards UDP packets with session tracking and 60-second idle timeout.
### 5. Packet Handler (`packet_handler.go`)
- Interface for reading/writing raw IP packets
- Mobile platforms implement:
- `ReadPacket()` - Read from TUN file descriptor
- `WritePacket()` - Write to TUN file descriptor
- `ProtectSocket(fd)` - Protect socket from VPN routing
- `Close()` - Clean up resources
### Packet Handler (`packet_handler.go`)
Interface for TUN I/O and socket protection.
### 6. Netstack Controller (`netstack.go`)
- Manages gVisor stack lifecycle
- Coordinates DNS filter and TCP/UDP forwarders
- Filters outbound packets (source=10.0.0.x)
- Drops return packets (handled by forwarders)
### Netstack Controller (`netstack.go`)
Manages gVisor stack and coordinates all components.
## Critical Design Decisions
## Socket Protection
### Socket Protection
Critical for preventing routing loops:
**Why It's Critical:**
Without socket protection, outbound connections would route back through the VPN, creating infinite loops:
```
Bad (without protect):
App → VPN → TCP Forwarder → net.Dial() → VPN → TCP Forwarder → LOOP!
Good (with protect):
App → VPN → TCP Forwarder → net.Dial() → [PROTECTED] → WiFi → Internet ✅
```
**Implementation:**
```go
// Protect socket BEFORE connect() is called
// Protection happens BEFORE connect()
dialer.Control = func(network, address string, c syscall.RawConn) error {
return c.Control(func(fd uintptr) {
protectSocket(int(fd)) // Android: VpnService.protect()
protectSocket(int(fd))
})
}
```
**All Protected Sockets:**
1. **TCP forwarder sockets** - User HTTP/HTTPS traffic
2. **UDP forwarder sockets** - User games/video/VoIP traffic
3. **ControlD API sockets** - Configuration fetch (api.controld.com)
4. **DoH upstream sockets** - DNS resolution (freedns.controld.com)
**All protected sockets:**
- TCP/UDP forwarder sockets (user traffic)
- ControlD API sockets (api.controld.com)
- DoH upstream sockets (freedns.controld.com)
**Timing is Critical:**
## Platform Configuration
Protection must happen **DURING** socket creation, not after:
```go
// ✅ CORRECT - Protection happens BEFORE connect()
dialer := &net.Dialer{
Control: func(network, address string, c syscall.RawConn) error {
return c.Control(func(fd uintptr) {
protectSocket(int(fd)) // Called before connect() syscall
})
},
}
conn, _ := dialer.Dial("tcp", "google.com:443") // Socket already protected!
// ❌ WRONG - Protection happens AFTER connect()
conn, _ := net.Dial("tcp", "google.com:443") // SYN already sent through VPN!
rawConn.Control(func(fd uintptr) {
protectSocket(int(fd)) // TOO LATE - routing loop already started
})
```
### Outbound vs Return Packets
**Outbound packets** (10.0.0.x → Internet):
- Source IP: 10.0.0.x
- Injected into gVisor netstack
- Handled by TCP/UDP forwarders
**Return packets** (Internet → 10.0.0.x):
- Source IP: NOT 10.0.0.x
- Dropped by readPackets()
- Return through forwarder's upstream connection automatically
### Address Mapping in gVisor
For inbound connections to the netstack:
- `id.LocalAddress/LocalPort` = **Destination** (where packet is going TO)
- `id.RemoteAddress/RemotePort` = **Source** (where packet is coming FROM)
Therefore, we dial `LocalAddress:LocalPort` (the destination).
## Packet Flow Examples
### DNS Query (Port 53)
```
1. App queries google.com
DNS query: 10.0.0.2:54321 → 10.0.0.1:53
2. VPN TUN captures packet
PacketHandler.readPacket() returns raw IP packet
3. DNS Filter detects port 53
Extracts UDP payload (DNS query bytes)
4. DNS Bridge processes query
- Parses DNS message (transaction ID: 12345)
- Stores in pending map
- Sends to query channel
5. DNS Handler receives query
- Forwards to ControlD proxy: localhost:5354
- ControlD applies policies and filters
- Queries DoH upstream (freedns.controld.com) via PROTECTED socket
- Returns filtered response
6. DNS Bridge matches response
- Finds pending query by transaction ID
- Sends response to waiting channel
7. DNS Filter builds response packet
- Wraps DNS bytes in UDP/IP packet
- Swaps src/dst: 10.0.0.1:53 → 10.0.0.2:54321
8. PacketHandler writes response to TUN
App receives DNS response with IP address
```
### TCP Connection (HTTP/HTTPS)
```
1. App connects to google.com:443
SYN packet: 10.0.0.2:12345 → 142.250.190.78:443
2. VPN TUN captures packet
PacketHandler.readPacket() returns raw IP packet
3. Netstack filters packet
- Checks source IP = 10.0.0.2 (outbound? YES)
- Not DNS (port != 53)
- Injects into gVisor netstack
4. gVisor calls TCP forwarder
TransportEndpointID:
- LocalAddress: 142.250.190.78:443 (destination)
- RemoteAddress: 10.0.0.2:12345 (source)
5. TCP Forwarder creates connection
- Creates gonet.TCPConn (TUN side)
- Dials net.Dial("tcp", "142.250.190.78:443")
- Protects socket BEFORE connect() → No routing loop!
- Connects via WiFi/Cellular (bypasses VPN)
6. Bidirectional copy
- TUN → Upstream: io.Copy(upstreamConn, tunConn)
- Upstream → TUN: io.Copy(tunConn, upstreamConn)
- All HTTP data flows through protected socket
7. Connection closes when either side finishes
```
### UDP Packet (Games/Video)
```
1. Game sends UDP packet
UDP: 10.0.0.2:54321 → game-server.com:9000
2. VPN TUN captures packet
PacketHandler.readPacket() returns raw IP packet
3. Netstack filters packet
- Checks source IP = 10.0.0.2 (outbound? YES)
- Not DNS (port != 53)
- Injects into gVisor netstack
4. gVisor calls UDP forwarder
TransportEndpointID:
- LocalAddress: game-server.com:9000 (destination)
- RemoteAddress: 10.0.0.2:54321 (source)
5. UDP Forwarder creates/reuses session
- Creates gonet.UDPConn (TUN side)
- Dials net.Dial("udp", "game-server.com:9000")
- Protects socket BEFORE connect() → No routing loop!
- Connects via WiFi/Cellular (bypasses VPN)
6. Bidirectional forwarding
- TUN → Upstream: Read from gonet, write to net.UDPConn
- Upstream → TUN: Read from net.UDPConn, write to gonet
- Session tracked with 60s idle timeout
7. Auto-cleanup after 60 seconds of inactivity
```
## VPN Configuration (Android)
### Android
```kotlin
val builder = Builder()
.setSession("ControlD VPN")
.addAddress("10.0.0.2", 24) // Device address
.addRoute("0.0.0.0", 0) // Route all traffic
.addDnsServer("10.0.0.1") // DNS to local TUN (ControlD)
.setMtu(1500)
vpnInterface = builder.establish()
```
**Why DNS = 10.0.0.1:**
- Apps query 10.0.0.1:53 (local TUN interface)
- Queries captured by DNS filter
- Filtered through ControlD policies
- Apps see "ControlD" as DNS provider (not Cloudflare/Google)
```
## Usage Example (Android)
```kotlin
// In VpnService.startVpn()
// 1. Create config.toml (required by ctrld)
val configFile = File(filesDir, "config.toml")
configFile.createNewFile()
configFile.writeText("") // Empty config, uses defaults
// 2. Build VPN interface
val builder = Builder()
.setSession("ControlD VPN")
Builder()
.addAddress("10.0.0.2", 24)
.addRoute("0.0.0.0", 0)
.addDnsServer("10.0.0.1") // CRITICAL: Use local TUN for DNS
.addDnsServer("10.0.0.1")
.setMtu(1500)
vpnInterface = builder.establish()
override fun protectSocket(fd: Long) {
protect(fd.toInt()) // VpnService.protect()
}
// 3. Get TUN file descriptor streams
inputStream = FileInputStream(vpnInterface.fileDescriptor)
outputStream = FileOutputStream(vpnInterface.fileDescriptor)
// DNS Proxy: 0.0.0.0:5354
```
// 4. Implement PacketAppCallback
### iOS
```swift
NEIPv4Settings(addresses: ["10.0.0.2"], ...)
NEDNSSettings(servers: ["10.0.0.1"])
includedRoutes = [NEIPv4Route.default()]
func protectSocket(_ fd: Int) throws {
let index = Int32(if_nametoindex("en0"))
setsockopt(Int32(fd), IPPROTO_IP, IP_BOUND_IF, &index, ...)
}
// DNS Proxy: 127.0.0.1:53
```
## Protocol Support
| Protocol | Support |
|----------|---------|
| DNS (UDP/TCP port 53) | ✅ Full |
| TCP (all ports) | ✅ Full |
| UDP (except 53, 80, 443) | ✅ Full |
| QUIC (UDP/443, UDP/80) | 🚫 Blocked |
| ICMP | ⚠️ Partial |
| IPv4 | ✅ Full |
| IPv6 | ✅ Full |
## QUIC Blocking
Drops UDP packets on ports 443 and 80 to force TCP fallback:
**Why:**
- Better DNS filtering (QUIC bypasses DNS)
- Visible SNI for content filtering
- Consistent ControlD policy enforcement
**Result:**
- Apps automatically fallback to TCP/TLS
- No user-visible errors
- Slightly slower initial connection, then normal
## Usage (Android)
```kotlin
// Create callback
val callback = object : PacketAppCallback {
override fun readPacket(): ByteArray {
val length = inputStream.channel.read(readBuffer)
val packet = ByteArray(length)
readBuffer.position(0)
readBuffer.get(packet, 0, length)
return packet
}
override fun writePacket(packet: ByteArray) {
outputStream.write(packet)
}
override fun readPacket(): ByteArray { ... }
override fun writePacket(packet: ByteArray) { ... }
override fun protectSocket(fd: Long) {
// CRITICAL: Protect socket from VPN routing
val success = protect(fd.toInt()) // VpnService.protect()
if (!success) throw Exception("Failed to protect socket")
protect(fd.toInt())
}
override fun closePacketIO() {
inputStream?.close()
outputStream?.close()
}
override fun exit(s: String) { }
override fun closePacketIO() { ... }
override fun exit(s: String) { ... }
override fun hostname(): String = "android-device"
override fun lanIp(): String = "10.0.0.2"
override fun macAddress(): String = "00:00:00:00:00:00"
}
// 5. Create and start packet capture controller
packetController = Ctrld_library.newPacketCaptureController(callback)
// Create controller
val controller = Ctrld_library.newPacketCaptureController(callback)
packetController.startWithPacketCapture(
// Start
controller.startWithPacketCapture(
callback,
"your-cd-uid", // From ControlD dashboard
"", // Provision ID (optional)
"", // Custom hostname (optional)
filesDir.absolutePath, // Home directory
"doh", // Upstream protocol (doh/dot/os)
2, // Log level (0-5)
"${filesDir.absolutePath}/ctrld.log"
cdUID,
"", "",
filesDir.absolutePath,
"doh",
2,
"$filesDir/ctrld.log"
)
// 6. Stop when disconnecting
packetController.stop(false, 0)
// Stop
controller.stop(false, 0)
```
## Protocol Support
## Usage (iOS)
| Protocol | Support | Details |
|----------|---------|---------|
| **DNS** | ✅ Full | Filtered through ControlD proxy (UDP/TCP port 53) |
| **TCP** | ✅ Full | All ports, bidirectional forwarding |
| **UDP** | ✅ Selective | All ports except 53, 80, 443 (see QUIC blocking) |
| **QUIC** | 🚫 Blocked | UDP ports 443 and 80 dropped to force TCP fallback |
| **ICMP** | ⚠️ Partial | Basic support (no forwarding yet) |
| **IPv4** | ✅ Full | Complete support |
| **IPv6** | ✅ Full | Complete support |
```swift
// DNS-only mode
let proxy = LocalProxy()
proxy.start(
cUID: cdUID,
deviceName: UIDevice.current.name,
upstreamProto: "doh",
logLevel: 3,
provisionID: ""
)
### QUIC Blocking
QUIC (Quick UDP Internet Connections) is blocked by dropping UDP packets on ports 443 and 80:
**Why Block QUIC:**
- QUIC bypasses traditional DNS lookups (uses Alt-Svc headers)
- Encrypts server name indication (SNI)
- Makes content filtering difficult
- Prevents some ControlD policies from working
**How It Works:**
```go
// In netstack.go readPackets()
if protocol == UDP {
if dstPort == 443 || dstPort == 80 {
// Drop QUIC/HTTP3 packets
// Apps automatically fallback to TCP (HTTP/2 or HTTP/1.1)
continue
}
}
// Firewall mode (full capture)
let proxy = LocalProxy()
proxy.startFirewall(
cUID: cdUID,
deviceName: UIDevice.current.name,
upstreamProto: "doh",
logLevel: 3,
provisionID: "",
packetFlow: packetFlow
)
```
**Result:**
- Chrome/apps attempt QUIC first
- QUIC packets dropped silently
- Apps fallback to TCP/TLS (HTTP/2)
- ControlD policies work correctly
- Slightly slower initial connection, then normal speed
## Performance
| Metric | Value |
|--------|-------|
| **DNS Timeout** | 5 seconds |
| **TCP Dial Timeout** | 30 seconds |
| **UDP Idle Timeout** | 60 seconds |
| **UDP Cleanup Interval** | 30 seconds |
| **MTU** | 1500 bytes |
| **Overhead per TCP connection** | ~2KB |
| **Overhead per UDP session** | ~1KB |
## Requirements
- Go 1.23+
- gVisor netstack v0.0.0-20240722211153-64c016c92987
- For Android: API 24+ (Android 7.0+)
- For iOS: iOS 12+
## No Root Required
This implementation uses **regular TCP/UDP sockets** instead of raw sockets, making it compatible with non-rooted Android/iOS devices. Socket protection via `VpnService.protect()` (Android) or `NEPacketTunnelFlow` (iOS) prevents routing loops.
- **Android**: API 24+ (Android 7.0+)
- **iOS**: iOS 12.0+
- **Go**: 1.23+
- **gVisor**: v0.0.0-20240722211153-64c016c92987
## Files
- `packet_handler.go` - Interface for TUN I/O and socket protection
- `netstack.go` - Main controller managing gVisor stack
- `dns_filter.go` - DNS packet detection and response building
- `dns_bridge.go` - DNS query/response bridging
- `tcp_forwarder.go` - TCP connection forwarding
- `udp_forwarder.go` - UDP packet forwarding with session tracking
- `packet_handler.go` - TUN I/O interface
- `netstack.go` - gVisor controller
- `dns_filter.go` - DNS packet detection
- `dns_bridge.go` - Transaction tracking
- `tcp_forwarder.go` - TCP forwarding
- `udp_forwarder.go` - UDP forwarding
## License
cmd/ctrld_library/netstack/netstack.go
+8
View File
@@ -8,6 +8,7 @@ import (
"sync"
"time"
"github.com/Control-D-Inc/ctrld"
"gvisor.dev/gvisor/pkg/buffer"
"gvisor.dev/gvisor/pkg/tcpip"
"gvisor.dev/gvisor/pkg/tcpip/header"
@@ -182,6 +183,8 @@ func NewNetstackController(handler PacketHandler, cfg *Config) (*NetstackControl
started: false,
}
ctrld.ProxyLogger.Load().Info().Msg("[Netstack] Controller created with TCP/UDP forwarders")
return nc, nil
}
@@ -204,6 +207,8 @@ func (nc *NetstackController) Start() error {
nc.wg.Add(1)
go nc.writePackets()
ctrld.ProxyLogger.Load().Info().Msg("[Netstack] Packet processing started (read/write goroutines)")
return nil
}
@@ -269,6 +274,7 @@ func (nc *NetstackController) readPackets() {
if isDNS && response != nil {
// DNS packet was handled, send response back to TUN
nc.packetHandler.WritePacket(response)
ctrld.ProxyLogger.Load().Debug().Msgf("[Netstack] DNS response sent (%d bytes)", len(response))
continue
}
@@ -300,6 +306,8 @@ func (nc *NetstackController) readPackets() {
if dstPort == 443 || dstPort == 80 {
// Block QUIC (UDP/443) and HTTP/3 (UDP/80)
// Apps will fallback to TCP automatically
dstIP := net.IPv4(packet[16], packet[17], packet[18], packet[19])
ctrld.ProxyLogger.Load().Debug().Msgf("[Netstack] Blocked QUIC packet to %s:%d", dstIP, dstPort)
continue
}
}
cmd/ctrld_library/netstack/tcp_forwarder.go
+5
View File
@@ -7,6 +7,7 @@ import (
"syscall"
"time"
"github.com/Control-D-Inc/ctrld"
"gvisor.dev/gvisor/pkg/tcpip/adapters/gonet"
"gvisor.dev/gvisor/pkg/tcpip/stack"
"gvisor.dev/gvisor/pkg/tcpip/transport/tcp"
@@ -102,6 +103,10 @@ func (f *TCPForwarder) handleConnection(ep *tcp.Endpoint, wq *waiter.Queue, id s
}
defer upstreamConn.Close()
// Log successful TCP connection
srcAddr := net.IP(id.RemoteAddress.AsSlice())
ctrld.ProxyLogger.Load().Debug().Msgf("[TCP] %s:%d -> %s:%d", srcAddr, id.RemotePort, dstAddr.IP, dstAddr.Port)
// Bidirectional copy
done := make(chan struct{}, 2)
go func() {
cmd/ctrld_library/netstack/udp_forwarder.go
+7
View File
@@ -8,6 +8,7 @@ import (
"syscall"
"time"
"github.com/Control-D-Inc/ctrld"
"gvisor.dev/gvisor/pkg/tcpip/adapters/gonet"
"gvisor.dev/gvisor/pkg/tcpip/stack"
"gvisor.dev/gvisor/pkg/tcpip/transport/udp"
@@ -77,6 +78,12 @@ func (f *UDPForwarder) handlePacket(req *udp.ForwarderRequest) {
return
}
f.connections[connKey] = conn
// Log new UDP session
srcAddr := net.IP(id.RemoteAddress.AsSlice())
dstAddr := net.IP(id.LocalAddress.AsSlice())
ctrld.ProxyLogger.Load().Debug().Msgf("[UDP] New session: %s:%d -> %s:%d (total: %d)",
srcAddr, id.RemotePort, dstAddr, id.LocalPort, len(f.connections))
}
conn.lastActivity = time.Now()
f.mu.Unlock()