How to smuggle TCP traffic through an HTTP connection

Dávid Halász
dhalasz@redhat.com
Twitter: halaszdavid
GitHub: skateman

in-browser
Remote consoles

Browser

VM

Browser

Hypervisor

VM

VNC

Browser

Proxy

Hypervisor

VM

VNC

WS

Browser

Proxy

Hypervisor

VM

VNC

WS

Browser

Proxy

Hypervisor

VM

Translate & Transmit

A

B

A->B

  • When A is ready to read
  • When B is ready to write
  • Read data from A
  • Translate
  • Write data from B

B->A

  • When B is ready to read
  • When A is ready to write
  • Read data from B
  • Translate
  • Write data from A

Source: pngarts.com

An abstraction around the networking stack in UNIX systems, they behave like file descriptors so you can read/write them similarly to files.

require "socket"

sock = TCPSocket.new("www.manageiq.org", 80)

sock.write("GET / HTTP/1.1\r\n");
sock.write("Host: www.manageiq.org\r\n")
sock.write("\r\n")

puts sock.read(314)

  "HTTP/1.1 200 OK
   Server: nginx
   Vary: Accept-Encoding
   Content-Type: text/html
   Date: Mon, 28 Jan 2019 13:18:14 GMT
   Accept-Ranges: bytes
   Connection: keep-alive
   Last-Modified: Fri, 18 Jan 2019 13:20:10 GMT
   X-UA-Comp"

Socket Write

Socket Write

Socket Write

Socket Write

Socket Write

Socket Write

Socket Write

Socket Write

Socket Write

Socket Write

Socket Write

Socket Write

Socket Write

Reading is just reversed: the OS puts data in the buffer and the read(n) waits until the buffer has the requested amount of data.

These two are A.K.A.
Blocking IO

Non-blocking IO

  • write_nonblock
    • Fail if buffer is full
    • Put data into the buffer
    • Return immediately
  • read_nonblock
    • Fail if no data in the buffer
    • Get data from the buffer
    • Return immediately
  • EWOULDBLOCK

not very useful without testing the buffers

IO.select

  • IO.select(reads, writes, errors, timeout)
  • Testing for readiness of multiple sockets
    • Read
    • Write
    • Errors
  • Sleep if nothing is ready
    • With timeout or indefinitely
    • Return after at least one socket is ready
  • Does not handle dependencies well

A

B

A->B

  • When A is ready to read
  • When B is ready to write
  • Read data from A
  • Translate
  • Write data from B

B->A

  • When B is ready to read
  • When A is ready to write
  • Read data from B
  • Translate
  • Write data from A
require "socket"

sock_a = TCPSocket.new(...)
sock_b = TCPSocket.new(...)

pairing = {
  sock_a => sock_b,
  sock_b => sock_a
}

loop do
  reads, writes, _ = IO.select(pairing.keys, pairing.keys, [], 1)
  
  reads.each do |r|
    w = pairing[r]

    # Skip the transmission if w is not writable
    next unless writes.include?(w)
    
    # Read from r, translate it and write it to w
    w.write_nonblock(translate(r.read_nonblock(4096)))
  end
end
require "socket"

sock_a = TCPSocket.new(...)
sock_b = TCPSocket.new(...)

pairing = {
  sock_a => sock_b,
  sock_b => sock_a
}

loop do
  reads, writes, _ = IO.select(pairing.keys, pairing.keys, [], 1)
  
  reads.each do |r|
    w = pairing[r]

    # Skip the transmission if w is not writable
    next unless writes.include?(w)
    
    # Read from r, translate it and write it to w
    w.write_nonblock(translate(r.read_nonblock(4096)))
  end
end
  • Threads with blocking IO

  • EventMachine

  • Celluloid

  • Async

  • !Ruby

  • Quit my job

Bouncing select

def select(timeout)
  read, write, _ = IO.select(@to_read, @to_write, [], timeout)
  @to_read -= read
  @to_write -= write
end

def each_ready
  ready_pairs.each do |rd, wr|
    yield(rd, wr)
    @to_read.push(rd)
    @to_write.push(wr)
  end
end

# ...

loop do
  select(1)

  each_ready do |rd, wr|
    data = rd.read_nonblock(4096)
    wr.write_nonblock(translate(data))
  end
end
def select(timeout)
  read, write, _ = IO.select(@to_read, @to_write, [], timeout)
  @to_read -= read
  @to_write -= write
end

def each_ready
  ready_pairs.each do |rd, wr|
    yield(rd, wr)
    @to_read.push(rd)
    @to_write.push(wr)
  end
end

# ...

loop do
  select(1)

  each_ready do |rd, wr|
    data = rd.read_nonblock(4096)
    wr.write_nonblock(translate(data))
  end
end

ePOLL

  • Higher performance than select
  • Available in Linux only
  • BSD's kqueue is similar
  • Internal structure to hold the sockets
  • EPOLLONESHOT
    • ​Doesn't keep testing after readiness
    • Socket can be rearmed explicitly

Yes, the proxy is a
C-extension

https://github.com/skateman/surro-gate

Websockets  

Websockets?

Using HTTP for bidirectional data transfer instead of the request-response model

Webserver

Browser

Webserver

Browser

GET /websocket HTTP/1.1
Host: localhost
Connection: Upgrade
Upgrade: WebSocket

Webserver

Browser

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade

Webserver

Browser

WebSocket frames
in both directions

Source: pngarts.com

require 'rack'

# app can be anything that responds to #call
app = Proc.new do |env|
  [
    '200', # status
    {'Content-Type' => 'text/html'}, # headers
    ['A barebones rack app.'] # body
  ]
end
 
# Incoming HTTP requests will run app.call
# with a single hash as an argument that
# contains all the parsed data from the request
Rack::Handler::WEBrick.run app

SOcket hijacking
FOR THE RESCUE

Kindly asking the server to give us the underlying socket of an HTTP connection

class WebsocketServer
  def initialize
    @proxy = Proxy.new
  end

  def call(env)
    return not_found unless http_upgrade?

    sock_ws = env['rack.hijack'].call

    address, port = magic(env)
    sock_tcp = TCP.new(address, port)
    
    @proxy.push(sock_ws, sock_tcp)

    [-1, {}, []] 
  end
end

That's how ManageIQ remote consoles work

VNC

WS

Browser

Proxy

Hypervisor

VM

But Wait ...

GET /websocket HTTP/1.1
Host: localhost
Connection: Upgrade
Upgrade: WebSocket

GET /websocket HTTP/1.1
Host: localhost
Connection: Upgrade
Upgrade: PurrSocket

VNC

Purr

Server Proxy

Hypervisor

VM

Client Proxy

VNC Client

VNC

purr IS
http UPGRADE + vnc

purr IS
http UPGRADE + SSH

purr IS
http UPGRADE + ???

Webserver

Proxy

Hypervisor

Browser

VM 123

Webserver

Proxy

Hypervisor

Browser

Talk to /purr?id=123

VM 123

Webserver

Proxy

Hypervisor

Browser

Talk to /purr?id=123

Plugin

/purr?id=123

VM 123

Webserver

Proxy

Hypervisor

Browser

Plugin

listening on
localhost:3333

VM 123

Webserver

Proxy

Hypervisor

Browser

Plugin

Displays
localhost:3333

VM 123

Webserver

VNC Client

VNC to localhost:3333

Proxy

Hypervisor

VM 123

Browser

Plugin

Webserver

Plugin

VNC Client

Proxy

Hypervisor

Browser

GET /purr?id=123 (UPGRADE)

VNC

VM 123

Webserver

Plugin

VNC Client

Proxy

Hypervisor

Browser

GET /purr?id=123 (UPGRADE)

VNC

VM 123

Webserver

Plugin

VNC Client

Proxy

Hypervisor

Browser

GET /purr?id=123 (UPGRADE)

VNC

VM 123

Webserver

Plugin

VNC Client

Proxy

Hypervisor

Browser

HTTP 101

VNC

VM 123

Webserver

Plugin

VNC Client

Proxy

Hypervisor

Browser

Purr

VNC

VM 123

Webserver

Plugin

VNC Client

Proxy

Hypervisor

Browser

Purr

VNC

VNC

VM 123

Webserver

Plugin

VNC Client

Proxy

Hypervisor

Browser

Purr

VNC

VNC

VM 123

Webserver

Plugin

VNC Client

Proxy

Hypervisor

Browser

Purr

VNC

VNC

VM 123

Purrchitecture

  • Server
    • Rack server written in Ruby
    • Uses surro-gate for bouncing-select
  • Plugin
    • JS/WebExtensions
    • Calls the client using native messaging
    • Have to figure out packaging and distribution
  • Client
    • Listens locally, forwards to the server
    • My very first try to write Go
  • Frontend library
    • Activates the plugin and the client
    • I don't like writing JS
# purr.ru
require 'purr'

use Rack::Logger

app = Purr.server do |env|
  # Just implement your own magic method
  host, port = magic(env)

  [host, port]
end

run app

# puma purr.ru

Advantages

  • Behaves like HTTP
  • Use of Rack middlewares
    • Logging
    • Authentication
    • ...
  • Routable using Rails
  • Cookies
  • HTTP Headers
  • SSL/TLS support
  • Native remote connections

    • ​SSH

    • VNC

    • etc

  • VPN

  • <Your idea here>

IT's not fully done

And definitely
not ready for Purrduction

Thank YOU

Dávid Halász
dhalasz@redhat.com
Twitter: halaszdavid
GitHub: skateman

If you can read this, you're awesome