dechunking - simple but potentially confusing

So say you've got some http data to analyze, but the content isn't lining up right - binary extractionfrom the http server response only produces corrupt files when using trace carving tools like chaosreader. Check the server response headers to see if the Transfer-Encoding header is present and set to "chunked". If it is, you will need to reassemble the response body from the chunks before subsequent decoding/decompression.

download link

#!/bin/env ruby
require 'stringio'
end_of_headers="\r\n\r\n"
outbuf=""
chunksize=-1

filename=ARGV[0]

f=File.new(filename,"r")

everything=f.gets(nil)

bodystart=everything.index(end_of_headers)
sio=StringIO.new(everything)

header=sio.read(bodystart + end_of_headers.length)
if header.match(/Transfer-Encoding: chunked/i)
until chunksize==0
  chunksize=sio.gets("\r\n").rstrip.hex
  outbuf+=sio.read(chunksize)
  2.times {sio.getc}
end
else
warn("#{filename} doesn't have a Transfer-Encoding: chunked header")
exit(1)
end

print outbuf