Capture Apple’s HTTP Streaming to File

When Apple introduced HTTP streaming to the world on their iOS platform, I always wondered what would involved in capturing it. After all, it’s just http data, how hard can it be? Well, I’m flying out tomorrow morning for work, and decided it’d be nice to watch the hour+ Celebrating Steve video that Apple just released on the plane ride. Only one problem: It was a streaming video, with no download link in sight.

I dove into the source, and found the <video> tag, and the source file, an HTTP Streaming .m3u8 playlist file. I downloaded the sl_mvp.m3u8 file it an opened it in TextEdit. Inside were the links to the various quality program stream playlists:

#EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=1308077

http://qthttp.apple.com.edgesuite.net/11ajdjnvkjnd10weoihf23ohfoihqw/2540/prog_index.m3u8

I figured the higher the bandwidth, the higher the quality, so I went with the highest. I then downloaded the prog_index.m3u8 file, which contained a sequential list of the transport stream chunks of the video. Each segment was about 10 seconds long. this file contained two important pieces of information: The base file name (fileSequenceXXX.ts), and the total number of chunks (486) for this stream. I figured this would be a perfect use for OS X’s built in curl utility and fired up the Terminal. I made a new folder for my video chunks, cd’d into it, and ran curl:

curl -O http://qthttp.apple.com.edgesuite.net/11ajdjnvkjnd10weoihf23ohfoihqw/2540/fileSequence[0-486].ts

hitting return went about downloading all 487 chunks. When all 1.6GB were done, I had to merge them together. Because these are transport streams, cat works well enough:

cat * > ~/output.ts

That command makes a file called output.ts in my home folder by merging all the small chunks in the folder together (I did use Automator to rename the files to three digit numbers just in case my shell’s file ordering was different than the Finder).

QuickTime X has no problem playing back this H264 video, AAC audio transport stream file directly, which is great: Mission accomplished. But being the perfectionist I am, I felt the need to try and maximize compatibility and get the .ts into a .mov or .mp4 container. VLC did this for me no problem using its wonderful export wizard, but for whatever reason when I played back the resultant .mov (or .mp4) in QuickTime I got black video. VLC played it fine. I’ll look into this more later, but here’s to hoping this info can help someone else!