Using Scripting to Download TWiT Shows

Once again the TWiT Army rides to the rescue. A number of you have been looking for automated ways to download every single Security Now episode. I’ve received tweets and emails from several listeners who have provided their own scripts. I haven’t tried them, but a cursory glance tells me they’ll probably work and are non-lethal.

Here’s an updated bash script (for The Tech Guy and Macbreak Weekly but you can change that) from Matthew Foot. Don’t forget to set your timezone before running it!

#!/bin/bash
# TWiTCap - captures TWiT show MP3 streams and saves to date/timestamped MP3 files.

# let's fix time/date and sync it - we shall be getting NO shows if the time is wrong!

# Set the timezone below to reflect your locale, eg, mine is Europe/London 

sudo timedatectl set-timezone Europe/London
sleep 1
echo "Date check in 1 second - compare to real time to confirm..."
sleep 1
date

while true
do
        hm_peta=$(TZ=":US/Pacific" date +'%H%M')
        dow=$(date +'%u')
        day=$(date +'%a')
        ymd=$(date +'%F')

        sleep 1
        if      [ $hm_peta -eq "1100" ] && [[ $dow -eq "6" ||  $dow -eq "7" ]]

        then    echo "Capturing The Tech Guy to MP3..."
                timeout 5h wget http://twit.am/listen -O TheTechGuy-$day-$ymd-$hm-US-Pacific.mp3

        elif    [ $hm_peta -eq "1100" ] && [ $dow -eq "2" ]
        then    echo "Capturing MacBreak Weekly and Security Now..."
                timeout 8h wget http://twit.am/listen -O MacBreakWeekly-SecurityNow-$day-$ymd-$hm-US-Pacific.mp3
        else    echo "Waiting for show to start..."
        fi
done

 

Finally, thanks to to Gary Nevills for the following two Powershell scripts. Gary writes:

“I thought I’d share my PowerShell scripts I use to download TWiT podcasts every week (and build up my library of shows). The download scripts work best in PowerShell 5.0 (Windows 10).

This one’s for downloading various shows in audio or video from TWiT’s website, you can also use it to view the download links (without downloading the shows), and highlight links that don’t redirect through PodTrac (see line 16).”

Download TWiT Shows

###########################
##### Begin Functions #####
###########################
Function find-url {
    param
    (
        [Parameter(Mandatory=$true)]
        [string] $DownloadType
    )
    #parse the webpage for the appropriate url
    $fileUrl = "$DownloadType not available for episode $num"
    $results | foreach {Select-String -InputObject $_ -Pattern "download>$DownloadType</a>"} | foreach {$fileUrl = $_}
    $fileUrl = $fileUrl -replace '.*(http.*.mp[3|4]).*', '$1'

    #uncomment below to show each url that doesn't go through podtrac
    #if ($fileUrl -notlike "*podtrac*" -and $fileUrl -notlike "$DownloadType not available for episode $num") {Write-Host $fileUrl}
    #uncomment below to display each url
    Write-Host $fileUrl
    
    #download the file if it is available and download was set to true
    if ($fileUrl -notlike "$DownloadType not available for episode $num" -and $downloadShow) {
        download-show -URL $fileUrl
    }
}
Function download-show {
    param
    (
        [Parameter(Mandatory=$true)]
        [string] $URL
    )
    #get the filename from the url and format it
    $fileName = [System.IO.Path]::GetFileName($fileUrl)
    $fileName = $fileName -replace "(.*[a-z]).*[0]($num).*(.mp[3|4])", '$1$2$3'
    #create the full output filename
    $outputFile = "$outputDirectory$DownloadType$fileName"

    #check if the directory to save file in exists, create it if necessary
    $fileDirectory = [System.IO.Path]::GetDirectoryName($outputFile)
    if (!(Test-Path $fileDirectory)) {New-Item -Path $fileDirectory -ItemType Directory | Out-Null}
        #test if file has already been downloaded, skip if it has
        if (!(Test-Path $outputFile)) {
           #download and save the file
           Invoke-WebRequest $URL -OutFile $outputFile
        }
}
##############################################
###############Begin Parameters###############
##############################################
#Set episode range
$numOfEpisodes = 1 .. 20

#Pick 1 show at a time
$show = 'Security Now'
#$show = 'Windows Weekly'
#$show = 'TWiET'
#$show = 'Coding 101'

#Set the output directoy
$baseOutputDirectory = 'E:userdocumentspodcasts'

#choose which formats to download
$audio = $true
$videoSdSmall = $false
$videoSdLarge = $false
$videoHd = $false

#Download shows true/false - useful if just auditing download URL's
$downloadShow = $true

##############################################
##############End Paraemeters#################
##############################################
################ Begin Script ################
##############################################

$outputDirectory = "$baseOutputDirectory$show"

foreach ($num in $numOfEpisodes) {

switch -regex ($num)
{
    {1..9 -contains $num} {$num = "{0:D1}" -f $num}
    default {$num = "{0:D2}" -f $num}
}
$searchResults = $null
$downloadPage = $null

switch ($show) {
    'Security Now' {$showUrl = "https://twit.tv/shows/security-now/episodes/"+$num+"?autostart=false"}
    'Windows Weekly' {$showUrl = "https://twit.tv/shows/windows-weekly/episodes/"+$num+"?autostart=false"}
    'TWiET' {$showUrl = "https://twit.tv/shows/this-week-in-enterprise-tech/episodes/"+$num+"?autostart=false"}
    'Coding 101' {$showUrl = "https://twit.tv/shows/coding-101/episodes/"+$num+"?autostart=false"}
}

Try {
    $downloadPage = Invoke-WebRequest $showUrl
    $results = $downloadPage.Content -split "`n"
}
Catch {
    Write-Error "Unable to find $showUrl"
    Continue
}
if ($audio) {find-url -DownloadType "Audio"}
if ($videoSdSmall) {find-url -DownloadType "SD Video Small"}
if ($videoSdLarge) {find-url -DownloadType "SD Video Large"}
if ($videoHd) {find-url -DownloadType "HD Video"}
}

And this one’s for getting all of the files from grc.com for security now:

Download A Range of Security Now episodes from grc.com

$numOfEpisodes = 554 .. 557
$outputDirectory = 'E:userdocumentspodcastssecurity now'


foreach ($num in $numOfEpisodes) {

$num3Digit = "{0:D3}" -f $num
$num4Digit = "{0:D4}" -f $num
Write-Host $num3Digit
#Write-Host $num4Digit

$url = "http://www.podtrac.com/pts/redirect.mp3/twit.cachefly.net/audio/sn/sn$num4Digit/sn$num4Digit.mp3"
$outputFile = "$outputDirectoryaudiosn$num4Digit.mp3"
try {if (!(Test-Path $outputFile)) {Invoke-WebRequest $url -OutFile $outputFile}}
catch {Write-Host "$num3Digit not available in audio"}

$url = "https://www.grc.com/sn/sn-$num3Digit-notes.pdf"
$outputFile = "$outputDirectoryshow notessn$num3Digit.pdf"
try {if (!(Test-Path $outputFile)) {Invoke-WebRequest $url -OutFile $outputFile}}
catch {Write-Host "$num3Digit not available in show notes"}

$url = "https://www.grc.com/sn/sn-$num3Digit.pdf"
$outputFile = "$outputDirectorytranscriptspdfsn$num3Digit.pdf"
try {if (!(Test-Path $outputFile)) {Invoke-WebRequest $url -OutFile $outputFile}}
catch {Write-Host "$num3Digit not available in pdf transcript"}

$url = "https://www.grc.com/sn/sn-$num3Digit.txt"
$outputFile = "$outputDirectorytranscriptstextsn$num3Digit.txt"
try {if (!(Test-Path $outputFile)) {Invoke-WebRequest $url -OutFile $outputFile}}
catch {Write-Host "$num3Digit not available in txt transcript"}

}

Thanks, Gary! I haven’t tried these, but I want to put them here so you Powershell gurus can access them and use them for yourselves.