Scanning the Dump

So I was minding my own business reading my feed and I noticed that @matt-a posted:

PSA: NEW HUGE PASSWORD DUMP. Are You One Of The 501,636,842 Earth Dwellers That Have Been Pwn3d?! You Might Want To Find Out. READ ME.

Now I have to write a ruby script to comb this 29 GB file of hashes. But I have a lot of things to check. I got one of the files over torrent, and extracted it (pwned-passwords-2.0.txt).

Here's what I'm using to check it:

Gemfile

source 'https://rubygems.org'
gem 'highline'

---

check.rb

require 'rubygems'
require 'bundler/setup'
require 'digest'

Bundler.require

filename = 'pwned-passwords-2.0.txt'
cli = HighLine.new
answers = []

loop do
  answer = cli.ask 'What do you want to check?  Empty line to start check.' do |q|
    q.echo = '*'
  end

  break if answer == ''
  answers << answer
end

shas = answers.map do |answer|
  Digest::SHA1.hexdigest(answer).upcase
end

File.open(filename, 'r') do |f|
  f.each_line do |line|
    exit if shas.empty?

    if line =~ /(#{shas.join('|')})/
      puts line
      sha = line.split(':').first
      shas.delete(sha)
    end
  end
end

Enjoy.